Pegasus Mail and Mercury Developer News
For archived developer news articles, click here.

According to the old joke, a camel is just a horse that was designed by a committee: when it came to OAUTH2, though, what the committee produced was more like a two-wheeled donkey.

OAUTH2 is a suite of documents that defines an authentication and authorization process - a set of rules and procedures that allows a user to control how a program can login to a service (for instance, to send mail), and what it can do while it is logged in. Many of its goals are entirely admirable:

• Allows the user to specify narrow areas of information that programs can access - for example, users might grant a mail program access to just their mail, but not to other things like their calendar or browsing history. Using older password-based approaches essentially allowed any application knowing the password to access all the user's data, and potentially do anything it wanted with it.


• Protects users from themselves by somewhat reducing their vulnerability when they use the same password on multiple sites, use weak passwords, or never change their passwords.


• Theoretically allows a better "user experience" by doing the actual login to the site using the site's own login facility, which is presumably more familiar to the user than a mail program's generic one. [Note - I currently dispute this one, and will explain why later on].

There are other more minor benefits that the developers of the framework make reference to, but these are the main ones the user will see. But you only get benefits from something like this if it is well-designed and implemented.

Very annoyingly, sites like GMail and Microsoft's outlook.com site have taken to calling OAUTH2 "modern authentication", as if this somehow marks it as a well-thought-out, balanced mechanism: unfortunately, neither is true.

OAUTH2 has had a very checkered history: originally a rather more complex framework called OAUTH 1, the process of developing OAUTH2 was so internally fraught that the lead author of the specification resigned and removed his name from the process before it was released. If you'd like to see a little history of OAUTH2, Wikipedia has an article here:

So why am I so critical of OAUTH2? Let me count the ways...

Before I start, though, I have to be clear up front about one thing: many of the goals of OAUTH2 are valid and worthwhile: my problems with it are exclusively to do with how it has been implemented. In my nearly thirty-five years of writing software in service of the Internet, OAUTH2 is the worst-conceived piece of software design I have ever encountered. More troublingly, it shows the increasing levels of control and power exercised by large, usually American corporations over the Internet, and the almost complete disregard they have for its historical openness and inclusiveness. OAUTH2 is a major step on the way to an Internet where the only players are large corporations, serving their own interests in the name of profit and power.

Issue one: The standard you're having when you're not having a standard.

The main problem with OAUTH2 can be seen in the first paragraph of the 'standard' that defines it - RFC6749:

— in other words, OAUTH2 is not a standard - it is instead a broad guide to how something claiming to be an "OAUTH2 implementation" should behave. Earlier protocols such as POP3, SMTP and IMAP, while undoubtedly flawed in the area of authentication, at least specified the detail of how the protocols worked: the OAUTH2 specification does not do this — rather, it defines broadly how an OAUTH2 implementation might work. It lays down little in the way of actual rules, and does not mandate any particular implementational detail. This leads me to my largest criticism of OAUTH2:

• OAUTH2 potentially requires client developers to produce different modules for every service to which they wish to provide the ability to connect. This is clearly a nonsense, and can be viewed as a cynical corporate attempt to squeeze out small-to-medium developers who simply won't have the resources to be able to provide custom-tuned implementations for every OAUTH2 provider.

Issue 2: Who polices the policemen?

The next issue is in some ways even worse: client developers, such as myself, are required to register their program with every OAUTH2 provider they wish to support. Once again, this can easily be viewed as a cynical power-grab: small-to-medium developers simply do not have the resources to be able to manage hundreds or even thousands of separate registrations in this way. At best, we will be limited to supporting a handful of the largest providers — which a sceptic like myself can easily construe as being just what those providers want.

But there's an even darker aspect to this requirement that is perhaps not immediately apparent: what happens if a site declines to accept your registration, or revokes your registration at a later time? Suddenly, a corporation (because the large players are all corporations) has control over whether or not your program is useful to your users: in other words, after thirty-five years of providing Pegasus Mail as a service to the broader Internet, I now find myself in a place where its usefulness and applicability is becoming subject to the whims of large, unaccountable corporations. To summarize my second main objection to OAUTH2, then:

• OAUTH2 requires obscene amounts of pointless administrative make-work by client developers, and places the viability of their products in the hands of corporate entities who are in no way accountable to anyone except their major shareholders.

Issue 3: "Make it horrible for them — it builds character."

Those two points would, on their own, have been enough to explain my antipathy for OAUTH2, but wait — there's more! OAUTH2 is needlessly complex: using my experience as an example, developing an even marginally useful OAUTH2 client for Pegasus Mail has required that I understand several Internet Standards Documents (or RFCs in industry parlance) that are severely jargon-ridden even according to the norms of such documents; because OAUTH2 appears to have been concocted almost entirely by web developers, it has also required me to write nearly 7,000 lines of code, just to support GMail —

• A full HTTP/HTTPS client for validation and token issue


• A full JSON parser to handle responses that could just as easily have been plain text with no reduction in functionality.


• A robust encrypted storage mechanism for token storage


• A large, detailed framework of my own to give me at least some chance of being able to support other providers without going through the whole damned process again.


• Changes to my POP3, SMTP and IMAP authentication routines

... and so it goes on. I estimate it has taken me more than 200 hours to get to the point of having something that works with a single provider - 200 stressful, frustrating hours for which nobody has paid me anything, and which have kept me away from other tasks that I consider a lot more important.

Things have not been helped by GMail: while it is not my intention to suggest that GMail is any worse than any other implementor, it is the one with which I am currently familiar, and hence the only one whose issues I feel competent to describe:

• GMail's developer support pages are years out of date: the Python sample they provide requires an obsolete version of Python to run, and gives little more than a basic overview of what you need to do. As with so many software tasks, devils abound in the details, and GMail's developer resources are sadly deficient in detail.


• The only practical method for the user to return GMail's permission to the application (copy and paste) is deprecated, and will presumably become unavailable sometime, but the other methods (such as starting up your own web server to handle the return - geez, give me a break, already!) are both over-complicated, and overkill.


• GMail do not document their status codes. When GMail returns an error code, there simply isn't any place I can look up what that error code means; furthermore, their errorcodes are inconsistently applied - a status return of "400", for instance, might mean any of the following:
  
  - You have provided an expired token - refresh it and try again.
  - The syntax of your command is incorrect
  - The syntax of the token block you provided is incorrect
  
  (That last one took me many hours to work out - it turned out that the keyword 'Bearer' is regarded as different from 'bearer').
  
  Without being able to determine what "status 400" actually means, it's nearly impossible for me to know how to handle it.


• Testing is difficult, because GMail will lock you out of your GMail account for a day at the slightest provocation, yet while developing a live protocol, a certain error rate is unavoidable. Testing that should have taken me a couple of hours ended up taking me more than a week as I waited for lockout timers to expire. Google provide a thing called the "OAUTH2 playground", but it has no documentation, seems to be exclusively intended for web developers, and I could not work out what use it was, nor whether it was in any way relevant to my efforts.


• Needless to say, there is no way of contacting anyone about anything during this process, other than to provide feedback which will doubtless be ignored (I'm sure mine was). Be warned - if you do OAUTH2 with GMail, you are on your own.


• According to their guidelines, I am probably going to have to submit Pegasus Mail for their approval and "validation" at least once a year, presumably at my own expense.

Finally, it's worth mentioning GMail's "improved user experience", which involves the user going through a quite scary sequence of dialogs that warn of the application deleting all their mail and insist that the user know and trust the program, all the while offering no fewer than three places where they encourage the user to back away... Look, it's fine to warn the user, but the GMail "user experience" looks more like one designed to frighten the average casual user away from allowing access to the application. More corporate heavy-handedness, anyone?

So now we have reached point three in my list of OAUTH2 donkeyisms:

• OAUTH2 is needlessly complex and convoluted, has little or no real-world standardization, and is at best dismally documented.

Amazingly, there is more, but if you've read this far in this document, I admire your stamina, and won't burden you with more ranting.

At a personal level, as I remarked above, developing an OAUTH2 client for GMail has been a frustrating, stressful experience that I regard as a complete waste of time I could have spent on any number of other more deserving tasks.

But I'm afraid there's probably little that can be done about it now. The agenda is already written, and the corporate spin machine will probably not even regard this little litany of complaints as worth reading.

I just hope the Internet can cope with this perversion and move past it at some stage.

-- David Harris --
May 2^nd 2022.

Addendum — May 9^th 2022

I have today attempted to submit my application, Pegasus Mail, to GMail to go through their "validation process". Clicking the button that said, simply enough, "Publish App" took me to a bewildering, convoluted multi-list screen of different terms, conditions and requirements — even including the demand that I make a Youtube video showing the code operating (seriously?!).

That, while burdensome, confusing and complicated would probably have been manageable — and of course, it's all freighted in terms of "protecting GMail users' data", so you cannot be seen to criticize it without appearing to be somehow irresponsible… But right at the end is the sucker punch — Google will charge you from "$10,000 to $75,000 or more" (their words, not mine) for this, and will require you to go through the process (and of course, pay the fee) annually.

I refer you back to my original contention that OAUTH2 is a step towards an Internet completely controlled by large corporations: perhaps you thought that statement was an exaggeration when you read it then — do you still think that now?

[ Page modified 9 May 2022 | Content © David Harris