Quite a lot of people around the world have been reporting that TTwitter doesn’t work on theirs or their client’s machines… with Twitter reporting a 401 error on each request!
I have been attempting to track down the cause of this problem for quite some time, and I have finally discovered where the issue resides!
TTwitter uses a third-party OAuth library to provide authentication and signing services for all requests sent via the Twitter API.
Part of the OAuth system involves generating a signature, which uses – amongst other things – the current Date and Time as part of this serialization.
I have discovered that this portion of the OAuth unit makes some fatal assumptions about the operating environment, in that it is hard-coded to interpret TDate and TDateTime values with very specific formatting.
The problem with this is that not every computer uses the same TDateTime format (the UK, for example, uses DD/MM/YYYY for dates, while the USA uses MM/DD/YYYY). This means that the value being serialized into the signature is incorrect, thus the signature is incorrect.
I will of course be fixing this in the near future (in the next few days, in fact), and will release an update to the TTwitter SVN repository!
Another announcement will be posted when this fix is available to all…. and I’m genuinely sorry it took so long to track down the cause of this fault.
Please keep in mind that bug tracking is difficult when you cannot replicate the problem. In my case, all of my systems use the standard UK TDateTime formatting, and thus I will never encounter the 401 issues. Indeed, the only reason I was able to isolate the cause was because of an e-mail from Volkan Atman (a Delphi developer using TTwitter in his project) detailing how all of his systems work perfectly with TTwitter, while clients in other countries are getting the 401 issues with the compiled executable.
I want to thank Volkan Atman for being so thorough in his bug report… and will post the body of his message below so others can see the kind of information you can provide in the future to help make bug tracking (and thus repairs) easier.
Hello Mr. Stuart,
I have been using your component. I also tried the new version. After compiled my Delphi program, which I used your component, I tried that at 3 different laptop (one of them XP SP3, the others XP SP2). Only one of them (XP SP3) asked me authentication, I remember it was a twitter page. But I forgot to take a screenshot. Anyway, there is no problem.
Then I sent the .exe and AppDetails.ini to my friend. He tried at two laptop (XP SP2 and vista). But he got 401 unauthorized error.
I know lots of people asked like these questions. But I didn’t see any problem like mine, so I want to ask to you.
If you can help, I would be happy.
Regards.
January 25, 2012 at 4:02 pm
Is this the RFC 822 date format?
function FormatDateTimeRFC822(const d: TDateTime): string;
const // as defined by Standard for the Format of ARPA Internet Text Messages,
// section 5.1: http://www.sendmail.org/rfc/0822.html#5
Days : Array [1..7] of string =
(‘Sun’, ‘Mon’, ‘Tue’, ‘Wed’, ‘Thu’, ‘Fri’, ‘Sat’);
Months: Array [1..12] of string =
(‘Jan’, ‘Feb’, ‘Mar’, ‘Apr’, ‘May’, ‘Jun’, ‘Jul’, ‘Aug’, ‘Sep’, ‘Oct’,
‘Nov’, ‘Dec’);
var
dd, mm, yy, hh, mn: Word;
tz : TIME_ZONE_INFORMATION;
dt: TTime;
gmt: string;
begin
DecodeDate(d, yy, mm, dd);
// compute GMT bias
GetTimeZoneInformation( tz );
hh := abs(tz.Bias div 60); // hours
mn := abs(tz.Bias mod 60); // minutes
dt := EncodeTime(hh, mn, 0, 0);
if ( (tz.Bias div 60) < 0) then
gmt := '-' + FormatDateTime('hhnn', dt)
else
gmt := '+' + FormatDateTime('hhnn', dt);
Result := Days[ DayOfWeek(d) ] + ', ' +
IntToStr( dd ) + ' ' +
Months[ mm ] + ' ' +
IntToStr( yy ) + ' ' +
FormatDateTime('hh:nn:ss', d) + ' ' + gmt;
end;
January 25, 2012 at 4:11 pm
The specific function of concern is on line 173 (to line 182) of Twitter_OAuth.pas in the TTwitter repository, as well as line 164 being of concern as it’s very likely this constant value is format-specific too!
I’m using a function similar to the one you provided to convert the Date/Time String values contained within Twitter’s returned packets for Tweets etc.
At least now I finally know where to look, so I can fix this problem (finally).
February 6, 2012 at 3:22 pm
I’m afraid I’m not following you. In the post, you speak about formattings. Later, you make lines 173 to 182 responsible – which are about converting a TDateTime to unix time, which has nothing at all to do with country locales.
You speak about different “TDateTime” formats – but TDateTime has just a single one. Days since a specific date before the digit, the fractional part of the day behind it. TDateTime is a double, not a string or a complicated bit masked thing!
All that dd/mm/yy or mm/dd/yy stuff is used in FormatDateTime to convert it to text to display to the user, but irrelevant when simply handling the TDateTime value.
So let’s throw this (imho wrong) assumption aside, and pay a closer look to DateTimeToUnix.
1. Why did you implement it yourself? Take a look at SysUtils.pas, Delhi already ships a procedure to convert a TDateTime to Integer.
2. I must admit I wrote one myself at some point
Comparing, I immediately noticed that if you use the Bias, you need to use DaylightSaving as well!
function ConvertDateTimeToUNIXTime(ADateTime: TDateTime): cardinal;
begin
if DaylightSavings then begin
Result := Round((ADateTime – 25569 + ((TZInfo.Bias + TZInfo.DaylightBias) / 1440)) * 86400)
end else begin
Result := Round((ADateTime – 25569 + ((TZInfo.Bias) / 1440)) * 86400);
end;
end;
from this units initialization:
begin
DaylightSavings := GetTimeZoneInformation(TZInfo) = TIME_ZONE_ID_DAYLIGHT;
end.
To sum this up – you should use the daylight savings to have correct values in summer, but other than that, I would say the given lines are pretty correct and not at fault.
What I can see is that GenerateNonce is calling GenerateTimestamp. Since each nonce per same timestamp needs to be unique, this means you’ve got duplicate and this invalid nonces for all requests within the same second! At least, GenerateNonce should include a long random string attached, like IntToStr(Random($FFFFFF)) or something like that to make collisions less likely (if you do not want a proper list of used up nonces). Or, an auto-incrementing value that is +1 on each GenerateNonce. Not so nice from the crypto standpoint, but at least preventing collisions safely. Well, or create a list of used-up nonces of course.
February 6, 2012 at 5:03 pm
I was pointing to lines to do with Date/Time handling, knowing that Date/Time handling is (at some point) responsible for the issue. I haven’t looked deep enough into the code as-yet (been busy with Lua4Delphi) so do not know for sure exactly where the problem is.
I never implemented anything in that OAuth unit myself! It was found online (open source)… not written by me!
I appreciate the input and you taking the time to go through this… I’ll give this a go as soon as I have a moment to look at it!
Cheers
January 25, 2012 at 4:23 pm
So you need this – and it’s reverse – I guess.
http://delphi.fosdal.com/2008/09/snippet-convert-java-time-to-tdatetime.html
January 25, 2012 at 4:27 pm
I’ll give it a look, thanks
January 25, 2012 at 5:09 pm
Big thanks Simon and keep the good work. We will be expecting the update
January 25, 2012 at 5:29 pm
No worries… just sorry it has taken this long to figure it out
January 26, 2012 at 8:10 am
I am pleased to help solving a problem. I thanked you for this component.
January 27, 2012 at 2:12 pm
Date format causing authentication problems on some computers – it sure was hard to find! Nice
January 27, 2012 at 2:50 pm
Sometimes it’s the simplest things which cause the most pervasive problems
February 3, 2012 at 3:53 pm
Hi Simon
Another way to trigger the 401 error is using special characters if you are logging & twitting.
Like this sample code:
procedure TfrmMainForm.Button1Click(Sender: TObject);
var
twit: String;
begin
twit := ‘Temperature 30º C’; // 401 ERROR
// twit := ‘Temperature 30 C’; NO ERROR
if Twitter1.Login then
Twitter1.PostTweet(False,twit, 0);
End;
(in this sample, the º character)
Regards
-Jordi
February 3, 2012 at 6:35 pm
Yes, this is due to the way unicode chars are encoded in the URL string! For some reason this same issue occurs when I use Twitter’s own “Developer Console” on their dev site! I’ve some ideas to solve that too!
February 22, 2012 at 7:51 pm
Any update to provide on TTwitter just yet? I realise Lua4Delphi has been a major project, and you’ve also been busy with an article for Blaise (congratulations on both accounts), but i’d love to hear if you’ve had chance to work on TTwitter in amongst that.
Regards,
Scott.
February 24, 2012 at 12:13 am
Addressed in my recent blog post.