Http header User-Agent

Question

I am trying to get list of browsers from the User-Agent strings in HTTP header. In many of the strings, the browser info is the second entry in the string, like the following:

(compatible;.MSIE.8.0;.Windows.NT.5.1;.Trident/4.0)

But in some of the strings, there is either no browser info, or the info comes as the 3rd entry like in the followings:

(Macintosh;.Intel.Mac.OS.X.10_6_1;.U;.so)
(Macintosh;.Intel.Mac.OS.X.10_6_1;.so)

How to approach this? Is there anything in Python for handling HTTP header fields? Many thanks.

so true, you only can optimize for some browsers, you don't have a general approach — Máthé Endre-Botond, May 31 '11 at 12:58
Also see this thread: http://stackoverflow.com/questions/927552/parsing-http-user-agent-string — Lars Nordin, May 24 '12 at 12:14

score 3 · Answer 1 · answered May 31 '11 at 13:15

I wrote a User Agent analyzer in PHP some time back, so it might be a bit off date, but hope it helps. I extracted the browser info, operating system and language, but I will only include the browser info here.

All major browser names are included in the UA string, but Mozilla is in almoust every one, for Firefox, use the string Firefox. So create an array with the content:

browserList = {'Opera': 'Opera', 
    'Internet Explorer': 'MSIE',
    'Firefox': 'Firefox',
    'Chrome': 'Chrome',
    'Not specified' => ''}

Then try to match these on the UA string. You can add more browsers, if you wan't to expand your statistics. As for the version number, in most cases it occurs right after the browser name. So try extracting the first number-dot-number right after the index you found the browser name.

Your visitor might be a crawler (a bot, like Google's), you can find these by matching against this list:

nuhk, Googlebot, Yammybot, Openbot, Slurp, MSNBot, Ask Jeeves/Teoma, ia_archiver

Hope this helps.

thank you guys for your replies and comments. I will see what can be done:) — Adia, May 31 '11 at 14:15

Http header User-Agent

1 Answers1