4

i want to do some global statistics on the apple app store. One bottleneck is getting getting at least the ids (these 9 digit numbers) of all currently active apps. Does anyone know a way to get a complete list of all ids of currently active apps on the iOS app store. Even better all ids of a specific category (say "education").

I tried several ways but none is reliable.

There is a list of apps at http://iphoneapplicationlist.com/apps/ipad/ but the list stops at November 2012. Also apple provides a search list at https://itunes.apple.com/us/genre/ios-education/id6017?mt=8 but this list turns out to be incomplete. (If I compare it with the other list there are more than 8000 apps missing, In particular my own one :-) ).

So my qeustion is: provided one can oneself do some php and javascript programming is there a good way of getting all app ids of all apps.

CloudyMarble
  • 36,908
  • 70
  • 97
  • 130
Jürgen Richter-Gebert
  • 57
  • 4

1 Answers1

0

I cannot find a way to get all mobile apps, but the iTunes sitemaps files seem to contain a list of apps that are recently updated. (I'm not sure exactly which apps are contained in the sitemaps files -- I will update this answer once I work it out.)

To get this list:

  1. Crawl https://itunes.apple.com/robots.txt. This is what it looks like: itunes.apple.com robots.txt file

  2. Crawl those sitemap files. They reference more sitemap files, which reference more sitemap files, etc, etc. Some sitemaps are in XML format, some are in gzipped XML format and some are in JSON format. Exclude any sitemap that contains itunes_music in the name, as those do not contain apps.

    There are about 12,500 sitemaps of interest. At 4 sitemaps per second, you can download the sitemaps in under an hour.

  3. The sitemaps contain loc elements and xhtml:link[href][rel=alternate] elements. Parse the XML, and get these values.

  4. Parse each URL for the locale/country code and IDs.

I discovered 719,344 IDs in this way.

Community
  • 1
  • 1
Brendon
  • 848
  • 8
  • 24