3

After reading this thread I decided to use pushstate api in my angularjs application which is fully API-based (independent frontend and independent backend).

Here is my test site: http://huyaks.com/index.html

I created a sitemap and uploaded to google webmaster tools.

From what I can see:

google indexed the main page, indexed the dynamic navigation (cool!) but did not index any of dynamic urls. Please take a look.

I examined the example site given in the related thread:

http://html5.gingerhost.com/london

As far as I can see, when I directly access a particular page the content which is presumed to be dynamic is returned by the server therefore it's indexed. But it's impossible in my case since my application is fully dynamic.

Could you, please, advise, what's the problem in my particular case and how to fix it?

Thanks in advance.

Note: this question is about pushState way. Please do not advise me to use escaped fragment or 3-d party services like prerender.io. I'd like to figure out how to use this approach.

Community
  • 1
  • 1
Sray
  • 665
  • 4
  • 13
  • 25

4 Answers4

1

Evidently Quentin didn't read the post you're referring to. The whole point of http://html5.gingerhost.com/london is that it uses pushState and proves that it doesn't require static html for the benefit of spiders.

"This site uses HTML5 wizrdry [sic] to load the 'actual content' asynchronusly [sic] to the rest of the code: this makes it faster for users, but it's still totally indexable by search engines."

Dodgy orthography aside, this demo shows that asynchronously-loaded content is indexable.

djadomi
  • 36
  • 2
  • thank you for your reply. However, this question is still unresolved for me. As I've mentioned before, that demo uses pre-rendered content as well (try to open view-source:http://html5.gingerhost.com/new-york in chrome and you'll see what I mean).Therefore it's indexed. In my particular case only dynamic navigation has been indexed. The dynamic pages have not been indexed. What I'm doing wrong? Could you take a look at my demo? – Sray Oct 27 '14 at 07:23
  • See my test which is indexed at https://www.google.com/search?q=%22What's+the+video+all+about%3F+Just+to+test+that+the+content+is+asynchronous%22 There's no static version of any of this. The data is at http://coger.me/data/data.json (which has CORS set up to allow it.) – djadomi Dec 02 '14 at 20:02
0

As far as I can see, when I directly access a particular page the content which is presumed to be dynamic is returned by the server

It isn't. You are loading a blank page with some JavaScript in it, and that JavaScript immediately loads the content that should appear for that URL.

You need to have the server produce the HTML you get after running the JavaScript and not depend on the JS.

Quentin
  • 914,110
  • 126
  • 1,211
  • 1,335
  • The part of my message was related to http://html5.gingerhost.com/london. They do return static content. >>>>You need to have the server produce the HTML you get after running the JavaScript and not depend on the JS. Do you say I have to feed static content for google bots anyway? – Sray Oct 05 '14 at 13:33
  • Yes. You need static content. – Quentin Oct 05 '14 at 13:48
  • OK, thank you for your reply. Let's say I'd like to use phantomjs. How can I launch it with pushstate angularjs? There's plenty of articles on how to deal with regular urls (hashbangs) but I don't quite understand how to tell search bots to use static content since escaped fragement is not used. – Sray Oct 05 '14 at 14:30
  • @Quentin that really is not 100% accurate anymore, you do not NEED to have static content. I have seen examples (one of which is listed in the OP) of sites that have content indexed that is 100% dynamic content. I personally have a site that is 100% Angular and API driven, that has pages indexed. I'm not quite ready to say the (relatively short-lived) days of requiring pre-rendered content are over, but if that was my business I'd be extremely nervous because Google and Bing are both able to read dynamic JS driven pages and are working to make pre-rendering unnecessary. – LocalPCGuy Oct 29 '14 at 02:54
  • Oops, I did notice that the site listed in the OP does render static data when the route is directly accessed. But I have definitely seen and built sites that are 100% dynamic and that same dynamic content is indexed by Google. I think that delivering static content is still fairly important, but IMO, growing less so with each passing month. – LocalPCGuy Oct 29 '14 at 03:18
0

Google does interpret Angular pages, as you can see on this quick demo page, where the title and meta description show up correctly in the search result.

It is very likely that if they interpret JS at all, they interpret it enough for thorough link analysis.

The fact that some pages are not indexed is due to the fact that Google does not index every page they analyze, even if you add it to a sitemap or submit it for indexing in webmaster tools. On the demo page, both the regular and the scope-bound link are currently not being indexed.

Update: so to answer the question specifically, there is no issue with pushState on the test site. Those pages simply do not contain value-adding content for Google. (See their general guidelines).

Sam
  • 399
  • 2
  • 10
0

Sray, I recently opened up the same question in another thread and was advised that Googlebot and Bingbot do index SPAs that use pushState. I haven't seen an example that ensures my confidence, but it's what I'm told. To then cover your bases as far as Facebook is concerned, use open graph meta tags.

I'm still not confident about pushing forward without sending HTML snippets to bots, but like you I've found no tutorial telling how to do this while using pushState or even suggesting it. But here's how I imagine it would work using Symfony2...

  1. Use prerender or another service to generate static snippets of all your pages. Store them somewhere accessible by your router.

  2. In your Symfony2 routing file, create a route that matches your SPA. I have a test SPA running at localhost.com/ng-test/, so my route would look like this:

    # Adding a trailing / to this route breaks it. Not sure why.
    # This is also not formatting correctly in StackOverflow. This is yaml.
    NgTestReroute:
    ----path: /ng-test/{one}/{two}/{three}/{four}
    ----defaults:
    --------_controller: DriverSideSiteBundle:NgTest:ngTestReroute
    --------'one': null
    --------'two': null
    --------'three': null
    --------'four': null
    ----methods: [GET]

  3. In your Symfony2 controller, check user-agent to see if it's googlebot or bingbot. You should be able to do this with the code below, and then use this list to target the bots you're interested in (http://www.searchenginedictionary.com/spider-names.shtml)...

    if(strstr(strtolower($_SERVER['HTTP_USER_AGENT']), "googlebot"))
    {
    // what to do
    }

  4. If your controller finds a match to a bot, send it the HTML snippet. Otherwise, as in the case with my AngularJS app, just send the user to the index page and Angular will correctly do the rest.

Also, has your question been answered? If it has, please select one so I and others can tell what worked for you.

HTML snippets for AngularJS app that uses pushState?

Community
  • 1
  • 1
Miguel Valencia
  • 221
  • 3
  • 14