1
  1. Website is PHP-based (soon it will have much more text for each post)
  2. Data is requested using REST and rendered using javascript templates
  3. Pages are "infinite (endless) scrolling" (it's actually finite..but consists of about 3k items)

For SEO purposes, I would need to somehow generate static HTML pages. I was thinking phantomjs (or any headless browser) which could help here, but there's a big problem with the endless scrolling, since the page must be scrolled down (or up) to render more items, phantomjs won't be able to render all the data, except data initially presented when a page loads (which is a fraction of the maximum page data).

What approach would work to statically render such a setup?
Do you guys even think that's possible?

Thanks.

vsync
  • 118,978
  • 58
  • 307
  • 400
  • http://stackoverflow.com/questions/16561582/how-to-scroll-down-with-phantomjs-to-load-dynamic-content – Artjom B. Nov 23 '14 at 18:58
  • ha..he uses interval and manually changes the scroll position..that would take a some time for a big "endless" page to be fully generated. I think this technique isn't good. A better one would be to have phantomjs go into a mode where the js renders all the items at once, without waiting for scroll. somehow. anyway to post you've mentioned is dealing with a particular case and what I was is a much broader solution. I don't want to deal with scrolling manually at all. – vsync Nov 23 '14 at 19:02
  • What mode would that be? Scrolling is instantaneous. The post is quite versatile and is the way to go for a page that you cannot change. Since you probably can change yours, I would suggest adding a global JavaScript function which loads all of the items by itself without needing PhantomJS to trigger a scroll. Maybe you can even directly deliver the complete data for the escaped_fragment. Maybe I misunderstood you and you ask about best practices for how many items you want to be pre-rendered. – Artjom B. Nov 23 '14 at 19:08
  • 1
    I am also wondering about the benefits of this. the rendered page might get insanely huge a lot of texts and images, and I wonder how would Google treat such a page. Maybe their algorithms for SEO won't like such pages at all, so doing all this might not be worth it. I do not know how their algorithms works for such cases, but maybe someone here does – vsync Nov 24 '14 at 10:06

1 Answers1

1

Fetching a website with infinite scrolling will be tricky:

  • The static file will be huge and Googlebot doesn't like large pages with too many links. That wouldn't give you a SEO edge.
  • Fetching very long website is going to be hard. Performance will suffer and it may take very long time and likely will fail quite often.

How about just modifying the website itself and have paginated version of posts for SEO purpose?

Advantages:

Jakozaur
  • 1,957
  • 3
  • 18
  • 20
  • I as afraid of this situation where google won't like such a generated long page. Well, we use REST where the server sends JSON ,and JS templates are building the DOM, each scroll to the bottom loads N new items. this is pretty easy to maintain, there's not much code at all. Moving to pages isn't a UX we want to have in our website, but I am wondering if there is a way to feed google bot with static pages. That would unfortunately mean the server would also have to render the same templates and we would end up with two separated templates doing the same thing. Hard to maintain. – vsync Nov 24 '14 at 10:13