1

Question

I'm trying to find an explanation for the following inconsistency on Google's PageSpeed Insights (Lighthouse):

The "Core Web Vitals Assessment" section at the top always lists a quite bad TTFB. For instance, it looks like this for a random static HTML document without any additional dependencies (700ms):

TTFB is 0.7s on Core Web Vitals

However, on the same result page Google also lists the "Initial server response time was short" test (in the "passed audits" section) much much better (40ms): initial server response time is only 40ms

In this example, that's a difference of 1750%!

How is this possible?

Additional Thoughts

Both metrics are supposed to describe the same, aren't they?

I do understand that the Core Web Vitals is supposed to be "what your real users are experiencing", i.e. analytical data collected on previous web site calls of real users. In contrast, the other value describes a single test snapshot as performed in that moment.

It's not as easy as "I just had a lucky shot, but usually it takes longer." though. I performed lots of tests from different locations, at different times, using various devices etc. and all manual tests are pretty fast. Only the Core Web Vitals are much worse and I can't find an explanation for it.

"Users having a slower Internet connection" isn't an explanation either, right? I could understand the difference until the last part of the page is there, but how can the very first byte be affected by this in such a drastic way?

Johnson_145
  • 1,994
  • 1
  • 17
  • 26

1 Answers1

4

Both metrics are supposed to describe the same, aren't they?

No they are not the same. Lighthouse deliberately avoids using the term TTFB and uses Server time because they are different.

Lighthouse typically normalises URLs. So if you enter https://web.dev/ttfb and that redirects to https://web.dev/ttfb/ by adding a trailing slash, then Lighthouse will run with the trailing slash. PageSpeed Insights will warn you of this, but your users may be using the pre-redirect URL so won't get this normalisation:

PageSpeed Insights example showing it was run with a redirect

Additionally many of your users will not come directly to the correct URL either. They may be going through link shorteners (e.g. t.co for Twitter), or via an Ad that runs through several redirects before actually requesting the page - and yes redirects count towards TTFB and so also FCP and LCP

Event without these, Lighthouse does not currently show TTFB, but only shows the server response time (it strips out any DNS and Redirect time to just show this portion).

"Users having a slower Internet connection" isn't an explanation either, right? I could understand the difference until the last part of the page is there, but how can the very first byte be affected by this in such a drastic way?

This absolutely can be a reason, even if there are no redirects discussed above. PageSpeed Insights runs from a server and is permanently connected to the internet, whereas your users may not be. It's true that Lighthouse attempts to simulate a slowed down connection, but that is an estimate. Real users may be connecting from far away countries, or patching mobile networks out in the countryside, where there is a significant delay from clicking on a link or typing a URL, before the page is even requested.

Repeatedly running a PSI test may give different results than to real users depending on how your infrastructure is set up. Is it cached at a CDN edge node so the PSI run is repeatedly hitting that cached version and very quick, whereas a user connecting to another CDN edge node may not have that cached and so have to wait until the CDN goes all the way back to the origin? Is the server always running or is there a booting up time if something has not been requested in some time, so again infrequently requested resources (whether for that CDN edge node, or just a lesser trafficked page) may have different server response times for repeat tests than for other users.

Finally, your screenshot shows it is not the page TTFB that is shown, but the origin TTFB as no page level data was available:

Zoom in on This URL/Origin section of above screenshot

Perhaps you have some pages that take longer to generate server-side as they do a lot of processing (for reasons given above, or other reasons) but you're testing a quick page in this test.

Barry Pollard
  • 40,655
  • 7
  • 76
  • 92
  • 1
    Could early hints also cause a fast server response time but not affect the TTFB that CrUX uses? Not sure if the testing tools trigger early hints, and the last I looked, the browser gathered TTFB was affected by early hints. So I suspect this is not the case. – Tony McCreath May 24 '23 at 22:53
  • 2
    CrUX will use Early Hints as TTFB (though that is changing!), but so should PSI. Either way any redirects or slowness before them is likely why there's a difference. – Barry Pollard May 25 '23 at 07:41
  • Thanks for the in-depth answer! I guess in my case it's probably a combination of your first and last points: I wasn't aware of redirects being relevant for the TTFB metric and I didn't pay attention to the `This URL` vs. `Origin` box at the top right. – Johnson_145 May 30 '23 at 10:20
  • The website I tested is relevant in one country only. Therefore we have only one server node and so this point shouldn't affect us. Actually, if it affects the metrics, it would probably work the other way around: Google may access us from server locations far away while real users will almost always be pretty close to our server. Very rarely individual users will indeed be affected by slow loading times caused by an empty cache. In our case this should be very very rare though and not relevant for the overall metric data. However, I guess in general you got a valid point here, too. – Johnson_145 May 30 '23 at 10:29
  • I kind of disagree that Lighthouse avoids using the term TTFB and uses Server time because they are different. Probably they really try to do so, but fail. As you can see in my screenshot, they link to a TTFB page right below the server time test section. The [linked article](https://developer.chrome.com/docs/lighthouse/performance/time-to-first-byte/?utm_source=lighthouse&utm_medium=lr) suggests that both terms are the same, too. That linking confused me a lot. The screenshot on that page suggests that they actually renamed the test to reduce ambiguity, but didn't tidy up the rest. – Johnson_145 May 30 '23 at 10:36
  • The "discover what real users are experiencing" section at the top IS based on real users and not a lab-based tool like Lighthouse. So the fact this is higher than the Lighthouse equivalent metric shows that real users ARE experiencing this as more of a problem (probably for one of the reasons I listed) than LH did in that example run. And yes you're right about the docs. That was raised previously here: https://github.com/GoogleChrome/lighthouse/issues/14255. Will see if I can clean it up. – Barry Pollard May 30 '23 at 14:26
  • 2
    FYI I've updated the Lighthouse docs on server response time. Also made a slight tweak to above answer to state that Lighthouse currently ONLY shows the server response time. So even if you test a site with a redirect, Lighthouse strips this redirect time out to only display the server response time proportion in the "Server response time" audit. – Barry Pollard Jun 15 '23 at 10:36
  • Your explanations are very helpful, thanks. I wish tools like PageSpeed Insights show at least some examplary detalisation regarding if those TTFB measurements had many redirects within them, as that might be a very useful (and actionable) clue – Kostiantyn Ko Jun 23 '23 at 09:36
  • Unfortunately there are some web privacy implications in measuring redirect times. Cross origin information leakage is something the web platform tries very hard to avoid. But I agree it makes it hard to know the exact cause of along TTFBs. – Barry Pollard Jun 23 '23 at 10:17