Load Runner and Browser Client Performance

Question

I'd like to confirm a few things:

1) Standard load runner scripts do NOT record the time the browser would spend rendering HTML and processing JS?

2) A GUI VUser would be needed to accomplish #1. Are there other ways?

3) The scripting for GUI Vuser is different than that of a standard vuser?

4) Is there any full proof way of determining if LoadRunner is capturing client time (as some analyzing the results not the executor/designer of the test)?

Thanks.

Answer 1

There is only one way to get the end user time inclusive of rendering, which is to sample at the end user interface, hence the GUI virtual user built using Quicktest Professional (if we talk about LR-based solutions).

A standard HTTP virtual user executes at the HTTP protocol level. It is above this level, within the browser, that JavaScript and rendering take place. No architectural proof of this needs to take place.

There is a hybrid model which does execute the Javascript and gets quite close to the end of the client, but still does not get to the exact edge needed for rendering time, this is the TruClient technology in Loadrunner 11. The trade-off in putting all of this functionality into the virtual user technology is that the resource requirements to run this virtual user type are close to that of running an actual browser.

Answer 2

1) Corrent. Web/HTML scripts do not process ANY content (JS or other) 2) QuickTest Professionaö (as mentioned already) 3) Yes 4) Yes

To explain the 4th YES you can record a script of any site that uses JavaScript heavily, do a replay and you'll see that no time is spent on UI rendering or JS processing. You could try the SunSpider JS benchmark as an example :)

The clients process time is generally simulated with lr_think_time() in LR Scripts.

Side note: As for measuring the Client-Render-Time it's a very difficult thing to do, and very inaccurate. The main reason being that YOU as a tester have NO CONTROL over the environment/hardware the actual user is going to use, so even if your tests show it's fast, the users setup may make it slow .. or wise versa ..

Answer 3

Re comments on using the new TruClient protocol

I have seen clients struggle with this when testing Rich Internet Applications

1. The technology is relatively new to LoadRunner
   • so quirky and hard to use at first (e.g. editing scripts as mix of JavaScript and C)
   • the Firefox JavaScript interface can be unforgiving and clunky
2. The VUs do not scale as well as a 'normal' HTTP user
   • each VU on my site reqd 60-70MB memory each
   • so running tests for 000s of users requires lots of hardware (cloud to the rescue ??)
3. As mentioned above the response time is closer in spirit to end user perceptions than a classic HTTP script.