Can I reuse my integration test suite to profile a Rails app?

Question

Most posts on Rails profiling recommend Ruby-Prof. To use Ruby-Prof I need to write at least one new test for each controller action, then manually compare the results to see what's taking the longest and might be a candidate for optimization.

This is good if I already know exactly what request I'm focusing on. It seems less good if I'm trying to identify the hot spots in the first place. Given that I already have a huge integration test suite covering all the app's functionality that I care about, it seems like what I really want to do is:

1. Run the entire test suite and capture the time spent in each controller action. (Or model method, or whatever level of granularity I want.)
2. Print two lists, of worst-case and average-case times in each controller action.
3. Sort each list and start investigating the longest-running controller actions, now using Ruby-prof or other profiling tools to drill down into the call stack. The worst-case times will identify request params that might be problematic (i.e., trigger slow code on the backend), without my having to think of them all when I write the performance test.

Is there some reason people don't use the integration test suite in this way, rather than basically duplicating it with a second performance test suite? I have not seen it suggested. Before I write code to do something like this (presumably with a before_action in ApplicationController, is there already a tool for this?

Answer 1

I think that the automated tests will not tell you anything about performance. You need real data. For example, your tests probably won't use indexes, but if you create 10,000 records without an index you may find a performance issue.

I need to write at least one new test for each controller action

Why would you performance test each controller action?

In my limited experience, performance testing was done after deploying the app and tested very specific things. I tested a chunk of code that was slow or code that I thought might be slow.

Also, if you use online performance tools it is not necessary to change your code. The online tool runs against an instance of the app that has been deployed.