How to get stats for several gem5 full system userland benchmark programs under Linux without having to reboot multiple times?

Question

I know how to modify my image, reboot, and re-run it, but that would make my experiments very slow, since boot takes a few minutes.

Is there a way to quickly switch:

• command line options
• the executable

that is being run after boot?

This is not trivial because the Linux kernel knows about:

• the state of the root filesystem
• the state of memory, and therefore of kernel CLI options that could be used to modify init

so I can't just switch those after a checkpoint.

This question is inspired from: https://www.mail-archive.com/gem5-users@gem5.org/msg16959.html

Answer 1

Here is a fully automated setup that can help you to do it.

The basic workflow is as follows:

• run your benchmark from the init executable that gets passed to the Linux kernel

  • https://unix.stackexchange.com/questions/122717/how-to-create-a-custom-linux-distro-that-runs-just-one-program-and-nothing-else/238579#238579
  • https://unix.stackexchange.com/questions/174062/can-the-init-process-be-a-shell-script-in-linux/395375#395375

• to run a single benchmark with different parameters without rebooting, do in your init script:

  m5 checkpoint
  m5 readfile | sh
  

  and set the contents of m5 readfile on a host filesystem file before restoring the checkpoint with:

  echo 'm5 resetstats && ./run-benchmark && m5 dumpstats' > path/to/script
  build/ARM/gem5.opt configs/example/fs.py --script path/to/script
  

  This is what the "configs/boot/hack_back_ckpt.rcS" but I think that script is overly complicated.

• to modify the executable without having to reboot, attach a second disk image and mount after the checkpoint is restored: How to attach multiple disk images in a simulation with gem5 fs.py?

  Another possibility would be 9P but it is not working currently: http://gem5.org/WA-gem5

• to only count only benchmark instructions, do:

  m5 resetstats
  ./run-benchmark
  m5 dumpstats
  

  If that is not precise enough, modify the source of your benchmark with m5ops magic instructions that do resetstats and dumpstats from within the benchmark as shown at: How to count the number of CPU clock cycles between the start and end of a benchmark in gem5?

• to make the first boot faster, you can boot with a simple CPU model like the default AtomicSimpleCPU and then switch to a more detailed, slower model after the checkpoint is restored: How to switch CPU models in gem5 after restoring a checkpoint and then observe the difference?