Obtaining physical address trace from GEM5

Question

I've been trying to extract physical address accessed by the application in order to analyze the row hits.

In doing so, I followed this page with little variation due to version change.

I fixed CacheConfig.py as:

system.monitor2 = CommMonitor()
system.monitor2.trace = MemTraceProbe(trace_file = "CT_mon2.trc.gz")
system.monitor2.slave = system.l2.mem_side

system.membus.slave = system.monitor2.master
system.l2.cpu_side = system.tol2bus.master

And ran a code:

build/X86/gem5.opt --debug-flag=CommMonitor configs/example/se.py --caches --l2cache --l2_size=2MB --mem-type=DDR4_2400_16x4 -c        tests/test-progs/mm/bin/x86/linux/mm --cpu-type=TimingSimpleCPU

The mm is a binary from a simple matrix multiplication:

// C program to multiply two square matrices. 
#include <stdio.h>
#define N 4 

// This function multiplies mat1[][] and mat2[][], 
// and stores the result in res[][] 
void multiply(int mat1[][N], int mat2[][N], int res[][N])
{
    int i, j, k;
    for (i = 0; i < N; i++)
    {
        for (j = 0; j < N; j++)
        {
            res[i][j] = 0;
            for (k = 0; k < N; k++)
                res[i][j] += mat1[i][k]*mat2[k][j];
        }
    }
}

int main()
{
    int mat1[N][N] = { {1, 1, 1, 1},
                    {2, 2, 2, 2},
                    {3, 3, 3, 3},
                    {4, 4, 4, 4}};

    int mat2[N][N] = { {1, 1, 1, 1},
                    {2, 2, 2, 2},
                    {3, 3, 3, 3},
                    {4, 4, 4, 4}};

    int res[N][N]; // To store result 
    int i, j;
    multiply(mat1, mat2, res);

    printf("Result matrix is \n");
    for (i = 0; i < N; i++)
    {
        for (j = 0; j < N; j++)
        printf("%d ", res[i][j]);
        printf("\n");
    }

    return 0;
}

After decoding the "CT_mon2.trc.gz", the memory trace are shown as:

5,u,15360,64,256,11500
6,u,183808,64,2,101000
5,u,18816,64,256,187000
6,u,183744,64,2,285000
5,u,18880,64,256,357000
6,u,171072,64,3,438000
6,u,171648,64,3,526000
6,u,172032,64,3,601000
6,u,174528,64,3,689000
5,u,18944,64,256,765000

The third one indicates physical address.

What I'm confusing is the "u" part. From decode stage, whatever that isn't read(r) or write(w) are notated as "u".

With debugging, commands were repeating with "UpgradeFailResp" and "ReadCleanReq".

I was expecting a trace with reads and writes, but I'm not sure what is happening here.

Can anyone tell me what am I missing?

Or even better way to obtain physical address will be a huge help.

Thanks, jwlee

Answer 1

5,u,15360,64,256,11500

You can find the meaning of these numbers from the packet decoding script in the util folder. For example, 5 means the master(port) id. "u" means it was not either read or write command.

If you want to know the command itself, one possible way is to edit the gem5/src/mem/comm_monitor.cc where timing requests and responses are handled. For example:

DPRINTF(CommMonitor, "cmd: %s, cmdIndex: %d, addr: %lld masterId: %d \n", pkt->cmdString(),pkt->cmdToIndex(),pkt->getAddr(), pkt->masterId());

pkt->cmdString() shows the command or you can simply use pkt->print() to see the packet information. You can investigate packet.cc for more information.

You need to rebuild gem5 every time you change anything in the src folder.

Answer 2

The reason you will see traffic beyond reads and writes has to do with the placement of the CommMonitor. In your system, the membus is likely the point of coherency, so you will get all sorts of traffic generated by the l2 cache that is meant for cache coherency operations with other l2 caches (if they existed). If you move your CommMonitor beneath the point of coherency, e.g. between the membus and and your memory controllers, you should see only read and write traffic.