My opencl kernel taking too long to finish, rocprof gave me below counters:
| Index | KernelName | gpu-id | queue-id | queue-index | pid | tid | grd | wgr | lds | scr | arch_vgpr | accum_vgpr | sgpr | wave_size | sig | obj | TCC_RW_REQ_sum | TCC_PROBE_sum | TCC_PROBE_ALL_sum | TCC_READ_sum | TA_DATA_STALLED_BY_TC_CYCLES_sum | TA_FLAT_WAVEFRONTS_sum | TD_SPI_STALL_sum | TCP_TOTAL_ATOMIC_WITH_RET_sum | TCP_TOTAL_ATOMIC_WITHOUT_RET_sum | TCP_TOTAL_WRITEBACK_INVALIDATES_sum | TCP_UTCL1_REQUEST_sum | TCC_WRITE_sum | TCC_ATOMIC_sum | TCC_HIT_sum | TCC_MISS_sum | L2CacheHit | TA_FLAT_ATOMIC_WAVEFRONTS_sum | TA_BUFFER_WAVEFRONTS_sum | TCP_UTCL1_TRANSLATION_MISS_sum | TCP_UTCL1_TRANSLATION_HIT_sum | TCP_UTCL1_PERMISSION_MISS_sum | TCP_TOTAL_CACHE_ACCESSES_sum | TCC_WRITEBACK_sum | TCC_EA_WR_UNCACHED_32B_sum | TCC_EA_WRREQ_IO_CREDIT_STALL_sum | TCC_EA_WRREQ_GMI_CREDIT_STALL_sum | TA_BUFFER_READ_WAVEFRONTS_sum | TA_BUFFER_WRITE_WAVEFRONTS_sum | TCP_TCP_LATENCY_sum | TCP_TA_TCP_STATE_READ_sum | TCP_TCC_READ_REQ_LATENCY_sum | TCP_TCC_WRITE_REQ_LATENCY_sum | FETCH_SIZE | WRITE_SIZE | WRITE_REQ_32B | FetchSize | WriteSize | MemWrites32B | TCC_EA_WRREQ_sum | TCC_EA_WRREQ_64B_sum | TCC_EA_RDREQ_sum | TCC_EA_RDREQ_32B_sum | VFetchInsts | VWriteInsts | MemUnitStalled | TA_FLAT_READ_WAVEFRONTS_sum | TA_FLAT_WRITE_WAVEFRONTS_sum | TD_TD_BUSY_sum | TD_TC_STALL_sum | TCP_GATE_EN1_sum | TCP_GATE_EN2_sum | TCP_TD_TCP_STALL_CYCLES_sum | TCA_CYCLE_sum | TCA_BUSY_sum | FlatVMemInsts | LDSInsts | ALUStalledByLDS | GPUBusy | VALUInsts | FlatLDSInsts | Wavefronts | TCC_EA_WRREQ_DRAM_CREDIT_STALL_sum | TCC_TOO_MANY_EA_WRREQS_STALL_sum | TCC_EA_WRREQ_LEVEL_sum | TCC_EA_RDREQ_LEVEL_sum | TA_BUFFER_ATOMIC_WAVEFRONTS_sum | TA_BUFFER_TOTAL_CYCLES_sum | TCP_TCC_READ_REQ_sum | TCP_TCC_WRITE_REQ_sum | TCP_TCC_ATOMIC_WITH_RET_REQ_sum | TCP_TCC_ATOMIC_WITHOUT_RET_REQ_sum | TCC_EA_ATOMIC_sum | TCC_EA_ATOMIC_LEVEL_sum | TCC_EA_RD_UNCACHED_32B_sum | TCC_EA_RDREQ_IO_CREDIT_STALL_sum | TA_BUFFER_COALESCED_READ_CYCLES_sum | TA_BUFFER_COALESCED_WRITE_CYCLES_sum | TCP_TCC_NC_READ_REQ_sum | TCP_TCC_NC_WRITE_REQ_sum | TCP_TCC_NC_ATOMIC_REQ_sum | TCP_TCC_UC_READ_REQ_sum | TCC_EA_RDREQ_GMI_CREDIT_STALL_sum | TCC_EA_RDREQ_DRAM_CREDIT_STALL_sum | TCC_TAG_STALL_sum | TCC_NORMAL_WRITEBACK_sum | TCP_TCC_UC_WRITE_REQ_sum | TCP_TCC_UC_ATOMIC_REQ_sum | TCP_TCC_CC_READ_REQ_sum | TCP_TCC_CC_WRITE_REQ_sum | TCC_ALL_TC_OP_WB_WRITEBACK_sum | TCC_NORMAL_EVICT_sum | TCC_ALL_TC_OP_INV_EVICT_sum | TCC_EA_RDREQ_DRAM_sum | TCP_TCC_CC_ATOMIC_REQ_sum | TCP_TCC_RW_READ_REQ_sum | TCP_TCC_RW_WRITE_REQ_sum | TCP_TCC_RW_ATOMIC_REQ_sum | TCC_EA_WRREQ_DRAM_sum | TCP_PENDING_STALL_CYCLES_sum | WriteUnitStalled | TCC_BUSY_avr | TCC_WRREQ_STALL_max | TCC_CYCLE_sum | TCC_BUSY_sum | TCC_REQ_sum | TCC_EA_WRREQ_STALL_sum | MemUnitBusy | TA_BUSY_avr | TA_BUSY_max | TA_BUSY_min | TA_TA_BUSY_sum | TA_TOTAL_WAVEFRONTS_sum | TD_LOAD_WAVEFRONT_sum | TD_ATOMIC_WAVEFRONT_sum | TCP_TCR_TCP_STALL_CYCLES_sum | TCP_READ_TAGCONFLICT_STALL_CYCLES_sum | TCP_WRITE_TAGCONFLICT_STALL_CYCLES_sum | TCP_ATOMIC_TAGCONFLICT_STALL_CYCLES_sum | SALUInsts | SFetchInsts | GDSInsts | VALUUtilization | VALUBusy | SALUBusy | LDSBankConflict | TCC_STREAMING_REQ_sum | TCC_NC_REQ_sum | TCC_UC_REQ_sum | TCC_CC_REQ_sum | TA_ADDR_STALLED_BY_TC_CYCLES_sum | TA_ADDR_STALLED_BY_TD_CYCLES_sum | TD_STORE_WAVEFRONT_sum | TD_COALESCABLE_WAVEFRONT_sum | TCP_VOLATILE_sum | TCP_TOTAL_ACCESSES_sum | TCP_TOTAL_READ_sum | TCP_TOTAL_WRITE_sum | DispatchNs | BeginNs | EndNs | CompleteNs |
| 0 | kernel1.kd | 0 | 0 | 0 | 352005 | 352009 | 6400 | 64 | 1024 | 0 | 48 | 0 | 32 | 64 | 0x7f4d1388e100 | 0x7f4c11844a00 | 25728.0000000000 | 0.0000000000 | 0.0000000000 | 25606.0000000000 | 12327.0000000000 | 1800.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 100.0000000000 | 2100.0000000000 | 300.0000000000 | 0.0000000000 | 25437.0000000000 | 467.0000000000 | 98.0813814054 | 0.0000000000 | 0.0000000000 | 100.0000000000 | 1700.0000000000 | 100.0000000000 | 2100.0000000000 | 64.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 456014.0000000000 | 1800.0000000000 | 185980.0000000000 | 56218.0000000000 | 49.9375000000 | 3.9062500000 | 125.0000000000 | 49.9375000000 | 3.9062500000 | 125.0000000000 | 63.0000000000 | 62.0000000000 | 799.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0224162944 | 1500.0000000000 | 300.0000000000 | 323668.0000000000 | 281804.0000000000 | 66528293.0000000000 | 403183.0000000000 | 0.0000000000 | 2580712.0000000000 | 424928.0000000000 | 18.0000000000 | 2021.0000000000 | 0.0000000000 | 100.0000000000 | 10410.0000000000 | 0.0000000000 | 100.0000000000 | 0.0000000000 | 0.0000000000 | 3720.0000000000 | 92654.0000000000 | 0.0000000000 | 0.0000000000 | 1152.0000000000 | 300.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 130.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 2872.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 32.0000000000 | 0.0000000000 | 0.0000000000 | 741.0000000000 | 0.0000000000 | 1152.0000000000 | 300.0000000000 | 0.0000000000 | 63.0000000000 | 216083.0000000000 | 0.0000000000 | 9780.5625000000 | 0.0000000000 | 21183648.0000000000 | 312978.0000000000 | 25466.0000000000 | 0.0000000000 | 0.1040613968 | 4230.6875000000 | 5511.0000000000 | 0.0000000000 | 67691.0000000000 | 1800.0000000000 | 1800.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 4344.0000000000 | 9.0000000000 | 0.0000000000 | 12.8938886488 | 1.5124873676 | 0.6309656319 | 0.0000000000 | 0.0000000000 | 112.0000000000 | 65.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 300.0000000000 | 1200.0000000000 | 57600.0000000000 | 57600.0000000000 | 48000.0000000000 | 9600.0000000000 | 3664882497116233 | 3664135843101786 | 3664882514625889 | 3664882514742848 |
| 1 | kernel2.kd | 0 | 0 | 2 | 352005 | 352009 | 6400 | 64 | 1024 | 16 | 72 | 0 | 96 | 64 | 0x7f4d1388e200 | 0x7f4c11844a40 | 58756.0000000000 | 0.0000000000 | 0.0000000000 | 58351.0000000000 | 12361.0000000000 | 3800.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 100.0000000000 | 4900.0000000000 | 700.0000000000 | 0.0000000000 | 56967.0000000000 | 1088.0000000000 | 98.0573111175 | 0.0000000000 | 200.0000000000 | 200.0000000000 | 3700.0000000000 | 200.0000000000 | 4900.0000000000 | 30.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 100.0000000000 | 100.0000000000 | 878610.0000000000 | 4000.0000000000 | 397328.0000000000 | 129527.0000000000 | 101.3125000000 | 0.0000000000 | 0.0000000000 | 101.3125000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 1621.0000000000 | 0.0000000000 | 1.0000000000 | 1.0000000000 | 0.0108338625 | 3500.0000000000 | 300.0000000000 | 729388.0000000000 | 634273.0000000000 | 557707761.0000000000 | 905423.0000000000 | 0.0000000000 | 23601924.0000000000 | 1053565.0000000000 | 38.0000000000 | 17652.8800000000 | 0.0000000000 | 100.0000000000 | 86416.8800000000 | 0.0000000000 | 100.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 163630.0000000000 | 0.0000000000 | 800.0000000000 | 2630.0000000000 | 700.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 144.0000000000 | 0.0000000000 | 400.0000000000 | 400.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 2588.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 1558.0000000000 | 0.0000000000 | 2630.0000000000 | 700.0000000000 | 0.0000000000 | 0.0000000000 | 434042.0000000000 | 0.0000000000 | 23384.9375000000 | 0.0000000000 | 188428736.0000000000 | 748318.0000000000 | 59052.0000000000 | 0.0000000000 | 0.0227353858 | 8258.3125000000 | 10710.0000000000 | 0.0000000000 | 132133.0000000000 | 4000.0000000000 | 4000.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 46104.9200000000 | 18.0000000000 | 0.0000000000 | 17.3315676902 | 1.4111824381 | 0.7528644267 | 0.0000000000 | 0.0000000000 | 224.0000000000 | 75.0000000000 | 0.0000000000 | 0.0000000000 | 0.0000000000 | 400.0000000000 | 2700.0000000000 | 126400.0000000000 | 126400.0000000000 | 115200.0000000000 | 11200.0000000000 | 3664882525999037 | 3664882529819153 | 3664882563472360 | 3664882563585605 |
I tried to understand the columns with rocprof --list-derived but its description to counters is very short n I couldn't comprehend much. For example, it says
TA_BUSY_avr : TA block is busy. Average over TA instances.
TA_BUSY_avr = avr(TA_TA_BUSY,16)
What's a TA block? and:
TD_TD_BUSY_sum : TD is processing or waiting for data. Perf_Windowing not supported for this counter. Sum over TD instances.
TD_TD_BUSY_sum = sum(TD_TD_BUSY,16)
What's a TD instance? and:
TCP_GATE_EN1_sum : TCP interface clocks are turned on. Not Windowed. Sum over TCP instances.
TCP_GATE_EN1_sum = sum(TCP_GATE_EN1,16)
What's a TCP instance? and:
TCA_CYCLE_sum : Number of cycles. Sum over all TCA instances
TCA_CYCLE_sum = sum(TCA_CYCLE,16)
What's a TCA instance? and:
TCC_WRREQ_STALL_max : Number of cycles a write request was stalled. Max over TCC instances.
TCC_WRREQ_STALL_max = max(TCC_EA_WRREQ_STALL,32)
What's a TCC?
How can i improve my kernels kernel1 and kernel2?
