2

My understanding is that when a NIC adapter receives new packets, the top half handler uses DMA to copy data from the RX buffer to the main memory. I think this handler should not exit or release the INT pin before the transmission is completed, otherwise new packets would corrupt the old ones.

However, DMA is generally considered asynchronous and itself requires the interrupt mechanism to notify the CPU that data transmission is done. Thus my question, is DMA actually synchronous here, or interrupt can in fact happen within another interrupt handler?

wlnirvana
  • 1,811
  • 20
  • 36

1 Answers1

3

In general, this synchronisation happens via ring descriptor between NIC(device driver) and host CPU. You will get packet path details here. I have explained the ring-descriptors below.

Edit:

Let me explain with Intel's ethernet Controller. If you look at section 3.2.3, where the RX descriptor format is given, it has status field which solves packet ownership problem. There are two major points to avoid contention and packet corruption as to who owns the packet (NIC driver or CPU).

DMA (from I/O device to Host memory): RX/TX Ring consists of 'hardware descriptors' and 'buffers' (carved from host memory). When we say DMA, controller transfer data, this happens from hardware FIFOs to this host memory.

  1. Let us assume my ring buffers (of 512 bytes) are not big enough to hold the complete incoming packet(1500 or Jumbo packet), in that case the packet may span across multiple ring buffers and with EOP(End Of Packet) status field, indicates that the complete packet is now received (considering all the sanity checks/checksums are already done).
  2. Second is who owns the packet now (driver or CPU for further consumption)? Now until the status flag DD (Descriptor Done) is set, it belongs to driver. Once set CPU can grab it for picking-and-poking.

This is specific to RX path. TX path is slightly different.

Consider it this way, there are multiple interrupts (IO, keyboard, mouse, etc) happening all the time in the system, but the time duration between two interrupts are so huge that CPU can do lot of other good stuff in between. And to further offload CPUs work DMA helps transferring data. So if an interrupt is raised and subroutine is called, all the subsequent interrupts can be masked as you are already inside that subroutine, but trust me these subroutine are very tiny they hardly consume any time until your next packet arrives. That means your packet arriving speed has to be higher than your processing speed.

Another Ex: for router/switches 99% time task is routing and switching hence subroutine and interrupt priorities are completely different, moreover all the time they are bombard with tons of packets and hence the subroutine in such cases will never come until there is another packet at bay. At least i have worked on such networking gears.

Milind Deore
  • 2,887
  • 5
  • 25
  • 40
  • Could you please elaborate on how DMA and interrupts play together? – wlnirvana Jul 31 '17 at 05:08
  • Let me rephrase to see if I understand it correctly. There are no more interrupts within an interrupt. The CPU cannot do other stuff until the completion of the top half handler, so in this sense, the DMA is synchronous. It is also named as "DMA" because the internal mechanism (which is the ring descriptor) does not require CPU participation during data transmission from NIC to RAM. – wlnirvana Jul 31 '17 at 06:40
  • Just a side question, does not the synchronous DMA result in a busy waiting or blocked (in the handler) CPU, because the CPU cannot do anything else until DMA is finished. But if this is case, DMA seems much less advantageous, as CPU sort of participates in the transmission by simply busy waiting. The first referenced link actually mentioned NAPI and poll mode driver. Is it supposed to work around this issue? – wlnirvana Jul 31 '17 at 07:28
  • This is one of the approach that you can follow. You can also follow [open source DPDK](http://dpdk.org/). – Milind Deore Jul 31 '17 at 08:10
  • 1
    Thank you very much for the detailed and patient explanation :) – wlnirvana Jul 31 '17 at 08:41