LightWeight IP: Buffer not freeing

Question

I'm using an TCP/IP stack called lwip. I have implemented a function below to send data packets, inspired from a similar callback function that receives data packets.

Each time a packet is received, I create a buffer using the pbuf_alloc function. Then, I send the packet using udp_sendto. Finally, I free the buffer using pbuf_free. (See the code below.)

For some reason, pbuf_free is not freeing the buffer. (I get a buffer overflow after n packets, where n is the pool size.) The lwip wiki warns that:

The network driver may also not assume that the pbuf memory is actually freed when it calls pbuf_free.

How can I force pbuf_free to free my buffer? How is the buffer overflow avoided?

(My implementation below.)

static err_t IAP_tftp_send_data_packet(struct udp_pcb *upcb, struct ip_addr *to, int to_port, int block)
{
  err_t err;
  struct pbuf *pkt_buf;
  char packet[TFTP_DATA_PKT_LEN_MAX];
  int bytesRead;
  int bytesToSend;

  /* Specify that we are sending data. */
  IAP_tftp_set_opcode(packet, TFTP_DATA); 

  /* Specify the block number that we are sending. */
  IAP_tftp_set_block(packet, block);

  bytesRead = IAP_tftp_set_data(packet, block);

  if(bytesRead != 0) {
    bytesToSend = TFTP_DATA_PKT_LEN_MAX - (512 - bytesRead + 1);
  } else {
    bytesToSend = TFTP_DATA_PKT_LEN_MAX - 512;
  }

  pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_POOL);

  if (!pkt_buf)
  {
    print("(TFTP) Buffer overflow!\r\n");
  }

  /* Copy the file data onto pkt_buf. */
  memcpy(pkt_buf->payload, packet, bytesToSend);

  err = udp_sendto(upcb, pkt_buf, to, to_port);

  /* free the buffer pbuf */
  printf("%d\n\r", pbuf_free(pkt_buf));

  return err;
}

Did you check the reference count to the buffer? I'm afraid that it is only freed if the reference count is 1. — Fred, Jun 18 '12 at 13:32
Possibly `udp_sendto` takes a reference and releases it asynchronously (in a timer?). Maybe you just need to give it a while? — ugoren, Jun 18 '12 at 13:51
@ugoren: I've tried waiting while there is a buffer overflow, but the buffer overflow remains. — Randomblue, Jun 18 '12 at 14:25
What does the printf() statement print when you call pbuf_free() ? — nos, Jul 19 '12 at 22:12

askmish · Accepted Answer · 2012-07-23T16:13:26.583

What version of lwIP are you using? Depending on different versions the answers vary a lot.

The memp_malloc() allocation function called inside the pbuf_alloc() has failed or the pbufs chaining has failed.So, it returns NULL.

pbuf_alloc() will also return NULL, if the passed arguments also contains NULL.(due to NULL arguments check).

In newer versions, could you show what value the MEMP_OVERFLOW_CHECK macro contains? The lwIP shows a diferent behavior when the macro value >= 2.

And another cause might be if you are using multi-threading, the locking mechanisms inside the pbuf_alloc() fail, might cause it to return NULL.

Some versions require that you call pbuf_init(), before calling pbuf_alloc().

You can try this:

pkt_buf = NULL;//Use NULL, just incase the NULL is not 0 as per your compiler.
pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_REF);
if(pkt_buf == NULL)
{
   printf("pbuf_alloc failed.\n");
}
else
{
   /* Do something with the allocated pbufs and free it. */
}

PBUF_REF will allocate no buffer memory for pbuf. The pbuf should be used in a single thread only and if the pbuf gets queued, then pbuf_take should be called to copy the buffer.

You can also try PBUF_RAM which will allocate buffer in RAM.

For more informtaion, you can also browse the source files of the version of lwIP, that you are using.

Let me know if this answer still didn't answer your question. — askmish, Aug 17 '12 at 06:14
I gave you the bounty because your answer is the most promising. I might have to ask you further details as I have time to investigate. Thanks. — Randomblue, Aug 21 '12 at 20:40
If you are comfortable and have enough time, I would suggest you to go through the source code of your version, rather than the wiki. The wiki is poorly maintained. — askmish, Aug 24 '12 at 03:42

score 6 · Answer 2 · answered Jun 18 '12 at 13:40

The easiest solution seems to be to make the buffer static, i.e. re-use the same buffer for each call:

static struct pbuf *pkt_buf = NULL;

if( pkt_buf == NULL )
    pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_POOL);
if( pkt_buf == NULL )
{
    print("(TFTP) Buffer overflow!\r\n");
}

If your scenario involves unloading/reloading the driver, it will leak memory. To fix that, make the buffer static outside the IAP_tftp_send_data_packet() function, and call pbuf_free() when the driver unloads (assuming lwip tells you).

score 0 · Answer 3 · answered Aug 21 '12 at 17:58

Just a passing thought, possibly completely nonsensical. In this code:

if(bytesRead != 0) {
    bytesToSend = TFTP_DATA_PKT_LEN_MAX - (512 - bytesRead + 1);
} else {
    bytesToSend = TFTP_DATA_PKT_LEN_MAX - 512;
}
pkt_buf = pbuf_alloc(PBUF_TRANSPORT, bytesToSend, PBUF_POOL);

...is it possible for bytesRead to assume the value 513 - TFTP_DATA_PKT_LEN_MAX ?

If it happened, wouldn't the request to allocate zero bytes fail? (this could be tested by printing the value of bytesToSend upon buffer overflow, and checking if it is nonzero).

score 0 · Answer 4 · answered Jul 26 '13 at 09:34

struct pbuf does not represent a continuous region of memory. It is rather a chain of memory locations. Thus this will not work in general case:

memcpy(pkt_buf->payload, packet, bytesToSend);

You need to scatter-copy your data. The memcpy() from the code snippet may overflow the payload buffer and cause all kinds of side effects including inability to free the pbuf chain cleanly.

LightWeight IP: Buffer not freeing

4 Answers4