3

I am trying to parse huge .pcap files (~1-2GB each). I have tried to use scapy but so far it is much to slow. I have benchmarked timings using the following code with different values for count

from scapy.all import *
from scapy.layers.dns import DNSRR, DNS, DNSQR
import time
t0 = time.time()
pcap = 'output.pcap'
pkts = rdpcap(pcap, count=1000000)
t1 = time.time()
print(t1-t0)

It seems rdpcap() can only handle about 800 packets / second. Are there any optimizations for scapy, other tools, or good ways of scaling this process? Each packet is supposed to be DNS.

deltap
  • 4,176
  • 7
  • 26
  • 35

1 Answers1

1

To really give you solid feedback I'd need you to post a larger chunk of your code. But from what I can see:

My first thought would be to use threading (if possible: I'm not sure what the rest of your code looks like)

My second thought would be to check the return type of your rdpcap() function and see the max size for that variable type. It could be that you are trying to fit more data into your buffer than you have space available.

keelyn roberts
  • 36
  • 1