Cannot read TcpPacket in pcap4j

Question

I am trying to use pcap4j to get TcpPackets from loopback address. I can read packets successfully but I cannot convert them to TcpPackets:

public class Main {
    public static void main(String[] args) throws Exception {
        PcapNetworkInterface device = null;
        try {
            PcapNetworkInterface nic = new NifSelector().selectNetworkInterface();
            System.out.println(nic);
            PcapHandle handle = nic.openLive(65536, PcapNetworkInterface.PromiscuousMode.PROMISCUOUS, 10000);
            handle.setFilter(
                    "tcp port 8080",
                    BpfProgram.BpfCompileMode.OPTIMIZE
            );

            while (handle.isOpen()) {
                Packet packet = handle.getNextPacketEx();
                if (packet != null) {

                    TcpPacket tcpPacket = packet.get(TcpPacket.class);
                    System.out.println(tcpPacket);

//                    EthernetPacket ethernetPacket = EthernetPacket.newPacket(packet.getRawData(), 0, packet.length());
//
//                    Dot1qVlanTagPacket dot1qVlanTagPacket = Dot1qVlanTagPacket.newPacket(ethernetPacket.getRawData(), 0, ethernetPacket.length());
//
//                    IpV4Packet ipV4Packet = IpV4Packet.newPacket(dot1qVlanTagPacket.getRawData(), 0, dot1qVlanTagPacket.length());
//
//                    TcpPacket tcpPacket1 = TcpPacket.newPacket(ipV4Packet.getRawData(), 0, ipV4Packet.length());
//                    System.out.println(tcpPacket1);
                }
            }

        } catch (Exception e) {
            System.out.println(e.getClass() + " " + e.getMessage());
        }
    }
}

Things to note:

I am using getNextPacketEx so I can benefit from packet factory

My dependencies in pom.xml looks like

<dependencies>
    <dependency>
        <groupId>org.pcap4j</groupId>
        <artifactId>pcap4j-core</artifactId>
        <version>1.8.2</version>
    </dependency>
    <dependency>
        <groupId>org.pcap4j</groupId>
        <artifactId>pcap4j-packetfactory-propertiesbased</artifactId>
        <version>1.8.2</version>
    </dependency>
</dependencies>

What I have also tried is to unwrap manually the packets ( see the commented code ). That works successfully until I get an IpV4Packet which contains fragmented data.

When I try to construct the TcpPacket.newPacket... I am receiving the error:

class org.pcap4j.packet.IllegalRawDataException The data offset must be equal or more than 5, but it is: 0

What I would like to do is read tcp packets, reassemble them and then read my http content that I want to sniff. Can anyone help?

UPDATE: This is a full dump of ipv4 packets for a simple Http get on localhost in port 8080:

name: [\Device\NPF_Loopback] description: [Adapter for loopback traffic capture] loopBack: [true]] up: [true]] running: [true]] local: [true]
[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 32 (MERIT-INP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (44 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 94 00 00 00 00 80 02 ff ff 97 b8 00 00 02 04 ff c3 01 03 03 08 01 01 04 02

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24590
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, false)
  Fragment offset: 4790 (38320 [bytes])
  TTL: 0
  Protocol: 32 (MERIT-INP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (44 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 1f 90 c3 df 1c b1 79 d7 16 42 e3 95 80 12 ff ff 01 1f 00 00 02 04 ff c3 01 03 03 08 01 01 04 02

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 95 1c b1 79 d8 50 10 27 f6 14 0c 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 106 (QNX)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (118 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 95 1c b1 79 d8 50 18 27 f6 c3 af 00 00 47 45 54 20 2f 67 72 65 65 74 69 6e 67 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 74 3a 20 6c 6f 63 61 6c 68 6f 73 74 3a 38 30 38 30 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 75 72 6c 2f 37 2e 35 35 2e 31 0d 0a 41 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24590
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, false)
  Fragment offset: 4790 (38320 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 1f 90 c3 df 1c b1 79 d8 16 42 e3 eb 50 10 27 f6 13 b6 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24590
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, false)
  Fragment offset: 4790 (38320 [bytes])
  TTL: 0
  Protocol: 175 (unknown)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (187 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 1f 90 c3 df 1c b1 79 d8 16 42 e3 eb 50 18 27 f6 5e cc 00 00 48 54 54 50 2f 31 2e 31 20 32 30 30 20 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 0d 0a 54 72 61 6e 73 66 65 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 63 68 75 6e 6b 65 64 0d 0a 44 61 74 65 3a 20 57 65 64 2c 20 32 38 20 4f 63 74 20 32 30 32 30 20 31 30 3a 31 32 3a 35 33 20 47 4d 54 0d 0a 0d 0a 32 33 0d 0a 7b 22 69 64 22 3a 36 36 2c 22 63 6f 6e 74 65 6e 74 22 3a 22 48 65 6c 6c 6f 2c 20 57 6f 72 6c 64 21 22 7d 0d 0a

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 eb 1c b1 7a 73 50 10 27 f5 13 1c 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24590
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, false)
  Fragment offset: 4790 (38320 [bytes])
  TTL: 0
  Protocol: 25 (Leaf-1)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (37 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 1f 90 c3 df 1c b1 7a 73 16 42 e3 eb 50 18 27 f6 ce f3 00 00 30 0d 0a 0d 0a

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 eb 1c b1 7a 78 50 10 27 f5 13 17 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 eb 1c b1 7a 78 50 11 27 f5 13 16 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24590
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, false)
  Fragment offset: 4790 (38320 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 1f 90 c3 df 1c b1 7a 78 16 42 e3 ec 50 10 27 f6 13 15 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24590
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, false)
  Fragment offset: 4790 (38320 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 1f 90 c3 df 1c b1 7a 78 16 42 e3 ec 50 11 27 f6 13 14 00 00

[IPv4 Header (32 bytes)]
  Version: 1 (unknown)
  IHL: 8 (32 [bytes])
  TOS: [precedence: 0 (Routine)] [tos: 0 (Default)] [mbz: 0]
  Total length: 0 [bytes]
  Identification: 24591
  Flags: (Reserved, Don't Fragment, More Fragment) = (true, true, true)
  Fragment offset: 5741 (45928 [bytes])
  TTL: 0
  Protocol: 20 (HMP)
  Header checksum: 0x0680
  Source address: /0.0.0.0
  Destination address: /0.0.0.0
  Option: [option-type: 0 (End of Option List)]
  Padding: 0x00 00 00 00 00 00 01 00 00 00 00
[Fragmented data (32 bytes)]
  Hex stream: 00 00 00 00 00 00 00 00 00 00 00 01 c3 df 1f 90 16 42 e3 ec 1c b1 7a 79 50 10 27 f5 13 15 00 00

mcernak · Accepted Answer · 2020-10-30T13:22:02.727

1

The pcap4j repository contains an example of how to deal with fragmented packets:
https://github.com/kaitoy/pcap4j/blob/v1/pcap4j-sample/src/main/java/org/pcap4j/sample/DefragmentEcho.java

Basically you need to group the IpV4Packets based on their header.identification

Packet packet = handle.getNextPacketEx();
Short id = packet.get(IpV4Packet.class).getHeader().getIdentification();

and once you have all the fragments, you can reassemble the whole payload:

//list of fragments with the same id
List<IpV4Packet> list; 
final IpV4Packet defragmentedIpV4Packet = IpV4Helper.defragment(list);

Edit (in response to the added pcap dump file)

I've noticed (in the output.pcap file that you've linked below in the comment) that you are using IPv6. I've tried using that file as a source of the packets, i.e.

PcapHandle handle = Pcaps.openOffline("output.pcap");

and those packets were not fully parsed. The reason for that was that the Protocol family was not recognized.

The correct value of IPv6 protocol family is platform specific and unless you specify a custom value, pcap4j uses this code to obtain a default value for respective platforms: Pcap4jPropertiesLoader.

Since the value 24 used in your captured packets is not listed among the default values, you need custom configuration.

One way to do that is via System.properties:

//associate value 24 with the IPv6 protocol family
System.setProperty("org.pcap4j.af.inet6", "24");
//set the class which should parse the packets for that family
System.setProperty("org.pcap4j.packet.Packet.classFor.org.pcap4j.packet.namednumber.ProtocolFamily.24", "org.pcap4j.packet.IpV6Packet");

After that, parsing via:

Packet packet = handle.getNextPacketEx();
TcpPacket tcpPacket = packet.get(TcpPacket.class)

should work normally.

edited Oct 30 '20 at 13:22

answered Oct 28 '20 at 08:46

mcernak

9,050
1
5
13

Hi mcernak! I updated my answer and posted the full packet dump for a simple http get. My problem is now that for example for packet 24591, the total length is always 0, the offset is always the same and the more flags is always true. Now I would have expected at some point that more flag is false and I can start to call your function to merge packets but that flag is never false so I don't know when to start to defragment. Is there any chance this library is bugged or maybe is something else? – adragomir Oct 28 '20 at 10:23
Maybe you should filter the packets based on protocol. Are you sure you want to be processing these protocols: `32 (MERIT-INP)`, `20 (HMP)`, `106 (QNX)`, `175 (unknown)`, `25 (Leaf-1)`? – mcernak Oct 28 '20 at 12:53
Hey mcernak! What I basically want to do is just capture the http traffic on localhost using pcap4j/ npcap but this is what the library is giving me. Basically the creator of library stated that if you have a packetfactory on classpath then TcpPacket tcpPacket = packet.get(TcpPacket.class); should work but it is not and these weird packets is all I got. If I print the packets content on screen I can see my http content along with a lot of unreadable characters. So I know I receive the correct sniff, but I just don't know how to extract it from these packets. What do you suggest mcernak? – adragomir Oct 28 '20 at 14:15
If I set the filter to `tcp port 8080`, I only see the packages with the TCP protocol. Could you capture your network exchange using `PcapDumper` into a pcap file and post it here? (e.g. `PcapDumper dumper = handle.dumpOpen("output.pcap"); ... Packet packet = handle.getNextPacket(); dumper.dump(packet, handle.getTimestamp()); ... dumper.close();` – mcernak Oct 29 '20 at 11:02
mcernak I added the captured pcap file here: https://drive.google.com/file/d/1JDsXTgQUpaMmE7a_aq-bLxfmcJYMCB2c/view?usp=sharing. Crazy enough I was able to open the file using wireshark and everuthing looks ok, but I don't get it how wireshark is able to reconstruct the packets although they have those bad offsets for packets. Or maybe there is a bug in the library pcap4j? – adragomir Oct 29 '20 at 20:51
I added an explanation of how to parse the pcap file that you've included. – mcernak Oct 31 '20 at 00:01
One humble question mcernak: in a generic network program how would I find the 24 value so I can set it ? – adragomir Oct 31 '20 at 10:58
I don't know of a general way, it seems to be on a case-by-case basis, see `org.pcap4j.Pcap4jPropertiesLoader#getDefaultAfInet6`. Just out of curiosity, what value do you get when you call `com.sun.jna.Platform.getOSType()`? – mcernak Oct 31 '20 at 11:44
hey, I get value 2 which goes and returns 23 for getDefaultAfInet6. – adragomir Oct 31 '20 at 12:36

Cannot read TcpPacket in pcap4j

1 Answers1