Questions tagged [netflow]

is a Cisco standard IP traffic reporting format, summarizing network sessions in terms of endpoints, port/protocol, bytes transferred, and other data.

NetFlow is a standard network session reporting format, developed by Cisco to summarize network communications between hosts. This data generally includes:

  • endpoint IP addresses, including the initiator
  • IP protocol (by IANA number)
  • port where applicable (generally only TCP or UDP)
  • bytes transferred according to direction

and other data. (A fuller listing can be found at the Wikipedia article on NetFlow)

A NetFlow exporter (which may include a router, switch, or other device; or a software exporter sniffing local traffic) maintains a list of tracked connections by analyzing the headers of packets visible to it. NetFlow does not require deep packet inspection; by default it summarizes only IP header and protocol header information. The exporting device periodically exports information on these connections to "collector" software, usually via UDP at the default port 2055. Such a collector may receive updates from multiple exporters on a push basis, with each datagram reporting identifying information for its exporter. Each datagram can include multiple sets of NetFlow sessions.

A number of free/OSS and commercial products are available to collect and analyze NetFlow data, as well as similar data sources such as sFlow and JFlow. Such products may support storage and search, monitoring for security purposes, traffic visualizations, network-based forensic procedures, or statistical analysis. For development, flowd is an open source flow collector implementation.

There are multiple versions of NetFlow, but NetFlow 5 and NetFlow 9 are the predominant ones. NetFlow 5 is a fixed format, with the same fields in the same position for every export. NetFlow 9 has a range of optional fields, some of which are vendor-specific. In order to decode a NetFlow 9 datagram, it is necessary to match the datagram to a numbered template (templates are exported in the datagram stream periodically). Each template will contain an in-order listing of the items to be found in NetFlow records exported according to that template. The NetFlow 9 standard is described in RFC 3954.

NetFlow 9 is the basis of the IPFIX standard, and has been adapted as the basis for formats such as NSEL (NetFlow Security Event Logging). Because IPFIX is built on NetFlow 9, libipfix may be useful to NetFlow developers.

101 questions
166
votes
1 answer

Netflow record can't get octets (jnca)

I'm using jnca library to collect NetFlow records sent by a router. The version of the NetFlow record sent by the router is version 9. When the NetFlow packet is observed from the Wireshark the flow sets with the template id 263 contains the data…
Asiri Liyana Arachchi
  • 2,663
  • 5
  • 24
  • 43
10
votes
2 answers

Converting a PCAP trace to NetFlow format

I would like to convert some PCAP traces to Netflow format for further analysis with netflow tools. Is there any way to do that? Specifically, I want to use "flow-export" tool in order to extract some fields of interest from a netflow trace as…
Regressor
  • 247
  • 1
  • 2
  • 11
4
votes
4 answers

Is there an open source netflow collector C++ library set?

I am looking for a C++ library set to develop my own C++ daemon in Linux for collecting NetFlow information. Does anyone know of an open source one or a library set that is available? Many thanks
aHunter
  • 3,490
  • 11
  • 39
  • 46
3
votes
1 answer

Collecting data from JSON array into ClickHouse table

I have some raw JSON data in ClickHouse table (actually, netflow V9 from netflow collector) it looks like this: {"AgentID":"10.1.8.1", "Header":{"Version":9,"Count":2}, "DataSets":[ …
3
votes
1 answer

Netflow sample data sets

Does anyone know of an open netflow data set, I want to use it to run a little experiment on it, and analyse some of the flows. I looked around but there is nothing. Or if there is a good method to capture netflow data without actually having a…
joh
  • 53
  • 2
  • 8
3
votes
1 answer

How to get flow record details of a netflow packet

I've used jflow to capture the netflow packets. By running the print example I could observe this kind of output. 13.243.146.68.41472 -> 10.100.0.126.13570 285212682 0 0.43.0.0.18 -> 0.199.0.0.4352 8321715 100 53.2.7.225.3571 -> 0.0.1.67.37446 323…
Asiri Liyana Arachchi
  • 2,663
  • 5
  • 24
  • 43
3
votes
3 answers

Java library for netflow

Is there a Java library for Netflow? I need to be able to objectify flows for analysis. (I really need to support sflow, too, but one hurdle at a time.) I found a project called "jflow" that has classes for Netflow v5 and v6, but I need to…
tdimmig
  • 680
  • 9
  • 24
3
votes
2 answers

C++,Selecting from 2d array based on value-entry

i'm using softflowd+nfdump to create netflow data and store this data in a 2d (string) array flows = new string *[flows_len]; for (int i=0;i
2
votes
0 answers

UDP missing packets

The problem: UDP packets are missing. How could I capture every single UDP packet that is hitting the port? I want to put received packet on the queue for preprocessing in the background and continue capturing new UDP packets without a single UDP…
Linas
  • 65
  • 1
  • 7
2
votes
2 answers

Storing ranged timeseries data in Postgres

I need to store netflow data in Postgresql. This is data about network traffic. Each record contains the following: Connection start time Connection end time Total data transferred Source/destination IPs/ASNs (There is a bunch more, but that is…
Adam Charnock
  • 1,596
  • 1
  • 13
  • 17
2
votes
1 answer

Determining throughput from pcap containing flow records

I have a single packet capture (acquired via tcpdump) that contains flow records between an exporter and a collector. I want to determine throughput across a given interface using the bytes (octets) field in the v9 record. I have filtered down to…
cjphlo
  • 45
  • 4
2
votes
0 answers

Determine ports used for connections on a node from netflow data

Given NetFlow data I would like to infer as much port information about the nodes as possible. Such as what ports are closed, what ports are being used for connections, and what ports are open. How can this be done? I think it is ok to assume my…
Hamzeh Alsalhi
  • 403
  • 1
  • 4
  • 10
2
votes
1 answer

How to obtain NFDUMP output in bytes using csv output

I need to obtain netflow record in csv format, but as a lot of traffic is in network, nfdump return information in MBytes. I have no choise but to use one option in NFDUMP to obtain traffic information in bytes. My current command is: nfdump -R…
MM Manuel
  • 375
  • 2
  • 4
  • 16
2
votes
0 answers

Pull netflow data from MySQL and detect anomalies

I am trying to create a DoS detection program using netflow data in a SQL database. So far I have the general idea on how to accomplish this but need some direction. The flow of the program thus far is:
The Sorrow
  • 21
  • 2
2
votes
1 answer

How do I retrieve the template from an IPFIX file using libfixbuf?

I am trying to use the CERT tools to read and parse an IPFIX file. The docs say to grab the "internal" template from the file, though every call relating to templates seems to require a template id (tid) or other information that I don't have. It's…
Fred
  • 31
  • 1
1
2 3 4 5 6 7