57

I started to explore the option of connecting with other using a p2p connection, so I coded a simple socket program in JAVA for android devices in which the users can share simple messages p2p (I didn't have any idea about NAT then). I got to know about NAT, so I now need to establish a TCP connection with another user which uses a server for discovery but payload is transferred p2p. I have also looked at XMPP(a very good and detailed explanation of how protocol works is here) and UPnP but I dont know how to implement them.

Another interesting question that arises is of BitTorrent because they can work on any device and even behind a NAT. I am not able to get any explanation of how BitTorrent works.

I have researched a lot but I am stuck.

My questions are:

  1. A detailed explanation of BitTorrent(like here, not how torrents work) and how is it able to work around NAT ?
  2. Is there a way to make a NAT entry programmatically ?
  3. Is socket programming sufficient for p2p ?
  4. How difficult is it to create your own protocol and how can I build one ?
  5. If two devices D1 and D2 want to communicate p2p and they know each other's IP. D1 sends a request to D2 and that can't get through the D2's NAT, but there should be an entry created in D1's NAT. So when D2 tries to send something D1's NAT should discover an entry with D2's IP. Then why is the packet not allowed by it ?
user3439988
  • 593
  • 1
  • 6
  • 9
  • Bittorrent uses [UDP hole punching.](http://en.wikipedia.org/wiki/UDP_hole_punching) I don't think there is any detailed explanation on how availible on the net. – Encombe May 21 '15 at 09:50
  • @Encombe I have read about hole punching. 1. Its not always successful 2. Messages cannot be transmitted using UDP, its very unreliable. – user3439988 May 21 '15 at 11:03
  • @Encombe Also could you tell me how these protocols deal with dynamic IP, and does a phone's IP remains the same for a single session or what is the timeout period? – user3439988 May 21 '15 at 11:07
  • I say you can do some research in other client's code, like Transmission, and see how it is implemented. For example, [here](https://trac.transmissionbt.com/browser/trunk/libtransmission/upnp.c). It can be hard, but in my opinion it will be constructive. – paulochf May 21 '15 at 14:20
  • Here is a link to XMPP article mentioned above. The mentioned link gives 404 error. https://web.archive.org/web/20170204074212/http://ceit.uq.edu.au/content/how-xmpp-works-step-step – Yashas Apr 10 '20 at 07:29

4 Answers4

27

Another interesting question that arises is of BitTorrent because they can work on any device and even behind a NAT. I am not able to get any explanation of how BitTorrent works.

This statement looks like you assume that bittorrent needs full connectivity to operate.

That is incorrect.

Behind a NAT device you will still be able to establish outgoing TCP connections. Which generally is sufficient for bittorrent as long as there are other, non-NATed (or NATed but properly port-forwarded) clients in the network that can accept incoming connnections.

NAT has no impact on the flow direction of the data because connections are bi-directional once they are established. It only is problematic for the initial connection setup.

This works perfectly fine for bittorrent because bittorent does not care from which specific node you get your data. Although better connectivity generally does improve performance.

If the identity of the node matters or one-on-one transfers are an important use-case then other p2p protocols usually attempt NAT traversal first and if that fails rely on 3rd party nodes relaying traffic between those nodes who cannot connect to each other directly.

Additionally, IPv6 support will become essential in the future to maintain end-to-end connectivity because more and more ISPs are starting to roll out carrier-grade NAT for IPv4 while IPv6 will remain non-NATed

the8472
  • 40,999
  • 5
  • 70
  • 122
  • 1
    My purpose for this statement was to find out how the 'initial connection setup' happens. I know that bittorrents use DHT for finding the connection. Suppose you know the IP address of the destination, even then you cant get through NAT. So you will need something to create an entry into the NAT forwarding table so that the data can flow in. You are absolutely right about the data going out and you are right that after initial connection setup has happened the data can flow without any problem Could you please suggest me something for the initial phase and for the time when IP changes? – user3439988 May 21 '15 at 19:25
  • "So you will need something to create an entry into the NAT forwarding table so that the data can flow in." for outgoing connections a local NAT device will create those automatically. That's why I said that outgoing connections will work if the remote is reachable. If that's not what you mean please amend your question with a more specific scenario outlining the whole path between two nodes and the actions you wish to perform. – the8472 May 21 '15 at 19:45
  • The problem is that unlike calling we cant send a message directly from one device to another even if we knew the IPs(because of NAT). Practical: We know each other's IP and are both behind NAT. I send you a message which is dropped by your NAT and so can't you. On the other hand if we have made an initial connection and entries do exist we can exchange message without any external aid. Please also read my question 5. Also if you want I could post the code (but its simple socket program server and client in android). – user3439988 May 21 '15 at 21:44
  • I already covered those things in my answer, please read it more carefully. – the8472 May 21 '15 at 22:01
  • I completely agree with everything you say. "It only is problematic for the initial connection setup". In bittorrent " if that fails rely on 3rd party nodes relaying traffic between those nodes who cannot connect to each other directly." But how did the first node get initiated. This is the same reason torrents dont work in my college. Also you mention "If the identity of the node matters" it actually does in my case. – user3439988 May 21 '15 at 22:15
  • Also I read about [Bleep](http://engineering.bittorrent.com/2014/09/17/how-does-bleep-work/) that they use SIP , but I can't find any implementation of that. Perhaps you can help. – user3439988 May 21 '15 at 22:17
  • 1
    Does it mean, that if there are no nodes in bittorrent network that aren't NATed, bittorrent won't work? – Aditya Agarwal Feb 07 '20 at 07:37
  • not necessarily. NATs with EIMs would be trivial to connect through – the8472 Feb 07 '20 at 22:19
16

One thing need to be clear is that 100% P2P between all type of NAT is impossible right now. There is no practical way to establish P2P connectivity between **Symmetric and Symmetric/PRC NAT. In this scenario connection is established through a relay server called TURN.

I am answering from your 2nd question because I don't know much about the first one.

2) Yes. You can send a packet through your NAT and there will be a mapping between your internal IP:Port to your NAT's external IP:Port. You can know these external IP:Port by sending a stun request. Note that this technique doesn't work for Symmetric NAT.

3)Yes socket programming sufficient for p2p.

4)Why do you need a protocol when there already exists several. ICE protocol is the best today for NAT traversal and I don't think it was easy to create. UPnP and NAT-PMP is really vulnerable in terms of security.

5)I think what happens is usually NAT blocks unknown packets coming to it. So when D1 sends a packet to D2, its NAT blocks all packets incoming from D1s IP:Port. That is why connection establishment fails. You have to employ hole punching technique for D1 and D2 to successfully establish P2P connectivity.

**By symmetric NAT I mean symmetric NAT with random port allocation.

Tahlil
  • 2,680
  • 6
  • 43
  • 84
9

There is a paper on "Peer-to-Peer Communication Across Network Address Translators" which describes the UDP hole punching method and extends it to be used over TCP as well.

Of course, you will always need a relay server for the cases where hole punching is not supported.

Fardin K.
  • 445
  • 9
  • 19
6

  1. Recent versions of BitTorrent use µTP, which is layered above UDP, not TCP. µTorrent uses a private extension (ut_holepunch) that performs UDP hole punching, most other implementations don't bother (with the notable exception of Tixati).

  2. Some NAT routers accept port forwarding requests using either the uPNP or the PMP protocol. Whether this is supported depends on the particular brand of router and its configuration.

  3. Yes, socket programming is enough for P2P.

  4. Difficult to answer. I suggest that you read the wikified and annotated BitTorrent specification for a start.

  5. Yes, this is the principle behind UDP hole punching.

jch
  • 5,382
  • 22
  • 41
  • All I gather from your answer is that using NAT traversal is the only solution apart form using uPNP and PMP. All three of them are not a 100% reliable solution. and uTP works on UDP, again unreliable. BitTorrent has made an app called [Bleep](http://engineering.bittorrent.com/2014/09/17/how-does-bleep-work/) (P2P chat). They used SIP for initiation. Also another solution someone suggested was to use UDP with error handling to work like TCP. Do you have any idea how to implement any of those? – user3439988 May 22 '15 at 05:43
  • Also do you know where I can find a sample code or text related to server sending data to the client after initiation in android, in which the client does NOT have to keep making request to the server every few seconds. – user3439988 May 22 '15 at 05:47
  • IIRC `ut_holepunch` is also supported by libtorrent/rasterbar and Tixati – Encombe May 22 '15 at 08:24
  • @Encombe, mentioned Tixati, since it's in the release notes, but couldn't find any information about libtorrent. Are you sure it implements `ut_holepunch`? – jch May 22 '15 at 11:21
  • 4
    @user3439988, if I were writing a P2P application today, I'd make it pure IPv6 and to hell with it. – jch May 22 '15 at 11:30
  • @jch I think I have seen `ut_holepunch` from qBittorrent in pcaps. Also, a search brings up; Arvid says: "holepunching support is always on in libtorrent." [here.](http://sourceforge.net/p/libtorrent/mailman/message/33113548/) You probably need to check libtorrent source code to confirm. – Encombe May 22 '15 at 12:20
  • It's not clear to me if he's speaking of `ut_holepunch` or PEX. (Together with µTP, PEX has the side-effect of causing holepunching if the timing is just right.) I'll check the sources at some point. – jch May 22 '15 at 12:36
  • By activating logging in µT(logger tab > Peer Traffic Logging > Log holepunch) I can confirm that holepunching works with BitTorrent/µTorrent, libtorrent(Deluge, qBittorrent, Bitlord), Tixati and Xunlei. [libtorrent source](http://sourceforge.net/p/libtorrent/code/11105/tree//trunk/src/bt_peer_connection.cpp#l1479) – Encombe Jun 01 '15 at 11:32