In your case I'd use TCP unless you actually have hands-on experience with fragmenting and re-assembling UDP packets manually, and you're willing to maintain the overhead introduced in your code (like having a re-assembling buffer and controlling the latency that this implies).
Furtherly, you should take the targetted network into consideration. Is it localhost only, a LAN, a WAN or even the internet. The less control you have over the network the higher the impact of favouring TCP in terms of roundtrip times, latency, packet loss, etc. By control I mean upper bounds or estimates of the number of crossed network segments (#routers), the number of different configurations (QoS, bandwidth-limiter, MTU, ...), and so on.
As a rule of thumb, UDP is great when all data necessary for an instant (defined below) fits in one packet (MTU is guaranteed to be 1280 in IPv6). An instant is a short snapshot in time, something that typically has a life-span of a round trip time. UDP is also great for conversations where both the query and the response are small entities.
So in this sense, I'd use UDP for something like DNS (short query, short answer), or financial transaction data (there's only so many within the life-span of 1 round trip time), or protocol meta data such as the number or identity hashes of participating clients (query/response short and there's only a handful within the roundtrip time).
Hope this helps.
Edit:
To answer your questions
- UDP (restrictions listed above)
- IPv6 offers path mtu detection, you'd simply use the PMTU, for IPv4 you'd have to roll your own:
- set the
IP_DONTFRAG
socket option
- send a packet that you would assume goes through
- think of a simple protocol to allow the receiver to tell you whether the packet has been received completely
- if no -> reduce the size, if yes -> increase the size
- after a few ping-pongs you have a safe estimate for the PMTU (of course you can send payload data along already)
- UDP will outperform TCP substantially if the nature of the network is stable and remains stable.
(Conversely) TCP will not win when the nature of the network keeps changing (latency variations, changing probability of packet loss, etc.)
But, on the same note, UDP will not win when the network segments are very far apart and QoS is used in some of the intermediate segments (QoS that is configured to favour more or less known TCP services over ``other'' stuff.
For some figures and inspiration you should check out udt.