5

I am trying to get more information about what is going on when I connect to a website at a lower level than what HttpWebRequest and HttpWebResponse gives me. I am using C#.

I would like to be able to see information about the dns lookup and the time it took to establish a connection (if a new connection was established). HttpWebRequest and HttpWebResponse work at a higher level than this and I want to ask if there is a way of getting the underlying TcpClient object (or whatever low level object they use).

If it is not possible, then is there a way to grab and manipulate a list of the connections that .net is maintaining without getting it through HttpWebRequest or HttpWebResponse?

I can't change the application I am working on to use TcpClient because it would be too time consuming to implement all the http stuff reliably.

Zane
  • 2,890
  • 2
  • 24
  • 23
  • 1
    Not sure about 'TcpClient' but I'm sure they use the 'Socket' class so look for it. Additionally, when you break-point and debug,you point over objects and see even their private members. That should give you some direction. – Poni Jan 03 '11 at 16:12
  • I hadn't thought about looking at the private members, thank you. Unfortunately I couldn't find much. I did see that the ServicePoint has a non-public variable called LastDnsResolve which I assume holds the time of the last DNS resolution. I may be able to use that for a small piece of my problem. – Zane Jan 03 '11 at 16:43

3 Answers3

4

The best that I can get you is to create an app.config file with the following information:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <system.diagnostics>
        <trace autoflush="true" />
            <sources>
                <source name="System.Net" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>
              <source name="System.Net.Sockets" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>  
           </sources>


            <sharedListeners>
                <add
                  name="MyTraceFile"
                  type="System.Diagnostics.TextWriterTraceListener"
                  initializeData="System.Net.trace.log"
                />
            </sharedListeners>
            <switches>
                <add name="System.Net" value="Verbose" />
              <add name="System.Net.Sockets" value="Verbose" />
            </switches>
    </system.diagnostics>
</configuration>

This will enable tracing and will kick out a log file named "System.Net.trace.log" in your app folder. You aren't going to get all of the information that you're looking for and its not easily consumable while the app is running but at least you don't need to have a third-party program running. Its not documented too much but there's some information out there at least.

Chris Haas
  • 53,986
  • 12
  • 141
  • 274
  • 2
    Building on this idea, you could write your own `TraceListener` which would enable you to track all of the logs in the same process at runtime. – Stephen Cleary Jan 03 '11 at 17:59
  • Thank you, this seems close. I have written a TraceListener as suggested and can get some information from the socket traces. Unfortunately however I can't match a particular trace item, which shows socket information, to the particular HttpWebResponse that it is associated with. – Zane Jan 03 '11 at 21:07
  • Frustratingly, I can see that .NET is using sockets behind the scenes, but can't access them. Is there a way to get an object from .NET using one of the unique object IDs that are displayed in the traces? – Zane Jan 03 '11 at 21:09
  • You can use `Trace.CorrelationManager.ActivityId`. It's a per-thread value that is automatically included in traces. I don't think that `HttpWebResponse` uses it, so you'd have to set it yourself. – Stephen Cleary Jan 04 '11 at 12:39
  • I know this is a really late accept but I never managed to get this working as planned. I think this was the best suggestion though. I ran into some trouble trying to parse the messages in my TraceListener and couldn't find a way to reliably string all the information about a single request together because many threads were running at once. – Zane Oct 21 '11 at 16:30
2

Use Wireshark, it's the best way to find out all that stuff.

fejesjoco
  • 11,763
  • 3
  • 35
  • 65
  • I agree. The point of HttpWebRequest is to abstract/hide the complicated parts. The tradeoff for ease of use is that you have to accept some "magic" at the lower levels. – Chris Haas Jan 03 '11 at 16:21
  • I agree. Instead of mucking about the internals of .net, just monitor the requests as they are going over the wire. This should give you exact timings as to when everything occurs. – NotMe Jan 03 '11 at 16:21
  • I do use wireshark, but I would like to do the same thing in .net: monitor the tcp connections. My question may have been unclear, I would like to be able to get this information repeatedly and automatically in .net and not just for once off troubleshooting. Unless you are suggesting that I use WinPcap or a Wireshark API (does that exist?). But that is taking it to another level and I would like to avoid that if possible. – Zane Jan 03 '11 at 16:36
1

If not Wireshark, then use Fiddler.

John Saunders
  • 160,644
  • 26
  • 247
  • 397
  • I use fiddler too. But that makes me think, how does fiddler get its statistical information? It reports on TCP/IP connection time and is written in C#. Does it use TcpClient/Sockets? Fiddler is very extensive in what it can do. I would feel too guilty to disassemble it, see how it's done and then use the same idea because I feel that's wrong (and it's probably illegal). – Zane Jan 03 '11 at 16:49
  • 2
    Fiddler is HTTP proxy - so whe Fiddled is turned on browser/any other HTTP client sends requests to local proxy (Fiddler) and it resends them to actual server. As result it knows everything about requests and responses since it send them itself. But it does not sound like what you need from original question - you seem to need HTTP/TCP stats for your process... – Alexei Levenkov Jan 03 '11 at 16:56