0

I am working on a small program to automate some IT tasks. The program works by having the user paste an email header text into a textbox and then parsing the text to find out important aspects of the email to inspect.

One functionality I cannot seem to figure out is the domain location. I would like to be able to look up domains to figure out their origin, activity, existence, etc. I am using the website VirusTotal for this because it is free to use and provides a decent amount of information.

For some reason, I cannot properly use the HttpWebRequest + HttpWebResponse methods for this site. I tried doing this on other sites like Google, Microsoft, Sony, etc. and they all worked just as expected by downloading the HTML code. Instead of downloading HTML code, all I get from the web response from VirusTotal is two ASCII characters (they look like black squares). What is causing this and what are my workarounds?

Here is my exact code for this

HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.virustotal.com/#/domain/" + domain);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            if (response.StatusCode == HttpStatusCode.OK)
            {
                Stream receiveStream = response.GetResponseStream();
                StreamReader readStream = null;
                if (response.CharacterSet == null)
                {
                    readStream = new StreamReader(receiveStream);
                }
                else
                {
                    readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
                }
                string page = readStream.ReadToEnd();
                response.Close();
                readStream.Close();
                MessageBox.Show(page);
            }
Tom Hood
  • 497
  • 7
  • 16
  • That might be a BOM. – SLaks May 24 '18 at 21:25
  • Characters after the `#` are not sent to the server. – SLaks May 24 '18 at 21:25
  • You should use their API. – SLaks May 24 '18 at 21:25
  • Thank you for the suggestions. It appears I found the solution with a different methods. I believe the site was using a compression to save on bandwidth space, so by adding one line of code I was able to quickly fix this. For anyone interested, the code is below and I put mine on the line right after the declaration of the HttpWebRequest object: `request.AutomaticDecompression = DecompressionMethods.GZip;` – Tom Hood May 24 '18 at 21:40
  • SLaks, I now see what you were saying. The address characters after the # are not sent to the server. Can you please explain more about what you mean when you say that I should use "their API"? I am not familiar with this. – Tom Hood May 24 '18 at 21:49

1 Answers1

0

See my comments under my question for more information. I have found a solution.

Additionally, this link was helpful.

Tom Hood
  • 497
  • 7
  • 16