73

i've got an integration test that grabs some json result from a 3rd party server. It's really simple and works great.

I was hoping to stop actually hitting this server and using Moq (or any Mocking library, like ninject, etc) to hijack and force the return result.

is this possible?

Here is some sample code :-

public Foo GoGetSomeJsonForMePleaseKThxBai()
{
    // prep stuff ...

    // Now get json please.
    HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("Http://some.fancypants.site/api/hiThere");
    httpWebRequest.Method = WebRequestMethods.Http.Get;
    
    string responseText;
    
    using (var httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse())
    {
        using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
        {
            json = streamReader.ReadToEnd().ToLowerInvariant();
        }
    }
    
    // Check the value of the json... etc..
}

and of course, this method is called from my test.

I was thinking that maybe I need to pass into this method (or a property of the class?) a mocked httpWebResponse or something but wasn't too sure if this was the way. Also, the response is a output from an httpWebRequest.GetResponse() method .. so maybe I just need to pass in a mocked HttpWebRequest ?.

any suggestions with some sample code would be most aprreciated!

moribvndvs
  • 42,191
  • 11
  • 135
  • 149
Pure.Krome
  • 84,693
  • 113
  • 396
  • 647

7 Answers7

75

You may wish to change your consuming code to take in an interface for a factory that creates requests and responses that can be mocked which wrap the actual implementation.

Update: Revisiting

I've been getting downvotes long after my answer was accepted, and I admit my original answer was poor quality and made a big assumption.

Mocking HttpWebRequest in 4.5+

The confusion from my original answer lies in the fact that you can mock HttpWebResponse in 4.5, but not earlier versions. Mocking it in 4.5 also utilizes obsolete constructors. So, the recommended course of action is to abstract the request and response. Anyways, below is a complete working test using .NET 4.5 with Moq 4.2.

[Test]
public void Create_should_create_request_and_respond_with_stream()
{
    // arrange
    var expected = "response content";
    var expectedBytes = Encoding.UTF8.GetBytes(expected);
    var responseStream = new MemoryStream();
    responseStream.Write(expectedBytes, 0, expectedBytes.Length);
    responseStream.Seek(0, SeekOrigin.Begin);

    var response = new Mock<HttpWebResponse>();
    response.Setup(c => c.GetResponseStream()).Returns(responseStream);

    var request = new Mock<HttpWebRequest>();
    request.Setup(c => c.GetResponse()).Returns(response.Object);

    var factory = new Mock<IHttpWebRequestFactory>();
    factory.Setup(c => c.Create(It.IsAny<string>()))
        .Returns(request.Object);

    // act
    var actualRequest = factory.Object.Create("http://www.google.com");
    actualRequest.Method = WebRequestMethods.Http.Get;

    string actual;

    using (var httpWebResponse = (HttpWebResponse)actualRequest.GetResponse())
    {
        using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
        {
            actual = streamReader.ReadToEnd();
        }
    }


    // assert
    actual.Should().Be(expected);
}

public interface IHttpWebRequestFactory
{
    HttpWebRequest Create(string uri);
}

Better answer: Abstract the Response and Request

Here's a safer bare-bones implementation of an abstraction that will work for prior versions (well, down to 3.5 at least):

[Test]
public void Create_should_create_request_and_respond_with_stream()
{
    // arrange
    var expected = "response content";
    var expectedBytes = Encoding.UTF8.GetBytes(expected);
    var responseStream = new MemoryStream();
    responseStream.Write(expectedBytes, 0, expectedBytes.Length);
    responseStream.Seek(0, SeekOrigin.Begin);

    var response = new Mock<IHttpWebResponse>();
    response.Setup(c => c.GetResponseStream()).Returns(responseStream);

    var request = new Mock<IHttpWebRequest>();
    request.Setup(c => c.GetResponse()).Returns(response.Object);

    var factory = new Mock<IHttpWebRequestFactory>();
    factory.Setup(c => c.Create(It.IsAny<string>()))
        .Returns(request.Object);

    // act
    var actualRequest = factory.Object.Create("http://www.google.com");
    actualRequest.Method = WebRequestMethods.Http.Get;

    string actual;

    using (var httpWebResponse = actualRequest.GetResponse())
    {
        using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
        {
            actual = streamReader.ReadToEnd();
        }
    }


    // assert
    actual.Should().Be(expected);
}

public interface IHttpWebRequest
{
    // expose the members you need
    string Method { get; set; }

    IHttpWebResponse GetResponse();
}

public interface IHttpWebResponse : IDisposable
{
    // expose the members you need
    Stream GetResponseStream();
}

public interface IHttpWebRequestFactory
{
    IHttpWebRequest Create(string uri);
}

// barebones implementation

private class HttpWebRequestFactory : IHttpWebRequestFactory
{
    public IHttpWebRequest Create(string uri)
    {
        return new WrapHttpWebRequest((HttpWebRequest)WebRequest.Create(uri));
    }
}

public class WrapHttpWebRequest : IHttpWebRequest
{
    private readonly HttpWebRequest _request;

    public WrapHttpWebRequest(HttpWebRequest request)
    {
        _request = request;
    }

    public string Method
    {
        get { return _request.Method; }
        set { _request.Method = value; }
    }

    public IHttpWebResponse GetResponse()
    {
        return new WrapHttpWebResponse((HttpWebResponse)_request.GetResponse());
    }
}

public class WrapHttpWebResponse : IHttpWebResponse
{
    private WebResponse _response;

    public WrapHttpWebResponse(HttpWebResponse response)
    {
        _response = response;
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    private void Dispose(bool disposing)
    {
        if (disposing)
        {
            if (_response != null)
            {
                ((IDisposable)_response).Dispose();
                _response = null;
            }
        }
    }

    public Stream GetResponseStream()
    {
        return _response.GetResponseStream();
    }
}
Community
  • 1
  • 1
moribvndvs
  • 42,191
  • 11
  • 135
  • 149
  • 8
    you can't create mock for HttpWebResponse. You wil get an error like this:"System.ArgumentException: Can not instantiate proxy of class: System.Net.HttpWebResponse. Could not find a parameterless constructor. Parameter name: constructorArguments" – Eduard Kibort Aug 08 '12 at 16:06
  • I am unable to find the .Net framework which supports Is IHttpWebRequest, IHttpWebRequestFactory, IHttpWebResponse. Any ideas? – Ajit Goel Apr 25 '16 at 19:41
  • 1
    @AjitGoel They are interfaces you would have to create, which I supplied in my example. Scroll down on the second code sample, it's there. – moribvndvs Apr 25 '16 at 20:54
  • 1
    @HackedByChinese: Dont know how I missed it. Need coffee to wake me up. – Ajit Goel Apr 25 '16 at 21:03
  • Is there a good reason for using IHttpWebRequestFactory? Can't we just keep the Create method inside the IHttpWebRequest interface? – Ivan Studenikin Apr 26 '16 at 09:37
  • @IvanStudenikin Whether it were included in the interface or not, some Create method would have to be implemented. – Suncat2000 Jan 05 '23 at 19:19
6

Rather than mocking out the HttpWebResponse is I would wrap the call behind an interface, and mock that interface.

If you are testing does the web response hit the site I want it too, that is a different test than if does class A call the WebResponse interface to get the needed data.

For mocking an interface I prefer Rhino mocks. See here on how to use it.

David Basarab
  • 72,212
  • 42
  • 129
  • 156
  • The method (above) is actually an implimented interface. So the class is `public class AwesomeClass : IThinkICan`. So you're suggestion i just mock that class instead, so i mock the return object? Right now, my test does hit the 3rd party, but i do not want it to do that. (ie. go from integration test to a unit test). – Pure.Krome Mar 22 '12 at 13:27
5

None of the Microsoft's HTTP stack was developed with unit testing and separation in mind.

You have three options:

  • Make the call to web as small as possible (i.e. send and get back data and pass to other methods) and test the rest. As far as the web call is concerned, there should be much magic happening there and very straightforward.
  • Wrap the HTTP call in another class and pass your mock object while testing.
  • Wrap HttpWebResponse and HttpWebRequest by two other classes. This is what the MVC team did with HttpContext.

Second option:

interface IWebCaller
{
    string CallWeb(string address);
}
Aliostad
  • 80,612
  • 21
  • 160
  • 208
3

If it helps please find below the code illustrated in the accepted answer using NSubstitute in place of Moq

using NSubstitute; /*+ other assemblies*/

[TestMethod]
public void Create_should_create_request_and_respond_with_stream()
{
   //Arrange
   var expected = "response content";
   var expectedBytes = Encoding.UTF8.GetBytes(expected);
   var responseStream = new MemoryStream();
   responseStream.Write(expectedBytes, 0, expectedBytes.Length);
   responseStream.Seek(0, SeekOrigin.Begin);

   var response = Substitute.For<HttpWebResponse>();
   response.GetResponseStream().Returns(responseStream);

   var request = Substitute.For<HttpWebRequest>();
   request.GetResponse().Returns(response);

   var factory = Substitute.For<IHttpWebRequestFactory>();
   factory.Create(Arg.Any<string>()).Returns(request);

   //Act
   var actualRequest = factory.Create("http://www.google.com");
   actualRequest.Method = WebRequestMethods.Http.Get;

   string actual;

   using (var httpWebResponse = (HttpWebResponse)actualRequest.GetResponse())
   {
       using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
       {
           actual = streamReader.ReadToEnd();
       }
   }

   //Assert
   Assert.AreEqual(expected, actual);
}

public interface IHttpWebRequestFactory
{
    HttpWebRequest Create(string uri);
}

Unit Test run and passes successfully.

Up vote given on the answer I've been looking for some time how to do this effectively.

John Saunders
  • 160,644
  • 26
  • 247
  • 397
David Hall
  • 126
  • 5
  • giving downvote because this is not working at all: ```var r = Substitute.For(); Castle.DynamicProxy.InvalidProxyConstructorArgumentsException: Can not instantiate proxy of class: System.Net.HttpWebResponse.``` – justmara Oct 21 '16 at 14:56
  • damn, cant edit my comment, so will add another: you cannot use substitute for .Net 3.5 variant of this class. higher versions got default constructor. but also in higher versions the default constructor is public so you may simple instantiate this class instead of mocking – justmara Oct 24 '16 at 10:00
1

You can actually return HttpWebResponse without mocking, see my answer here. It does not require any "external" proxying interfaces, only the "standard" WebRequest WebResponse and ICreateWebRequest.

If you don't need access to HttpWebResponse and can deal with just WebResponse it is even easier; we do that in our unit tests to return "prefabricated" content responses for consumption. I had to "go the extra mile" in order to return actual HTTP status codes, to simulate e.g. 404 responses which requires you use HttpWebResponse so you can access the StatusCode property et al.

The other solutions assuming everything is HttpWebXXX ignores everything supported by WebRequest.Create() except HTTP, which can be a handler for any registered prefix you care to use (via WebRequest.RegisterPrefix() and if you are ignoring that, you are missing out, because it is a great way to expose other content streams you otherwise have no way to access, e.g. Embeeded Resource Streams, File streams, etc.

Also, explicitly casting the return of WebRequest.Create() to HttpWebRequest is a path to breakage, since the method return type is WebRequest and again, shows some ignorance of how that API actually works.

Community
  • 1
  • 1
escape-llc
  • 1,295
  • 1
  • 12
  • 25
1

Although there is an accepted answer, I would like to share my view. There are many times that modifying the source code is not an option as suggested. Furthermore the answer introduces to say the least a lot of production code.

So I prefer to use the free harmonyLib and patch the CLR without any change in the production code, by intercepting the WebRequest.Create static method.

My modified version looks like:

        public static bool CreateWithSomeContent(ref WebRequest __result){
            var expected = "response content";
            var expectedBytes = Encoding.UTF8.GetBytes(expected);
            var responseStream = new MemoryStream();
            responseStream.Write(expectedBytes, 0, expectedBytes.Length);
            responseStream.Seek(0, SeekOrigin.Begin);

            var response = new Mock<HttpWebResponse>();
            response.Setup(c => c.GetResponseStream()).Returns(responseStream);

            var requestMock = new Mock<HttpWebRequest>();
            requestMock.Setup(c => c.GetResponse()).Returns(response.Object);
            __result = requestMock.Object;
            return false;
        }

        [Test]
        public void Create_should_create_request_and_respond_with_stream(){
            //arrange
            var harmony = new Harmony(nameof(Create_should_create_request_and_respond_with_stream));
            var methodInfo = typeof(WebRequest).GetMethod("Create",new Type[]{typeof(string)});
            harmony.Patch(methodInfo, new HarmonyMethod(GetType(), nameof(CreateWithSomeContent)){});

            //act
            var actualRequest = WebRequest.Create("");
            string actual;
            using (var httpWebResponse = (HttpWebResponse)actualRequest.GetResponse()){
                using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream())){
                    actual = streamReader.ReadToEnd();
                }
            }
            
            // assert
            // actual.Should().Be("response content");
        }
Apostolis Bekiaris
  • 2,145
  • 2
  • 18
  • 21
-1

I Found a great solution in this blog post:

It´s very easy to use, you just need to do this:

string response = "my response string here";
WebRequest.RegisterPrefix("test", new TestWebRequestCreate());
TestWebRequest request = TestWebRequestCreate.CreateTestRequest(response);

And copy those files to your project:

    class TestWebRequestCreate : IWebRequestCreate
{
    static WebRequest nextRequest;
    static object lockObject = new object();

    static public WebRequest NextRequest
    {
        get { return nextRequest ;}
        set
        {
            lock (lockObject)
            {
                nextRequest = value;
            }
        }
    }

    /// <summary>See <see cref="IWebRequestCreate.Create"/>.</summary>
    public WebRequest Create(Uri uri)
    {
        return nextRequest;
    }

    /// <summary>Utility method for creating a TestWebRequest and setting
    /// it to be the next WebRequest to use.</summary>
    /// <param name="response">The response the TestWebRequest will return.</param>
    public static TestWebRequest CreateTestRequest(string response)
    {
        TestWebRequest request = new TestWebRequest(response);
        NextRequest = request;
        return request;
    }
}

class TestWebRequest : WebRequest
{
    MemoryStream requestStream = new MemoryStream();
    MemoryStream responseStream;

    public override string Method { get; set; }
    public override string ContentType { get; set; }
    public override long ContentLength { get; set; }

    /// <summary>Initializes a new instance of <see cref="TestWebRequest"/>
    /// with the response to return.</summary>
    public TestWebRequest(string response)
    {
        responseStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(response));
    }

    /// <summary>Returns the request contents as a string.</summary>
    public string ContentAsString()
    {
        return System.Text.Encoding.UTF8.GetString(requestStream.ToArray());
    }

    /// <summary>See <see cref="WebRequest.GetRequestStream"/>.</summary>
    public override Stream GetRequestStream()
    {
        return requestStream;
    }

    /// <summary>See <see cref="WebRequest.GetResponse"/>.</summary>
    public override WebResponse GetResponse()
    {
        return new TestWebReponse(responseStream);
    }
}

class TestWebReponse : WebResponse
{
    Stream responseStream;

    /// <summary>Initializes a new instance of <see cref="TestWebReponse"/>
    /// with the response stream to return.</summary>
    public TestWebReponse(Stream responseStream)
    {
        this.responseStream = responseStream;
    }

    /// <summary>See <see cref="WebResponse.GetResponseStream"/>.</summary>
    public override Stream GetResponseStream()
    {
        return responseStream;
    }
}
Guilherme Torres Castro
  • 15,135
  • 7
  • 59
  • 96
  • this makes no reference to HttpWebResponse. as soon as you cast TestWebResponse as HttpWebResponse it will be null. – Christo Jun 08 '15 at 09:55
  • @Christo when you call `WebRequest.RegisterPrefix("test", new TestWebRequestCreate());` you are registering a prefix for requests. So you have to mock the url too: using **test**://example.com instead of **http**://example.com – Guilherme Torres Castro Jun 08 '15 at 18:45
  • the post you found is cool and useful for testing WebRequest/Response etc. but the original question was how can i test ```HttpWebRequest```. if you use this code to test the code in the question it would fail trying to explicitly convert ```TestWebReponse``` to ```HttpWebRequest```. line 4 of his method – Christo Jun 09 '15 at 09:07
  • @Christo I think you mean convert `TestWebReponse` to `HttpWebResponse` and `TestWebRequest` to `HttpWebRequest`. In this case you just have to change the inheritance from `TestWebRequest` and `TestWebReponse` to inherit from `HttpWebRequest` and `HttpWebResponse` respectively – Guilherme Torres Castro Jun 09 '15 at 14:47
  • sorry for the typo. i did mean that. i suggest you test these suggestions as i dont think that would work. – Christo Jun 09 '15 at 15:25