How to Bypass CefSharp and libcef to handle POST request Manually?

Question

I am new to "Web Programming" and I would like to know if it is possible to handle POST request and response manually in CEFSharp. The Scenario is that the website that I am interacting with using CEFSharp is creating the request, But I want to take control after that if it is possible. I Imagine that ResourceRequestHandler must be some thing like the follwing, which one must convert the CEFSharp IRequest to .NET HttpRequestMessage somehow:

public class CustomResourceRequestHandler : CefSharp.Handler.ResourceRequestHandler
{
private static readonly HttpClient hc = new();
private readonly AutoResetEvent are;

public CustomResourceRequestHandler(AutoResetEvent are)
{
    this.are = are;
}

protected override CefReturnValue OnBeforeResourceLoad(IWebBrowser chromiumWebBrowser, IBrowser browser, IFrame frame, IRequest request, IRequestCallback callback)
{
    var hrm = new HttpRequestMessage(HttpMethod.Post, request.Url);
    
    //How to add Headers properly?
    foreach (string rh in request.Headers)
        hrm.Headers.Add(rh, request.Headers[rh]);
    
    //How to add content properly?
    
    Task.Factory.StartNew(async() =>
    {
        are.WaitOne();
        var response = hc.SendAsync(hrm);
        await response;
        // Do Something with response.Result
    });

    return CefReturnValue.Cancel;
}
}

The questions are: First, Is this even possible? since the underlying socket is changed, doesn't the webserver complain? Second, How to copy IRequest to HttpRequestMessage Properly? I Added the headers, and don't know if it's done correctly, But I am struggling to convert the content field. Third, Is copying Headers and Content enough?

What exactly are you trying to achieve? What is the purpose of using httpclient to make the request? — amaitland, Oct 18 '21 at 00:24
@amaitland Awsome work on CEFSharp library BTW. I am doing a time critical task and want to do time profiling. I want to log the time network packets hit the wire(or network card driver) and the exact time the response packets are received. I will replace the HttpRequest with raw sockets next. CEFSharp is crafting the correct request for me. — Al Beir, Oct 18 '21 at 08:14
Have you checked the performance metrics available thru DevTools? You should be able to access those metrics programatically https://chromedevtools.github.io/devtools-protocol/tot/Performance/ https://github.com/cefsharp/CefSharp/issues/3165 — amaitland, Oct 18 '21 at 21:08
Otherwise I think you'd be better off using/writing a proxy. https://stackoverflow.com/questions/226784/how-to-create-a-simple-proxy-in-c — amaitland, Oct 18 '21 at 21:11

score 0 · Answer 1 · answered Oct 19 '21 at 14:53

After tinkering with CEFSharp and .NET HttpRequest, this is a semi-working example. I had to copy Cookies, Headers, and Content to make it work. Calling are.Set() will trigger the manual request and reading response. Before going for something like this, check @amailand comments in question which may suggest better methods for you.

protected override CefReturnValue OnBeforeResourceLoad(IWebBrowser chromiumWebBrowser, IBrowser browser, IFrame frame, IRequest request, IRequestCallback callback)
    {
        Task.Factory.StartNew(async() =>
        {
            //Cookies
            var cm = chromiumWebBrowser.GetCookieManager();
            var t = cm.VisitUrlCookiesAsync(request.ReferrerUrl, true);
            Task.WaitAll(t);
            var cookies = t.Result;

            var cc = new CookieContainer();
            using HttpClientHandler hch = new() 
            { 
                CookieContainer = cc, 
                AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
            };
            using HttpClient hc = new(hch);

            foreach (var cookie in cookies)
                cc.Add(new System.Net.Cookie(cookie.Name, cookie.Value, cookie.Path, cookie.Domain));

            var hrm = new HttpRequestMessage(HttpMethod.Post, request.Url);
            
            //Headers
            foreach (string rh in request.Headers)
                hrm.Headers.TryAddWithoutValidation(rh, request.Headers[rh]);

            //Check if more headers should be added manually

            //Content
            hrm.Content = new StringContent(Encoding.UTF8.GetString(request.PostData.Elements[0].Bytes),
                Encoding.UTF8, "application/json");
            are.WaitOne();
            var response = hc.SendAsync(hrm);
            await response;

            if (response.Result.StatusCode != HttpStatusCode.OK) return;
            var t1 = response.Result.Content.ReadAsByteArrayAsync();
            Task.WaitAll(t1);
            // result is in t1.Result
        });

        return CefReturnValue.Cancel;
    }

How to Bypass CefSharp and libcef to handle POST request Manually?

1 Answers1