2

I successfully have scrapysharp working in a console app.

I created a new MVC web app in VS2013 with no authentication or anything else special. I used nuget to add ScrapySharp and then have this code in my Home Controller. I get no response to my pageresult. Does anyone have any ideas why? How do I get scrapysharp to work in a MVC web app?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Mvc;
using ScrapySharp.Html.Forms;
using ScrapySharp.Network;
using ScrapySharp;
using HtmlAgilityPack;
using ScrapySharp.Extensions;

namespace scrapyB.Controllers
{
    public class HomeController : Controller
    {
        public ActionResult Index()
        {
           var  browser = new ScrapingBrowser();
            browser.AllowAutoRedirect = true;
            browser.AllowMetaRedirect = true;
           var pageresult = browser.NavigateToPage(new Uri("https://google.com"));
           var x = pageresult;
            return View();
        }

        public ActionResult About()
        {
            ViewBag.Message = "Your application description page.";

            return View();
        }

        public ActionResult Contact()
        {
            ViewBag.Message = "Your contact page.";

            return View();
        }
    }
}

Edit Actually I just tried a windows Forms application and NavigateToPage doesn't return a result either. It's weird but it seems as it only works in a console app. Does anyone know how to get ScrapySharp to work in something other than a console app?

nvoigt
  • 75,013
  • 26
  • 93
  • 142
bhs8227
  • 81
  • 9
  • I love how people mark questions with a -1 and give no insight on how to solve an issue. So helpful. – bhs8227 Sep 07 '16 at 21:25
  • Please note that the model-view-controller tag is for questions about the pattern. There is a specific tag for the ASP.NET-MVC implementation. –  Sep 07 '16 at 23:35
  • Thank you. I selected MVC wasn't thinking. My fault. I see it is already fixed. I appreciate that. – bhs8227 Sep 07 '16 at 23:45
  • What are you looking to do with ScrapySharp in a web project? It seems conceptually weird to me have a project which both delivers webpages to clients while also simulating a web browser on the server. – Will Ray Sep 08 '16 at 00:15
  • Scrapysharp allows me to easily request pages. Login, Get forms from the page and change form field values and then submit the form. It also gets the viewstate values from the page in question with out me having to try to figure out how to do the same with httpclient or httpwebclient. I am having to deal with a third party site that doesn't have an api. If some one knows a better way I am open to that too. – bhs8227 Sep 08 '16 at 00:57
  • @bhs8227 Cool! Sounds like it's meant to be used in a console app for things like UI tests and whatnot. I can't answer your question but I would suggest that a console app will give you what you need without all the baggage that comes along with an MVC project. – Will Ray Sep 08 '16 at 01:08
  • It is very cool. If it was for my own purposes a console app would be fine. But I need to add a way to interface with this third party site in an existing asp.net mvc project. I can't expect customers to install an intermediate console app to exchange data between the two sites. – bhs8227 Sep 08 '16 at 01:12

1 Answers1

3

Ok so scrapysharp has a NavigateToPageAsync function. This returns a result as expected.

Form.submit() uses NavigateToPage as well but you can instead use NavigateToPageAsync and pass in the correct parameters to submit the form.

bhs8227
  • 81
  • 9