-1

I want to parse some websites and get list of all pages on current domain, like a:

sample.com/

sample.com/page1/

sample.com/page2.html

But I can't find samples, how to build this sitemap or tree using C# and ASP.NET

I found only one example: http://www.codeproject.com/Articles/13486/A-Simple-Crawler-Using-C-Sockets

But I can't understand, how author use it

if(Directory.Exists(strUri) == true) { //some code DirectoryInfo dir = new DirectoryInfo(folderName); FileInfo[] fia = dir.GetFiles("*.txt"); }

When I use this code, result in if is always false. When I use only

GetFiles function

I have Exception:

URI formats are not supported

Who have any ideas?

Sergey Svezentsev
  • 5
  • 4

1 Answers1

0

remember that on a Web ambient, you cant read files that way, you need to use

Server.MapPath(url)

to get the physical address to the files, then you can do the loop you are using.

thepanch
  • 353
  • 2
  • 13
  • This is not my sites. Are you sure that `MapPath` can work with url? – Sergey Svezentsev Apr 07 '16 at 01:42
  • you can't do that, a virtual address is not a "folder", and a simple http doesn't have it, However. There ARE many extensions to the HTTP protocol. WebDAV IS a protocol for mapping an HTTP URI to a directory in some location. Check out http://stackoverflow.com/questions/7399358/webdav-request-show-files-in-folder for how to query a WebDAV service for the content of a fold – thepanch Apr 07 '16 at 14:23