I am making a SEO project using PHP where I need to crawl every pages/directories of a website. But for this, I have to know or list all the directories of that website. Is it possible? Can we do it with PHP?
Asked
Active
Viewed 216 times
1 Answers
3
No. HTTP provides no way for a client (regardless of the programming language it is written in) to ask for directory listings.
This is why search engines crawl links and make use of sitemaps.
A PHP program could inspect the directory structure of the file system of the computer it is running on, but even that wouldn't give you a good view in the general case as most websites are not simply a bunch of files served up by mapping URLs directly onto a filesystem (for example, the Front Controller design pattern is quite common).
Quentin
- 914,110
- 126
- 1,211
- 1,335