1

I would like to download a html from twitter. I use wget but I can only download a small part of the members because it needs more time for loading the list. How can I download all this html?

I use now, stores the information in usa.dat

wget -c -N -p -O usa.dat https://twitter.com/IABM1995/lists/usa/members
Enric Agud Pique
  • 1,087
  • 7
  • 13
  • 30
  • 3
    Assuming I understand your requirement: I think that is being dynamically generated when the user scrolls down and this I think is through js and wget does not support js at this time. Does this help: http://stackoverflow.com/questions/5852220/how-to-download-dynamic-generated-content-from-webpage – Khanna111 Feb 25 '15 at 06:50
  • I will check this link Khanna111, best – Enric Agud Pique Feb 26 '15 at 21:32

1 Answers1

0

check this

scrape websites with infinite scrolling

Community
  • 1
  • 1
candymanuu
  • 110
  • 7