5
import twint
import os, requests, re, time

c = twint.Config()
c.Username = <anyusername> #Replace with an actual uname in quotes
c.Store_object = True
c.Limit = 10
try:
    twint.run.Followers(c)
except:
    print("Unexpected error:", sys.exc_info()[0])
f = twint.output.follows_list
print(f)

Output

CRITICAL:root:twint.feed:Follow:IndexError
[]

Have done

pip install twint
pip install --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint

Googling, a lot of people have faced this error - but I cannot really find a solution

Running the twint command line also gives the same error

twint -u <uname> --followers
CRITICAL:root:twint.feed:Follow:IndexError

This is not only followers. Anything I try, I get a similar error.

Running Python 3.8.1 on Windows 10
Twint - latest version - 2.1.21

user93353
  • 13,733
  • 8
  • 60
  • 122

3 Answers3

14

Legacy mobile Twitter version will shut down on December 15th 2020. (M2 mobile web)

That's the headline of a reddit thread talking about the recent shutdown of the M2 Mobile Web ("Legacy") Twitter version; from this date on Twitter will only support these browsers.

If you take a look at the GitHub repository of Twint:

_usr.followers = int(ur['data']['user']['legacy']['followers_count'])

... it uses the legacy attribute and therefore won't work from the 15th of December 2020.

A contributor to the GitHub repo, namely @himanshudabas, also mentioned this in a reply to an identical issue:

That's because twitter killed the no js (mobile version) version of twitter on 15th December. Followers were scraped using that version.

You're not the only one facing this problem (obviously), and unfortunately with the Twint module there is no workaround (currently?).

Speaking of the current situation you would need to either use the official Twitter API or alternative wrappers / unofficial APIs.

J. M. Arnold
  • 6,261
  • 3
  • 20
  • 38
  • They are talking about an issue where you get only limited number of followers - partial data. I am not getting anything. i.e. it errors out right at the beginning. I am unable retrieve even 1 follower. – user93353 Jan 04 '21 at 01:13
  • @user93353 Correct - as written, due to the recent change there is no possibility to scrape followers via Twint; it uses the legacy version! – J. M. Arnold Jan 04 '21 at 01:32
  • @user93353 and the one comment I specifically mentioned is in regards to this change as well. – J. M. Arnold Jan 04 '21 at 01:32
  • @user93353 I rechecked the sources mentioned in my reply and they specifically tackle the issue of getting **no data** returned - how should they? The legacy version has been terminated and hence the function isn’t able to work! – J. M. Arnold Jan 04 '21 at 10:17
  • 1
    Hi @J.M.Arnold, any updates about that? I used to love Twint ... – Revolucion for Monica Feb 22 '21 at 00:01
  • 1
    @RevolucionforMonica No, unfortunately there haven’t been any updates (as far as I’m aware). – J. M. Arnold Feb 22 '21 at 05:41
1

Looks like it could be answered here.

That's because twitter killed the no js (mobile version) version of twitter on 15th December. Followers were scraped using that version.

In fact, it looks like the following twitter parser uses html processing (BeautifulSoup as we see).

Also there is a workaround, which described there. Probably, it is dead too, due we see nojs version of site in the description.

Looks like, you have to use Twitter API.

Also next time you can just trace your requests instead of asking quesions here, and see what actually happens. Probably Wireshark can be a little bit difficult with https, but you can just hook requests directly from python. Here is an example: Log all requests from the python-requests module.

CPPCPPCPPCPPCPPCPPCPPCPPCPPCPP
  • 977
  • 2
  • 17
1

Cloning the git repo of twint instead of pip install works. Any tweet can be retrieved using a python script instead of command line.

git clone --depth=1 https://github.com/twintproject/twint.git
cd twint
pip3 install . -r requirements.txt

Refer here

Swetha Magesh
  • 51
  • 5