CRITICAL:root:twint.feed:Follow:IndexError

[benjaminvanrenterghem]

Issue Template

Please use this template!

Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

• Python version is 3.6;
• Updated Twint with pip3 install --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint;
• I have searched the issues and there are no duplicates of this issue/question/request.

Command Ran

Please provide the exact command ran including the username/search/code so I may reproduce the issue.

twint -u coindesk --followers --elasticsearch localhost:9200

Description of Issue

Please use as much detail as possible.

Got 30k results afterwards error:
CRITICAL:root:twint.feed:Follow:IndexError

Rerunning command doesn't give any immediate issues, just starts from 0 again

Environment Details

Using Windows, Linux? What OS version? Running this in Anaconda? Jupyter Notebook? Terminal?

Ubuntu 18.04, Python 3.6, ES 7, kibana 7

[pielco11]

Hi @0xSnipe and thanks for reaching us out,

That error message appears because Twitter does not return more followers, and so in the newly returned bunch of data, no followers found.

You could wait a while until Twitter does not "unsuspend" your IP, and then resume the scraping

#340

[benjaminvanrenterghem]

Very interesting thread, thanks for your quick reply.
Perhaps it would benefit from something like this, instead of the fixed sleep?
https://github.com/litl/backoff

I'll see what I can do to make that happen, if it's not beyond my python knowledge

[pielco11]

That's a really interesting library! Thanks for sharing

Feel free to provide any kind of patch, either by code or concept!

[ghost]

Has anyone found a solution for this?

I get this error after fetching just 20 usernames.

[pielco11]

@KayHS not yet

I suggest you to apply a workaround as explained in #340 or use the Twitter API

[edwardqianofficial]

Hi, just wondering if I resume the scrape, am i able start from where I stop? Since When I tried to resume the program, the scraped results would start over again.

[pielco11]

You have to specify config.Resume = "session_number_123.session" (or whatever fits your needs) first, and then you can resume that session from where you stopped

[ahaseeb7838]

I'm facing same issue, I'm getting this error,

CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable
CRITICAL:root:twint.feed:Follow:IndexError

I'm running this command
twint -u paulchuckle2 --followers -o PaulChuckle.csv --csv --resume username_followers_resume.txt

I'm getting above error after running the command with out getting any follower.
Can anyone help me about the issue ?

[ahaseeb7838]

I'm running the script on AWS EC2 Instance so I have also changes my IP but still getting this error,

CRITICAL:root:twint.get:User:'NoneType' object is not subscriptable CRITICAL:root:twint.feed:Follow:IndexError

c = twint.Config() c.Username = "KatiePrice" c.Followers = True c.Resume = "resume.txt" c.Output = "KatiePrice.csv" twint.run.Followers(c)

I'm running above code. Total Followers are 1.9M but I'm receiving this error after 6 Lac followers.

[theshouryagupta]

I am trying to extract the followers from a twitter account. I am getting the following error, it was working fine yesterday.
CRITICAL:root:twint.feed:Follow:IndexError

[himanshudabas]

That's because twitter killed the no js (mobile version) version of twitter on 15th December. Followers were scraped using that version.

[mayankrichu]

is there a new update?

[drumstick90]

Same problem. Other twint features are working fine but I need this one

[itsnotashes]

The problem persists...

[brenorb]

That's because twitter killed the no js (mobile version) version of twitter on 15th December. Followers were scraped using that version.

So the whole lib is now useless? Nothing I try is working at the moment.