Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise a meaningful error when trying to flatten non-tweets #667

Open
gdhpearson opened this issue Nov 15, 2022 · 2 comments
Open

Raise a meaningful error when trying to flatten non-tweets #667

gdhpearson opened this issue Nov 15, 2022 · 2 comments
Labels
good first issue Good for newcomers

Comments

@gdhpearson
Copy link

Using Jupyter in VSCode. Several of the twarc functions lead to the entire of the data being printed to the console which makes debugging a pain and generally clogs things up.

An example code that does this.

from twarc.client2 import Twarc2
from twarc.expansions import ensure_flattened


from twarc.expansions import ensure_flattened
twarc = Twarc2(SOME LOGIN INFO)

listOfIds = [SOMEIDS]

for id in listOfIds
  search = twarc.liking_users(id, max_results=100)
  for page in search:
      for profile in ensure_flattened(page):
          # Do something with the tweet
          allLikes.append({tid:profile['username']})

This code leads to every returned profile being printed in full to the console. Example of the output here.

https://imgur.com/a/X82V5K9

@SamHames
Copy link
Contributor

Hello - it looks ensure_flattened isn't tested on the user profile responses like that endpoint generates - you aren't iterating over tweet objects, but over user profile objects. I wouldn't recommend using it for this particular purpose.

My suggestion is to iterate over the response yourself:

for profile in page["data"]:
      # Do something with the user profile
      allLikes.append({id: profile['username']})

@igorbrigadir
Copy link
Contributor

igorbrigadir commented Nov 17, 2022

Maybe it's not clear with the error message, but this is an error that prints out the data for debugging:

Unable to expand dictionary without includes: {'data': ...

the list of users is not something you can "flatten" the same way as tweets. But maybe the error message shouldn't print the entire response because that seems to hide the error.

I think that's worth fixing in a small PR

@edsu edsu added the good first issue Good for newcomers label Nov 17, 2022
@edsu edsu changed the title Code prints all data to console without any print function when using Jupyter Raise a meaningful error when trying to flatten non-tweet data Nov 17, 2022
@edsu edsu changed the title Raise a meaningful error when trying to flatten non-tweet data Raise a meaningful error when trying to flatten non-tweets Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants