twitter stream JSON decoding

Question

So, I have created geo map box from which I gather Tweets, and I want to get even more precise locations using long;lat.

I need to get out only coordinates (long;lat separate) without rest of "coordinates" data.

Im using tweepy and I understand Iam not decoding it right but I can't seem to understand why It doesn't work.

and this where and how i keep failing

input JSON

    {  
   u'contributors':None,
   u'truncated':False,
   u'text':   u'Stundas tikai l\u012bdz 12.00 \U0001f64c\U0001f389\U0001f389\U0001f389 (@ R\u012bgas Valsts v\u0101cu \u0123imn\u0101zija - @rvv_gimnazija in R\u012bga) https://t.co/XCp8OzqQgk',
   u'in_reply_to_status_id':None,
   u'id':599100313690320896,
   u'favorite_count':0,
   u'source':   u'<a href="http://foursquare.com" rel="nofollow">Foursquare</a>',
   u'retweeted':False,
   u'coordinates':{  
      u'type':u'Point',
      u'coordinates':[  
         24.062859,
         56.94697
      ]
   },

My Code

class listener(StreamListener):
    def on_data(self, data):

        tweet = json.loads(data)


        #print time.time()
        text = tweet['text']
        name = tweet['user']['name']
        screenName = tweet['user']['screen_name']
        location = tweet['coordinates']['coordinates'][0]

        print name.encode('utf-8')
        print text.encode('utf-8')
        print location
        print '\n'

        # into the data file
        with open('minedData', 'a') as outfile:
            json.dump({ 'location':location, 'time': time.time(), 'screenName': screenName, 'text': text, 'name': name}, outfile, indent = 4, sort_keys=True)
            #outfile.write(',')
            outfile.write('\n')

        return True

    def on_error(self, status):
        print status


auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter(locations=[23.47,56.66,25.148411,57.407558])

The error

Traceback (most recent call last):
  File "loc3.py", line 45, in <module>
    twitterStream.filter(locations=[23.47,56.66,25.148411,57.407558])
  File "/Library/Python/2.7/site-packages/tweepy/streaming.py", line 428, in filter
    self._start(async)
  File "/Library/Python/2.7/site-packages/tweepy/streaming.py", line 346, in _start
    self._run()
  File "/Library/Python/2.7/site-packages/tweepy/streaming.py", line 255, in _run
    self._read_loop(resp)
  File "/Library/Python/2.7/site-packages/tweepy/streaming.py", line 309, in _read_loop
    self._data(next_status_obj)
  File "/Library/Python/2.7/site-packages/tweepy/streaming.py", line 289, in _data
    if self.listener.on_data(data) is False:
  File "loc3.py", line 23, in on_data
    location = tweet['coordinates']['coordinates'][0]
TypeError: 'NoneType' object has no attribute '__getitem__'

@wanderlust Hmm, isn't that what's in the "my code" section? — tripleee, May 15 '15 at 08:09
@wanderlust yes, basically I left out only imports and authorizations — andris, May 15 '15 at 08:13

score 0 · Answer 1 · answered May 15 '15 at 08:23

0

By looking at other examples, it looks like the argument you receive in on_data is already parsed into a dict, not raw JSON. So there is no JSON to read and therefore tweet ends up empty.

The quick and simple fix is to change

def on_data(self, data):
    tweet = json.loads(data)

into simply

def on_data(self, tweet):

and take it from there.

I also note that your coordinates for the bounding box seem to be in the wrong order -- the location should be specified by the southwest and northeast coordinates.

answered May 15 '15 at 08:23

tripleee

175,061
34
275
318

Tank You for insight, but when I do to this way it give me err `string indices must be integers` this `text = tweet[0]` also doesn't do. I tried son formatting in python like in documentations - `print tweet["geo"]["coordinates"][0]` worked well on test son string but not here on stream. – andris May 15 '15 at 12:52
This code cannot produce that error message. The object is not a string (unless you have made unrelated changes) and the index *is* an integer. – tripleee May 15 '15 at 13:44
only thing I have changed is `def on_data(self, data): tweet = json.loads(data)` to `def on_data(self, tweet):` and it gives error about `text = tweet['text']` `tring indices must be integers` I didn't changes anything code is like one above just these simple changes – andris May 15 '15 at 16:03

twitter stream JSON decoding

1 Answers1