Tweets streaming to .txt file with Python

Question

I have the below code and want to write the stream of tweets to a text file. Is there a way to include the output to text file within the same code and save it in the working directory? I am an IDE lover and really don't like using the console. I am new to python (2 weeks), I am an R / R Studio user.

I know I could use:

filename.py > output.txt

I am currently using Rodeo, Python 3.6.1.

import oauth2 as oauth
import urllib.request as urllib

api_key = "##"
api_secret = "##"
access_token_key = "##-##"
access_token_secret = "##"

_debug = 0

oauth_token    = oauth.Token(key=access_token_key, secret=access_token_secret)
oauth_consumer = oauth.Consumer(key=api_key, secret=api_secret)

signature_method_hmac_sha1 = oauth.SignatureMethod_HMAC_SHA1()

http_method = "GET"


http_handler  = urllib.HTTPHandler(debuglevel=_debug)
https_handler = urllib.HTTPSHandler(debuglevel=_debug)

'''
Construct, sign, and open a twitter request
using the hard-coded credentials above.
'''
def twitterreq(url, method, parameters):
  req = oauth.Request.from_consumer_and_token(oauth_consumer,
                                             token=oauth_token,
                                             http_method=http_method,
                                             http_url=url, 
                                             parameters=parameters)

  req.sign_request(signature_method_hmac_sha1, oauth_consumer, oauth_token)

  headers = req.to_header()

  if http_method == "POST":
    encoded_post_data = req.to_postdata()
  else:
    encoded_post_data = None
    url = req.to_url()

  opener = urllib.OpenerDirector()
  opener.add_handler(http_handler)
  opener.add_handler(https_handler)

  response = opener.open(url, encoded_post_data)

    f = open("output.txt", "wb")

def fetchsamples():
  url = "https://stream.twitter.com/1.1/statuses/sample.json"
  parameters = []
  response = twitterreq(url, "GET", parameters)
  for line in response:
    f.write(line)

if __name__ == '__main__':
  fetchsamples()

# f.close()

You should create a [Minimal, Complete, and Verifiable](http://stackoverflow.com/help/mcve) example. — Stephen Rauch, May 24 '17 at 03:29
@StephenRauch. I did a mix of your implementation and some google searches. in my edit above. I opened the txt "con" and then processed the streaming line by line. I think this avoided the string vs bytes conflict. How can I specify the path to write the .txt? Right now it writes it to my wd. Thanks a lot! — Daniel Vargas, May 24 '17 at 03:47
provide `response` object example, we need to know which objects are stored there: `bytes` or `str` — Azat Ibrakov, May 24 '17 at 03:51
You said you could just use `filename.py > output.txt` but do not want to? This is the most efficient method, have you tried doing that? If you plan to do any major programming in the future, you should get used to the commandline/terminal/etc... — Jeremy, May 24 '17 at 05:33
@Jeremy, yes I did it in the console first. I just like having all coding in a single place. — Daniel Vargas, May 25 '17 at 01:34

score 0 · Answer 1 · answered May 25 '17 at 03:39

Besides the comment I made previously, I would suggesting checking out this stack overflow question: how to direct output into a txt file in python in windows

To quote:

If you want to do it in Python then you would write:
with open('out.txt', 'w') as f:
  f.write(something)`
Obviously this is just a trivial example. You'd clearly do more inside the with block.

Tweets streaming to .txt file with Python

1 Answers1