0

I am having trouble efficiently storing the PhoneNumber value in the block below

    response = requests.get(api)
    data = json.loads(response.text)

So after doing the above, I have a dictionary 'data' which looks like this:

{'Key1': 'Some Value ', 'Key2': [{'PhoneNumber': '180000000000'}]}

The goal is to get just the PhoneNumber value i.e. '180000000000', and my first reaction was the below which works, but I know this will not scale well. Wondering if within the json.loads call above I can pass an argument or if anyone has other advice on how to more effectively get this value I am interested. 

    data = data['Key2'][0]['PhoneNumber'])

So after the above data is now = '180000000000' and I want to store it in s3, I had old code (below) for a JSON file, and this works but is pointless. I just need to store the phone numbers as text file ( I know how to use with open for .txt but I am confused on how to use the io.Bytes and if it should be SringIO instead of BytesIO) 

    inmemory = io.BytesIO() 
    with gzip.GzipFile(fileobj=inmemory, mode='wb') as fh:

        with io.TextIOWrapper(fh, encoding='utf-8',errors='replace') as wrapper:
            wrapper.write(json.dumps(data, ensure_ascii=False,indent=2))
    inmemory.seek(0)
    s3_resource.Object(s3bucket, s3path + '.json.gz').upload_fileobj(inmemory)  
    inmemory.close()

And when it is stored like this, I have when I pull the value from S3, I have to use 'with gzip.open' and decode it ect

0004
  • 1,156
  • 1
  • 14
  • 49
  • You're making things harder than they need to be — that's a list holding a dictionary. Don't turn the data you parsed back into string: `data['Key2'][0]['PhoneNumber']` is what you need. – Mark Dec 31 '19 at 04:04
  • is there any point in using json.loads ? – 0004 Dec 31 '19 at 04:21
  • Yes, assuming you are starting with a json string — `loads` converts it to a python data structure. – Mark Dec 31 '19 at 04:22
  • I may be misunderstanding, but this reads like two separate questions (since the edit): one asks how to more efficiently retrieve the phone number, and the other about `BytesIO`. Perhaps this should be split into two separate SO posts? – chris Dec 31 '19 at 04:23
  • Hi Chris, you are correct , I apologize on second thought that should be a separate question – 0004 Dec 31 '19 at 04:24
  • @MarkMeyer , to be honest I am not certain if I am starting with a 'json string'(I am new I am sorry) when I input print(type(response)) print(type(response.text)) into my console I get : . So really, I am not sure if there is a difference between a string and a json string? – 0004 Dec 31 '19 at 04:28
  • @chris - I broke out the second question here https://stackoverflow.com/questions/59539656/using-io-library-to-load-string-variable-as-a-txt-file-to-from-s3 – 0004 Dec 31 '19 at 04:39

1 Answers1

1

Stringifying data['Key2'] is unnecessary, standard dictionary and list access syntax should work for your case. Condensed into one line:

>>> data = {'Key1': 'Some Value ', 'Key2': [{'PhoneNumber': '180000000000'}]}
>>> data['Key2'][0]['PhoneNumber']
'180000000000'

You can (and probably should) handle error cases in here, as well, perhaps by splitting this out into multiple steps or by trapping the KeyError and IndexError together. As one option, take a look at the newish suppress util in contextlib (docs). Would look something like this:

>>> with suppress(KeyError, IndexError):
...     ph = data['Key2'][0]['PhoneNumber']
...
>>> ph
'180000000000'
chris
  • 1,915
  • 2
  • 13
  • 18
  • Hi Chris, thank you for your help, I will input a check because your response did make things more efficient and I updated the code in my question to reflect so now, wondering if there is still room for improvement and if json.loads was necessary because the inital block is an api response in text, then a dictionary then back as a string (ie when it is set back to 'data' after the dictionary parsing) . Also, what the best way to write that data to/from s3 – 0004 Dec 31 '19 at 04:24
  • I broke out the second question here: https://stackoverflow.com/questions/59539656/using-io-library-to-load-string-variable-as-a-txt-file-to-from-s3 – 0004 Dec 31 '19 at 04:39
  • Why would I want to handle error cases in here? I will lookinto suppress , ty for docs link – 0004 Dec 31 '19 at 04:40
  • You'd want to check for errors in case the response body doesn't contain the data exactly as you expect. (For instance, if `data['Key2']` is `[]`.) This isn't necessary if something else is validating your input. – chris Dec 31 '19 at 14:39