0

I have just learned about pandas dataframes. I have a nested dictionary that has values i extract in a for loop. What is the best way to transform this data into a dataframe? 

   if result != None:
    for ads in result['data']['ad_list']:

        #print getOnlineProvider(currency)
        if(ads['data']['online_provider'] in onlineprovider):

            #"NATIONAL_BANK") or 
            #(ads['data']['online_provider'] =="SPECIFIC_BANK") or 
            #(ads['data']['online_provider'] =="ALTCOIN_DASH") or 
            #(ads['data']['online_provider'] =="CASH_DEPOSIT")):

            trader = ads['data']['profile']['name']
            adusdprice = ads['data']['temp_price_usd']
            delta =  float(adusdprice) - float(btcprice)
            deltapercent =  (1 -  (float(btcprice)/float(adusdprice))) * 100
            adid = ads['data']['ad_id']
            maxamount = ads['data']['max_amount']
            minamount = ads['data']['min_amount']
            paywindow = ads['data']['payment_window_minutes']

            print("Trader:" + trader + "||" + "Ad ID:" + str(adid) + "||"+ "paymethod: " + ads['data']['online_provider'] + "||" + "USD price: " + adusdprice + "||"+ "min-max: " + str(minamount) + "-" + str(maxamount) +"||" + "PayWindow: " + str(paywindow) + "||" + "Price delta: " + str(delta) + "||"+ "Pct Delta: " + str(deltapercent) + "%") 

print("=====================================================")

I've tried creating a dataframe from dict with 

pd.DataFrame.from_dict(result['data']['ad_list']['data'])

but this isnt working, as far as i can tell i need to do it row by row as i extract each value to get it properly..

this is the result of the dataframe i get with the above command.

{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 90, u'hidden_by_op...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...
{u'public_view': u'https://localbitcoins.com/a...  {u'require_feedback_score': 0, u'hidden_by_ope...

while im looking to get something in a pandas dataframe format similar to what im printing out in the for loop...

Trader:mr_ahmed_btc (1000+; 99%)||Ad ID:736624||paymethod: CASH_DEPOSIT||USD price: 3866.49||min-max: 200-9185||PayWindow: 270||Price delta: 260.05||Pct Delta: 6.72573833115%

Charles Landau
  • 4,187
  • 1
  • 8
  • 24
joe
  • 73
  • 2
  • 8

1 Answers1

0

from_dict with no kwargs expects dicts to be in the form:

{"colname1": [1,2,3,4], "colname2": [2,3,4,5]}

It's not easy to tell from your question what the structure of result['data']['ad_list']['data'] but my best guess is that it's something like:

[
    {
      "dog": "cat",
      "cat": "dog",
      "bug": 234
    },
    {
        "dog": "hot",
        "cat": "tin roof",
        "bug": 432
    }
]

In which case use:

df = pd.DataFrame(result['data']['ad_list']['data'])

This constructs the dataframe directly from the dict.

Edit: To your question about pretty printing, check out Pretty Printing a pandas dataframe

Simple printing of a dataframe is just print(df)

But I don't know if that will solve your problem. It sounds like result['data']['ad_list']['data'] might be a list of JSON strings, in which case you would need to do something like:

import json data = json.loads(result['data']['ad_list']['data']) df = pd.DataFrame(data)

Charles Landau
  • 4,187
  • 1
  • 8
  • 24