Can't merge two dictionaries into one dictionary

Question

So I got problem that I can't merge two dictionaries into one dictionary. This is my code:

def create_app():
    app = Flask(__name__)

    @app.route('/', methods=['POST'])
    def index():
        news_site = request.get_json()
        print(news_site)

        scrapdata = {}
        scrapdata2 = {}

        if news_site.get('kompas', True) and news_site.get('detik', True) == True:
            scrapdata = kompas_fun()
            scrapdata2 = detik_fun()
            fscrapdata = {**scrapdata, **scrapdata2}
            return jsonify(fscrapdata)

        elif news_site.get('kompas', True) == False:
            scrapdata = detik_fun()
            fscrapdata = {**scrapdata}
            return jsonify(fscrapdata)

        elif news_site.get('detik', True) == False:
            scrapdata = kompas_fun()
            fscrapdata = {**scrapdata}
            return jsonify(fscrapdata)

        else:
            return jsonify({'value': 'error'})

    return app

When input matched with if condition, it's just return with scrapdata2 dictionary only instead merge both scrapdata and scrapdata2. So it's just scrapdata2 dictionary as the result

I already tried solution from How do I merge two dictionaries in a single expression in Python (taking union of dictionaries)?, but still not work. Or use reduce, update and ChainMap but still not work.

Edit:For input I'm using Postman using json, this is the input:

{
    "kompas":true,
    "detik":true
}

This is the output from my code above, but I only post few from many dictionary from the output:

{
    "data": [
        {
            "author": "Sachril Agustin Berutu",
            "category": "detikNews",
            "content": "Beredar dokumen hasil tes swab Habib Rizieq Shihab (HRS) positif Corona. Wali Kota Bogor Bima Arya mengaku tidak mengetahui apakah dokumen tes swab Habib Rizieq itu asli atau palsu., \"Kita tidak bisa pastikan itu asli apa palsu,\" kata Bima saat dihubungi, Rabu (2/12/2020)., \n",
            "date": "02 Des 2020",
            "headline": "\nDialog 212 Dimulai, Habib Rizieq Hadir di Lokasi Terpisah\n",
            "link": "https://news.detik.com/berita/d-5277968/dialog-212-dimulai-habib-rizieq-hadir-di-lokasi-terpisah",
            "tag": "habib rizieq",
            "time": "09:58 WIB",
            "total comment": "240"
        },
        {
            "author": "Achmad Dwi Afriyadi",
            "category": "detikFinance",
            "content": "Industri hulu migas merupakan industri yang penuh ketidakpastian. Untuk menarik investasi, pemerintah berupaya mengurangi ketidakpastian tersebut., Menteri ESDM Arifin Tasrif mengatakan, ketidakpastian sendiri berasal eksternal dan internal.",
            "date": "02 Des 2020",
            "headline": "\nCara Pemerintah 'Manjakan' Investor Migas\n",
            "link": "https://finance.detik.com/energi/d-5278096/cara-pemerintah-manjakan-investor-migas",
            "tag": "migas",
            "time": "11:18 WIB",
            "total comment": "0"
        }
    ],
    "news_portal": "detik"
}

Both kompas_fun() and detik_fun() returning a dictionary from web scraping from two news website. But the output only from detik_fun(). The kompas_fun() is works and returning a list but not merged with result from detik_fun.

This is end from kompas_fun() which appending and converting result to dictionary:

      arti.append({
        'headline': title,
        'content':content,
        'writer': writer,
        'editor': editor,
        'category' : cat,
        'tag' : tag1,
        'total comment': comment,
        'date': dates,
        'time': times,
        'read count': rcount,
        'link': lnk
        
      })
  df = pd.DataFrame(arti)
  list_dct = df.to_dict(orient='records')
  dct = {"date": scrapdate, 'news_portal': 'kompas', "data": list_dct}
  return dct

From detik_fun() is same like above. The different is just `'news_portal': 'detik'.

Any help would be appreciated.

Try This `fscrapdata = {k:v for k, v in scrapdata.items()+scrapdata2.items()}` — TheLazyScripter, Dec 01 '20 at 18:43
@TheLazyScripter it give me `TypeError: unsupported operand type(s) for +: 'dict_items' and 'dict_items'` error — Yoel Regen, Dec 02 '20 at 05:31
@Stuart yes. But already made both of them have different keys but still same — Yoel Regen, Dec 02 '20 at 05:32
Sorry that's a difference between python versions. convert each to a list as follows `{k:v for k, v in list(scrapdata.items())+list(scrapdata2.items())}` — TheLazyScripter, Dec 02 '20 at 05:38
@TheLazyScripter the result is same like my code above. Not merged — Yoel Regen, Dec 02 '20 at 05:51
@PIG208 It's like a header for which one is `kompas` and `detik` dictionary when both of them merged — Yoel Regen, Dec 02 '20 at 06:06
Have you tried to examine the dictionary content, explicity, in the code sequence by `print(scrapdata)` before the line `fscrapdata = {**scrapdata, **scrapdata2}`? — Nour-Allah Hussein, Dec 02 '20 at 06:11
@Nour-AllahHussein `scrapdata` had the same content, structure, etc like `scrapdata2`. Which is like the output I post above — Yoel Regen, Dec 02 '20 at 06:18
If both has same 'keys', merging results would be the final one which is `scrapdata2` — Nour-Allah Hussein, Dec 02 '20 at 06:21
I already made them different keys. Either keys inside the dictionary like `author`, `category` or keys from `kompas_fun()` and `detik_fun()` like `data`, `date` and `news_portal`. But the result is same — Yoel Regen, Dec 02 '20 at 06:25

PIG208 · Answer 1 · 2020-12-02T07:04:41.847

Since we are only concerned with merging the two dictionaries here, let's limit our discussion on this code block.

if news_site.get('kompas', True) and news_site.get('detik', True) == True:
    scrapdata = kompas_fun()
    scrapdata2 = detik_fun()
    fscrapdata = {**scrapdata, **scrapdata2}
    return jsonify(fscrapdata)

fscrapdata = {**scrapdata, **scrapdata2} doesn't not work because scrapdata2 replaces the data in scrapdata. The code below illustrates how this might happen.

print({**{"data":[1,2,3]}, **{"data":[4,5,6]}})

Output:

{'data': [4, 5, 6]}

Both of the dictionaries have "data" as a key, and thus the latter would replace the value of the former.

You can't put two keys with the same name but with different values in a dictionary. But you can store the dictionaries as items in a list or as values in another dictionary.

# Make a new list containing both of them
scrapdata = kompas_fun()
scrapdata2 = detik_fun()
fscrapdata = [scrapdata, scrapdata2]

# Make a new dictionary containing and assign each of them a unique key.
scrapdata = kompas_fun()
scrapdata2 = detik_fun()
fscrapdata = {"1":scrapdata, "2":scrapdata2}

However, if this is not what you are looking for, you could also consider appending the elements for each key.

scrapdata = kompas_fun()
scrapdata2 = detik_fun()
fscrapdata = {}
for k, v in list(scrapdata1.items()) + list(scrapdata2.items()):
    # Create a new list if the current key doesn't have a list yet. Do no change otherwise.
    fscrapdata[k] = fscrapdata[k] if k in fscrapdata else []
    fscrapdata[k].append(v)

This will produce a result as follows:

{'data': [[{'author': 'Sachril Agustin Berutu',
            'category': 'detikNews',
            ......
            'total comment': '240'},
           {'author': 'Achmad Dwi Afriyadi',
            'category': 'detikFinance',
            ......
            'total comment': '0'}],
          [{'author': 'Sachril Agustin Berutu',
            'category': 'kompasNews',
            ......
            'total comment': '240'},
           {'author': 'Achmad Dwi Afriyadi',
            'category': 'kompasFinance',
            ......
            'total comment': '0'}]],
 'news_portal': ['detik', 'kompas']}

score 0 · Accepted Answer · answered Dec 04 '20 at 09:17

For anyone that might have same problem like me, it the end I'm using update and made the value key from kompas_fun() and detik_fun() different each of them.

def create_app():
    app = Flask(__name__)

    @app.route('/', methods=['POST'])
    def index():
        news_site = request.get_json()
        print(news_site)

        scrapdata = {}
        scrapdata2 = {}

        if news_site.get('kompas', True) and news_site.get('detik', True) == True:
            scrapdata = kompas_fun()
            scrapdata2 = detik_fun()
            for d in (scrapdata, scrapdata2): fscrapdata.update(d)
            return jsonify(fscrapdata)

        elif news_site.get('kompas', True) == False:
            scrapdata = detik_fun()
            fscrapdata = {**scrapdata}
            return jsonify(fscrapdata)

        elif news_site.get('detik', True) == False:
            scrapdata = kompas_fun()
            fscrapdata = {**scrapdata}
            return jsonify(fscrapdata)

        else:
            return jsonify({'value': 'error'})

    return app

At first time idk why using this method doesn't work, I'm 100% sure already saved edit on my code before run. And this is not 100% solved my problem since what I want is both result from kompas_fun() and detik_fun() have same key and merged with same json format like my post above.

Big thanks for @PIG208 solution, but I must have the same json format like in my post as an output.

score -1 · Answer 3 · answered Dec 01 '20 at 19:01

-1

Use the .update method():

Here is an example :

d1 = {1:1, 2:2}
d2 = {3:3, 4:4}

d1.update(d2)

print(d1)

Output is :

{1: 1, 2: 2, 3: 3, 4: 4}

answered Dec 01 '20 at 19:01

Malo

1,233
1
8
25

Already said on my post that I already using `update` but not working – Yoel Regen Dec 02 '20 at 05:33
I think `update` doesn't work here because it replaces values for the conflicting keys. – PIG208 Dec 02 '20 at 05:50

Can't merge two dictionaries into one dictionary

3 Answers3