Updating dictionary without overwriting existing key-value pair?

Question

I have an input file as below, from which I can construct my dictionary

General format

<IP_1>
KEY_1=VALUE_1
KEY_2=VALUE_2

<IP_2>
KEY_1=VALUE_1
KEY_2=VALUE_2

Example

192.168.1.1
USER_NAME=admin
PASSWORD=admin123

192.168.1.2
USER_NAME=user
PASSWORD=user123

Expected dictionary should look like this:

>>print dictionary_of_ip
{'192.168.1.1':{'USER_NAME'='admin','PASSWORD'='admin123'},
 '192.168.1.2':{'USER_NAME'='user','PASSWORD'='user123'}}

Essentially a dictionary within dictionary

Below is my code:

def generate_key_value_pair(filePath, sep='='):
    dict_of_ip = {}
    slave_properties = {}
    with open(filePath, "rt") as f:
        for line in f:
            stripped_line = line.strip()
            if stripped_line and stripped_line[0].isdigit():
                #print 'Found Ip'
                ip = stripped_line
                dict_of_ip[ip] = ''
            elif stripped_line and stripped_line[0].isupper():
                #print "Found attributes")
                key_value = stripped_line.split(sep)
                key = key_value[0].strip()
                value = key_value[1].strip()
                slave_properties[key] = value
                dict_of_ip[ip] = slave_properties

    return dict_of_ip

I am able to get the first of IP and their attributes as expected, but the second set of values from second IP are overwriting the first one.

>>print dict_of_ip
{'192.168.1.1': {'USER_NAME': 'user', 'PASSWORD': 'user123'},
 '192.168.1.2': {'USER_NAME': 'user', 'PASSWORD': 'user123'}}

dict_of_ip[ip] = slave_properties is causing the overwrite. How do I prevent the values from the '192.168.1.2' key overwriting the first one?

`if ip not in dict_of_ip: dict_of_ip[ip] = slave_properties` should do it... — Jean-François Fabre, Mar 02 '17 at 19:00
Hint: How many different dictionaries does the variable `slave_properties` refer to over its lifetime? — jwodder, Mar 02 '17 at 19:01
so you say: if key already exists then don't update it, only create. See my comment, just test for key already in dict... — Jean-François Fabre, Mar 02 '17 at 19:02
Possible duplicate of [Updating nested dictionaries when data has existing key](http://stackoverflow.com/questions/27118687/updating-nested-dictionaries-when-data-has-existing-key) — Tadhg McDonald-Jensen, Mar 02 '17 at 19:07
@Jean-FrançoisFabre , i tried with your suggestion , but i received the output as {'192.168.1.1': '', '192.168.1.2': ''} — Abhay Hegde, Mar 02 '17 at 19:31

score 0 · Accepted Answer · answered Mar 02 '17 at 19:07

Try this:

def generate_key_value_pair(filePath, sep='='):
    dict_of_ip = {}
    with open(filePath, "rt") as f:
        for line in f:
            stripped_line = line.strip()
            if stripped_line and stripped_line[0].isdigit():
                #print 'Found Ip'
                slave_properties = {}
                ip = stripped_line
                dict_of_ip[ip] = ''
            elif stripped_line and stripped_line[0].isupper():
                #print "Found attributes")
                key_value = stripped_line.split(sep)
                key = key_value[0].strip()
                value = key_value[1].strip()
                slave_properties[key] = value
                dict_of_ip[ip] = slave_properties

    return dict_of_ip

You where using the same (modified) dict. I didn't change your code logic on purpose, just changed the slave_properties = {} to where it should be

You can even remove your slave_properties and use the dict alone

def generate_key_value_pair(filePath, sep='='):
    dict_of_ip = {}
    with open(filePath, "rt") as f:
        for line in f:
            stripped_line = line.strip()
            if stripped_line and stripped_line[0].isdigit():
                #print 'Found Ip'
                ip = stripped_line
                dict_of_ip[ip] = {}
            elif stripped_line and stripped_line[0].isupper():
                #print "Found attributes")
                key_value = stripped_line.split(sep)
                key = key_value[0].strip()
                value = key_value[1].strip()
                dict_of_ip[ip][key] = value

    return dict_of_ip

score 0 · Answer 2 · answered Mar 02 '17 at 19:09

0

You do not do slave_properties = {} in the loop, so you are referring to the same dict object when you think you are you are creating a new dict with

slave_properties[key] = value
dict_of_ip[ip] = slave_properties

answered Mar 02 '17 at 19:09

Jmills

2,414
2
19
22

SashaZd · Answer 3 · 2017-03-02T19:18:20.370

This becomes easier and more efficient if you use a Python's High-Performance datatypes. For instance, here's the same code with the defaultdict instead of creating your own collection type.

from collections import defaultdict

dict_of_ip = defaultdict(dict)   # This creates a dictionary of dictionaries for you.

ip = None  
for line in f:
    stripped_line = line.strip() 
    if stripped_line and stripped_line[0].isdigit():
        ip = stripped_line  # The value for the ip only changes if a new 
                            # IP is detected. Otherwise the algorithm proceeds
                            # with the older IP address to the nested dict. 

    elif stripped_line and stripped_line[0].isupper():
        key_value = stripped_line.split(sep)  
        key = key_value[0].strip()
        value = key_value[1].strip()
        dict_of_ip[ip][key] = value # IP set in the earlier if-loop.

More specifically though, the reason you were getting the error was you were editting the same slave_properties dictionary for each subdictionary. So changes in one propagate to another.

Jan · Answer 4 · 2017-03-02T19:26:08.500

You could choose a regex way in combination with a dict comprehension:

import re

string = """
192.168.1.1
USER_NAME=admin
PASSWORD=admin123

192.168.1.2
USER_NAME=user
PASSWORD=user123
"""

regex = re.compile(r"""
    ^
    (?P<ip>\d+\.\d+\.\d+\.\d+)[\n\r]
    USER_NAME=(?P<user>.+)[\r\n]
    PASSWORD=(?P<password>.+)
    """, re.MULTILINE | re.VERBOSE)

users = {match.group('ip'):
            {'USER_NAME': match.group('user'), 
            'PASSWORD': match.group('password')}
            for match in regex.finditer(string)}

print(users)
# {'192.168.1.2': {'USER_NAME': 'user', 'PASSWORD': 'user123'}, '192.168.1.1': {'USER_NAME': 'admin', 'PASSWORD': 'admin123'}}

See it working on ideone.com. Here's the corresponding demo on regex101.com.

score 0 · Answer 5 · answered Mar 02 '17 at 19:23

Just move the initialization of slave_properties to right after an IP address is recognized. That way it will start out empty for each one encountered (I also removed the unnecessary initialization you had there of dict_of_ip[ip]).

from pprint import pprint

def generate_key_value_pair(filePath, sep='='):
    dict_of_ip = {}

    with open(filePath, "rt") as f:
        for line in f:
            line = line.strip()
            if line and line[0].isdigit():  # ip?
                slave_properties = {}  # initialize
                ip = line
            elif line and line[0].isupper():
                key_value = line.split(sep)
                key = key_value[0].strip()
                value = key_value[1].strip()
                slave_properties[key] = value
                dict_of_ip[ip] = slave_properties

    return dict_of_ip

result = generate_key_value_pair('ips.txt')
pprint(result)

Output:

{'192.168.1.1': {'PASSWORD': 'admin123', 'USER_NAME': 'admin'},
 '192.168.1.2': {'PASSWORD': 'user123', 'USER_NAME': 'user'}}

score 0 · Answer 6 · answered Mar 02 '17 at 19:24

A compact solution using slices and a dictcomp:

with open("data.txt", "r") as f:
    f = [i.strip() for i in f.readlines()]
    ipdict = {ip: {'USER_NAME': user[10:], 'PASSWORD': password[9:]} 
              for ip, user, password in zip(f[0::4], f[1::4], f[2::4])}

Updating dictionary without overwriting existing key-value pair?

6 Answers6