Dynamic nested dictionaries

Question

Just to begin I know there are a couple similarly-titled questions on here, but none are explained quite this way nor is the problem scope the same.

I want to add nested dictionary entries dynamically.

The use case is thus: I have a python script that is monitoring the network. A dictionary is created for each observed IP protocol (tcp, udp, icmp) that is observed. Then a sub-dictionary is created with a key that is the destination port (if one exists) for each of these IP protocols (80, 443, etc.) (note it doesn't matter whether it sees the server port as the source or destination, but normally servers are the destination so I picked HTTP and HTTPS as examples). For each of these destination ports a key is created corresponding to the server IP (e.g., the IP for www.google.com). And then another dictionary with the timestamp the session was first observed as the key and the data/value for the key being the client's IP.

However, I want need this to be populated as time goes on since I won't have the data before execution nor at initialization.

The output would look something akin to:

{  'icmp' :
   {  'echo-request' :
       {  '<ip_of_www.google.com>' :
           {   '<timestamp>' : <some_client_ip> }
       }
      'echo-reply' :
       {  '<ip_of_www.google.com>' :
           {   '<timestamp>' : <some_client_ip> }
       }
   }
   'tcp' :
   {
      '80' 
       {  '<ip_of_www.google.com>' :
           {   '<timestamp>' : <some_client_ip> }
           {   '<timestamp>' : <some_client_ip> }
       }
      '443' 
       {  '<ip_of_encrypted.google.com>' :
           {   '<timestamp>' : <some_client_ip> }
           {   '<timestamp>' : <some_client_ip> }
           {   '<timestamp>' : <some_client_ip> }
           {   '<timestamp>' : <some_client_ip> }
       }
   }
}

Thanks!

1) update the dictionary as time goes on. 2) note: that timestamp as key doesn't make much sense. 3) you have an invalid structure. you can have only one value for a key. — Karoly Horvath, Dec 01 '14 at 22:09
Regarding #1: That's exactly what I'm asking for... Regarding #2: a timestamp as a key doesn't have to make sense, this was an exaggerated example Regarding #3: I'm not sure what you mean by this. Of course there's only one key, but the value for each key can be another dictionary (hence the nesting attribute). Where did I show/demonstrate/expect/ask about a multi-valued key? — Nick, Dec 02 '14 at 14:57
Are you commenting on a key value being a string in one instance and an integer in another? Unless you're talking about that scenario I don't quite understand what you mean. — Nick, Dec 02 '14 at 15:11
Are you simply mentioning I am missing a curly bracket in front of ? I hand-typed that whole setup just as an example of what I was looking for, I did not have that output from any code that I wrote? — Nick, Dec 02 '14 at 22:10

Anton Zuenko · Answer 1 · 2014-12-01T22:36:16.490

5

Here you are:

def set_nested(dict, value, *path):
    for level in path[:-1]:
        dict = dict.setdefault(level, {})

    dict[path[-1]] = value


d = {}

set_nested(d, '127.0.0.1', 'icmp', 'echo', 'google.com', '1 dec 2014')
set_nested(d, '127.0.0.1', 'icmp', 'echo', 'google.com', '2 dec 2014')
set_nested(d, '127.0.0.1', 'icmp', 'echo', 'yahoo.com', '2 dec 2014')
set_nested(d, 'error', 'udp')

from pprint import pprint
pprint(d)

Output:

{'icmp': {'echo': {'google.com': {'1 dec 2014': '127.0.0.1',
                                  '2 dec 2014': '127.0.0.1'},
                   'yahoo.com': {'2 dec 2014': '127.0.0.1'}}},
 'udp': 'error'}

I'd also suggest you to have a look at json and tinydb if you want to store and query results.

edited Dec 01 '14 at 22:36

answered Dec 01 '14 at 22:28

Anton Zuenko

721
3
13

Looks good. I'll try this out and see how it goes. I wasn't previously familiar with the *path -styled parameter that gobbles up the rest of the elements. That itself is worth a +1 :) – Nick Dec 02 '14 at 15:09
When I run your code, but add one more call to `set_nested` as follows: `set_nested(d, 'not-an-error', 'udp', 'test')`, I end up with the same output at the `pprint(d)` part... how would you modify to work around that? – johnny_be Feb 20 '16 at 19:39

Dynamic nested dictionaries

1 Answers1

Linked