Solving nested dict iteration by list comprehension or any other method

Question

I have a dict containing another dict inside it

d1 = {'a':{'p':1, 'q':2, 'r':'abc'}, 
      'b':{'p':5, 'q':6, 'r':["google", "pypi.org"]}
     }
url1 = "https://google.com"
url2 = "https://abc.com"

Now what I want to do is run a check on values of r from both the dict values but I don't want any code redundancy.How is that possible ?

What I am doing right now is :-

for k, v in d1.iteritems():
    if isinstance(v['r'], list):
        for l in v['r']:
            if url1.find(l):
                ..Do something..
            else:
                continue
    else:
        if url1.find(v['r'):
                ..Do Something same as above..
            else:
                continue

Now the problem arises with the same Do something repeated 2 times , Is there a way to solve redundancy with comprehension or by any other method , except function making and calling .

Edit-- The code is already inside a large function definition , so do provide other solutions than making another function and calling it.

That's what functions were invented for. Just wrap `..Do something..` like that `def do_something(): ..Do something..`. I don't understand the `except function` part. — freakish, May 29 '14 at 09:44
@freakish- The problem is that the following code is already inside a function and it is returning something — Ayush, May 29 '14 at 09:46
So? You can define functions inside functions (if necessary, for example if `do something` depends on the context) and then call them. It's Python. — freakish, May 29 '14 at 09:46
@freakish- The function will get slower then which I don't want , please do provide solution other than making function. — Ayush, May 29 '14 at 09:48
The what? Slower then what? The overhead of creating and calling a function is negligible. This micro-optimization is just a waste of time. You have an elegant solution here. — freakish, May 29 '14 at 09:51

Mitul Shah · Answer 1 · 2014-05-29T09:59:37.340

2

You can convert the non-list items, ie. string in this case to a list and then simply iterate over that list. And you don't need that else: continue part:

for k, v in d1.iteritems():
    value = v['r'] if isinstance(v['r'], list) else [v['r']]
    for l in value:
        if url1.find(l):
           #Do something..

edited May 29 '14 at 09:59

answered May 29 '14 at 09:52

Mitul Shah

1,556
1
12
34

score 2 · Answer 2 · answered May 29 '14 at 11:08

If you are dead serious about performance in python code and are willing to accept certain stylistic compromises, the following form will run just as fast as the manually inlined code, assuming you use pypy:

def inner():
    pass

for k, v in d1.items():
    if isinstance(v['r'], list):
        for l in v['r']:
            if url1.find(l):
                inner()
            else:
                continue
    else:
        if url1.find(v['r']):
            inner()
        else:
            continue

For a slightly more realistic example including non-empty inner and some timing code, please see this link.

Note that, just as you wrote, this version is significantly slower than the inlined one under CPython, which of course does no JIT inlining.

Hans Then · Accepted Answer · 2014-05-29T13:12:15.253

1

Freakish is right. Using a functions will be the best solution here. The overhead of using function calls will be negligible and it will probably be less than creating new lists and looping over lists of length one.

However, if you do want to avoid code duplication at all costs and avoid multiple function calls, you may want to consider rewriting your code to a generator function. This will yield the items you want to process one at a time.

def loop(d1):
    for k, v in d1.iteritems():
        if isinstance(v['r'], list):
            for l in v['r']:
                if url1.find(l):
                    yield l
                else:
                    continue
        else:
            if url1.find(v['r']):
                yield v['r']
            else:
                continue

for item in loop(d1):
    print "do something"

edited May 29 '14 at 13:12

answered May 29 '14 at 09:53

Hans Then

10,935
3
32
51

This is a better option but i don't want to create another function as the code is already inside a larger function . – Ayush May 29 '14 at 09:57
You can create a nested or a top level function. It will be called only once (and not once per loop iteration as in the suggestion by freakish). – Hans Then May 29 '14 at 09:58
@AyushTiwari If you don't want to create another function, then duplicate the code and be a bad coder. You don't seem to understand what functions are for. – freakish May 29 '14 at 09:59
@HansThen I've never suggested doing this once per loop iteration. I said doing this inside of the big functions but **before** the loop. Otherwise it doesn't make any sense. BTW: the generator is just another word for "create a function". It does create functions under the hood and call them (i.e. `__iter__`). – freakish May 29 '14 at 10:00
I'm sorry, I thought you suggested to put the "Do something" inside a function definition and instead of executing "Do something" to call the function? Doesn't this mean one function call per loop iteration? – Hans Then May 29 '14 at 10:02
@HansThen Oh, sorry, I've misread you. I thought you've written "it will be *defined* only once". But I don't understand your comment now. `do something` is the common piece of code. How do you intended not to call it once per loop iteration? – freakish May 29 '14 at 10:04
Yes, you are right about the implicit function calls in using a generator. This is not to suggest something more efficient. As I have stated, I do not think your suggestion can be beaten in terms of performance. – Hans Then May 29 '14 at 10:04
@freakish-Have a look-http://stackoverflow.com/questions/14055843/what-is-the-performance-overhead-of-nested-functions – Ayush May 29 '14 at 10:09
@AyushTiwari Cool story, bro. Have you actually measured it on your function? Because those functions there **do nothing**. If your function already does lots of heavy stuff, then you won't even see the difference. Benchmarks, jeezus... You are trying to squish these few microseconds. On the other hand you are wasting your time as a coder and you are looking for a way to write bad and hard to maintain code. – freakish May 29 '14 at 10:12
@AyushTiwari If you are concerned about the performance hit of a nested function call, use a top level function to factor out the common code. Anyway, time your actual results before you decide you need the performance optimizations. – Hans Then May 29 '14 at 10:17

Solving nested dict iteration by list comprehension or any other method

3 Answers3