Compare two lists of lists and fill in blank values

Question

I'm reading data from an API and have a list of lists like this:

listData = [[datetime.datetime(2018, 1, 1, 5, 0), -6.78125], 
[datetime.datetime(2018, 1, 1, 7, 0), -6.125], 
[datetime.datetime(2018, 1, 1, 8, 0), -5.90625]]

I need to create a complete list filling in the missing values. I've created a destination, like this:

listDest = [[datetime.datetime(2018, 1, 1, 5, 0), None], 
[datetime.datetime(2018, 1, 1, 6, 0), None], 
[datetime.datetime(2018, 1, 1, 7, 0), None],
[datetime.datetime(2018, 1, 1, 8, 0), None]]

The end result should look like this:

[[datetime.datetime(2018, 1, 1, 5, 0), -6.78125],
[datetime.datetime(2018, 1, 1, 6, 0), None],
[datetime.datetime(2018, 1, 1, 7, 0), -6.125], 
[datetime.datetime(2018, 1, 1, 8, 0), -5.90625]]

Here is the code I've tried:

for blankTime, blankValue in listDest:
    for dataTime, dataValue in listData:
        if blankTime == dataTime:
            blankIndex = listDest.index(blankTime)
            dataIndex = listData.index(dataTime)
            listDest[blankIndex] = tempRm7[dataIndex]

This returns the following error, which is confusing since I know that value is in both lists.

ValueError: datetime.datetime(2018, 1, 1, 5, 0) is not in list

I attempted to adapt the methods in this answer but that's for a 1D list and I couldn't figure out how to make it work for my 2D list.

Are both `listData` and `listDest` sorted by the datetime? – Andrej Kesely Jun 08 '20 at 22:56 — Andrej Kesely, Jun 08 '20 at 22:56
@AndrejKesely yes, both are sorted. – LShaver Jun 08 '20 at 22:59 — LShaver, Jun 08 '20 at 22:59

score 1 · Accepted Answer · answered Jun 08 '20 at 23:02

If both lists are sorted, you can merge them and then group them (using heapq.merge/itertools.groupby):

import datetime
from heapq import merge
from itertools import groupby


listData = [[datetime.datetime(2018, 1, 1, 5, 0), -6.78125],
            [datetime.datetime(2018, 1, 1, 7, 0), -6.125],
            [datetime.datetime(2018, 1, 1, 8, 0), -5.90625]]

listDest = [[datetime.datetime(2018, 1, 1, 5, 0), None],
            [datetime.datetime(2018, 1, 1, 6, 0), None],
            [datetime.datetime(2018, 1, 1, 7, 0), None],
            [datetime.datetime(2018, 1, 1, 8, 0), None]]

out = [next(g) for _, g in groupby(merge(listData, listDest, key=lambda k: k[0]), lambda k: k[0])]

# pretty print to screen:
from pprint import pprint
pprint(out)

Prints:

[[datetime.datetime(2018, 1, 1, 5, 0), -6.78125],
 [datetime.datetime(2018, 1, 1, 6, 0), None],
 [datetime.datetime(2018, 1, 1, 7, 0), -6.125],
 [datetime.datetime(2018, 1, 1, 8, 0), -5.90625]]

Perfect, thanks! Is this what's called "list comprehension"? I need to spend some time understanding exactly how that works. The syntax is inscrutable to me. — LShaver, Jun 10 '20 at 14:37

Compare two lists of lists and fill in blank values

1 Answers1