remove dict from list if one value is same and other value is lower

Question

I have a list with dicts with version numbers

my_list = [{'version': 'v1.2.3', 'major': '1.2'},
           {'version': 'v1.2.7', 'major': '1.2'},
           {'version': 'v1.3.7', 'major': '1.3'},
           {'version': 'v1.4.1a1', 'major': '1.4'},
           {'version': 'v1.3.8b1', 'major': '1.3'},
           {'version': 'v1.3.2', 'major': '1.3'}]

In the end I want this list only to have the latest version of each major version and have all alpha/beta versions removed.

my_list = [{'version': 'v1.2.7', 'major': '1.2'},
           {'version': 'v1.3.7', 'major': '1.3'}]

My first thought was to make a new list and iterate through my list, if a major is not in the new list, it will be added, if it's in the new list, it will compare and replace. But I think there may be a more pythonic way to do so.

EDIT: there are also alpha and beta versions which I'd like to remove completely from the list.

I asked you a question. Can you answer it please? `Is order important for the output?` — cs95, Nov 24 '17 at 09:42

Ma0 · Accepted Answer · 2017-11-24T09:45:03.907

4

Another case where itertools.groupby comes to the resque:

from itertools import groupby

my_list = [{'version': 'v1.2.3', 'major': '1.2'},
           {'version': 'v1.2.7', 'major': '1.2'},
           {'version': 'v1.3.7', 'major': '1.3'},
           {'version': 'v1.4.1a1', 'major': '1.4'},
           {'version': 'v1.3.8b1', 'major': '1.3'},
           {'version': 'v1.3.2', 'major': '1.3'}]

my_list_ = list(filter(lambda x: all(beta not in x['version'] for beta in ('a', 'b')), my_list))  # removing beta-versions

version_f = lambda y: [0 if any(beta in x for beta in ('a', 'b')) else int(x) for x in y['version'].replace('v', '').split('.')]
grouper = lambda x: x['major']

d = [max(k, key=version_f) for _, k in groupby(sorted(my_list, key=grouper), key=grouper)]
print(d)  # -> [{'version': 'v1.2.7', 'major': '1.2'}, {'version': 'v1.3.7', 'major': '1.3'}, {'version': 'v1.4.1a1', 'major': '1.4'}]

Notes:

Do not use the name list. You are overwriting the Python built-in.
If the list-comprehension is too big for your taste, break it down using good-old for loops. There is no crime in that.
As @Coldspeed mentions, comparing versions is not so straightforward, so to showcase that, I took the liberty to modify your input a bit (added {'version': 'v1.3.12', 'major': '1.3'}) and made the lambda a bit more clever¹.

^1.This works based on the Python built-in ordering scheme for lists of integers ([1, 3, 10] > [1, 3, 7] returns True).

edited Nov 24 '17 at 09:45

answered Nov 24 '17 at 09:09

Ma0

15,057
4
35
65

Yes, this would be the most straightforward answer if ordering wasn't important with the output. – cs95 Nov 24 '17 at 09:10
Keep in mind, however, that comparing versions isn't so straightforward. 10.0 is greater than 2.0 but your approach isn't going to pick up on that. – cs95 Nov 24 '17 at 09:10
@cᴏʟᴅsᴘᴇᴇᴅ Is that better, or am I still missing something? – Ma0 Nov 24 '17 at 09:15
There's a question on SO that you might find useful - https://stackoverflow.com/questions/1714027/version-number-comparison – cs95 Nov 24 '17 at 09:17
2

@cᴏʟᴅsᴘᴇᴇᴅ Both examples you provided work. [Have a look](https://repl.it/repls/FloralwhitePaleFallowdeer). And the post you linked, utilizes the same principle – Ma0 Nov 24 '17 at 09:20
Okay, I'm satisfied. :) – cs95 Nov 24 '17 at 09:22
@cᴏʟᴅsᴘᴇᴇᴅ Then I am satisfied too. Let's see what OP thinks ;) – Ma0 Nov 24 '17 at 09:22
This is almost perfect! But I just saw, that I have also a1 and b2 versions which I like to completely ignore in that case. Your current solution throws an error because it's not an int. – Andreas Hubert Nov 24 '17 at 09:26
1

@AndreasHubert Can you please edit your question then to include those *edge-cases* too? – Ma0 Nov 24 '17 at 09:27
1

No problem, I love nitpicking ;) BTW, no need to call `list()` inside `max`. Finally, a lambda is an anonymous function. If you name this anonymous function, you might as well use a function. – Eric Duminil Nov 24 '17 at 09:40
1

@AndreasHubert How about now? – Ma0 Nov 24 '17 at 09:46
@Ev.Kounis perfect! Thanks so much! Do you maybe know also good tutorials if it comes to such solutions? Because I never get this on my own with such things and always end up on StackOverflow :/ – Andreas Hubert Nov 24 '17 at 09:49
@AndreasHubert The difficult part you shown that you can do your self (the description of the algorithm in your question). Coding it, is not a big deal really; just break it down into small pieces and do one chunk at a time. – Ma0 Nov 24 '17 at 09:52
@Ev.Kounis Expressing what I want is easy, but still I have problems to get my head around in python about sorting and grouping and filtering etc. And again I'm stuck right now ... will make a new question here :D – Andreas Hubert Nov 24 '17 at 13:34

Eric Duminil · Answer 2 · 2017-11-24T10:23:32.697

Your idea

Your idea is actually a good one. It is efficient and can be implemented in a relatively Pythonic way:

import re
releases = [{'version': 'v1.2.3', 'major': '1.2'},
            {'version': 'v1.2.7', 'major': '1.2'},
            {'version': 'v1.3.7', 'major': '1.3'},
            {'version': 'v1.4.1a1', 'major': '1.4'},
            {'version': 'v1.3.8b1', 'major': '1.3'},
            {'version': 'v1.3.2', 'major': '1.3'}]

stable_releases = [r for r in releases if 'a' not in r['version']
                                      and 'b' not in r['version']]

latest = {}

def major_minor_build(version):
    return [int(d) for d in re.findall('\d+', version)]

for release in stable_releases:
    version, major = release['version'], release['major']
    latest[major] = max([version, latest.get(major, '')],
                                  key=major_minor_build)

print(latest)
# {'1.2': 'v1.2.7', '1.3': 'v1.3.7'}

The output data is a dict of (major, latest) pairs, which probably is easier to work with than a list of dicts.

SetuptoolsVersion

Versions can be tricky. Instead of reinventing the wheel, we could use pkg_resources.SetuptoolsVersion. Comparisons are already implemented, so max and sort won't need any key. As a bonus, is_prerelease is True if the version is alpha or beta:

from pkg_resources import SetuptoolsVersion, parse_version
from itertools import groupby

def get_major(release):
    return release._version.release[:2]

mylist = [{'version': 'v1.2.3', 'major': '1.2'},
         {'version': 'v1.2.7', 'major': '1.2'},
         {'version': 'v1.3.7', 'major': '1.3'},
         {'version': 'v1.4.1a1', 'major': '1.4'},
         {'version': 'v1.3.8b1', 'major': '1.3'},
         {'version': 'v1.3.2', 'major': '1.3'}]

releases = [parse_version(r['version']) for r in mylist]
stable_releases = [r for r in releases if not r.is_prerelease]
stable_releases.sort()

print({major:max(group) for major, group in groupby(stable_releases, key=get_major)})
# {(1, 2): <Version('1.2.7')>, (1, 3): <Version('1.3.7')>}

remove dict from list if one value is same and other value is lower

2 Answers2

Your idea

SetuptoolsVersion