1

I have a list of dictionaries as 

x = [{'name': 'attack', 'value': 10}, {'name': 'attack_side', 'value': 12}, ..., \
{'name': 'goals', 'value': 5}, ........]

How can I efficiently determine whether or not the list has a dictionary with dic['name'] is 'goals' for example?

A solution:

value = None
for dic in x:
        if dic['name'] == 'goals':
            value = int(dic['value'])
            break

The list x is a stats for a football player. So, a player that have not scored any goal will not have a dictionary with dic['name'] is 'goals'.

playerStats = [....], each element in playerStats has similar form of x, but may not have a dictionary with goals value.

There will be around 800 lookups

An actual solution using @ggorlen's answer

import numpy
import time

playerStats = numpy.load("/home/asus/Arief_tempo/others/Python/PL_data/players_stats.npy").item()
playerGoals = {}

a = time.perf_counter()
for name in playerStats.keys():
    lookup = {d['name']: d['value'] for d in playerStats[name]['stats']}
    try:        
        playerGoals[name] = int(lookup['goals'])
    except:
        playerGoals[name] = 0

print(time.perf_counter() - a)

Here is an actual example of an element in playerStats, the x I mentioned above can be associated to playerStats['Mohamed Salah']['stats'], the 'id' won't be used here. He has 'goals', but other players may not hace 'goals' as key.

playerStats['Mohamed Salah']
{'id': 5178.0, 'stats': [{'name': 'accurate_back_zone_pass', 'additionalInfo': {}, 'description': 'Todo: accurate_back_zone_pass', 'value': 70.0}, {'name': 'accurate_chipped_pass', 'additionalInfo': {}, 'description': 'Todo: accurate_chipped_pass', 'value': 10.0}, {'name': 'accurate_corners_intobox', 'additionalInfo': {}, 'description': 'Todo: accurate_corners_intobox', 'value': 6.0}, {'name': 'accurate_cross', 'additionalInfo': {}, 'description': 'Todo: accurate_cross', 'value': 11.0}, {'name': 'accurate_cross_nocorner', 'additionalInfo': {}, 'description': 'Todo: accurate_cross_nocorner', 'value': 5.0}, {'name': 'accurate_flick_on', 'additionalInfo': {}, 'description': 'Todo: accurate_flick_on', 'value': 5.0}, {'name': 'accurate_freekick_cross', 'additionalInfo': {}, 'description': 'Todo: accurate_freekick_cross', 'value': 0.0}, {'name': 'accurate_fwd_zone_pass', 'additionalInfo': {}, 'description': 'Todo: accurate_fwd_zone_pass', 'value': 377.0}, {'name': 'accurate_layoffs', 'additionalInfo': {}, 'description': 'Todo: accurate_layoffs', 'value': 19.0}, {'name': 'accurate_long_balls', 'additionalInfo': {}, 'description': 'Todo: accurate_long_balls', 'value': 7.0}, {'name': 'accurate_pass', 'additionalInfo': {}, 'description': 'Todo: accurate_pass', 'value': 436.0}, {'name': 'accurate_through_ball', 'additionalInfo': {}, 'description': 'Todo: accurate_through_ball', 'value': 8.0}, {'name': 'accurate_throws', 'additionalInfo': {}, 'description': 'Todo: accurate_throws', 'value': 3.0}, {'name': 'aerial_lost', 'additionalInfo': {}, 'description': 'Todo: aerial_lost', 'value': 32.0}, {'name': 'aerial_won', 'additionalInfo': {}, 'description': 'Todo: aerial_won', 'value': 9.0}, {'name': 'appearances', 'additionalInfo': {}, 'description': 'Todo: appearances', 'value': 21.0}, {'name': 'assist_penalty_won', 'additionalInfo': {}, 'description': 'Todo: assist_penalty_won', 'value': 1.0}, {'name': 'attempted_tackle_foul', 'additionalInfo': {}, 'description': 'Todo: attempted_tackle_foul', 'value': 5.0}, {'name': 'attempts_conceded_ibox', 'additionalInfo': {}, 'description': 'Todo: attempts_conceded_ibox', 'value': 95.0}, {'name': 'attempts_conceded_obox', 'additionalInfo': {}, 'description': 'Todo: attempts_conceded_obox', 'value': 61.0}, {'name': 'attempts_ibox', 'additionalInfo': {}, 'description': 'Todo: attempts_ibox', 'value': 50.0}, {'name': 'attempts_obox', 'additionalInfo': {}, 'description': 'Todo: attempts_obox', 'value': 19.0}, {'name': 'att_assist_openplay', 'additionalInfo': {}, 'description': 'Todo: att_assist_openplay', 'value': 34.0}, {'name': 'att_assist_setplay', 'additionalInfo': {}, 'description': 'Todo: att_assist_setplay', 'value': 3.0}, {'name': 'att_bx_centre', 'additionalInfo': {}, 'description': 'Todo: att_bx_centre', 'value': 23.0}, {'name': 'att_bx_left', 'additionalInfo': {}, 'description': 'Todo: att_bx_left', 'value': 2.0}, {'name': 'att_bx_right', 'additionalInfo': {}, 'description': 'Todo: att_bx_right', 'value': 10.0}, {'name': 'att_cmiss_high', 'additionalInfo': {}, 'description': 'Todo: att_cmiss_high', 'value': 2.0}, {'name': 'att_cmiss_left', 'additionalInfo': {}, 'description': 'Todo: att_cmiss_left', 'value': 2.0}, {'name': 'att_cmiss_right', 'additionalInfo': {}, 'description': 'Todo: att_cmiss_right', 'value': 3.0}, {'name': 'att_fastbreak', 'additionalInfo': {}, 'description': 'Todo: att_fastbreak', 'value': 6.0}, {'name': 'att_freekick_total', 'additionalInfo': {}, 'description': 'Todo: att_freekick_total', 'value': 1.0}, {'name': 'att_goal_high_centre', 'additionalInfo': {}, 'description': 'Todo: att_goal_high_centre', 'value': 1.0}, {'name': 'att_goal_high_left', 'additionalInfo': {}, 'description': 'Todo: att_goal_high_left', 'value': 1.0}, {'name': 'att_goal_low_centre', 'additionalInfo': {}, 'description': 'Todo: att_goal_low_centre', 'value': 4.0}, {'name': 'att_goal_low_left', 'additionalInfo': {}, 'description': 'Todo: att_goal_low_left', 'value': 5.0}, {'name': 'att_goal_low_right', 'additionalInfo': {}, 'description': 'Todo: att_goal_low_right', 'value': 2.0}, {'name': 'att_hd_target', 'additionalInfo': {}, 'description': 'Todo: att_hd_target', 'value': 3.0}, {'name': 'att_hd_total', 'additionalInfo': {}, 'description': 'Todo: att_hd_total', 'value': 3.0}, {'name': 'att_ibox_blocked', 'additionalInfo': {}, 'description': 'Todo: att_ibox_blocked', 'value': 7.0}, {'name': 'att_ibox_goal', 'additionalInfo': {}, 'description': 'Todo: att_ibox_goal', 'value': 13.0}, {'name': 'att_ibox_miss', 'additionalInfo': {}, 'description': 'Todo: att_ibox_miss', 'value': 12.0}, {'name': 'att_ibox_target', 'additionalInfo': {}, 'description': 'Todo: att_ibox_target', 'value': 18.0}, {'name': 'att_lf_goal', 'additionalInfo': {}, 'description': 'Todo: att_lf_goal', 'value': 11.0}, {'name': 'att_lf_target', 'additionalInfo': {}, 'description': 'Todo: att_lf_target', 'value': 11.0}, {'name': 'att_lf_total', 'additionalInfo': {}, 'description': 'Todo: att_lf_total', 'value': 56.0}, {'name': 'att_lg_centre', 'additionalInfo': {}, 'description': 'Todo: att_lg_centre', 'value': 1.0}, {'name': 'att_miss_high', 'additionalInfo': {}, 'description': 'Todo: att_miss_high', 'value': 4.0}, {'name': 'att_miss_high_left', 'additionalInfo': {}, 'description': 'Todo: att_miss_high_left', 'value': 3.0}, {'name': 'att_miss_high_right', 'additionalInfo': {}, 'description': 'Todo: att_miss_high_right', 'value': 1.0}, {'name': 'att_miss_left', 'additionalInfo': {}, 'description': 'Todo: att_miss_left', 'value': 4.0}, {'name': 'att_miss_right', 'additionalInfo': {}, 'description': 'Todo: att_miss_right', 'value': 6.0}, {'name': 'att_obox_blocked', 'additionalInfo': {}, 'description': 'Todo: att_obox_blocked', 'value': 10.0}, {'name': 'att_obox_miss', 'additionalInfo': {}, 'description': 'Todo: att_obox_miss', 'value': 6.0}, {'name': 'att_obox_target', 'additionalInfo': {}, 'description': 'Todo: att_obox_target', 'value': 3.0}, {'name': 'att_obx_centre', 'additionalInfo': {}, 'description': 'Todo: att_obx_centre', 'value': 16.0}, {'name': 'att_obx_right', 'additionalInfo': {}, 'description': 'Todo: att_obx_right', 'value': 2.0}, {'name': 'att_one_on_one', 'additionalInfo': {}, 'description': 'Todo: att_one_on_one', 'value': 2.0}, {'name': 'att_openplay', 'additionalInfo': {}, 'description': 'Todo: att_openplay', 'value': 58.0}, {'name': 'att_pen_goal', 'additionalInfo': {}, 'description': 'Todo: att_pen_goal', 'value': 2.0}, {'name': 'att_rf_goal', 'additionalInfo': {}, 'description': 'Todo: att_rf_goal', 'value': 2.0}, {'name': 'att_rf_target', 'additionalInfo': {}, 'description': 'Todo: att_rf_target', 'value': 7.0}, {'name': 'att_rf_total', 'additionalInfo': {}, 'description': 'Todo: att_rf_total', 'value': 10.0}, {'name': 'att_setpiece', 'additionalInfo': {}, 'description': 'Todo: att_setpiece', 'value': 3.0}, {'name': 'att_sv_high_centre', 'additionalInfo': {}, 'description': 'Todo: att_sv_high_centre', 'value': 1.0}, {'name': 'att_sv_high_left', 'additionalInfo': {}, 'description': 'Todo: att_sv_high_left', 'value': 1.0}, {'name': 'att_sv_low_centre', 'additionalInfo': {}, 'description': 'Todo: att_sv_low_centre', 'value': 8.0}, {'name': 'att_sv_low_left', 'additionalInfo': {}, 'description': 'Todo: att_sv_low_left', 'value': 5.0}, {'name': 'att_sv_low_right', 'additionalInfo': {}, 'description': 'Todo: att_sv_low_right', 'value': 6.0}, {'name': 'backward_pass', 'additionalInfo': {}, 'description': 'Todo: backward_pass', 'value': 151.0}, {'name': 'ball_recovery', 'additionalInfo': {}, 'description': 'Todo: ball_recovery', 'value': 61.0}, {'name': 'big_chance_created', 'additionalInfo': {}, 'description': 'Todo: big_chance_created', 'value': 13.0}, {'name': 'big_chance_missed', 'additionalInfo': {}, 'description': 'Todo: big_chance_missed', 'value': 11.0}, {'name': 'big_chance_scored', 'additionalInfo': {}, 'description': 'Todo: big_chance_scored', 'value': 8.0}, {'name': 'blocked_cross', 'additionalInfo': {}, 'description': 'Todo: blocked_cross', 'value': 2.0}, {'name': 'blocked_pass', 'additionalInfo': {}, 'description': 'Todo: blocked_pass', 'value': 25.0}, {'name': 'blocked_scoring_att', 'additionalInfo': {}, 'description': 'Todo: blocked_scoring_att', 'value': 17.0}, {'name': 'challenge_lost', 'additionalInfo': {}, 'description': 'Todo: challenge_lost', 'value': 5.0}, {'name': 'clean_sheet', 'additionalInfo': {}, 'description': 'Todo: clean_sheet', 'value': 9.0}, {'name': 'corner_taken', 'additionalInfo': {}, 'description': 'Todo: corner_taken', 'value': 23.0}, {'name': 'crosses_18yard', 'additionalInfo': {}, 'description': 'Todo: crosses_18yard', 'value': 8.0}, {'name': 'crosses_18yardplus', 'additionalInfo': {}, 'description': 'Todo: crosses_18yardplus', 'value': 4.0}, {'name': 'dispossessed', 'additionalInfo': {}, 'description': 'Todo: dispossessed', 'value': 72.0}, {'name': 'draws', 'additionalInfo': {}, 'description': 'Todo: draws', 'value': 3.0}, {'name': 'duel_lost', 'additionalInfo': {}, 'description': 'Todo: duel_lost', 'value': 143.0}, {'name': 'duel_won', 'additionalInfo': {}, 'description': 'Todo: duel_won', 'value': 80.0}, {'name': 'effective_blocked_cross', 'additionalInfo': {}, 'description': 'Todo: effective_blocked_cross', 'value': 2.0}, {'name': 'effective_clearance', 'additionalInfo': {}, 'description': 'Todo: effective_clearance', 'value': 2.0}, {'name': 'effective_head_clearance', 'additionalInfo': {}, 'description': 'Todo: effective_head_clearance', 'value': 1.0}, {'name': 'final_third_entries', 'additionalInfo': {}, 'description': 'Todo: final_third_entries', 'value': 36.0}, {'name': 'fouled_final_third', 'additionalInfo': {}, 'description': 'Todo: fouled_final_third', 'value': 12.0}, {'name': 'fouls', 'additionalInfo': {}, 'description': 'Todo: fouls', 'value': 15.0}, {'name': 'freekick_cross', 'additionalInfo': {}, 'description': 'Todo: freekick_cross', 'value': 4.0}, {'name': 'fwd_pass', 'additionalInfo': {}, 'description': 'Todo: fwd_pass', 'value': 114.0}, {'name': 'game_started', 'additionalInfo': {}, 'description': 'Todo: game_started', 'value': 20.0}, {'name': 'goals', 'additionalInfo': {}, 'description': 'Todo: goals', 'value': 13.0}, {'name': 'goals_conceded', 'additionalInfo': {}, 'description': 'Todo: goals_conceded', 'value': 9.0}, {'name': 'goals_conceded_ibox', 'additionalInfo': {}, 'description': 'Todo: goals_conceded_ibox', 'value': 9.0}, {'name': 'goals_openplay', 'additionalInfo': {}, 'description': 'Todo: goals_openplay', 'value': 9.0}, {'name': 'goal_assist', 'additionalInfo': {}, 'description': 'Todo: goal_assist', 'value': 7.0}, {'name': 'goal_assist_deadball', 'additionalInfo': {}, 'description': 'Todo: goal_assist_deadball', 'value': 1.0}, {'name': 'goal_assist_intentional', 'additionalInfo': {}, 'description': 'Todo: goal_assist_intentional', 'value': 7.0}, {'name': 'goal_assist_openplay', 'additionalInfo': {}, 'description': 'Todo: goal_assist_openplay', 'value': 5.0}, {'name': 'goal_assist_setplay', 'additionalInfo': {}, 'description': 'Todo: goal_assist_setplay', 'value': 2.0}, {'name': 'goal_fastbreak', 'additionalInfo': {}, 'description': 'Todo: goal_fastbreak', 'value': 1.0}, {'name': 'hand_ball', 'additionalInfo': {}, 'description': 'Todo: hand_ball', 'value': 1.0}, {'name': 'head_clearance', 'additionalInfo': {}, 'description': 'Todo: head_clearance', 'value': 1.0}, {'name': 'head_pass', 'additionalInfo': {}, 'description': 'Todo: head_pass', 'value': 25.0}, {'name': 'interception', 'additionalInfo': {}, 'description': 'Todo: interception', 'value': 4.0}, {'name': 'interception_won', 'additionalInfo': {}, 'description': 'Todo: interception_won', 'value': 4.0}, {'name': 'leftside_pass', 'additionalInfo': {}, 'description': 'Todo: leftside_pass', 'value': 216.0}, {'name': 'long_pass_own_to_opp', 'additionalInfo': {}, 'description': 'Todo: long_pass_own_to_opp', 'value': 13.0}, {'name': 'long_pass_own_to_opp_success', 'additionalInfo': {}, 'description': 'Todo: long_pass_own_to_opp_success', 'value': 10.0}, {'name': 'losses', 'additionalInfo': {}, 'description': 'Todo: losses', 'value': 1.0}, {'name': 'mins_played', 'additionalInfo': {}, 'description': 'Todo: mins_played', 'value': 1759.0}, {'name': 'offtarget_att_assist', 'additionalInfo': {}, 'description': 'Todo: offtarget_att_assist', 'value': 14.0}, {'name': 'ontarget_att_assist', 'additionalInfo': {}, 'description': 'Todo: ontarget_att_assist', 'value': 30.0}, {'name': 'ontarget_scoring_att', 'additionalInfo': {}, 'description': 'Todo: ontarget_scoring_att', 'value': 34.0}, {'name': 'open_play_pass', 'additionalInfo': {}, 'description': 'Todo: open_play_pass', 'value': 556.0}, {'name': 'overrun', 'additionalInfo': {}, 'description': 'Todo: overrun', 'value': 7.0}, {'name': 'passes_left', 'additionalInfo': {}, 'description': 'Todo: passes_left', 'value': 47.0}, {'name': 'passes_right', 'additionalInfo': {}, 'description': 'Todo: passes_right', 'value': 119.0}, {'name': 'penalty_won', 'additionalInfo': {}, 'description': 'Todo: penalty_won', 'value': 3.0}, {'name': 'pen_area_entries', 'additionalInfo': {}, 'description': 'Todo: pen_area_entries', 'value': 93.0}, {'name': 'poss_lost_all', 'additionalInfo': {}, 'description': 'Todo: poss_lost_all', 'value': 323.0}, {'name': 'poss_lost_ctrl', 'additionalInfo': {}, 'description': 'Todo: poss_lost_ctrl', 'value': 323.0}, {'name': 'poss_won_att_3rd', 'additionalInfo': {}, 'description': 'Todo: poss_won_att_3rd', 'value': 20.0}, {'name': 'poss_won_def_3rd', 'additionalInfo': {}, 'description': 'Todo: poss_won_def_3rd', 'value': 14.0}, {'name': 'poss_won_mid_3rd', 'additionalInfo': {}, 'description': 'Todo: poss_won_mid_3rd', 'value': 27.0}, {'name': 'put_through', 'additionalInfo': {}, 'description': 'Todo: put_through', 'value': 25.0}, {'name': 'rightside_pass', 'additionalInfo': {}, 'description': 'Todo: rightside_pass', 'value': 92.0}, {'name': 'shot_fastbreak', 'additionalInfo': {}, 'description': 'Todo: shot_fastbreak', 'value': 6.0}, {'name': 'shot_off_target', 'additionalInfo': {}, 'description': 'Todo: shot_off_target', 'value': 18.0}, {'name': 'successful_final_third_passes', 'additionalInfo': {}, 'description': 'Todo: successful_final_third_passes', 'value': 261.0}, {'name': 'successful_open_play_pass', 'additionalInfo': {}, 'description': 'Todo: successful_open_play_pass', 'value': 419.0}, {'name': 'successful_put_through', 'additionalInfo': {}, 'description': 'Todo: successful_put_through', 'value': 6.0}, {'name': 'total_att_assist', 'additionalInfo': {}, 'description': 'Todo: total_att_assist', 'value': 44.0}, {'name': 'total_back_zone_pass', 'additionalInfo': {}, 'description': 'Todo: total_back_zone_pass', 'value': 82.0}, {'name': 'total_chipped_pass', 'additionalInfo': {}, 'description': 'Todo: total_chipped_pass', 'value': 30.0}, {'name': 'total_clearance', 'additionalInfo': {}, 'description': 'Todo: total_clearance', 'value': 2.0}, {'name': 'total_contest', 'additionalInfo': {}, 'description': 'Todo: total_contest', 'value': 61.0}, {'name': 'total_corners_intobox', 'additionalInfo': {}, 'description': 'Todo: total_corners_intobox', 'value': 20.0}, {'name': 'total_cross', 'additionalInfo': {}, 'description': 'Todo: total_cross', 'value': 36.0}, {'name': 'total_cross_nocorner', 'additionalInfo': {}, 'description': 'Todo: total_cross_nocorner', 'value': 16.0}, {'name': 'total_distance_in_m', 'additionalInfo': {}, 'description': 'Todo: total_distance_in_m', 'value': 10282.4443359375}, {'name': 'total_fastbreak', 'additionalInfo': {}, 'description': 'Todo: total_fastbreak', 'value': 9.0}, {'name': 'total_final_third_passes', 'additionalInfo': {}, 'description': 'Todo: total_final_third_passes', 'value': 363.0}, {'name': 'total_flick_on', 'additionalInfo': {}, 'description': 'Todo: total_flick_on', 'value': 13.0}, {'name': 'total_fwd_zone_pass', 'additionalInfo': {}, 'description': 'Todo: total_fwd_zone_pass', 'value': 527.0}, {'name': 'total_launches', 'additionalInfo': {}, 'description': 'Todo: total_launches', 'value': 2.0}, {'name': 'total_layoffs', 'additionalInfo': {}, 'description': 'Todo: total_layoffs', 'value': 23.0}, {'name': 'total_long_balls', 'additionalInfo': {}, 'description': 'Todo: total_long_balls', 'value': 15.0}, {'name': 'total_offside', 'additionalInfo': {}, 'description': 'Todo: total_offside', 'value': 18.0}, {'name': 'total_pass', 'additionalInfo': {}, 'description': 'Todo: total_pass', 'value': 573.0}, {'name': 'total_pull_back', 'additionalInfo': {}, 'description': 'Todo: total_pull_back', 'value': 1.0}, {'name': 'total_scoring_att', 'additionalInfo': {}, 'description': 'Todo: total_scoring_att', 'value': 69.0}, {'name': 'total_sub_off', 'additionalInfo': {}, 'description': 'Todo: total_sub_off', 'value': 6.0}, {'name': 'total_sub_on', 'additionalInfo': {}, 'description': 'Todo: total_sub_on', 'value': 1.0}, {'name': 'total_tackle', 'additionalInfo': {}, 'description': 'Todo: total_tackle', 'value': 10.0}, {'name': 'total_through_ball', 'additionalInfo': {}, 'description': 'Todo: total_through_ball', 'value': 11.0}, {'name': 'total_throws', 'additionalInfo': {}, 'description': 'Todo: total_throws', 'value': 5.0}, {'name': 'touches', 'additionalInfo': {}, 'description': 'Todo: touches', 'value': 957.0}, {'name': 'touches_in_opp_box', 'additionalInfo': {}, 'description': 'Todo: touches_in_opp_box', 'value': 165.0}, {'name': 'turnover', 'additionalInfo': {}, 'description': 'Todo: turnover', 'value': 59.0}, {'name': 'unsuccessful_touch', 'additionalInfo': {}, 'description': 'Todo: unsuccessful_touch', 'value': 59.0}, {'name': 'was_fouled', 'additionalInfo': {}, 'description': 'Todo: was_fouled', 'value': 21.0}, {'name': 'wins', 'additionalInfo': {}, 'description': 'Todo: wins', 'value': 17.0}, {'name': 'won_contest', 'additionalInfo': {}, 'description': 'Todo: won_contest', 'value': 41.0}, {'name': 'won_corners', 'additionalInfo': {}, 'description': 'Todo: won_corners', 'value': 23.0}, {'name': 'won_tackle', 'additionalInfo': {}, 'description': 'Todo: won_tackle', 'value': 5.0}]}
  • What you already have is a good and efficient solution. – Primusa Jan 05 '19 at 04:43
  • There is not more efficient way. List objects require O(N) time to check for containment, although, if the list is sorted, you can use bisection search to do it in logn time. You could use a dict to do it in O(1) – juanpa.arrivillaga Jan 05 '19 at 04:44
  • 2
    Possible duplicate of [Python list of dictionaries search](https://stackoverflow.com/questions/8653516/python-list-of-dictionaries-search) – Feelsbadman Jan 05 '19 at 04:45
  • 3
    I think it's worth asking whether you are going to look this up once, or whether you are going to do the lookup millions of times. If it is just once, your solution (as well as the solutions posted below) are fine. If you are going to do lots of lookups on the same data, then you should consider constructing a mapping of `name` -> value and using _that_ for your lookup table rather than the list. – mgilson Jan 05 '19 at 04:45
  • @mgilson I am going to do it for many more similar lists –  Jan 05 '19 at 04:46
  • @AriefAnbiya -- If it isn't the same lists, then maybe you are out of luck unless you can coax the producer of these lists to give it to you in a different format. – mgilson Jan 05 '19 at 04:47
  • @mgilson there will be a list of football player stats, each player's stat is also a list containing dictionaries for each type of stat (goals, assists, passes, etc), but a player with no goal will have no dictionary for goals stat –  Jan 05 '19 at 04:49
  • 2
    @AriefAnbiya there maybe an underlying problem with your code/app design if you're storing data like this and have to keep looking it up using this method for `more similar lists`. May I ask if you're storing it this way or are you simply parsing data from some other source? – Yang K Jan 05 '19 at 04:50
  • 1
    @AriefAnbiya sounds like you should re-design your data structures, because a list of dicts is sort of the worst of both worlds – juanpa.arrivillaga Jan 05 '19 at 04:50
  • @AriefAnbiya I'm thinking a class might be a good idea here? – Yang K Jan 05 '19 at 04:51

3 Answers3

4

A dictionary of dictionaries offers O(1) lookup time:

x = [
    {'name': 'attack', 'value': 10}, 
    {'name': 'attack_side', 'value': 12},
    {'name': 'goals', 'value': 5}
]

lookup = {d['name']: d for d in x}

print(lookup['goals'])

Output:

{'name': 'goals', 'value': 5}

Notes:

  • Use lookup = {d['name']: d['value'] for d in x} if you only need to retrieve the value key instead of the whole dictionary.
  • Build the dictionary once and use it for all your lookups.
  • If you're only doing one lookup, you already have the optimal solution.
  • There must be no duplicate name keys.
ggorlen
  • 44,755
  • 7
  • 76
  • 106
  • @ggorlen your code works fine, and take up `0.04325622000033036` seconds –  Jan 05 '19 at 05:10
  • I am doing for about 800 lookups, since there are around 800 players –  Jan 05 '19 at 05:11
  • This is the way to go, then. Building one dictionary and doing 800 lookups on it should give you a huge performance boost. Your post seems to indicate that each player has a different list, though, so you'll need to [post the full scenario](https://en.wikipedia.org/wiki/XY_problem) and describe how many lookups are being done on each sub-list if that's the case. – ggorlen Jan 05 '19 at 05:13
  • No, there are about 800 players, for each player..I will lookup for a dictionary with `'goals'`. Each player has a list similar to `x` –  Jan 05 '19 at 05:15
  • 1
    All right, so that means you're only doing one lookup for each `x`? Then don't bother making a dictionary. – ggorlen Jan 05 '19 at 05:16
  • I think better to say that 'each player is associated with list similar of `x`' –  Jan 05 '19 at 05:18
  • 1
    I'm thinking this is likely an x-y problem. You're asking how to perform fast list lookups but it sounds like you might need to redesign your application to store data in lookup tables from the start. – ggorlen Jan 05 '19 at 05:19
  • @AriefAnbiya are you doing anything else with the data or are you just looking up the number of goals for each player? – Yang K Jan 05 '19 at 05:20
  • @AriefAnbiya also, are you making this list of player stats because if you are then this is pretty bad design. You might have to reconsider your application design – Yang K Jan 05 '19 at 05:21
  • @YangK I scraped this data. Not going to make an app of it, only for analysis. thanks –  Jan 05 '19 at 05:23
  • The edit you made to your post is not how I intended this to be used. You're only doing one lookup on the dict, which makes no sense. – ggorlen Jan 05 '19 at 05:23
  • 1
    @AriefAnbiya if you had scraped this data, you should have found a better way to store it rather than putting multiple dictionaries in a list with each list pertaining to one player. – Yang K Jan 05 '19 at 05:25
  • @YangK did not put it that way, it was like that after scraping using Python –  Jan 05 '19 at 05:26
  • 1
    mind providing a link to the scraped data to check if there is a better way to do it? – Yang K Jan 05 '19 at 05:27
  • @ggorlen , the `playerStats[name]['stats']` is `x`. And I am looping over all the players (`playerStats[name]` is a dictionary `{'id': ..., 'stats': x_like}`) –  Jan 05 '19 at 05:28
  • Yeah I'm seconding Yang K. Post the scraped data link and show us exactly how you're using it. What you have here is highly suspicious. – ggorlen Jan 05 '19 at 05:35
  • @AriefAnbiya Thanks, but that's not what we're asking for. The problem here is that all of your data is being put into containers that are not conducive to lookups. If you give us your original data source, the raw data (API, URL, database, whatever), we can determine if there is a way to build these structures for quicker lookups right from the beginning. Once the data is in lists, your hands are tied and you're stuck looping over them. – ggorlen Jan 05 '19 at 05:46
  • The url is https://www.premierleague.com/players/ . But right now I intend to know best way to lookup in the current structure. –  Jan 05 '19 at 05:50
  • 1
    I think I know what your answer means, you want the dictionary to be restructured from `{'name': 'goals', ...}` to just `{'goals': value}` to make it easier to access the values –  Jan 05 '19 at 05:51
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/186213/discussion-between-ggorlen-and-arief-anbiya). – ggorlen Jan 05 '19 at 05:55
3

Try something like:

Code:

goals = next((d for d in data if d['name'] == 'goals'), None)

This uses a generator expression to look through the list, and returns the first matching instance, or None if nothing matches.

Test Code:

data = [
    {'name': 'attack', 'value': 10},
    {'name': 'attack_side', 'value': 12},
    {'name': 'goals', 'value': 5}
]

goals = next((d for d in data if d['name'] == 'goals'), None)
print(goals)

no_goals = next((d for d in data if d['name'] == 'no_goals'), None)
print(no_goals)

Results:

{'name': 'goals', 'value': 5}
None
Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
  • 3
    How is this "more efficient"? This is probably *slower* although the same time complexity – juanpa.arrivillaga Jan 05 '19 at 04:44
  • 2
    Why probably slower? – Stephen Rauch Jan 05 '19 at 04:45
  • Although it is more condensed, seems more slower rather than efficient – Sheshank S. Jan 05 '19 at 04:45
  • @juanpa.arrivillaga, In my quick tests, for a list of any size (len > 30) the generator in this case is definitely faster. – Stephen Rauch Jan 05 '19 at 04:51
  • 1
    I'd be curious to see these tests. How about `len = 10,000?` If generators were faster than plain old loops with `break`, it seems like we'd have dumped loops long ago. – ggorlen Jan 05 '19 at 04:54
  • Yeah, can't reproduce. Looping over a list is always (negligibly) faster. – juanpa.arrivillaga Jan 05 '19 at 04:56
  • With the searched for element at position 10000, the loop takes almost twice as long to find it. Let me clean up my code and add to post... – Stephen Rauch Jan 05 '19 at 04:56
  • @juanpa.arrivillaga, Me being in error is most certainly possible. I am using the OPs code... But the timing maybe fishy. – Stephen Rauch Jan 05 '19 at 04:59
  • https://stackoverflow.com/a/48958217/6243352 has a speed test that might be relevant. Shows LC as faster than gen as faster than regular loop. I'm not convinced just yet. – ggorlen Jan 05 '19 at 05:05
  • 1
    @juanpa.arrivillaga, timing was suspect. Using timeit, the generator appears to be about the same performance for the looping, but is slower to setup. At somewhere between 1000 and 10000 elements the timings are the same. but shorter than that, the loop wins... Thanks for the insights. – Stephen Rauch Jan 05 '19 at 05:21
  • Thanks. But I thought `generator` is known to be fast in Python. –  Jan 05 '19 at 05:22
0

I'd suggest a different approach: convert the list of dictionary in a different format, precisely in a dict:

x = [{'name': 'attack', 'value': 10}, {'name': 'attack_side', 'value': 12},{'name': 'goals', 'value': 5}]

player_x = { ability['name']: ability['value'] for ability in x }
#=> {'attack': 10, 'attack_side': 12, 'goals': 5}

Then it is easy to get:

print(player_x.get('attack', None)) #=> 10
print(player_x.get('whathever', None)) #=> None

Or also:

for ability in player_x.keys():
  print(ability, player_x[ability])

# attack 10
# attack_side 12
# goals 5
iGian
  • 11,023
  • 3
  • 21
  • 36