-2

I am writing a lambda in python where I am searching elements within dictionary without exact match.

for security_data_item in security_list_per_prefix_raw_m:
    investment_vehicle_id = security_data_item.get_investment_vehicle_id()
    unique_id = investment_vehicle_id


    ext_security_dict_keys = list(dict(filter(lambda item: item[0].startswith(unique_id), 
    security_dict.items())).keys())

When I am running this my lambda is getting time-out. security_dict will have too many items. I can not use exact match security_dict.get(unique_id) Is there any better way I can search? I also tried this ext_security_dict_keys = [val for key, val in security_dict.items() if re.search(unique_id, key)]

Mark
  • 7,785
  • 2
  • 14
  • 34
user3420305
  • 11
  • 4
  • 3
    Partial search defeats the purpose of using a hashed data type such as a dictionary. Approximately how many items will there be? What is the maximum length of a key? What is the minimum length of a `unique_id` you want to partially search for? Where does `security_dict` come from? Can you use another data type instead of a dict? – Selcuk Jul 28 '23 at 02:08
  • 2
    "I can not use exact match" then you have no choice but to iterate through the dictionary and check one-by-one. – juanpa.arrivillaga Jul 28 '23 at 02:08
  • 1
    Use a different data structure, [such as a trie](https://stackoverflow.com/questions/11015320). To the extent that this question is distinct and answerable, I don't think it's really a Python-specific question, but a general data structures and algorithms question. – Karl Knechtel Jul 28 '23 at 02:09
  • @juanpa.arrivillaga Of course they have other choices. They could use a trie, as Karl Knechtel mentioned, or even generate additional lookup keys in the same dict, even though it is not ideal. – Selcuk Jul 28 '23 at 02:12
  • Not sure how I can use trie in this case? – user3420305 Jul 28 '23 at 02:19
  • You are comparing the beginning of the keys in the `security_dict` with the `unique_id`. Does this mean that every key in `security_dict` is of the form `unique_id-something`? If so, can you pre-convert the dictionary to have `unique_id` as a key, like `dict[unique_id, list[item]]`? – ken Jul 28 '23 at 02:21
  • 1
    @user3420305 You should clarify your question by answering the questions above, first. – Selcuk Jul 28 '23 at 02:22
  • @Ken unique_id in security_dict might be 'F000000EU6XNAS' however my list would have value = 'F000000EU6' which I wanted to search in security_dict without exact match. I did not get this part - ``` dict[unique_id, list[item]]``` – user3420305 Jul 28 '23 at 03:05
  • 1
    @Selcuk I meant "no other choice if you continue to use a dictionary mapping those keys to those values", of course, you can use a different data structure or approach. – juanpa.arrivillaga Jul 28 '23 at 04:58
  • @user3420305 do you know in advance what the length of the overlap will be? – juanpa.arrivillaga Jul 28 '23 at 05:03

1 Answers1

0

You're going through too many data structure iteration/creation. You can get the result directly from iterating on the dictionary (it iterates on its keys)

*ext_security_dict_keys, = filter(lambda k:k.startswith(unique_id),security_dict)

or

ext_security_dict_keys = [k for k in security_dict if k.startswith(unique_id)]
Alain T.
  • 40,517
  • 4
  • 31
  • 51