0

I'm writing a program in Python. The algorithm that I use is very slow (and it can't be faster for what I want to do) so how quickly the program will execute is important (in other words: computational complexity matters) and this is what I need to do:

  1. I have a list.
  2. I need to iterate the list and do some operation on every element.
  3. When iterating the list, I need to remove some elements from the list.

This can be done very easily, but the problem is that I want removing element to have computational complexity O(1), not O(n), so that the entire process of iterating the list and removing some elements has computational complexity O(n). Now, I don't know what the internal implementation of python lists is, but I'm almost sure that when I remove element like that:

del list[5]

or like that:

list.pop(5)

It will probably have to move all of the elements after fifth element one position back which results in computational complexity O(n).

If it was C++, then I would use list that is implemented using pointers, so that I could go through the entire list and remove the elements that I want to remove in the most quickest way. But in Python, there are no pointers, so I can't implement this list on my own. So is there a way to do the above process in Python with computational complexity O(n)?

In other words:

In C++, there are two ways how you can implement a list: using pointers and using an array. The first implementation is faster for some operations, and the second is faster for other operations. I need to use the first implementation, but I don't know if it's possible in Python.

Blorgbeard
  • 101,031
  • 48
  • 228
  • 272
Damian
  • 178
  • 11
  • 1
    All Python lists are implemented as arrays of pointers pointing to objects elsewhere in memory. – Patrick Haugh Oct 15 '17 at 17:17
  • The "pointers" implementation you referred to is called a linked-list. That's what you need to google for. I think the question I linked has some useful answers though. – Blorgbeard Oct 15 '17 at 17:18
  • You could also check out [llist module](https://pythonhosted.org/llist/) – coder Oct 15 '17 at 17:23
  • 2
    *“The algorithm that I use is very slow […]”* – Chances are, the linear time complexity for removing items *won’t* be a bottleneck for your algorithm. Unless you actually profile your application and figure out that this is really a part where you lose time, you shouldn’t spend time on optimizing it. – If you implement an already slow algorithm in Python, you are probably *not* having any problem with using data structures that are implemented in native code (which lists are). – poke Oct 15 '17 at 17:23
  • Rather than making things complicated with linked lists, create a new list with the elements that you wish to keep. – jonatan Oct 15 '17 at 17:43
  • Thanks for all the answers. @poke "Chances are, the linear time complexity for removing items won’t be a bottleneck for your algorithm." - I haven't implemented the algorithm yet but I can estimate computational complexity right now and this is the part of the algorithm where it's the slowest. More precisely, linear complexity wouldn't be a problem, but this removing part is inside some loop which is in another loop etc. and using linked list makes computation complexity of the whole algorithm from O(n^4) to O(n^3) which makes the whole algorithm much more faster. – Damian Oct 15 '17 at 18:18
  • @jonatan - good solution, but I think this doesn't solve the problem in my case (my problem is a little bit different than what I described in the first post). – Damian Oct 15 '17 at 18:19
  • @coder - this module should solve the problem, thanks. – Damian Oct 15 '17 at 18:19
  • Theoretical time complexity often ends up very irrelevant when looking at the algorithm running on real-life data, you shouldn’t be too fixated on this. And even more so in a language like Python, that is run on a virtual machine. I wouldn’t be suprised if the linked list (implemented in Python) would end up being more expensive than the native array implementation of a list, even if it is theoretically of lower complexity. – poke Oct 15 '17 at 18:46

0 Answers0