1

Is there any way to create many objects at once in Python?

Let say, that I want to populate a list with 1.000.000 object instances. Instead of initializing objects one-by-one in loop, is there any way to create them in one call?

Iterating one-by-one takes some time and I am looking for a way other than running many threads to reduce this time.

My adventure with Python started a month ago, so I am quite unaware of many Python specific tricks. Any hints how to deal with this problem?

macsz
  • 21
  • 1
  • 3
  • 1
    What about lazily creating objects as you use them instead of creating from the start? check [defaultDict](https://docs.python.org/3/library/collections.html#collections.defaultdict) for example. – user3557327 Aug 16 '14 at 18:38
  • some packages such as numpy let you create a large array quickly, but generally, you have to create them one at a time. List comprehensions like `[MyClass(x) for x in range(1000000)]` tend to be faster than for loops because there are fewer name lookups. – tdelaney Aug 16 '14 at 18:40
  • 1
    Is there enough memory for 1,000,000 objects? What kind of post processing? Can they be dealt with 1 by 1? Is random access to any of the objects needed? What kind of objects? Are they muteable or immutable? Do they depend on some value of X when created? Do you need to search them for a value? Do you need to be able to order them or save them? Do you need to insert values at X location? Are they to be kept unique (like a set)? – dawg Aug 16 '14 at 20:27
  • There is an existing list of N objects that contain some data, that by creating an BLL object for each existing objects, is parsed to JSON and returned. Random access is not needed, I am looking for a solution to boost parsing process that is being carried out by creating N BLL's. Thought that something like mass object allocation could work. – macsz Aug 17 '14 at 13:10
  • VTR. This question and the linked duplicate are both talking about a list of objects, but this one is asking about the objects while the other one is asking about the list. – wjandrea Mar 22 '21 at 00:46

3 Answers3

4

You must iterate N times to make N objects, either in a for loop or list comp.

objectList = [object() for _ in range(1000000)]

For example to instatiate a list of int

intList = [int() for _ in range(1000000)]

For a small list of 10 int:

>>> [int() for _ in range(10)]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Cory Kramer
  • 114,268
  • 16
  • 167
  • 218
1
objectList = [None] * 1000  # Make a list of 1000 None's

ADDED

If you are wanting to preallocate a list to avoid the overhead of reallocation the list as it continues to grow, this construct is appropriate as is will allocate the entire memory for the list and initialize each member of the list using the same object.

If you need the initialize each member to a different object, you have to initialize each list element separately via a loop, list comprehension, etc.

Memory reallocation under Windows tends to be a good bit slower than under Unix based systems, so this approach can be faster than simply initialing the loop as this construction runs at native speed.

wjandrea
  • 28,235
  • 9
  • 60
  • 81
Gary Walker
  • 8,831
  • 3
  • 19
  • 41
0

I would use a generator:

>>> def obj(x):
...    print 'obj called with', x
...    return x
... 
>>> gen=(obj(x) for x in xrange(10))

Notice at this point, obj has not been called. Once you actually convert to a list (or much better still, iterate over it) the objects will be created in turn:

>>> list(gen)
obj called with 0
obj called with 1
obj called with 2
obj called with 3
obj called with 4
obj called with 5
obj called with 6
obj called with 7
obj called with 8
obj called with 9
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
dawg
  • 98,345
  • 23
  • 131
  • 206