I'm running the following simple script to get some familiarity with tracemalloc:
import tracemalloc
import linecache
def display_custom(snapshot):
... # skipped for shorter code
tracemalloc.start()
a = [3.] * 512
b = [3.] * 513
c = [3.] * 514
d = [3.] * 515
e = [3.] * 516
f = [3.] * 517
g = [3.] * 518
h = [3.] * 519
i = [3.] * 520
j = [3.] * 521
k = [3.] * 522
l = [3.] * 523
m = [3.] * 524
n = [3.] * 525
o = [3.] * 526
snapshot = tracemalloc.take_snapshot()
display_custom(snapshot)
display_custom is just a slight modification of display_top from https://docs.python.org/3/library/tracemalloc.html. I get the following output:
Top 15 lines
#1 (Line 72): size=5264 B, count=2
h = [3.] * 519
#2 (Line 79): size=4208 B, count=1
o = [3.] * 526
#3 (Line 78): size=4200 B, count=1
n = [3.] * 525
#4 (Line 77): size=4192 B, count=1
m = [3.] * 524
#5 (Line 76): size=4184 B, count=1
l = [3.] * 523
#6 (Line 75): size=4176 B, count=1
k = [3.] * 522
#7 (Line 74): size=4168 B, count=1
j = [3.] * 521
#8 (Line 73): size=4160 B, count=1
i = [3.] * 520
#9 (Line 71): size=4144 B, count=1
g = [3.] * 518
#10 (Line 70): size=4136 B, count=1
f = [3.] * 517
#11 (Line 69): size=4128 B, count=1
e = [3.] * 516
#12 (Line 68): size=4120 B, count=1
d = [3.] * 515
#13 (Line 67): size=4112 B, count=1
c = [3.] * 514
#14 (Line 66): size=4104 B, count=1
b = [3.] * 513
#15 (Line 65): size=4096 B, count=1
a = [3.] * 512
Total allocated size: 63392 B
What exactly happens with line 72? Why are two blocks used when other, larger lists only use one block? Why is the memory 5264 bytes? (for all entries with one block memory is just 8*number of elements)
Below is a shortened version of my actual code for reproducing. The only difference to my code above is that
- I don't import an (unused) package,
- I removed two unused function definitions (different version of the pretty-printing function) and
- I removed some lines that were commented out.
Nothing else has changed so I wouldn't expect the memory allocations to change. The result similar to what I got before (but not identical).
import tracemalloc
import linecache
def display_custom(snapshot, key_type='lineno', limit=10):
snapshot = snapshot.filter_traces((
tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
tracemalloc.Filter(False, "<unknown>"),
))
top_stats = snapshot.statistics(key_type)
print("Top %s lines" % limit)
for index, stat in enumerate(top_stats[:limit], 1):
frame = stat.traceback[0]
print("#%s (Line %s): size=%d B, count=%d"
% (index, frame.lineno, stat.size, stat.count))
line = linecache.getline(frame.filename, frame.lineno).strip()
if line:
print(' %s' % line)
other = top_stats[limit:]
if other:
size = sum(stat.size for stat in other)
print("%s other: %d B" % (len(other), size))
total = sum(stat.size for stat in top_stats)
print("Total allocated size: %d B" % (total))
tracemalloc.start()
a = [3.] * 512
b = [3.] * 513
c = [3.] * 514
d = [3.] * 515
e = [3.] * 516
f = [3.] * 517
g = [3.] * 518
h = [3.] * 519
i = [3.] * 520
j = [3.] * 521
k = [3.] * 522
l = [3.] * 523
m = [3.] * 524
n = [3.] * 525
o = [3.] * 526
snapshot = tracemalloc.take_snapshot()
display_custom(snapshot, limit=15)
The output is:
Top 15 lines
#1 (Line 40): size=5280 B, count=2
j = [3.] * 521
#2 (Line 45): size=4264 B, count=2
o = [3.] * 526
#3 (Line 44): size=4256 B, count=2
n = [3.] * 525
#4 (Line 43): size=4192 B, count=1
m = [3.] * 524
#5 (Line 42): size=4184 B, count=1
l = [3.] * 523
#6 (Line 41): size=4176 B, count=1
k = [3.] * 522
#7 (Line 39): size=4160 B, count=1
i = [3.] * 520
#8 (Line 38): size=4152 B, count=1
h = [3.] * 519
#9 (Line 37): size=4144 B, count=1
g = [3.] * 518
#10 (Line 36): size=4136 B, count=1
f = [3.] * 517
#11 (Line 35): size=4128 B, count=1
e = [3.] * 516
#12 (Line 34): size=4120 B, count=1
d = [3.] * 515
#13 (Line 33): size=4112 B, count=1
c = [3.] * 514
#14 (Line 32): size=4104 B, count=1
b = [3.] * 513
#15 (Line 31): size=4096 B, count=1
a = [3.] * 512
Total allocated size: 63504 B
