Stack Overflow
Stack Overflow

Questions tagged [tracemalloc]

A standard library module for Python with utilities for tracing memory blocks. Also add the python tag for increased visibility.

tracemalloc is a module for the Python language and is a debug tool to trace memory blocks allocated by Python. It is available since Python 3.4.

https://docs.python.org/3/library/tracemalloc.html

25 questions
6
votes
1 answer

Does Python not reuse memory here? What does tracemalloc's output mean?

I create a list of a million int objects, then replace each with its negated value. tracemalloc reports 28 MB extra memory (28 bytes per new int object). Why? Does Python not reuse the memory of the garbage-collected int objects for the new ones? Or…
Kelly Bundy
  • 23,480
  • 7
  • 29
  • 65
4
votes
1 answer

How big is the overhead of tracemalloc?

I would like to log in a production system the current memory usage of a Python script. AWS has Container Insights, but they are extremely well-hidden and I'm not sure how to use them properly within other dashboards / logging- and altering systems.…
Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
3
votes
1 answer

Measuring the allocated memory with tracemalloc

The Python module tracemalloc offers a detailed insight into the allocated memory of the program. For example, one use case is to record the current and peak memory usage: import tracemalloc tracemalloc.start() # ...code... current, peak =…
Hermi
  • 350
  • 2
  • 5
  • 16
2
votes
0 answers

Understanding Python's memory allocation

I'm trying to track down a suspected memory leak in a Python application which uses numpy and pandas as two of the main libraries. I can see that the application uses more and more memory over time (as DataFrames are processed). Memory consumption…
orange
  • 7,755
  • 14
  • 75
  • 139
2
votes
1 answer

Python tracemalloc, what is size and count?

I am using the tracemalloc library to pinpoint memory concerns in my application. This is the code that I am using. tracemalloc.start() snapshot = tracemalloc.take_snapshot() top_stats = snapshot.statistics('lineno') …
GeekGeek4
  • 149
  • 9
2
votes
0 answers

tracemalloc doesn't reflect memory usage of flask server?

I have a memory leak in my flask server (it's a server that serve a tensorflow and a pytorch machine learning model). It has a dread memory leak. So I use tracemalloc to track the memory. The problem is tracemalloc doesn't detect any memory leak.…
user2459179
  • 197
  • 2
  • 2
  • 9
1
vote
1 answer

python3 -- RAM usage (htop and tracemalloc give different values)

I have a code where I need to save RAM usage so I've been tracing RAM usage through tracemalloc.get_traced_memory. However, I have found that what tracemalloc.get_traced_memory gives is very different from the RAM usage I see through htop. In…
SWMin
  • 11
  • 1
1
vote
0 answers

Can I use tracemalloc to line-by-line profile a subprocess?

I am attempting to write a generic,python script that memory profiles a target script, but I can't figure out how to get line-by-line memory stats from the script being run by subprocess. # mem_profiler.py import subprocess import tracemalloc …
leftoverBits
  • 11
  • 2
1
vote
0 answers

Understanding tracemalloc output for small lists

I'm running the following simple script to get some familiarity with tracemalloc: import tracemalloc import linecache def display_custom(snapshot): ... # skipped for shorter code tracemalloc.start() a = [3.] * 512 b = [3.] * 513 c = [3.] *…
mpr
  • 33
  • 5
1
vote
0 answers

what kind of memory is allocated by tracemalloc module

what kind of memory is allocated by tracemalloc module during program execution? is RAM memory? According to the documentation: they are allocated memory blocks. But is it RAM memory?
vincere
  • 11
  • 1
1
vote
0 answers

memory leak that persists after colab cell executes

I'm encountering a subtle memory leak, and unable to determine the source using tracemalloc. I run the following code in google colab, which is meant to optimize hyperparameters for a custom ppo agent. Also, the speed at which the leak happens…
watch-this
  • 1
  • 4
  • 20
1
vote
1 answer

Python tracemalloc's "compare_to" function delivers always "StatisticDiff" objects with len(traceback)=1

Using Python's 3.5 tracemalloc module as follows tracemalloc.start(25) # (I also tried PYTHONTRACEMALLOC=25) snapshot_start = tracemalloc.take_snapshot() ... # my code is running snapshot_stop = tracemalloc.take_snapshot() diff =…
Michael Sanders
  • 29
  • 5
1
vote
1 answer

ResourceWarning for a file that is unclosed but UnitTest is throwing it

I have a function like this: def init_cars(self, directory=''): #some_code cars = set([line.rstrip('\n') for line in open(directory + "../myfolder/all_cars.txt")]) #some_more_code I am writing unittest and when I run them, I get the…
Saffik
  • 911
  • 4
  • 19
  • 45
1
vote
0 answers

Memory Leak in python undetected by tracemalloc

I have a python(3.6.5) package that creates multiple objects. The main process seems to have a memory leak based on what I see in task scheduler as well as a bail out criteria that I have put in place if memory every exceeds 1GB. Code for detecting…
NightFury
  • 19
  • 3
1
2