2

Why running a DQN, the memory of my program increase at every model.fit() call. using memory_profiler on the train() function in my DQN I get this:

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
   124    315.7 MiB    315.7 MiB           1       @profile
   125                                             def train(self):
   126                                         
   127    315.7 MiB      0.0 MiB           1           if len(self.replaybuffer) < MIN_REPLAY_BUFFER:
   128                                                     return
   129                                         
   130                                                 #get idexes from replay buffer
   131    315.7 MiB      0.0 MiB           1           rpbIdexes = np.random.choice(range(len(self.replaybuffer)), size=BATCHSIZE, replace=False)

   132                                         
   133                                                 #predict in bulk for speed up
   134    315.7 MiB      0.0 MiB          35           nextObservations = np.array([self.replaybuffer[rpbId][4] for rpbId in rpbIdexes])

   135    315.7 MiB      0.0 MiB           1           if self.useTarget:
   136    315.7 MiB      0.0 MiB           1               nextPredictions = self.targetmodel(nextObservations.reshape(-1, 4))

   137                                                 else:
   138                                                     nextPredictions = self.model(nextObservations.reshape(-1, 4))

   139                                         
   140                                                 # x and y arrays for fitting
   141    315.7 MiB      0.0 MiB           1           x, y = [], []
   142    315.7 MiB      0.0 MiB          33           for idx, rpbId in enumerate(rpbIdexes):
   143    315.7 MiB      0.0 MiB          32               observation = self.replaybuffer[rpbId][0]
   144    315.7 MiB      0.0 MiB          32               x.append(observation)
   145                                         
   146    315.7 MiB      0.0 MiB          32               nextPrediction = np.max(nextPredictions[idx])

   147                                         
   148    315.7 MiB      0.0 MiB          32               done = self.replaybuffer[rpbId][5]
   149    315.7 MiB      0.0 MiB          32               reward = self.replaybuffer[rpbId][3]
   150    315.7 MiB      0.0 MiB          32               if done:
   151    315.7 MiB      0.0 MiB           2                   target = reward
   152                                                     else:
   153    315.7 MiB      0.0 MiB          30                   target = reward + self.gamma * nextPrediction

   154                                                     # change the prediciton
   155    315.7 MiB      0.0 MiB          32               takenAction = self.replaybuffer[rpbId][2]
   156    315.7 MiB      0.0 MiB          32               prediction = self.replaybuffer[rpbId][1]
   157                                         
   158    315.7 MiB      0.0 MiB          32               prediction = prediction.numpy()
   159    315.7 MiB      0.0 MiB          32               prediction[takenAction] = target
   160    315.7 MiB      0.0 MiB          32               y.append(prediction)
   161    315.7 MiB      0.0 MiB           1           x = tf.convert_to_tensor(x)
   162    315.7 MiB      0.0 MiB           1           y= tf.convert_to_tensor(y)
   163    316.1 MiB      0.4 MiB           1           history = self.model.fit(x=x, y=y, batch_size=BATCHSIZE, verbose=0, shuffle=False)

   164    316.1 MiB      0.0 MiB           1           return history

Every time model.fit() gets called a small amount of MiB is "leaked". When running for a long time, the memory gets full.

Im using tensforflow 2.12.0, python 3.10, and its running on my CPU.

I have tried to convert the x and y input to tensors but this did not help. Also I've tried to clear the garbage collector and clear the keras session (keras.backend.clear_Session()).

MrHurricane
  • 21
  • 2
  • What are you doing with the `history` variable that you return? The results shown there just imply that `history` takes 400k. It's not indicative of a problem. – Tim Roberts Apr 08 '23 at 23:16
  • I extract the loss form history to make a plot. – MrHurricane Apr 08 '23 at 23:18
  • every time Train() gets called the MiB number is different. When I do `print(sys.getsizeof(history))` it is always 48 bytes. so it does not show `history` takes 400k. – MrHurricane Apr 08 '23 at 23:31
  • Just before the end of the function, use `del` on as many variable as possible. If it is not enough you can add `gc.collect` after the `del` – Caridorc Apr 09 '23 at 00:07
  • 1
    getsizeof does not represent the actual size of an object, especially a custom object (note that history is actually a Tensorflow History object). In addition, getsizeof is not recursive, meaning nested objects' sizes are not taken into account. Finally, TensorFlow is a very complex framework, it may choose to internally record certain information that are not exposed to user in order to make things more convenient or performant. – seermer Apr 09 '23 at 02:42
  • If you're passing parts of `history` into a plot, then that data has to stick around until the plot is dismissed. @seermer's point is exactly right. The Python `history` object is just a handle. Python can't see the C memory underneath. Same thing happens with numpy arrrays. – Tim Roberts Apr 09 '23 at 03:27

0 Answers0