0

I am working on a script that streams data and stores it into an array, however in order to conserve on memory I am hoping to unload some of this data into a CSV file. Here is what that function looks like:

def array_dump(file, array, targetSize, dumpSize):
    if np.size(array) > targetSize:
        with open(file, 'a') as csvFile:
            for val in array[:dumpSize]:
                csvFile.write(str(val))
                csvFile.write('\n')
            csvFile.close()

        array = np.delete(array, np.s_[:dumpSize])

This function does successfully write to the CSV file, however it does not update the array. Here is the full Python script:

import csv
import numpy as np
from datetime import datetime

def array_dump(file, array, targetSize, dumpSize):
    if np.size(array) > targetSize:
        with open(file, 'a') as csvFile:
            for val in array[:dumpSize]:
                csvFile.write(str(val))
                csvFile.write('\n')
            csvFile.close()
        #array = array[dumpSize:]
        array = np.delete(array, np.s_[:dumpSize])
        print(np.size(array))


currencyPair = "EUR_USD"
global fileName
now = datetime.now()
fileName = "%s-%s-%s-%s.csv" % (currencyPair, now.year, now.month, now.day)
file = open(fileName, 'a')
file.close()

global a 
a = np.array([0.69126532,  0.51056279,  0.35202518,  0.13374911,  0.12558003,
        0.06426236,  0.8885468 ,  0.18412806,  0.96355032,  0.79755331,
        0.86043666,  0.51713643,  0.43547321,  0.34684145,  0.88541084,
        0.04065618,  0.25212305,  0.64831108,  0.9623507 ,  0.10511934,
        0.09817052,  0.15486658,  0.56793785,  0.50717163,  0.38677482,
        0.63625237,  0.33494367,  0.71391515,  0.53597436,  0.44326044,
        0.40208125,  0.32963563,  0.78210655,  0.23943754,  0.74833578,
        0.94591554,  0.29412408,  0.71934844,  0.99787723,  0.92264647,
        0.94351218,  0.19185273,  0.14808887,  0.91515938,  0.48011523,
        0.33024851,  0.39953296,  0.18908139,  0.7347978 ,  0.09855765,
        0.06552421])

#print(a)
print(np.size(a))
array_dump(fileName, a, 50, 25)
print(np.size(a))

Is there any way I can use this function to unload, delete and update my array? Thanks.

ng150716
  • 2,195
  • 5
  • 40
  • 61
  • That sample is not very long, but it would make your question contain a [mcve]. It's always encouraged to have all relevant code in the question, I believe you're better off if you include it here. – Andras Deak -- Слава Україні Oct 10 '16 at 19:44
  • 1
    Your question reduces to this one: http://stackoverflow.com/questions/986006/how-do-i-pass-a-variable-by-reference – tsj Oct 10 '16 at 19:50
  • What @tsj means is that you're rebinding `array` to a local variable in the loop. Otherwise it should work fine. – Andras Deak -- Слава Україні Oct 10 '16 at 19:51
  • 1
    Just return `array` from the function. `np.delete` creates a new array, so you can't modify the argument in place. – hpaulj Oct 10 '16 at 20:58
  • It looks like you're trying to use `global` to force `array_dump` to update `a`. You need to make the `global` declaration within the same scope where you're assigning to the variable, i.e. declare `global array` before `array = np.delete(array, np.s_[:dumpSize])` within the body of `array_dump`, and rename `a` to `array` outside of the function. However as @hpaulj says, you're better off just returning the array - global variables are considered bad practice by many, since they tend to make code harder to read and debug. – ali_m Oct 11 '16 at 00:14
  • Also, it's best not to call your variable `array` in order to avoid confusion with `np.array` or the `array` module. – ali_m Oct 11 '16 at 00:17

0 Answers0