Does creating mid-step temp variables matter? How is dynamic memory allocation done in python for string?

Question

Sample name of file uploaded to watchdog folder: 300_processtest_20201008_092912_rx.csv

I get file path from watchdog handler as follow:

        elif event.event_type == 'created':
        temp = {}
        print ("Received created event - %s." % event.src_path)
        file = event.src_path
        req_id, d_type = get_requestID(file)
        temp['requestID'] = "req_id"
        temp['data_type'] = d_type

        with open(file) as csv_file:
            csv_reader = csv.DictReader(csv_file, delimiter='|')
            temp['data'] = list(csv_reader)
        r = requests.post(url = UPDATE_ENDPOINT, json = json.loads(json.dumps(temp))) 



def get_requestID(file_path):

     split_string = file_path.rpartition("/")
     file_name = split_string[2]
     remove_extension = file_name.rpartition(".")
     split_name = split_extension[0].rsplit("_",2)
     data_type = split_name[2]
     request_id = split_name[1]
     print(request_id, data_type)

     # Does doing above thing in one line, saves memory usage?
     #split_name = file_path.rpartition("/")[2].rpartition(".")[0].rpartition("_",2)
     #data_type = split_name[2]
     #request_id = split_name[1]
     return request_id, data_type

I am wondering which would be a better way to write the code. I am not sure how memory allocation works in python specially for string (string interning) mentioned here. If that is true

Method 1 using temp variables takes much more memory but is easily readable.

Method 2 doing it in one line is a bit difficult to read and understand.

Initially I thought both method would require same memory ie. creating temp variables doesn't require additional memory. Does it? Makes me wonder I never really paid attention to how python works.

Instead of splitting on path separators yourself, see https://docs.python.org/3/library/os.path.html — kaya3, Apr 28 '21 at 01:36
file path is something I am getting from watchdog event. Need the path for logging. This function simply parses req_id from file name to insert the data in mongodb — badass 1996, Apr 28 '21 at 13:53
Can you add one example of the file that you're passing in the function? — Nk03, Apr 28 '21 at 14:29

score 0 · Answer 1 · answered Apr 28 '21 at 14:32

Take the advantage of pathlib module -

from pathlib import Path

def get_requestID(file_path):
    file_path =  Path(file_path)
    file_name = file_path.name
    split_extension = file_path.suffix
    request_id1 = split_extension[0].rpartition("_")
    request_id2 = file_path.rpartition("/")[2].rpartition(".")[0].rpartition("_")
    return request_id2[2]

Does creating mid-step temp variables matter? How is dynamic memory allocation done in python for string?

1 Answers1