2

I have the following code, that goes through the stages of reading a file in line by line, converting it into a list, editing a field in that list, but the last stage is rewriting the original file, with this edit included.

I would be interested in answers that suggest the easiest/simplest fix (without the use of pandas/numpy etc) and using the original code provided.

My current algorithm includes having to create a new file with all the records, except the one linked to this username, and then writing in this list to it. This seems arduous and unnecessary. Any solutions gratefully received!

The task is explained more clearly in the comments:

Code

 """ ==============TASK
    ALLOW THE USER TO CHANGE OR EDIT THEIR PASSWORD
    1. Search for any given username
    2. Edit the password field for that given username
    3. Save the new updated username and password to file / updated
    """

    import csv
    def main():
        #1. This code snippet asks the user for a username and then allows them to change password for that record
        updatedlist=[]
        with open("fakefacebook.txt",newline="") as f:
          reader=csv.reader(f)
          print("CHANGE PASSWORD?!")
          username=input("Enter the username for the required user:")
          for row in reader: #for every row in the file
              for field in row:

                    if field==username: #if a field is == to the required username
                        updatedlist.append(row) #add each row, line by line, into a list called 'udpatedlist'
                        newpassword=input("Enter new password")
                        #print(updatedlist[0][1]) (this is how we get at the password field, noting it is a nested list)
                        updatedlist[0][1] = newpassword #set the field for password to the new password


          updatepassword(updatedlist)

    def updatepassword(updatedlist):
        with open("fakefacebook.txt","w",newline="") as f:
            Writer=csv.writer(f)
            Writer.writerows(updatedlist)
            print("File has been updated")


    main()

Note: at the moment the code just allows the user to change password (and this is changed in the list). The list is comprised of just that user's record. It overwrites this single record (with the changed password) to the text file, instead of what is required (original file contents + just this edit)

File Contents

username,password,email,no_of_likes
marvR,pass123,marv@gmail.com,400
smithC,open123,cart@gmail.com,200
blogsJ,2bg123,blog@gmail.com,99

Required output

If tested with: marvR change password to: boo123

new file should contain:

username,password,email,no_of_likes
marvR,**boo123**,marv@gmail.com,400
smithC,open123,cart@gmail.com,200
blogsJ,2bg123,blog@gmail.com,99

Any comments/explanations about the best way to approach this for teaching beginners would also be appreciated. It seems odd that Python hasn't developed a module of some sort to make editing a field in a file easier than this 3 step algorithm which is really quite arduous for beginners (i'm talking 13-14 year olds) to work out

JClarke
  • 788
  • 1
  • 9
  • 22
Compoot
  • 2,227
  • 6
  • 31
  • 63

3 Answers3

3

One of the related issues was not addressed so far. You have mentioned "teaching beginners", so please consider this answer as a supplement to other answers.

When a file like password file is edited, the original file should not be overwritten like in the code presented here. A data loss is possible if a write fails for any reason including disk full or power outage. Even more important, the file should be always in a consistent state, i.e. nobody should ever see a partially written file.

To achieve that:

  1. read the data from file
  2. modify it in memory process the data as needed
  3. save the result to a new temporary file, close that file (with does it automatically), you might even want to call os.fsync()
  4. if there was any error, remove the temporary file and stop here
  5. only if everything is fine, atomically rename the temporary file to the original file preferrably with os.replace() (Python 3.3+). Atomic operation are done in one single step.

UPDATE: modified updatepassword code with some comments:

FILE = "fakefacebook.txt"

def updatepassword(updatedlist):
    # create the tempfile in the same directory (os.replace requires the same filesystem)
    tempfile = FILE + ".tmp"
    try:
        # "x" = fail with the FileExistsError if the file exists already
        with open(tempfile, "x", newline="") as f:
            writer = csv.writer(f)
            writer.writerows(updatedlist)
            f.flush()               # flush the internal buffers
            os.fsync(f.fileno())    # write to disk
        os.replace(tempfile, FILE)
        print("File has been updated")
        tempfile = None     # tempfile renamed
    except FileExistsError:
        print("Another copy is running or stale tempfile exists")
        tempfile = None     # tempfile does not belong to this process
    except OSError as err:
        print("Error: {}".format(err))
    finally:
        if tempfile:
            try:
                os.unlink(tempfile)
            except OSError:
                print("Could not delete the tempfile")
VPfB
  • 14,927
  • 6
  • 41
  • 75
  • Very helpful - can you explain "atomically". And are you able to post a solution as below, using my code and extending it to demonstrate this solution. That would be most helpful and I could accept answer. The last answer here works, I assume, but I want to take on board your point about the preference to save first to a temporary file. – Compoot Jul 25 '17 at 19:23
  • 1
    Also, it is the "modify in memory" bit that needs expanding. What is the best method to modify? Like I have done? (and then what?) For instance, editing the field in the list, and then writing that individual list to another file, and then searching the original file for every row except the given username, and then creating a new file. (seems way too long, hence the question for a more efficient answer) – Compoot Jul 25 '17 at 19:47
  • @MissComputing "Modify in memory" is probably not the best term. I just mean compute the output from the input in general sense. – VPfB Jul 25 '17 at 19:57
  • @MissComputing "atomic operation" is a well known term in computer science. I'm sure there is plenty of well written explanations, just ask google. Basically it is an operation that can be executed either completely in one step or not at all. – VPfB Jul 25 '17 at 20:07
1

@VPfB's answer should be taken into consideration as well.

This is just the answer to updating the records without replacing your existing records in the file. But there are better ways to do this.

import csv

def main():
    #1. This code snippet asks the user for a username and then allows them to change password for that record
    updated_list = []
    cached_list = []

    with open("fakefacebook.txt", newline="") as f:
        reader = list(csv.reader(f)) # Convert iterable to list to make things easier.
        print("CHANGE PASSWORD?!")
        username=input("Enter the username for the required user: ")
        cached_list = reader # store copy of data.

        for row in reader: #for every row in the file
            for field in row:  
                if field == username: #if a field is == to the required username
                    updated_list.append(row) #add each row, line by line, into a list called 'udpated_list'
                    newpassword = input("Enter new password: ")
                    #print(updatedlist[0][1]) (this is how we get at the password field, noting it is a nested list)
                    updated_list[0][1] = newpassword #set the field for password to the new password

        update_password(updated_list, cached_list)

def update_password(updated_list, cached_list):
    for index, row in enumerate(cached_list):
        for field in row:
            if field == updated_list[0]:
                cached_list[index] = updated_list # Replace old record with updated record.

    with open("fakefacebook.txt","w", newline="") as f:
        Writer=csv.writer(f)
        Writer.writerows(cached_list)
        print("File has been updated")


main()
JClarke
  • 788
  • 1
  • 9
  • 22
  • It appears your code simply overwrites the file with nothing. https://repl.it/JiDj/0 –  Jul 25 '17 at 19:40
  • Compare this with your code and you'll see where you made the error. https://repl.it/JiDj/2 – JClarke Jul 25 '17 at 19:52
  • Thanks - just tried your version. I see the ommisions, but yours still does not remove the required row for the entered username? – Compoot Jul 25 '17 at 19:53
  • As above. Are we missing something? Your code doesn't remove the row (e.g. marvR entered as username, should remove marvR's record, but it doesn't) –  Jul 25 '17 at 19:55
  • 1
    Looks like repl.it makes to repl everytime you save. https://repl.it/JiDj/7 – JClarke Jul 25 '17 at 19:55
  • 2
    Remove? It updates the password. Base on your required output the code works, at least as I've seen. I'm not sure what you're doing different. – JClarke Jul 25 '17 at 19:56
  • 1
    Works! Thank you! ....any comments that you could add to your answer (that I've accepted now any way) would be hugely appreciated. Are there more efficient methods to do the same thing, without using pandas/numpy etc. In any case - thanks! – Compoot Jul 25 '17 at 19:58
0

You can use a dictionary instead of a list data structure here. Dictionaries map values to user-defined keys rather than integers. So instead of saying iterating through a list to find the correct username, you could simply store the entire file in a data structure and use random access to change the desired username:

from collections import defaultdict
def main():
        inputDict=defaultdict(string)
        with open("fakefacebook.txt",newline="") as f:
          reader=csv.reader(f)
          for row in reader: #for every row in the file
              line = row.split(',') #splits the CSV row into an array

              #sets the key to be the username, and everything else 
              #to be value, and then adds it to inputDict
              inputDict[line[0]] = line[1:] 

          print("CHANGE PASSWORD?!")
          username=input("Enter the username for the required user:")
          newpassword=input("Enter new password")
          if (inputDict[username]):
              inputDict[username] = newpassword
          else: print("No such username!")


          updatepassword(inputDict)

You would have to modify your updatepassword() to work with a dictionary.

Aditya Gune
  • 520
  • 4
  • 7
  • Correct me if I'm wrong, but this does not deal with writing to or editing a file. I'm aware Dictionaries could be used to solve this problem, but the issue in question is editing a "file". Thanks so much – Compoot Jul 25 '17 at 19:24
  • Correct, this doesn't improve on writing/editing a file. AFAIK, there is no way to selectively edit one specific record of a file in python without traversing the whole file, simply because the script will have no way of knowing where in your file the target occurs. If you know exactly where your username will appear you can try [mmap](https://docs.python.org/3/library/mmap.html), but again, it depends on you knowing the precise location of the data within the file. Otherwise, you will still need to follow the three steps of 1. read in the file 2. locate and modify data 3. re-write to file – Aditya Gune Jul 26 '17 at 19:26