Write data to a file in Python

Question

How can I save data to a file (that I will plot later on) effectively?

I have got this from my research:

#Open new data file
f = open("data2.txt", "w")
f.write( str(yEst)  )      # str() converts to string
f.close()

Here is a bit of my code so far:

for i in drange2(0, 2*math.pi + 0.0634665182543392 , 0.0634665182543392):
    for x in range(1,N+1):
       yEst = yEst + a * cos(x* i)
#print yEst
f.write( str(yEst)  )      # str() converts to string
yEst=0
f.close()

Now, when I go and open my file "data2.txt" I can not read the data because it is not 'organized'. How can I go to the next line using the f.write( str(yEst) ) so that I have a column that contains my 'yEst' data into the file "data2.txt? Thank you in advance for your consideration :)

PS: yEst looks like (in the data2.txt file): 48.901347147148.605785828748.114506165947.429486 .. and I want it as a column: -->

48.9013471471 (new line)
48.6057858287 (new line)
48.1145061659 (new line)
47.4294863684    
etc ..

I have a hard time understand exactly what your problem is. Could you rephrase? — Lanaru, Aug 28 '12 at 18:07
What do you mean, "you cannot go to the line"? And what do you mean by re-using the array `yEst`? Are you trying to save some data in a file you will read later? — Pierre GM, Aug 28 '12 at 18:08
In general, try to use tutorials and reference material to answer these sorts of basic questions first. — Marcin, Aug 28 '12 at 18:09
Can you show us (a bit of) what `yEst` looks like? In general I'd say that it is a good idea to save data as plain text, unless it is a huge amount (gigabytes). — Roland Smith, Aug 28 '12 at 18:12
@RolandSmith Why? It's usually better to use an ORM unless there's too much data or a compelling reason that makes that unviable. — Marcin, Aug 28 '12 at 18:34
@Marcin Using plain text allows you to inspect data files without needing a program to do it. It also enables you to create test cases easily; you only need a text editor. As Doug McIlroy said as part of the UNIX philosophy "Write programs to handle text streams, because that is a universal interface." — Roland Smith, Aug 28 '12 at 18:40
@RolandSmith No, it's not a universal interface, unless the program is merely doing textual transformations. — Marcin, Aug 28 '12 at 18:49

Hannele · Accepted Answer · 2012-08-28T19:03:09.083

13

Do you mean to say you'd like each point of data on its own line, like this?

48.9013471471
48.6057858287
48.1145061659
47.4294863684

If so, then you might like to try something like this:

for i in drange2(0, 2*math.pi + 0.0634665182543392 , 0.0634665182543392):
    for x in range(1,N+1):
       yEst = yEst + a * cos(x* i) 
       f.write( str(yEst) + "\n"  )
f.close()

Firstly, to write each data point to the line, it needs to be 'inside' the loop - that's why I've added extra space to the line before I call f.write.

The second thing I added is + "\n" - this will add the new line character to the end of the line. You can change it to whatever you'd like! A couple of examples:

f.write( str(yEst) + " " ) will add one space after each data point:

48.9013471471 48.6057858287 48.1145061659 47.4294863684

f.write( str(yEst) + "|" ) will add one pipe character after each data point:

48.9013471471|48.6057858287|48.1145061659|47.4294863684|

If, on the other hand, you'd rather just save the data as an array, try the below:

yEst = 0 # or some initial value
yEstArray = []
for i in drange2(0, 2*math.pi + 0.0634665182543392 , 0.0634665182543392):
    for x in range(1,N+1):
       yEst = yEst + a * cos(x* i) 
       yEstArray.append(yEst)

Then, you can iterate over the array like so:

for yEst in yEstArray:
    do something with yEst

or

for yEst, index in enumerate(yEstArray):
    do something with yEst and its index

edited Aug 28 '12 at 19:03

answered Aug 28 '12 at 18:44

Hannele

9,301
6
48
68

1

@Faycal F: the thing you want to remember is that Python does **not** automatically add a new line character (`\n`) when you write a line in a file. You have to do it yourself. – Pierre GM Aug 28 '12 at 18:47
1

You can also use os.linesep to get the system specific line separator. – brian buck Aug 28 '12 at 18:49
@brianbuck Handy! Although, for the beginner's sake, you'd also need to add `import os` before that statement. – Hannele Aug 28 '12 at 18:51
@Pierre GM: Yes, same goes for C, I would have used arrays here instead if I knew how but that is all I can do so far and it works. It slows down the process but I guess it should be all right for a small program :) – Faycal F Aug 28 '12 at 18:54
@FaycalF Are you saying that rather than putting the data in an array, you're writing it to a file and then reading it back out? – Hannele Aug 28 '12 at 18:56
@FaycalF I've updated with my answer with some stub code on how to use arrays instead, I hope you find it helpful (if not now, perhaps in the future!) – Hannele Aug 28 '12 at 19:00
Yes .. I have read previously that I need the "array" library and for this task I can only use these "viz NumPy, SciPy and MatPlotLib". – Faycal F Aug 28 '12 at 19:02
@FaycalF How odd that someone would say so - it's very much built-in behaviour! The above code doesn't require anything to be imported. – Hannele Aug 28 '12 at 19:04

score 6 · Answer 2 · answered Aug 28 '12 at 18:06

6

You'll want pickle for data serialization.

answered Aug 28 '12 at 18:06

Dhaivat Pandya

6,499
4
29
43

4

Wait, what? Looks like a bit too advanced for the problem at hand, and I'm not sure it's really relevant... – Pierre GM Aug 28 '12 at 18:10
If his data format changes, he'll be better off than if he just dumps stuff in CSV right now. – Dhaivat Pandya Aug 28 '12 at 18:42
It's still better to learn how to walk before starting to run. I agree that a pickle could be useful, but that's not really the topic you want to address after only 3h of Python, hence my -1. – Pierre GM Aug 28 '12 at 18:49
@PierreGM So, you're just going to vote down every other answer on this question? – Marcin Aug 28 '12 at 18:51
@Marcin Nope, I just upvoted Hannele's answer as I judged it was the most appropriate for the OP's problem. Nothing personal. – Pierre GM Aug 28 '12 at 18:58
On second thought, I agree. It is important to learn how to do stuff by hand. – Dhaivat Pandya Aug 28 '12 at 19:37

score 6 · Answer 3 · answered Aug 28 '12 at 18:13

6

Python has three main built-in ways to persist data:

pickle which serialises objects to files;
sqlite - an embedded SQL database, which is supported by many ORM systems (which with that addition is an excellent alternative to pickle for object storage); and
json/yaml - serialisation intended for the web, with basic datastructures and types.

answered Aug 28 '12 at 18:13

Marcin

48,559
18
128
201

@PierreGM (a) The question has changed since I posted this (b) just because the OP thinks they want to save data to a text file, it does not mean that it is the best solution to their problem. – Marcin Aug 28 '12 at 18:50
In the long run, probably not. For the problem at hand, yes. – Pierre GM Aug 28 '12 at 18:56
The OP wants to learn some basics of I/O, for example, writing a file. Once the bases are understood, then you can start suggesting more advanced alternatives and their advantages. But only then. – Pierre GM Aug 28 '12 at 19:08
@PierreGM Really? Unless you can find that in the question, you're just making up additional requirements. OP wants to persist data - it seems that the details arise because they don't know of any other way to do it. – Marcin Aug 28 '12 at 19:09
We should move to chat, but it's failing on my network right now. Anyhow, call that a hunch if you like, but I don't think the OP was looking for a long term solution. You don't know how the data was to be plotted after, how the OP wanted to check the validity of the results. While your suggestion is the best in the long run, I don't think it was appropriate at that point. – Pierre GM Aug 28 '12 at 19:26
1

+1 for the JSON option. It's both plain text, so easy to understand, and structured, so easy to use, and a long term learning investment--JSON is a good format to know. As a novice, I found more details on how to use it [here](http://stackoverflow.com/a/12309296/674976). – texnic Jun 19 '16 at 13:35

Write data to a file in Python

3 Answers3