60

What is the best way to write the contents of a StringIO buffer to a file ?

I currently do something like:

buf = StringIO()
fd = open('file.xml', 'w')
# populate buf
fd.write(buf.getvalue ())

But then buf.getvalue() would make a copy of the contents?

phoenix
  • 7,988
  • 6
  • 39
  • 45
gauteh
  • 16,435
  • 4
  • 30
  • 34
  • 6
    `StringIO` objects are always stored in main memory. If you don't want that, don't use `StringIO` and write directly to the file. – Philipp Jul 15 '10 at 07:18
  • 1
    @Philipp: Yes, but using `buf.getvalue()` in this way might (?) create a copy of the contents. – gauteh Jun 30 '15 at 09:21
  • @Philipp maybe programmer wants to do some operations on StringIO object before writing to file? – alercelik Sep 04 '20 at 11:49

2 Answers2

97

Use shutil.copyfileobj:

with open('file.xml', 'w') as fd:
  buf.seek(0)
  shutil.copyfileobj(buf, fd)

or shutil.copyfileobj(buf, fd, -1) to copy from a file object without using chunks of limited size (used to avoid uncontrolled memory consumption).

phoenix
  • 7,988
  • 6
  • 39
  • 45
Steven
  • 28,002
  • 5
  • 61
  • 51
13

Python 3:

from io import StringIO
...
with open('file.xml', mode='w') as f:
    print(buf.getvalue(), file=f)

Python 2.x:

from StringIO import StringIO
...
with open('file.xml', mode='w') as f:
    f.write(buf.getvalue())
Demitri
  • 13,134
  • 4
  • 40
  • 41
  • 3
    This makes a copy of `buf.getvalue()`. – gauteh Oct 29 '19 at 08:13
  • 1
    @gauteh Thanks; good point. I think this is useful for small-ish data to avoid the need for an additional import, though it strikes me as odd that `shutil.copyfileobj` is the best solution. – Demitri Oct 31 '19 at 14:09
  • 2
    mode='w' is ok for text files , like the file.xml, but if the content is not text, one should write to binary with mode='wb' – Ricardo Rivaldo Feb 23 '21 at 01:33