0

I wrote a python script that takes care of creating a file atomically as described in this other question. The file that gets created by the script is somewhat big (about ~1.5MB) as seen below:

$ ./my_script.py --output_file some_directory/my_file.txt
$ ls -l --block-size=K some_directory/my_file.txt
-rw------- 1 foouser foogroup 1477K Aug  7 17:39 some_directory/my_file.txt

Unfortunately, due to some legacy infrastructure and the way things are set up, this script needs to be called from within a TCL script. Here is the way in which I am currently doing that:

#!/usr/bin/env tclsh
set PY_SCRIPT my_script.py
puts [exec $PY_SCRIPT --output some_directory/my_file.txt]

The problem that I am seeing is that the generated file does not seem to be created atomically anymore (atomicity is a requirement for my application). The TCL puts command documentation says the following:

Tcl buffers output internally, so characters written with puts may not appear immediately on the output file or device; Tcl will normally delay output until the buffer is full or the channel is closed.

I believe this explains why my file is not being atomically created anymore. I'm not an expert in TCL, so I am not sure how to accomplish atomic file creation in the TCL script.

Any suggestions?

AndresM
  • 1,293
  • 10
  • 19
  • Why is there any need for `puts`? You're still using my_script.py to produce all the output you need, right? Shouldn't you just exec that script in tcl? – Scott Mermelstein Aug 08 '17 at 17:14
  • You are aware that files that size are not normally written atomically? The usual size of a true atomic write is around 16 kB, though the OS might provide a view that simulates atomicity. Everything is more noticeable on Windows, due to the impact of anti-virus software (which often spoils the atomicity properties of writes). – Donal Fellows Aug 08 '17 at 17:20
  • @DonalFellows Based on their linked question, OP seems to be misusing the concept of atomic writes to simply mean "either all or none of the file is written". – Scott Mermelstein Aug 08 '17 at 17:52

2 Answers2

1

I think what you want is just flush stdout.

Undo
  • 25,519
  • 37
  • 106
  • 129
Larry Smith
  • 41
  • 1
0

A direct translation of the Python code that you link to would be something like this (untested code):

package require Tclx

set f [open $tmpFile w]
puts -nonewline $f $text
sync $f    ; # flush is performed implicitly
close $f

file rename -force $tmpFile $myFile

The TclX package is needed to get the sync command. (This will probably not work on Windows.)

Peter Lewerin
  • 13,140
  • 1
  • 24
  • 27
  • Wouldn't a `flush $f` be sufficient here (no TclX dependency), to clear the buffer; `rename` will serve for the `all-writes-or-none` requirement? – mrcalvin Feb 27 '19 at 20:10
  • @mrcalvin: not sure anymore: I've mostly left Tcl behind now, and I can't figure this out just by looking at the code. Sorry. – Peter Lewerin Mar 23 '19 at 10:33