I have two incoming streaming, both of them are big. can I flush the first data, let say from the file's position 1 to 512 and the other from 513 to 1023? Note that that both streams are coming parallelly.
thanks!
Asked
Active
Viewed 3,127 times
1
-
If concurrency is so vital then why aren't you using mmap? – Ignacio Vazquez-Abrams Mar 01 '12 at 03:24
-
Look up the Global Interpreter Lock. Python doesn't really have concurrency... – Endophage Mar 01 '12 at 03:39
-
@Endophage: It does as soon as you hit C code, which file I/O is. – Ignacio Vazquez-Abrams Mar 01 '12 at 03:41
-
@IgnacioVazquez-Abrams afaik it kind of does when it hits C code. The file I/O itself is will run concurrently, but if the data is being read into some Python object, that will cause the GIL to kick in. – Endophage Mar 01 '12 at 05:43
2 Answers
1
Not sure exactly what you mean by parallel, but you can write to two different file objects that have the same file open. I assume it doesn't matter if you are using threads or an event loop.
>>> f1 = open("/tmp/foo", "a")
>>> f2 = open("/tmp/foo", "a")
>>> f1.write("a\n")
>>> f2.write("b\n")
>>> f1.close()
>>> f2.close()
>>> print open("/tmp/foo").read()
a
b
Ben
- 2,422
- 2
- 16
- 23
0
Sure, you can do it. Are you guaranteed correct results? No. Use locks or mmap as suggested by Ignacio.
Here's a good discussion too: Python multiple threads accessing same file
