-1

I have a number (number unknown) of data files in a directory. Each data file has the following content.

FILE TYPE:  1   
COLUMNS: 7  
TITLE: TRACK HISTORY    

COLUMN TYPE VARIABLE (UNITS)    
------ ---- -------- -------    
1 2 ParticleResidenceTime (s)   
2 10 ParticleID - 

3 10 ParticleXPosition (m)  
4 10 ParticleYPosition (m)  
5 10 ParticleZPosition (m)  
6 10 ParticleDiameter (m)

7 10 ParticleDensity (kg/m3)    

---------------------------------------------   

3.00E-01    1.01E+05    -5.32E-02   -1.19E-01 -4.21E-02 1.28E-04    1.50E+03
3.00E-01    1.36E+05    -5.73E-02   -1.30E-01   -2.69E-02   1.50E-04    1.50E+03
3.00E-01    1.53E+05    -5.53E-02   -8.33E-02   -8.47E-03   1.39E-04    1.50E+03

Each data file has around 300k lines like above. I need to consolidate all these files into one file. with only 3 columns in them and 1 header. The 3 columns i need the columns 3, 4, 5 which are the particle x y z position data. The data starts on the 16th line of each file.

so the eventual merged file would look something like the below.

X            Y          Z  ( i guess i could add this header at the end manually too)
-5.32E-02 -1.19E-01 -4.21E-02
-5.12E-02 -1.39E-01 -4.21E-02
-5.32E-02 -1.19E-01 -4.21E-02
-5.32E-02 -1.19E-01 -4.21E-02

An Empty Line after data from file 1 following which data from file 2 will start

-5.32E-02 -1.19E-01 -4.21E-02
-5.12E-02 -1.39E-01 -4.21E-02
-5.32E-02 -1.19E-01 -4.21E-02
-5.32E-02 -1.19E-01 -4.21E-02

An Empty Line after data from file 3 following which data from file 4 will start

-5.32E-02 -1.19E-01 -4.21E-02
-5.12E-02 -1.39E-01 -4.21E-02
-5.32E-02 -1.19E-01 -4.21E-02
-5.32E-02 -1.19E-01 -4.21E-02

and so on till the data from all the files is put in this one file.

The script needs to do the following.

  1. first find how many files there are in that directory so that it can use that as the counter for a for loop (if used)
  2. Open a new file and add the data in columns 3, 4, 5 from line 16 till end from file 1 in the directory.
  3. add an empty line
  4. move to next file(since these data are time dependent data and the data needs to be accumulated in the order of time. the files will be sorted in the directory.)
  5. add columns 3, 4, 5 from the second file from line 16 till end
  6. add an empty line
  7. Repeat until the last file in the directory.

I would appreciate if someone showed me how to do this using Python.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Suresh
  • 65
  • 1
  • 6
  • 4
    It looks like you want us to write some code for you. While many users are willing to produce code for a coder in distress, they usually only help when the poster has already tried to solve the problem on their own. A good way to demonstrate this effort is to include the code you've written so far, example input (if there is any), the expected output, and the output you actually get (console output, stack traces, compiler errors - whatever is applicable). The more detail you provide, the more answers you are likely to receive. – Martijn Pieters Jun 07 '13 at 20:17

1 Answers1

0

I won't write the code for you, but here are some links that should explain what you need to know. Reading and writing files from Python

Count the number files in a directory

Get the file creation times and dates in python

In addition, some string manipulation - the split method should come in handy.

Hope this gets you started!

Community
  • 1
  • 1
user2461391
  • 1,433
  • 3
  • 16
  • 26