How do I copy specific strings from a text file with python?

Question

I am working in the field of astronomy, and the process that I use to unzip the images that I get from the telescopes can be very tedious. The format that the images come in is 'fits.fz' which stands for fits.fits-zipped. I want to decompress these into just '.fits'. I have already I'm working on a program that simplifies this process of decompressing. I have created a graphical interface with two buttons through Python and Tkinter. The first button creates a text file named 'list.txt' and then executes a pre-existing .bat file which dumps the names of every file in a specific directory that ends with 'fits.fz' into 'list.txt'. The first button is also supposed to copy the specific names of the files into a very specific place in another bat file. The other .bat file is called 'Decompress.bat' and is supposed to use the following command for each file in 'list.txt':

 C:\ds9\ds9.exe 
 C:\directory\FITS FILE HERE 
-savefits

I would like for the python program to be able to copy specific sections from a line of code and paste them where 'FITS FILE HERE' is.

The following is the function that is executed when the first button is pressed.

 f = open('C:/jah/list.txt')
 f1 = open('C:/jah/decompress.bat', 'a')

 def begin_wombocombo(): #Is function for first button
     open('C:/jah/list.txt', 'w').close() #Clears 'list.txt'
     open('C:/jah/decompress.bat', 'w').close() #Clears 'decompress.bat'
     subprocess.call([r'C:/jah/newbat.bat']) #Dumps directory into 'list.txt'

     doIHaveToCopyTheLine=False #Bool for whether or not the program has to copy line
     for line in f.readlines(): #loops through all instances to find fz files and then pastes them into decompress.bat
       if 'fits.fz' in line:
         doIHaveToCopyTheLine=True
       if doIHaveToCopyTheLine:
         f1.write(line)
     f1.close()
     f.close()

The issue with this is that it only copies the lines of text that has the fits.fz files. This means that it copies everything else on the line such as when the file was created. Is there any way to simply copy and paste the fits.fz file alone? How would I go about working these strings into the .bat file?

Thank you for your time, and btw the second button just executes 'decompress.bat' which is the file with the commands to unzip the images.

Maybe you just need to use `os.listdir()` instead of `subprocess.call([r'C:/jah/newbat.bat'])` (don't forget to `import os` in first line) — CrafterKolyan, Jun 19 '19 at 23:28
@CrafterKolyan If I use os.listdir(), how would I go about pasting it into the decompress.bat file? Edit: formatting — Zach042, Jun 19 '19 at 23:30
Seems like you could simplify this process by directly calling ds9.exe from python, rather than writing out bat files - or is the process more complex than this? — Blorgbeard, Jun 19 '19 at 23:32
@Blorgbeard I am fairly new to python so I am not sure how to go about doing this. Could you point me in the right direction? — Zach042, Jun 19 '19 at 23:34
@CrafterKolyan Okay thank you for your help. It will make things easier to just use Python. Do you have any pointers as how I should go about adding the specific strings to the bat file? — Zach042, Jun 19 '19 at 23:35
Is the process basically: execute `ds9.exe {path/to/filename.fits.fz} -savefits` for every file matching `*.fits.fz`? — Blorgbeard, Jun 19 '19 at 23:36
OK - that can be done in a one-liner batch file, or a few more lines (but more readable lines) of python. Examples: [batch file](https://stackoverflow.com/questions/14237548/batch-script-run-command-on-each-file-in-directory), [python](https://stackoverflow.com/questions/1120707/using-python-to-execute-a-command-on-every-file-in-a-folder/1120736) — Blorgbeard, Jun 19 '19 at 23:41
@Blorgbeard I am sorry, I really am new to Python and this is my first time creating a program like this. Could you help me to understand how I would go about doing this with a few more lines of Python code? Thank you so much. — Zach042, Jun 19 '19 at 23:48
@Blorgbeard I'm trying to implement it right now. I don't see anything happening though. Do I need to replace "filename" or "os.path.abspath"? Edit: Is there anything else I need to add? I'm very sorry and thank you so much for being so helpful! — Zach042, Jun 20 '19 at 00:02

Blorgbeard · Accepted Answer · 2019-06-20T00:20:05.730

0

I think in Python, something like this would do the trick, without writing out batch files etc.

import os
import subprocess

target_directory = 'C:\\directory\\' # change this as required
zipped_files = [x for x in os.listdir(target_directory) 
                   if x.lower().endswith('.fits.fz')]
for filename in zipped_files:
    subprocess.call([r'C:\ds9\ds9.exe', os.path.abspath(filename), '-savefits'])

edited Jun 20 '19 at 00:20

answered Jun 19 '19 at 23:48

Blorgbeard

101,031
48
228
272

I'm trying to implement it right now. I don't see anything happening though. Do I need to replace "filename" or "os.path.abspath"? – Zach042 Jun 19 '19 at 23:59
That's odd. I'm currently on linux, and I obviously don't have `ds9.exe`, but from my testing it should work. Do you see no output at all? – Blorgbeard Jun 20 '19 at 00:12
Oh, it might be the backslashes in the string, try adding the `r` that I just edited in. – Blorgbeard Jun 20 '19 at 00:14
Ah, and this is working on the current directory - you might have to pass the directory with the files in it to `listdir()`. – Blorgbeard Jun 20 '19 at 00:17
Yes that was it. I also just needed to edit the file path. "C:/SAOImageDS9/ds9.exe" – Zach042 Jun 20 '19 at 00:20
I am having an issue where once the program runs, DS9 is telling me that it cannot load the image. I don't know if it's looking in the right directory – Zach042 Jun 20 '19 at 00:20
You could add a `print(os.path.abspath(filename))` before the call to make sure it's passing the right path. – Blorgbeard Jun 20 '19 at 00:22
For some reason it keeps searching in the cwd. I added all of your changes and it won't work for some reason. – Zach042 Jun 20 '19 at 00:33
I also added in `print(os.path.abspath(filename))` and it just keeps going to the cwd. if you want you can email me at zteagarden@gmail.com – Zach042 Jun 20 '19 at 00:38
The program opens DS9 and everything, but is being really stubborn about looking for files in the cwd. – Zach042 Jun 20 '19 at 00:50
Hmm, is it DS9 that's complaining? Can you see the print output? Does it look right? If you're running this from your Tkinter program, maybe try just executing it as a standalone script on the command line. – Blorgbeard Jun 20 '19 at 01:00
Whoooo. It actually worked. I had this idea that seemed crazy, but I noticed that it was seeing the files, but was attempting to open them in the cwd. I simply moved the file to the directory where the fz files were and boom it worked. DS9 is opening and closing and it appears to be saving each image. The only issue is that I can't tell where the images are being saved. – Zach042 Jun 20 '19 at 01:27
This is the command I would be using for a batch file. `C:\ds9\ds9.exe C:\Directory\ogg0m404-kb82-20190616-0148-e91.fits.fz -savefits C:\Directory\ogg0m404-kb82-20190616-0148-e91.fits -exit` The difference here is that the batch file tells DS9 to save it as .fits whereas the Python code does not. Is there anything I can do to add this to the Python program? – Zach042 Jun 20 '19 at 01:36
I just searched my entire PC for the files, but they have not been saving for some reason. I think I might need your help. Thank you so much for donating so much of your time to my project. I really appreciate it. – Zach042 Jun 20 '19 at 02:00
Ok I fixed it. It took a long time and it's low key weird, but I'll give you the code so that you can write the answer because you deserve to have it. – Zach042 Jun 20 '19 at 04:52
`target_directory = 'C:/Directorty/' # change this as required target_write_directory = 'C:/Directory/' zipped_files = [x for x in os.listdir(target_directory)if x.lower().endswith('.fits.fz')] for filename in zipped_files: print(os.path.abspath(filename)) subprocess.call([r'C:/SAOImageDS9/ds9.exe', os.path.abspath(filename), '-savefits', os.path.abspath(target_write_directory + filename[:-3]), '-exit'])` – Zach042 Jun 20 '19 at 04:54
Hey great to see you figured it out! It seems like the changes you had to make were because of how ds9.exe works - I'll leave the answer as it is since it matches the original question more closely, I think. Interesting that you had to move the file to the working folder though, not sure why that would be. – Blorgbeard Jun 20 '19 at 15:27
BTW, you can mark this answer as "accepted" (and the question as "answered") by clicking the check-mark beside the answer :) – Blorgbeard Jun 20 '19 at 15:28

How do I copy specific strings from a text file with python?

1 Answers1