0

I have a Java script that executes a k-shell which calls a program that writes file A. Thereafter, the same Java script parses output file A. No other process read or writes file A.

The problem is that file A is not always ready to be parsed in time, which I assume is because the file system is still finishing its job even after the program & calling shell return. This only occurs when there are a lot of processes running.

lsof does not seem to work on my system. If I open a file with an editor, lsof shows me the editor process, but if I pass the filename to lsof I get: lsof: no file use located: filename

I don't think a fileLock will work, as I would have to unlock it before either returning from the program or the k-shell, by which time the file still might not have been closed completely.

My thought was to write a short script that would rename (mv) the file, and then have the parser parse the renamed file, my understanding being that a file cannot be renamed until it is completely written. I put the mv command in a while loop that checks to see if the stdOut from the mv command has anything in it, theory being that if it is empty, we're good to go.

#! /bin/ksh
# Move a file. The idea is that this script will not return until the mv is
# complete. If mv is successful, mv.out.txt should be empty.

mv $1/camber.out $1/camber.out.copy > mv.out.txt
while [[ -s mv.out.txt ]] ; 
do
    echo "mv did not happen"
    mv $1/camber.out $1/camber.out.rename > mv.out.txt
done
echo "mv should have taken place"

exit

I have yet to see an instance where the mv was held up. Any comments, suggestions, hints, or insults would be greatly appreciated. Thanks in advance to all.

dazedNconfused
  • 3
  • 2
  • The file should be available after mv returns, whether is did a rename or a cp. Secondly, I think that you are misunderstanding a few things. "mv a b > c" means "move a to b, redirect all output from mv to a file called c". mv does not print anything, normally, so this file will be empty. The -s test is checking whether a file exists, and has a size greater than 0. This means "if the first mv did say something, Try moving the original file to .rename instead, and check the mv output again". – folkol May 27 '15 at 06:09
  • Also, are you trying to do this from JavaScript, or from Java? Please paste the code that you have so far. – folkol May 27 '15 at 06:15
  • @folkol I am using a vendor's package which implements the system call via its own method: public void run(java.lang.String cmd) throws java.lang.Exception Runs a command. This function will not return until the sub-process has finished. In my code, "cmd" would be the commands to run the k-shell that runs the program, and the rename script above. My code would be in JavaScript. Sorry I can't paste it all... – dazedNconfused May 27 '15 at 16:52
  • @folkol I just made a few tweaks to my code above, redirecting to stdErr using "2> mv.out.txt". I did a small test, deliberately changing the from fileName in the first "mv" to something that doesn't exist, and I got something in mv.out.txt, so that part seems to work. – dazedNconfused May 27 '15 at 16:55
  • Are you actually running JavaScript? If you are running Java, see Tomboyo's answer below. – folkol May 27 '15 at 16:57
  • I'm a self-taught old-school code who is being forced to learn IT & CS stuff, so sorry for the delay while I drink from the firehose. It appears my code above is irrelevant as Linux apparently doesn't much care how many processes are accessing a file and when. – dazedNconfused May 29 '15 at 05:26

1 Answers1

2

I lack the ability to comment, so an answer this shall be. Regardless, what it sounds like is a problem with asynchronous execution- the java program calls the bash script, which is handled as a separate program by the OS and thus runs concurrently with the java program. For everything to run correctly, you just need to make sure that the bash script is run synchronously- that is, the script must finish before Java continues. I believe this SO answer related to blocking should do what you need. The solution is this:

ProcessBuilder pb = new ProcessBuilder("myscript.sh");
Process p = pb.start();     // Start the process.
p.waitFor();                // Wait for the process to finish.
System.out.println("Script executed successfully");

That should force the Java program to sleep until the bash script is finished.

The same logic may need to be applied to the k-shell, as you pointed out in your comment below. You can use the wait command, which takes a job as an optional parameter, to wait on a process (if you do not pass in a specific job, the process will wait for all child processes to finish).

Community
  • 1
  • 1
Tomboyo
  • 171
  • 2
  • 12
  • Thanl you!I believe you are correct, EXCEPT that I am concerned that the program is returning control to the KSH before the write is actually complete. So even if the script is finished, the write from the program may not be. Problem is, I don't really know... – dazedNconfused May 27 '15 at 17:23
  • You're right. You should be able to do something similar in the ksh. Updated answer to reflect this, as well. Check out http://www.computerhope.com/unix/uksh.htm for specifics on the wait command and ways to identify a job, if you need it. – Tomboyo May 28 '15 at 01:48
  • The JavaScript has access to Java methods. The wrapper package I'm using has a system call method which does what you described, so this is the answer. Apparently the file system we are using has some asynchronicity issues when it is really getting hammered (as in what we are doing). I've implemented the wait command, and so far we are ok, although as the fault was somewhat infrequent, I don't have an absolute answer. Thanks tomboyo & folkol! You've been great. – dazedNconfused May 29 '15 at 05:35