1

The content of the shell script is like this : 

date
cur_date=`date '+%Y-%m-%d'` 
cur_time=`date '+%Y-%m-%d %H %M %S'`
start_date='2014-01-01'  
end_date=`date '+%Y-%m-%d'`
FiveDaysBack=`date --date='-7 day' "+%Y-%m-%d"`

rm -f *.sql
mysqldump -h192.168.200.150 torr > torr1.sql &
mysqldump -h192.168.200.100 torr > torr2.sql &
...
...
...
wait && echo `date` complete.

I'm running this script as :

nohup sh etl.sh &

Executing the script results in all the sql dump going into the nohup.out file and not to the torr1.sql, torr2.sql et all. So I get a huge nohup.out file where I wanted the dump to go the torr.sql files.

Please note that this script does a lot more than this and I'm not looking for an alternate arrangement for obtaining mysql dumps. All I'd like to know is why is the mysqldump output not going into the specified files and instead being directed to nohup.out. I'd like to resolve just that.

fixxxer
  • 15,568
  • 15
  • 58
  • 76

3 Answers3

0

Re-answer:

I'd use 'screen' instead of 'nohup'. It's a more powerful solution that allows you to revisit your active terminal if it is closed. See here or here for a discussion.

Community
  • 1
  • 1
philshem
  • 24,761
  • 8
  • 61
  • 127
  • the script is over 300 lines in length and it does much more than mysqldump. For clarity about the problem - the nohup.out is getting the mysqldumps and not torr1.sql and torr2.sql .And I'm not able to get to the cause of that. Have updated the question as well. – fixxxer Feb 11 '14 at 10:01
  • You can try 'screen' instead of nohup. This is a little more elegant. – philshem Feb 11 '14 at 10:26
  • I made a major edit to my answer that suggests screen instead of nohup. – philshem Feb 11 '14 at 10:29
  • Will take a look after I solve the pain-in-the-ass of a problem above. Thanks for the suggestion. – fixxxer Feb 11 '14 at 11:22
  • If you are using nohup to keep the session alive, then you can just switch to screen. Type 'screen', hit enter, and then run the command without any prefix. Just make sure to not type 'exit' or the screen session will end. – philshem Feb 11 '14 at 13:59
0

The last line should be with two & characters: wait && echo $(date) complete.

This still does not answer your problem. Maybe mysqldump writes to stderr or your etl.sh did something funny with the stdout.

I reproduced your problem with this script:

sleep 10
rm non_existing > stdout.txt

I started this script (I saved it as rm_nothing.sh) with

 nohup rm_nothing.sh

The output of rm did not appear in stdout.txt.

Please try to add 2>&1

mysqldump -h192.168.200.150 torr > torr1.sql 2>&1 &
Walter A
  • 19,067
  • 2
  • 23
  • 43
  • Even with 2>&1 it is sending the output to nohup.out instead of the sql files. – fixxxer Feb 12 '14 at 02:46
  • 1
    Strange! I guess you did not change the filedescriptors. I also think you have the correct rights for creating the files. You can test this by removing the torr?.sql files and see if they are created after the script. When they are created and empty the script seems to write to another fd. You want a solution, not questions. You can try changing the redirection into using an extra parameter: --result-file=file_name, -r file_name Direct output to a given file. (...) The result file is created and its previous contents overwritten, even if an error occurs while generating the dump. – Walter A Feb 12 '14 at 10:03
  • Weirdly, the torr.sql files are not even being created. And even with the "--result-file" parameter the output is going to the nohup.out file. Like you said, the file descriptors are behaving unnaturally. – fixxxer Feb 13 '14 at 02:30
  • Sounds like you need to ge debugging with echo statements. You said it is a large script, so it is hard to say where the things get mixued up. Of course you looked in the nohup.out for stderr messages, I can't believe you have a problem with file permissions. So please try to insert a lot of debuglines in your script, all trying to write to torr1.sql. echo "Debug: Just started (pwd=${PWD})" >> torr1.sql ... echo "Debug: Before calling weirdfunction (pwd=${PWD})" >> torr1.sql ... – Walter A Feb 13 '14 at 09:09
  • Walter, these are the starting lines of the script. Plus there are no functions in it. I'm hoping for the best. Much thanks. – fixxxer Feb 13 '14 at 13:17
  • 1
    Try narrowing down the problem. Test with mysqldump -h Test with echo in front of the lines with sqldump? Do you have the same problem when you copy the file to strange.sh and delete everything except the lines you posted above? And what if you also delete the wait command? Can mysqldump be started in parallel (try deleting the second) ? Change mysqldump into mytest.sh and make a runnable file mytest.sh with sleep 5; echo Awake. Try without nohup (eth.sh > all_output.txt) Try nohup without the sh (nohup eth.sh &) Use #!/bin/ksh or #!/bin/bash as first line Try set -x – Walter A Feb 13 '14 at 15:43
  • Walter - Thanks for trying so hard to help. I should have posted the entire command. – fixxxer Feb 14 '14 at 06:11
0

Okay, this is going to sound stupid.
1. sorry for not pasting the entire command, Walter and philshem. Perhaps it would have helped.
2. The entire command was :

mysqldump --insert-ignore --skip-add-drop-table --no-create-info --lock-tables=false -ucrbt_se -pcrbt_se -h10.2.2.150 blah master_backup --where "date(END_TIME)>='$FiveDays'" --result-file=torr1.sql

and Nano was wrapping the text around. As a result of the large size of the dump, I had never waited to see the error that MySQL threw in the end, which indicated the wrong parameter to the command.

2 days wasted. Damn.

fixxxer
  • 15,568
  • 15
  • 58
  • 76