0

Jetty 9 is installed on EC2 linux server, jetty.sh was copied to /etc/init.d and proper links made at /etc/rc1.d, /etc/rc2.d.

I'm connecting to the instance using SSH, and running jetty through sudo service jetty start which loads the service correctly, even after logging out of the SSH session.

But when running a remote SSH command ssh -t -i key.pem ec2-user@instance.domain.com 'sudo service jetty restart' on my instance, it starts but then stops right after. Here's the log:

2013-11-14 18:03:01 main DispatcherServlet [INFO] FrameworkServlet 'restapi': initialization completed in 1376 ms
2013-11-14 18:03:01.824:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@67fc3482{/,file:/usr/share/jetty/jetty-distribution-9.0.6.v20130930/webapps/ROOT/,AVAILABLE}{/ROOT}
2013-11-14 18:03:01.941:INFO:oejs.ServerConnector:main: Started ServerConnector@10614f3d{HTTP/1.1}{0.0.0.0:8080}2013-11-14 18:03:02.226:INFO:oejs.ServerConnector:main: Started ServerConnector@666c1838{SSL-http/1.1}{0.0.0.0:8443}2013-11-14 18:03:05.670:INFO:oejs.ServerConnector:Thread-3: Stopped ServerConnector@10614f3d{HTTP/1.1}{0.0.0.0:8080}
2013-11-14 18:03:05.671:INFO:oejs.ServerConnector:Thread-3: Stopped ServerConnector@666c1838{SSL-http/1.1}{0.0.0.0:8443}

I've tried executing remotely sudo nohup service jetty restart &, same result.

How can I restart remotely (SSH) and keep it running?

Kof
  • 23,893
  • 9
  • 56
  • 81
  • I think launching the shell allow sudo to get the user info for look it is in the sudoers file. Maybe you are already using the root account already when you log in with ssh, so using `sudo` may be useless. The other possibility it to install [expect](http://expect.nist.gov/) and look for examples [like this one](http://bash.cyberciti.biz/security/expect-ssh-login-script/ "SSH login expect shell script to supply username and password"). If those things don't work, try to look [here](http://stackoverflow.com/a/3872762/2284570) (also please notify @ me, I don't check for answers manually). – user2284570 Nov 11 '13 at 23:07
  • Try running a full shell rather than just using `sudo` on its own: `ssh -t -i key.pem ec2-user@instance.example.com "sudo su - -c 'service jetty start'"`? – pgl Nov 12 '13 at 11:38
  • @pgl : "start a shell" that's the base of my comment below... – user2284570 Nov 12 '13 at 18:10
  • Does logging in and interactively running `sudo service jetty restart` work? – ishaaq Nov 13 '13 at 23:23
  • @Kof Again: When you use ssh, are you already using the root account? Do you use password or public key for ssh login? – user2284570 Nov 14 '13 at 00:04
  • @ishaaq: yes, it works correctly and doesn't stop when I log out. – Kof Nov 14 '13 at 05:44
  • @user2284570: no, I log in as ec2-user. – Kof Nov 14 '13 at 05:44
  • what's wrong with `ssh -i key.pem ec2-user@instance.domain.com 'sudo service jetty restart'` ? Note that I have removed `-t` from the ssh command.(presuming that current state of jetty is `running`, else the `restart` will fail eventually.) – slayedbylucifer Nov 14 '13 at 07:39
  • @slayedbylucifer: it fails with `sudo: sorry, you must have a tty to run sudo` – Kof Nov 14 '13 at 08:02
  • yours seem to be redhat based distro. Not recommended but try this: comment out "Defaults requiretty" by running 'visudo' on target server and then run the ssh command without `-t`. It will display password in clear text. This is not recommended, but could try this for testing purpose so we will know whether it's `ssh` issue or `sudo` issue – slayedbylucifer Nov 14 '13 at 08:56
  • @slayedbylucifer: it fails loading, in the log it says: `2013-11-14 12:41:15.581:WARN:oejuc.AbstractLifeCycle:FAILED SslSelectChannelConnector@0.0.0.0:8443: java.net.BindException: Address already in use`, I guess that it requires more su permissions – Kof Nov 14 '13 at 12:48
  • @Kof have you tried `nohup`? – user2284570 Nov 16 '13 at 14:59

6 Answers6

2

I replicated your issue on CentOS (RPM based) and it works like a charm. I installed jetty and configured it on port 7070.

Stopping Jetty:

# ssh -t -i key.pem ec2-user@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com 'sudo service jetty stop'
Stopping Jetty: OK
Connection to ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com closed.

Nothing is listening on port 7070:

# ssh -t -i key.pem ec2-user@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com 'sudo netstat -anp | grep 7070'
Connection to ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com closed.

Starting jetty:

# ssh -t -i key.pem ec2-user@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com 'sudo service jetty start'
Starting Jetty: . OK Thu Nov 14 08:57:19 EST 2013
Connection to ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com closed.

port 7070 is in use now:

# ssh -t -i key.pem ec2-user@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com 'sudo netstat -anp | grep 7070'
tcp        0      0 :::7070                     :::*                        LISTEN      2431/java           
Connection to ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com closed.

Here are the corresponding logs:

2013-11-14 08:57:15.610:INFO:oejsh.ContextHandler:main: Started o.e.j.s.h.MovedContextHandler@1c9c8aa{/oldContextPath,null,AVAILABLE}
2013-11-14 08:57:16.283:INFO:oejs.ServerConnector:main: Started ServerConnector@176982e{HTTP/1.1}{0.0.0.0:7070}
2013-11-14 08:57:16.392:INFO:oejs.ServerConnector:main: Started ServerConnector@166320b{SSL-http/1.1}{0.0.0.0:8443}

And here is the platform I used:

# ssh -t -i key.pem ec2-user@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com 'sudo lsb_release -a'
LSB Version:    :core-4.0-ia32:core-4.0-noarch:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-ia32:printing-4.0-noarch
Distributor ID: CentOS
Description:    CentOS release 6.3 (Final)
Release:    6.3
Codename:   Final
Connection to ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com closed.

And finally, here is hte restart command:

# ssh -t -i key.pem ec2-user@ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com 'sudo service jetty restart'
Stopping Jetty: OK
Starting Jetty: . OK Thu Nov 14 09:13:00 EST 2013
Connection to ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com closed.

I ran all above commands via sudo to make sure that sudo is not the issue.

  • Were there any changes on your instance?
  • Were you able to start jetty via SSH n sudo in the past successfully? if yes, then what has changed since then?
  • If this is new installation, then spin-up a new instance and check whether you still face the same issue.

BTW, I am not a java guy and followed these instruction to install jetty 9.

slayedbylucifer
  • 22,878
  • 16
  • 94
  • 123
  • 1
    Thank you for the detailed reply, but what you offer is what I tried in the first place. It probably worked on your instance because it's Centos, mine is Linux (see org post). Like I said, it looks OK when running the SSH command, but port isn't open after SHH connection is closed and logs shows Jetty was stopped (see org post for log). – Kof Nov 14 '13 at 18:07
  • @Kof : Linux on Amazon ec2 use red hat and I've seen this:[`With Red Hat Cloud Access, customers have the ability to use the advantages provided by public cloud providers, while continuing to benefit from the high level of support delivered directly by Red Hat.`](http://www.redhat.com/promo/amazon/) I don't understand why you didn't got help form their support... – user2284570 Nov 15 '13 at 17:40
1

I think that the init script teminate before Jetty is fully launched so it still need the shell during some seconds after the script termination.
Just try:

ssh -t -i key.pem ec2-user@instance.domain.com 'sh -c "sudo service jetty restart;sleep 8"'

Again, you may try to redirect input from your script

ssh -t -i key.pem ec2-user@instance.domain.com <<'FIN'
#commands to run on remote host
/bin/bash --login -i -s << FAIM
su -l -s /bin/bash << FEIN
your_password
nohup service jetty restart &
bg
disown -a
sleep 20
FEIN
sleep 5
FAIM
FIN

An another alternative is to ask to the red hat support, since you're paying for it with your Amazon cloud service.

Community
  • 1
  • 1
user2284570
  • 2,891
  • 3
  • 26
  • 74
  • This produced the same result, only with 8 seconds between the 'Started' message to 'Stopped' message. Jetty stops once SSH session is closed. – Kof Nov 16 '13 at 12:18
  • @Kof And for the second one? Since it consist of redirecting /dev/stdin (no sh -c), it think it should work as when you log in manually. – user2284570 Nov 16 '13 at 13:41
  • Result: `Pseudo-terminal will not be allocated because stdin is not a terminal.` `sudo: sorry, you must have a tty to run sudo` – Kof Nov 16 '13 at 13:53
  • 1
    @Kof comment out `Default requiretty` in the sudoers file – user2284570 Nov 16 '13 at 14:25
  • @user2284580: Thank you - that last clue did help. Removing `requiretty` allowed running SSH command without `-t` parameter, which caused `ssh -i key.pem ec2-user@instance.domain.com 'sudo service jetty restart' >/dev/null` to work. Notice that without `>/dev/null`, I would not get command prompt back. Thanks to Artur R. Czechowski for this suggestion. Please update your answer with the right solution (removing `requiretty`, removing `-t` and redirecting output to null) and I'll accept your answer. – Kof Nov 18 '13 at 07:50
1

Try to redirect I/O using following command:

ssh -t -i key.pem ec2-user@instance.domain.com 'sudo service jetty restart </dev/null >/dev/null 2>/dev/null'

If it helps, you can try with only >/dev/null.

ArturFH
  • 1,697
  • 15
  • 28
1

Working Solution:

  1. On EC2 instance, edit the sudoers file with what you want.
  2. Then, restart Jetty remotely with this command: ssh -i key.pem ec2-user@instance.domain.com 'sudo service jetty restart', notice that -t was removed.
  3. If command prompt doesn't come back after restart finishes, add a >/dev/null at the end.
user2284570
  • 2,891
  • 3
  • 26
  • 74
0

try this , use -f to force ssh into background :

 ssh -f -t -i key.pem ec2-user@instance.domain.com 'sudo service jetty restart'
michael501
  • 1,452
  • 9
  • 18
  • It doesn't do the restart, instead it writes: `Pseudo-terminal will not be allocated because stdin is not a terminal. sudo: sorry, you must have a tty to run sudo` - there's a conflict between -f and -t, which forces pseudo-tty allocation. – Kof Nov 13 '13 at 07:01
  • create a bash script on instance.domain.com and instead of sudo … run the script – michael501 Nov 13 '13 at 18:13
  • Impossible, because ec2-user doesn't have the required permissions. Jetty failed loading when I tried due to file access errors, etc. – Kof Nov 13 '13 at 21:40
  • @Kof michael mean create the script manually while logged in on the server... – user2284570 Nov 15 '13 at 17:59
  • @user2284570: Understood that, that's what I did - Jetty fails loading because it doesn't have permissions for some resources. – Kof Nov 16 '13 at 12:15
0

Does this work, from my comment above:

ssh -t -i key.pem ec2-user@instance.example.com "sudo su - -c 'service jetty start'"

?

pgl
  • 7,551
  • 2
  • 23
  • 31
  • Tried that just now, same issue - it stops as soon as the SSH connection is closed. – Kof Nov 13 '13 at 08:44