We are running a Golang application that internally runs an autoupdate module to recieve over the air updates. At the time of a new update, the autoupdate module downloads the new version from s3 and invokes an autoupdate bash script present in the same working directory as the executable. The steps performed by the autoupdate bash script are explained in terms of process management handled with and without systemd.
Execution without Systemd
- Kills the currently running golang app via PID
- Runs the newly downloaded golang binary.
- The new version exposes an health check API which is invoked by script to makesure this version is healthy.
- If not healthy, rollback is performed. The new version is stopped and older version is started.
In golang, the script execution code is written such that when autoupdate script is invoked, it is disowned from the lifecycle of the golang application. i.e bash script (child process) continues execution even after the parent(golang app) is killed. It works fine when running from the GoLand IDE.
Issue : Execution via Systemd
We needed a cleaner system to handle the start ,stop, restart-on failure functionalities. So we decided to run the app as a systemd service.The steps are the same as above but performed using systemctl
- Kills the currently running golang app : "systemctl stop goapp.service"
- Updates the systemd service with the new executable path : "systemctl daemon-reload"
- Restarts the systemd service : "systemctl restart goapp.service"
- If not healthy, rollback is performed. The new version is stopped, systemd service file is updated with older version and older version is started.
On executing the application via systemd, the script exits as soon as the systemctl service is stopped using command "sudo systemctl stop goapp.service" during step 1 mentioned above. This means that the lifecycle of the script executed by the golang app is clearly coupled to the lifecycle of the systemd service. How can I detach it from the of scope of systemd service?
Systemd service file
[Unit]
Description=Goapp
After=docker.service
Requires=docker.service
Requires=docker-mysql.service
Requires=docker-redis.service
StartLimitInterval=200
StartLimitBurst=5
[Service]
User=root
Environment=AWS_SHARED_CREDENTIALS_FILE=/home/username/.aws/credentials
Restart=on-failure
RestartSec=30
WorkingDirectory=/home/username/go/src/goapp-v2.0.0
ExecStart=/home/username/go/src/goapp-v2.0.0/goexecutable --autoupdate=true
[Install]
WantedBy=multi-user.target
The Golang code section that invokes the autoupdate bashscript
func scriptExecutor(argsliceString []string, remoteVersion *semver.Version, localVersion *semver.Version) {
newVersion := fmt.Sprint(remoteVersion)
previousVersion := fmt.Sprint(localVersion)
argslice := append(argsliceString, newVersion, previousVersion)
scriptParams := append([]string{"./autoupdate-script.sh"}, argslice...)
Info("Autoupdater - New version ", newVersion)
Info("Autoupdater - Existing Version ", previousVersion)
Info("Autoupdater - Executing shell script to upgrade go app.Passing parameters ", scriptParams) // ./autoupdate-script.sh goexecutable 7.1.0 7.0.0
cmd := exec.Command("sudo", scriptParams...)
err := cmd.Start()
if err!=nil{
Info(err)
}
cmd.Process.Release()
}
Posts on "how to run a script in Golang and disown/detach it" have been posted but could not find any posts on this type of an issue. Please help me resolve the issue or correct me if there are any errors in my understanding.
