1

I can start my web app with a make run just fine. When I start it with Systemd I can see the initialization steps succeed, I see the lisp * prompt but instantly after I get this error:

* ;
; compilation unit aborted
; caught 1 fatal ERROR condition

What's going on, what am I missing?

My .service file:

[Unit]
Description=my app

[Service]
WorkingDirectory=/home/vince/projets/myapp/
ExecStart=/usr/bin/make run
Type=simple
Restart=on-failure

My run script does the following:

;; (eval-when (:compile-toplevel :load-toplevel :execute))

(load "myapp.asd")

(unless (ql:quickload "myapp")
  (uiop:quit 1))

(handler-case
    (progn
      (uiop:format! t "-------- start app…") ;; so far so good
      (myapp:startapp :port (ignore-errors (parse-integer (uiop:getenv "PORT")))))

  (error (c)
    (format *error-output* "~&An error occured: ~a~&" c)
    ;; edit:
    (trivial-backtrace:print-backtrace c)))

This starts the Hunchentoot web server.

and I start SBCL with the following switches (not that in makes a difference, with or without the switches it works fine):

LISP ?= /usr/bin/sbcl  --core /home/vince/projets/ciel/ciel --disable-debugger --userinit /home/vince/.sbclrc

status:

$ sudo systemctl status myapp.service     *[master] 
● cmdcollectivites.service - myapp. 
   Loaded: loaded (/etc/systemd/system/myapp.service; static; vendor preset: enabled)
   Active: inactive (dead)

Jun 09 11:22:48 pommier make[31598]: Loading config file...
Jun 09 11:22:48 pommier make[31598]: Starting the web server on port 9999
Jun 09 11:22:48 pommier make[31598]: ✔ Ready. You can access the application!
Jun 09 11:22:48 pommier make[31598]: * ;
Jun 09 11:22:48 pommier make[31598]: ; compilation unit aborted
Jun 09 11:22:48 pommier make[31598]: ;   caught 1 fatal ERROR condition
lines 1-14/14 (END)

There's something that Systemd expects but that CL is not giving. What about the Lisp prompt?

SBCL 1.4.5-debian (some issues with the 2.0…)

Using handler-bind doesn't change the output.

(handler-bind ((error (lambda (c)
                        (format *error-output* "~&An error occured: ~a~&" c)
                        (format *error-output* "~&Backtrace: ~&")
                        (trivial-backtrace:print-backtrace c))))

  (progn
      (uiop:format! t "-------- start app… --------------")
      (cmdcollectivites:startapp :port (ignore-errors (parse-integer (uiop:getenv "PORT"))))))
Ehvince
  • 17,274
  • 7
  • 58
  • 79
  • It might be a warning during loading? I would strongly suggest compiling your app into an executable to make your life easier in the long run. – fstamour Jun 08 '22 at 18:09
  • Why is the `eval-when` empty? Also, wrap a `handler-bind` around everything with an error handler that prints a backtrace so you can see where it's blowing up. – ignis volens Jun 09 '22 at 12:25
  • @ignisvolens indeed, I remove the `eval-when`. I actually have an `handler-case` around my app startup function that prints on error output (it was on my script but edited out of the snippet, it's back), and I don't see its output ("An error occurred: ~a" c). I'll try adding a backtrace. – Ehvince Jun 09 '22 at 14:53
  • @ignisvolens I added a call to `trivial-backtrace:print-backtrace`, but I don't see any. It's like if systemd killed my app. – Ehvince Jun 09 '22 at 15:00
  • 1
    @Ehvince the fact you get a 'compilation unit aborted' message, from sbcl, means sbcl is dying: systemd is not killing it. You can't get a backtrace from `handler-case` as the stack is already unwound: you need to use `handler-bind` so your handler runs *before* the stack is unwound and while there is a backtrace to print. – ignis volens Jun 09 '22 at 18:03
  • @ignisvolens good point. Doesn't change anything. If my setup seems legit, it's time for me to look for a reproducible example… – Ehvince Jun 09 '22 at 20:23

1 Answers1

2

I got tricked by the * prompt. I knew what's going on since I wrote about it on StackOverflow: Deploying Common Lisp Web Applications and on the Cookbook: https://lispcookbook.github.io/cl-cookbook/scripting.html#for-web-apps

But in this case I saw this log:

* ;
; compilation unit aborted
; caught 1 fatal ERROR condition

I thought the REPL was waiting for us, and that there was actually an error. Wrong.

The issue was that the Hunchentoot server was started in the background (as always), and nothing was telling the Lisp image to stay with us. It exited right away.

The solution is to put the server thread in the foreground, here using bordeaux-threads:

(bt:join-thread (find-if (lambda (th)
                               (search "hunchentoot" (bt:thread-name th)))
                             (bt:all-threads)))

But still… it isn't clear to me why we get "caught 1 fatal error" with Systemd, why the process stops when it works as expected when run from the terminal, where we have the REPL waiting on the foreground.

Things to keep in mind when starting your app:

  • you probably rely on Quicklisp. Then on your .sbclrc. Then on QL's snippet which uses (user-homedir-pathname). But you use Systemd from root, and you probably didn't install QL in your root home directory. So, adapt this line, or use this sbcl switch:

    sbcl --userinit /path/to/your/.sbclrc

  • the --disable-debugger switch is good for production (but handy to keep for debugging…)

  • don't forget you can build a binary, so you won't face these two issues.

Ehvince
  • 17,274
  • 7
  • 58
  • 79
  • Also related: https://stackoverflow.com/questions/30422451/sbcl-deploying-hunchentoot-application-as-executable – Ehvince Jun 09 '22 at 21:52