0

My project has some code that requires AOT compilation.

Below is a shortened version of my deps.edn. The only way I have found to compile some code before each test run is to add a -e (compile,'my.project.namespace) pair to the :main-opts. This requires knowledge of the exact namespaces that require compilation, which changes periodically during development. Even simple experimentation in a unit test may require a temporary change to the deps.edn which is very annoying.

{:paths   ["src" "classes"]
 :deps    {}
 :aliases {:test           {:extra-paths ["test"]}
           :test/cognitect {:extra-deps {com.cognitect/test-runner {:git/url "..." :sha "..."}}
                            :main-opts  ["-e" "(compile,'my.project.namespace1)"
                                         "-e" "(compile,'my.project.namespace2)"
                                         "-e" "(compile,'my.project.namespace3)"
                                         "-m" "cognitect.test-runner"]
                            :exec-fn    cognitect.test-runner.api/test}}

This would be invoked with clj -M:test:test/cognitect.

Is there an easy way to compile the entire project (including tests) when -M (or -X) is invoked with a certain alias?

I know that -e can handle any Clojure expression so one option would be to write an entire mini program that will enumerate all the namespaces and call compile on them. I would be okay with this as long as the code is simple. Is this a good idea?

In general, how would one setup a deps.edn to AOT compile the entire project?

Erp12
  • 600
  • 3
  • 16

2 Answers2

1

Take a look at tools.build: https://clojure.org/guides/tools_build

This is the core team's answer to this problem.

Sean Corfield
  • 6,297
  • 22
  • 31
  • I did some learning about `tools.build` and I agree that it is the solution. Unfortunately I'm not sure the ecosystem is not mature enough for me to recommend it yet. For example, my IDE (Cursive) doesn't recognize the `build.clj` file as being part of the project because it isn't under a "source root". Another minor point: I can't find a way to implicitly invoke `tools.build` before all invocations of the test runner so (depending on the code change) it may not be possible to run the tests in the usual ways (from inside the IDE or using one of the conventional commands) – Erp12 Jul 16 '21 at 17:52
1

Sean Corfield's answer seems to be the best direction. Given how new tools.build is I thought it might be helpful to expand a bit for others to see how this would be done and highlight an issue that I encountered.

(def class-dir "target/classes")

(def basis (b/create-basis {:project "deps.edn"
                            ;; Include any aliases that will bring deps required for 
                            ;; compilation onto the classpath.
                            :aliases [:spark-deps :test]}))

(defn compile-for-tests
  []
  (b/compile-clj {:basis basis
                  :src-dirs ["src" "test"]
                  :class-dir class-dir
                  ;; Filters which classes get written into `class-dir` by their namespace prefix.
                  :filter-nses ['my.project]})`)

I beleive that the above is typically all that you need, however my project requires 2 phases of compilation. One to compile some :gen-class namespaces and a second to compile the rest of the codebase (some of which imports those :gen-class classes as a Java class).

Also, those :gen-class namespaces seem to need to be compiled one at a time (even though they don't reference each other) in order to not get ClassNotFoundException. I can't explain why this is, but I will update this answer if I ever figure it out. Below is the exact function I am currently using.

(defn ns-under-prefix
  [prefix paths]
  (let [all-ns (mapcat #(-> % b/resolve-path jio/file find/find-namespaces-in-dir) paths)]
    (filter #(str/starts-with? (name %) (name prefix)) all-ns)))


(defn compile-for-tests
  [_]
  ;; Compiling one at a time is required... for some reason.
  (doseq [nmspc (ns-under-prefix 'erp12.clark.core.aot ["src"])]
    (println "Compiling" nmspc)
    (b/compile-clj {:basis basis
                    :src-dirs ["src"]
                    :class-dir class-dir
                    :ns-compile [nmspc]}))
  (println "Compiled AOT namespaces.")
  ;; Compile everything else. Relies on the AOT classes already existing.
  (b/compile-clj {:basis basis
                  :src-dirs ["src" "test"]
                  :class-dir class-dir
                  :filter-nses ['erp12.clark.core]}))
Erp12
  • 600
  • 3
  • 16