I have a docker image that is built on one of two machines in CI (call them "A" and "B") which uses the python package trimap, which uses annoy under the hood.
The test stage in CI is then run using the built container on one of those machines separately (so build on "A" -> run on "B", or any combination thereof are possible).
The problem arises when running docker build on machine "A" and then running the test in the built container on machine "B", which causes this Illegal Instruction issue (also relevant). Running docker build as well as the tests on machine "B" works, and going from build on "B" and running tests on "A" is also OK.
I'm building annoy from source when I build the docker image (it's one of the recommended fixes), my current hypothesis is that it's compiling against a certain CPU instruction that is missing on the other host. Could this be true? I would have assumed that docker abstracts this away in order to maintain interoperability.
