Questions tagged [singularity-container]

For questions about programmatic problems relating to the Singularity scientific computing container solution.

Singularity is a container solution created by necessity for scientific and application driven workloads. Singularity provides four primary functions (taken from the preceding link):

  1. Mobility of compute: the ability to define, create and maintain a workflow and be confident that the workflow can be executed on different hosts, operating systems (as long as it is Linux) and service providers.
  2. Reproducibility: once a contained workflow has been defined, the container image can be snapshotted, archived, and locked down such that it can be used later and you can be confident that the code within the container has not changed.
  3. User freedom: users can define their own working environment and literally copy that environment image (single file) to a shared resource, and run their workflow inside that image.
  4. Support on existing traditional HPC: Singularity supports existing and traditional HPC resources as easily as installing a single package onto the host operating system. Custom configurations may be achieved via a single configuration file, and the defaults are tuned to be generally applicable for shared environments.
365 questions
25
votes
3 answers

how to build singularity container from dockerfile

I want to build singularity container from dockerfile. I have pulled and run docker images from docker hub with singularity. singularity pull docker://ubuntu:latest I have also build the image from singularity recipe file. singularity build …
alex
  • 303
  • 1
  • 4
  • 6
15
votes
3 answers

Containerize a conda environment in a Singularity container

I've come across several instances where it would be really helpful to containerize a conda environment for long-term reproducibility. As I'm normally running in high-performance computing systems, they need to be Singularity containers for security…
LucasBoatwright
  • 1,456
  • 1
  • 16
  • 20
14
votes
1 answer

Binding ports when running Docker images in Singularity

I am currently working on a distributed graph processing platform which maintains an Akka cluster inside of docker containers and have recently been granted access to a large cluster to test this. Unfortunately, this cluster does not run docker,…
ben steer
  • 197
  • 1
  • 10
12
votes
5 answers

Is there a "multi-user" Docker mode, e.g. for scientific clusters?

I want to use Docker for isolating scientific applications for the use in a HPC Unix cluster. Scientific software often has exotic dependencies so isolating them with Docker appears to be a good idea. The programs are to be run as jobs and not as…
Manuel
  • 6,461
  • 7
  • 40
  • 54
7
votes
3 answers

Converting docker tar file into a singularity image?

I am new to both Docker and Singularity. I recently created the canonical main.c. #include #include int main(void){ printf("Hello Docker World!\n"); return 0; } I statically compiled this code, i.e. gcc -static…
irritable_phd_syndrome
  • 4,631
  • 3
  • 32
  • 60
6
votes
1 answer

Singularity equivalent to "docker image list"

Is there a singularity equivalent to docker image list? The Singularity 3.8 documentation states that All cache entries are named using a content hash which makes sense when checking for identical layers/images, but has no semantic meaning…
user163859
  • 190
  • 2
  • 7
6
votes
1 answer

How to activate an Anaconda environment in a Singularity recipe

I am trying to create a singularity image and recipe that will create an anaconda environment and then activate said environment so I can build the python wheel of a project in that environment so it's 100% installed and functional after the…
Tom V.
  • 175
  • 2
  • 7
6
votes
1 answer

singularity extract, edit, and rebuild image

I have a singularity container that has been made for me (to run tensorflow on comet GPU nodes) but I need to modify the keras install for my purposes. I understand that .simg files are not editable (and that the writable .img format is…
saladi
  • 3,103
  • 6
  • 36
  • 61
6
votes
1 answer

Snakemake + docker example, how to use volumes

Lets have a simple snakefile like rule targets: input: "plots/dataset1.pdf", "plots/dataset2.pdf" rule plot: input: "raw/{dataset}.csv" output: "plots/{dataset}.pdf" shell: "somecommand…
mox
  • 447
  • 6
  • 15
6
votes
3 answers

awk command fails in snakemake --use-singularity

I am trying to combine Snakemake with Singularity, and I noticed that a simple awk command no longer works when using singularity. The $1 in the last line gets replaced by bash instead of being used as the first field by awk. Here is a minimal…
sssascha
  • 90
  • 1
  • 4
5
votes
1 answer

Accessing host Python Virtual Environment from a Singularity container

I have a Singularity container with many tools along with Python and Jupyter installed in it. But I want to access the external virtual environments I have created for python while using the other tools in Singularity. The reason being I am…
kosmos
  • 359
  • 5
  • 13
5
votes
0 answers

singularity : how to free loop devices?

On a server, we run continuous integration jobs as runscripts of singularity image, e.g. wget url_to_mysingularityimage.sif chmod +x ./mysingularityimage.sif ./mysingularityimage.sif In case this is relevant: in our specific case, the runscript is…
Vince
  • 3,979
  • 10
  • 41
  • 69
5
votes
1 answer

Why does `singularity run/exec` automatically bind specific some directories? What is the use case?

I'm familiar with containers, but new to Singularity and I found myself fighting a broken Python installation in a Singularity container tonight. It turns out that this was because $HOME was being mounted into my container without my knowledge. I…
Vorticity
  • 4,582
  • 4
  • 32
  • 49
5
votes
1 answer

Use SSH key of host during Singularity/Apptainer build

When building a Singularity/Apptainer image from a definition file, is there a portable way to make a SSH key of the host system available during the build? To give some context: I have a definition file where in the %post section I'm cloning a…
luator
  • 4,769
  • 3
  • 30
  • 51
5
votes
2 answers

File ownership and permissions in Singularity containers

When I run singularity exec foo.simg whoami I get my own username from the host, unlike in Docker where I would get root or the user specified by the container. If I look at /etc/passwd inside this Singularity container, an entry has been added to…
rgov
  • 3,516
  • 1
  • 31
  • 51
1
2 3
24 25