AWS Batch enables you to run batch computing workloads on the AWS Cloud. Batch computing is a common way for developers, scientists, and engineers to access large amounts of compute resources. AWS Batch removes the undifferentiated heavy lifting of configuring and managing the required infrastructure.
Questions tagged [aws-batch]
420 questions
74
votes
5 answers
ECS unable to assume role
From the console, I am invoking a lambda which submits a batch job. The batch job fails, indicating that ECS is unable to assume the role that is provided to execute the job definition.
For the role, I've added the lambda and ECS services.
The error…
Nate Reed
- 6,761
- 12
- 53
- 67
35
votes
9 answers
Why are AWS Batch Jobs stuck in RUNNABLE?
I use a computing environment of 0-256 m3.medium on demand instances. My Job definition requires 1 CPU and 3 GB of Ram, which m3.medium has.
What are possible reasons why AWS Batch Jobs are stuck in state RUNNABLE?
AWS says:
A job that resides in…
arm
- 605
- 1
- 7
- 16
32
votes
4 answers
configured logging driver does not support reading : Docker
I am running my docker container in AWS ECS. When i try to execute the below command to read the logs from container, i am facing the below error.
command: docker logs -f "Container ID"
Error response from daemon: configured logging driver does not…
palani.p
- 521
- 1
- 7
- 13
18
votes
3 answers
How to define root volume size in AWS batch
I'm using AWS Batch and I found the root volume size too low for my task.
I tried creating a new computing environment/job queue but there's not any option to set volume size. I tried changing launch configuration from here but the new launch…
gc5
- 9,468
- 24
- 90
- 151
17
votes
2 answers
Docker image different size when pushed to ECR than locally
I have a docker image that is 1.46GB on my local machine, but when this is pushed to AWS ECR (either via my local machine or via CicleCI deployment) it is only 537.05MB. I'm pretty new to Docker and to AWS, so any help in figuring out as to why this…
Robert Farrimond
- 308
- 2
- 9
12
votes
4 answers
What is the best way to run python scripts in AWS?
I have three python scripts, 1.py, 2.py, and 3.py, each having 3 runtime arguments to be passed.
All three python programs are independent of each other. All 3 may run in a sequential manner in a batch or it may happen any two may run depending upon…
Parijat Bose
- 380
- 1
- 6
- 22
12
votes
2 answers
How pass named arguments with parameters to aws batch?
Consider piece of AWS batch job definition:
MyJobDefinition:
Type: "AWS::Batch::JobDefinition"
Properties:
Type: container
Parameters: {}
JobDefinitionName: "my-job-name"
…
Cherry
- 31,309
- 66
- 224
- 364
10
votes
1 answer
How to clear aws batch job history in dashbord
In aws batch Job queues dashboard, it shows all job status failed and succeeded job count for 24 hours. Is it possible to reset counter to zero?
WoodChopper
- 4,265
- 6
- 31
- 55
10
votes
1 answer
Aws Batch not logging to Cloudwatch
I have a docker image through which I make an aws batch job definition. Within the docker I am printing some statements.
The job runs and there's a success message. But the logs aren't there in Cloudwatch. It says that no events found. Not even the…
Piyush Dugar
- 101
- 1
- 5
9
votes
1 answer
How to get Subnet list from VPC with terraform
I've tried to get all subnet ids to add aws batch with terraform with following code:
data "aws_subnet_ids" "test_subnet_ids" {
vpc_id = "default"
}
data "aws_subnet" "test_subnet" {
count = "${length(data.aws_subnet_ids.test_subnet_ids.ids)}"
…
PPShein
- 13,309
- 42
- 142
- 227
9
votes
1 answer
Use of Snakemake workflows in AWS Batch
I wanted to ask the Snakemake community if anybody has had success implementing Snakemake workflows in AWS Batch.
Page 4 of a recent publication from Oct 2018 seems to suggest that Snakemake does not work on AWS as it cannot handle resource…
user10101904
- 427
- 2
- 12
9
votes
1 answer
How to override the docker image in an AWS Batch job?
In AWS Batch, I am trying to override the image hardcoded in the job definition at the time of submitting a job. But I dont see any documentation around that.
When I try to add an image field during job submission, I get either:
Parameter…
GuSuku
- 1,371
- 1
- 14
- 30
9
votes
3 answers
Get latest job revision while submitting AWS batch job without specifying the exact revision number
I am using AWSBatch Java client com.amazonaws.services.batch (AWS SDK for Java - 1.11.483) to submit jobs programmatically.
However, our scientists keep updating the job definition.
Every time there is a new job definition, I have to update the…
IamAshay
- 1,377
- 1
- 7
- 16
9
votes
1 answer
AWS batch - how to limit number of concurrent jobs
I am looking for a way to limit the number of batch jobs that are running by holding the remaining jobs in the queue. Is it possible with aws batch?
manji369
- 186
- 4
- 16
9
votes
1 answer
How to submit aws batch job periodically
We are looking for aws batch job and we want to submit this batch job on a certain predefined interval. For e.g.: we want to auto submit this job every 4 hours.
What could be a preferred way to achieve it.
Rahul Garg
- 4,069
- 1
- 34
- 31