Questions tagged [azure-batch]

The azure-batch tag should be used to ask questions regarding all aspects of the Azure Batch service including job scheduling, compute node provisioning, and pool orchestration. Extended ecosystem questions are also welcome that are related to Azure Batch accelerator offerings, Azure Portal Batch blades, and Batch Explorer.

Azure Batch is a job scheduling and compute management orchestration solution for the Microsoft Azure cloud. The service is available in all Azure regions and can deploy with nearly all Azure VM sizes available, including those with specialized hardware such as GPUs and Infiniband/RDMA interfaces.

Azure Batch is a managed job scheduling solution where the underlying compute resources are allocated as pools. Pools may be comprised of Linux or Windows VMs and any number of dedicated and low priority nodes and can be automatically scaled through autoscale formulas. An Azure Batch Job can be thought of as a queue for tasks. Each Azure Batch Job can be assigned to a pool. Any number of tasks can be assigned to jobs, including multi-instance tasks (i.e., multi-node) which support MPI jobs. Azure Batch also supports containerized execution.

For a full introductory treatment of Azure Batch, see the Azure Batch Technical Overview. The service team also provides a GitHub repository for issue tracking, feature requests, and change logs.

416 questions
22
votes
1 answer

Run Selenium parallel test on Azure batch

I am using latest version of R on windows 7. I would like to run many test in parallel using RSelenium so, my question is: What is the recommended way to run many RSelenium tests? Let's say I would like to run 1000 tests and each step takes 1…
Mislav
  • 1,533
  • 16
  • 37
13
votes
1 answer

Azure functions are slow

I’m testing the CPU performance of functions, so I made a function that finds the prime numbers in a number. It is triggered by Azure Service Bus. On my local machine it runs in 350ms. The function, when running in a consumption plan, takes around…
12
votes
2 answers

Difference among Azure batch, scheduler and web job and when to use what

I could see primarily there are 3 options in Windows Azure to schedule jobs. Batch, scheduler and web jobs. Is there any link or video explaining what are the differences and what to use when and benefits? Thanks in advance
8
votes
2 answers

Azure Batch Pool: How do I use a custom VM Image via Python?

I want to create my Pool using Python. I can do this when using an image (Ubuntu Server 16.04) from the marketplace, but I want to use a custom image (but also Ubuntu Server 16.04) -- one which I have prepared with the desired libraries and…
pookie
  • 3,796
  • 6
  • 49
  • 105
7
votes
4 answers

VM Scaleset vs Azure Batch

VM scale set can be used to create multiple VM's based on the business requirement and, Also, Azure batch is also used to execute job in multiple VM's. What is the exact difference between Azure Batch and VM Scale set?
Balanjaneyulu K
  • 3,660
  • 5
  • 24
  • 45
6
votes
1 answer

What is the Azure API version

I'm trying to access the result of a GET request provided by Azure, as shown in the example : https://msdn.microsoft.com/sv-se/library/azure/dn820159.aspx My problem is that the api-version is a mandatory argument, but I have no idea about what to…
Romain
  • 799
  • 1
  • 9
  • 29
5
votes
0 answers

Unusable Compute Node in Batch pool

I am trying to create a pool with an application package installed on all nodes, however I keep getting that the compute nodes are unusable. My guess was that there was an error with the installation of application packages that caused the nodes to…
J.B
  • 137
  • 2
  • 16
5
votes
3 answers

How to install .net 4.6.1 on compute node in azure batch

I am stuck on creating azure batch pool with .net 4.6.1. I went through those very good resources: Compute Node - Install .NET 4.6.1 How to get the Windows 2016 Preview OS and I found out that there is a way to use .net 4.6.1 without installing…
adam.bielasty
  • 671
  • 9
  • 20
5
votes
1 answer

Azure web jobs VS azure batch VS worker roles

What are the pros/cons to decide on the approach for data processing on top of Azure? We see Azure web jobs, Azure batch and Azure worker roles, but literally unsure on pros/cons of each choice, especially on scalability and cost. Assuming e have…
user656
  • 347
  • 1
  • 4
  • 17
4
votes
0 answers

Is it possible to configure Azure Data Factory to run custom activity in a container-enabled pools in Azure Batch?

I'm building a data pipeline with custom activities and I would like to run them in containers, in a specially prepared images. My first attempt to configure pipeline failed with : Container-enabled compute node requires task container…
StanislawSwierc
  • 2,571
  • 17
  • 23
4
votes
0 answers

Azure Batch: Send email if job fails

As stated in the question, similar to Azure data factory, I want to have a feature where if my job fails I will get an email of it. How to configure that in Azure batch?
Gagan
  • 1,775
  • 5
  • 31
  • 59
4
votes
1 answer

How to retrieve stdout / stderr.txt files from Azure Batch compute nodes using python SDK?

The .NET documentation for Batch includes a method for retrieving files off a node in the pool: link The corresponding classes in the python SDK don't have any methods. What is the best way of returning the stderr.txt file for a task when it fails?
RedPanda
  • 522
  • 6
  • 15
4
votes
2 answers

Install Microsoft ODBC Driver 13 for SQL Server on azure batch node

I am deploying Ubuntu Server 16.04 Canonical nodes to Azure Batch. My Tasks need to access an Azure SQL server database and my code is written in Python. Before I can use pyodbc, I need to install the Microsoft ODBC Driver 13 for SQL Server. I've…
pookie
  • 3,796
  • 6
  • 49
  • 105
4
votes
1 answer

Retry deleting pool or jobs using Azure Batch?

I am using this Microsoft tutorial for a starting point for using Azure Batch pools, jobs, and containers. I have altered their code for deleting pools and jobs slighty to // Cleanup Batch Account Resources // Clean up Job await…
B. Witter
  • 564
  • 6
  • 19
3
votes
1 answer

Using Managed Identity in Azure Batch to Authenticate with Key Vault using Python in a Batch Pool

I am trying to use the system-assigned managed identity of azure batch to access the Azure Key Vault. I have found some code online, but I didn't know if this is possible or the certificate route is the only possibility. I have enabled a managed…
1
2 3
27 28