Azure Batch for Internet Data Collection Part 2: Application Package and Pool

Azure Batch Solution Configuration

To build out the solution design in my previous blog article, these are the following steps and design decisions.

  1. Create a batch account

To create an Azure Batch application, you can follow the steps outlined in the article Create a Batch account with the Azure portal

Provisioned Batch account
AzureBatchBlogpic1

I previously created a storage account for my .NET console applications

  1. Applications blade

    Azure Batch for Internet Data Collection Part 2- Application Package and Pool 2
    Add a compiled application that can be deployed to batch pools.

    1. To add a .NET console application, I compiled my application in Visual Studio
    2. Go to the .\bin\debug folder and zip up all files.
    3. Upload the zip file and enter Application Id and Version.
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 3
    4. In this case, I have added a 2nd package into the DataCollector Application Id
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 4
  2. Pools

    Azure Batch for Internet Data Collection Part 2- Application Package and Pool 5

    • Click Add
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 6
    • Enter a Pool ID and necessary configuration
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 7
    • Windows Server 2016 VM
      Standard_A2 2 cores 3.5GB RAM
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 8
    • 10 dedicated nodes (VMs)
    • 20 low priority nodes (VMs). Since these VMs are lower cost and are taken from a surplus capacity, they can be taken away at any moment. In other words, preemption. Any running tasks during preemption are requeued against another node.Since the VM size is A2 with 2 cores, there are 60 total cores among the dedicated and low priority nodes.
    • The max tasks per node are 4. In my case, each task is an execution of the .NET console application.
    • Task scheduling policy is set to spread which will create tasks across nodes in a round robin fashion vs pack which will fill a node of tasks before scheduling to another available node.
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 9
      Set the application package to the one just uploaded so that it will be deployed to each node as it is provisioned.
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 10
      Click OK
      Nodes in the pool in the process of being created
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 11
      Still in the process of completing. 2 are read but 8 are starting. At this point, 10 dedicated nodes are almost ready, but no low priority nodes have been made available yet as there is no guarantee.
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 12
      10 nodes are idle and ready for execution. This took about several minutes.
      Azure Batch for Internet Data Collection Part 2- Application Package and Pool 13

Next: Azure Batch for Internet Data Collection Part 3: Jobs and Tasks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s