Although Microsoft-hosted agents are generally a good option for Build and Release; there are certain cases as:
- Security: you might want to control policies and access control to the VM.
- Networking: you might need to deploy from Azure DevOps to a restricted network such as on-premises or, in Azure, a private App Service Environment.
- Software: the software required for your pipeline to run is big or slow to install/add at runtime or simply not possible.
- Performance: you need a bigger machine because your build and/or release process is too CPU and/or memory and/or disk (and disk space) intensive. Microsoft-hosted agents have 10 GB of free space which becomes a constrain when using for certain Infrastructure components.
to just outline some of them. This is guide is aimed to explain how you can automate the self-hosted agent deployment. I’ll show you and explain a script I put together to configure it automatically as well as the ARM Template to get it deployed, along the path I will explain some of the basic for a self-hosted agent.
Microsoft-hosted Agents
Before getting in configuring your own agents, it’s a good start to understand what Azure DevOps provides out of the box in order to make the necessary adjustments. Microsoft-hosted agents are Azure Virtual Machines, generally, Standard_DS2_v2 (2 vCPU, 7GB RAM) according to the documentation and, the pool that I generally (Hosted VS2017) use has these software installed. Having this information is at least a starting point on what specs you should consider.
Let’s review some concepts
It’s import to go through a couple of concepts which will be helpful to understand why I did or recommend doing certain things in certain manners. These are mixed concepts from Azure DevOps (former VSTS) or Azure itself. so bear with me as we do some topic browsing.
Azure DevOps – Personal Access Token (PAT)
Odd enough, the authentication between the self-hosted agent and Azure DevOps is through a Personal Access Token, pretty much the most insecure method but it’s what it’s. Having this in mind, it’s important to know how-to create a new PAT, for that you can refer to Azure DevOps Services documentation. Take not of the token once created because that’s gonna be the only time you will see it.
Take away: You must create a PAT before installing a self-hosted agent since it will be required by the installation process.
Azure DevOps – Capabilities
Every agent in VSTS has a list of capabilities it supports (basically what it can do) which can be reviewed at the Agent Pool hub (https://[Account Name].visualstudio.com/_settings/agentpools). There are two types of capabilities:
- System Capabilities (Automated): these are the ones automatically discovered by the agent (either by finding specific software or by just adding all Environmental Variables)
- User Capabilities (Hardcoded): these are the ones manually configured in the server.
So, to me, ideally we should try to use Environmental Variables as much as possible so, in this way, capabilities are added automatically and you don’t need to worry on keeping them up to date through the administration portal. If we take a look at the capabilities an agent from “Hosted VS2017” pool (that’s a Microsoft-hosted one), these are the capabilities we see:
Basically, each one of those but Agent.Name and Agent.Version are environmental variables so if we want to build our own agent we should make use of them so everything just keeps working. Microsoft documentation has more information about this topic.
Take away: You should define environmental variables if you want Azure DevOps agent to automatically pick up capabilities.
Azure DevOps – Demands
Because of “Demands” is that “Capabilities” is important. This is how our Build or Release process specifies which capabilities the hosted-agent must satisfy in order to be able to run the pipeline successfully. If these are not met, the job will not even start:
Azure – Temporary Drive
Every Virtual Machine is Azure is deployed with a D drive; this is a temporary storage which has no cost associated and it’s basically used for the system page file. In other words, every machine comes with a free space for non-persistent content since whatever you have there, it might get lost (sounds like the agent working dir, right?). Just to give you an idea, at this moment, temporary space for DSv2-series SKU is:
Size | Temp storage (SSD) GiB |
---|---|
Standard_DS1_v2 | 7 |
Standard_DS2_v2 | 14 |
Standard_DS3_v2 | 28 |
Standard_DS4_v2 | 56 |
Standard_DS5_v2 | 112 |
Take away: Azure VMs come with a default D drive for temporary storage which we could leaverage for our agent working directory since the data there shouldn’t be persistent and, if it’s lost, the agent will regenerate the folder.
Windows Self-Host Agent
I personally hate to do things manually and evenmore if they require several steps, that’s why I put together a full automation to deploy my own hosted agent. The main reason for my need was the fact I had to execute a Release pipeline against an ASE with ILB (basically, non-public network) and this pipeline needed to execute the following tasks:
- Azure Resource Group deployment for deploying ARM Templates against Azure
- Azure App Service deployment for deploying the code for the different App Services (using msdeploy)
- Azure SQL database deployment for running TSQL and applying DACPACs to Azure SQL
- Azure Powershell for running some scripts to fill gaps not available with ARM Template today.
- Run GRUNT for adjusting configuration files.
In order to be able to run these tasks certain software and powershell modules needed to be present in the agent to successfuly run (or attempt to run), that’s how I found:
Task | Needs |
---|---|
Azure Resource Group deployment |
|
Azure App Service deployment |
|
Azure SQL database deployment | for running inline TSQL:
for applying DACPAC:
|
Azure Powershell |
|
Run Grunt |
|
Run Pester |
|
And after all this install Git for Windows and Azure DevOps Agent….. way too much work….
How the automation works
The automation is made of an Azure ARM Template which uses a Custom Script extension for executing a Powershell script which downloads and installs the software (if needed), then add whatever it’s defined to the system environmental variable PATH and creates the specified environmental variables (for configuring the capabilities). So, the components are:
- azuredeploy.json is the ARM Template which deploys the VM and configure a Custom Extension
- Configure-Agent.ps1 and software.json are the two pieces required to install and configure (bootstrapping) the hosted agent.
Before going further you might ask: “Hey, why did you write all this if there’s an Azure extension to install the agent? Check this“. Well, yes, it’s true but a couple of things: this extension doesn’t have some many configuration options (such as ability to run the agent as a service or choose the location of the working directory {remember I wanna make use of the Azure temporary storage}). Also, this extension just install the Agent but not all the other software or configurations for which it just did very few of what I needed. Finally, it’s more fun to write it 🙂
You can find all you need in my project: azure-devops-self-hosted-agent
AzureDeploy.json
Azure ARM Template to deploy a Windows Server 2016 VM with a Custom Script extension to configure Azure DevOps agent; this template includes:
- a Network Interface
- a Storage Account for Diagnostics
- a Virtual Machine with Managed Disks and small-disks (31GB for OS Drive)
- a Custom Script extesion
This template isn’t creating the VNET but it should exist before hand, this is because in most of the corporate deployments, VNET is in a separate resource group with more controls around it.
Also, information around your Azure DevOps account is required:
- Team Account which is a part of the URL you always acccess. If your URL is https://xyz.visualstudio.com then your team account is xyz.
- PAT which is passed as a Secure String to protect it.
To access the documentation around this file, click here
Software.json
It’s what defines the different available software to be installed; it’s vital for the automation to work properly as it contains the URL, command lines, environmental variables to define, etc. It’s actually that important, than the powershell script dynamically generates some of its parameters based on this file.
To access the documentation around this file, click here
Configure-Agent.ps1
This is the script installs the different software (and its configurations) together with Git for Windows and Azure DevOps agent. It has three base parameters (TeamAccount, PoolName and PATToken) but the rest of the parameters are dynamically generated based on the object in Software.json. (I could have Choco but it felt too much work to create the packages and their dependencies, plus I need it to learn it first)
Azure DevOps agent, and all software, will be downloaded to “C:\Packages\AzureDevOpsAgent“, will be configure to Run-as-a-Service and will set its working directory to “D:\agent_work” (Remember? Azure temporary drive). Agent name will be set to the computer name.
Something I didn’t mentioned but important, specially on corporate environments, this script, if you configure it, does hash validation of the downloaded content in order to ensure the downloaded software is what you expected (also to improve download time since if a piece of software is already downloaded, it won’t download the software again); if hash check fails, the whole process is aborted. So make sure, if you use hashes, that they actually match.
To access the documentation around this file, click here
Example of an execution
If you have everything that you need, if not get it from the repo, you are ready to run this template. This is how the execution looks like:
For deploying this Standard_DS2_v2 VM took 21 minutes and 15 seconds:
and after that, you just can go to Azure DevOps Services and verify the agent is there together with its capabilities:
Wrap Up
Of course, I created this whole automation just to make my life easier but, at the same time, I tried to make it as extensible as possible that’s why software.json showed up (I know it might not be easy to get) and that’s why I documented every file, so you can use this as a base to accomplish the configuration you need or collaborate to the repo to include more and more software to be installed.
Resources
- azure-devops-self-hosted-agent repository where you will find all the script and both json files
- Deploy an agent on Windows
- Software on Microsoft-hosted agents: Visual Studio 2017 on Windows Server 2016 (Hosted VS2017)
July 25, 2019 at 12:14
Very nice blog post. I absolutely love this site. Continue the good work!|
LikeLike
July 25, 2019 at 10:50
Hello it’s me, I am also visiting this website on a regular basis, this web page is truly fastidious and the users are in fact sharing nice thoughts.|
LikeLike
July 21, 2019 at 19:33
Can I just say what a relief to discover someone that really
knows what they are discussing on the web. You definitely know how
to bring an issue to light and make it important. More and more people must look at this and understand this side of your story.
I was surprised you’re not more popular
since you certainly have the gift.
LikeLike
July 19, 2019 at 23:09
Hi, I just looked at your site and really impressed by it’s design and information. You are doing a great job by providing such content to the people. Thank you so much.
LikeLike
July 19, 2019 at 17:52
Hi, yes this piece of writing is truly nice and I have learned lot of things from it regarding blogging. thanks.|
LikeLike
July 19, 2019 at 11:08
Truly good site thank you so much for your time in publishing the posts for all of us to learn about.
LikeLike
July 15, 2019 at 08:25
Nice post. I be taught something more difficult on totally different blogs everyday. It should at all times be stimulating to read content material from different writers and observe a little something from their store. I’d want to use some with the content on my blog whether or not you don’t mind. Natually I’ll provide you with a hyperlink in your net blog. Thanks for sharing.
LikeLike
July 14, 2019 at 17:11
Hello are using WordPress for your site platform? I’m new to the blog world but I’m trying to get started and create my own. Do you require any html coding expertise to make your own blog? Any help would be greatly appreciated!|
LikeLike
July 25, 2019 at 21:27
For now, I’m just using WordPress platform itself; it seemed easier for a start. You don’t need HTML coding, it’s like a word processor but it definitely helps, but not a must, a desidered.
LikeLike
July 14, 2019 at 16:33
I was wondering if you ever considered changing the layout of your site? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having one or 2 pictures. Maybe you could space it out better?|
LikeLike
July 25, 2019 at 21:28
Nop, I haven’t but definitely interested in know more. Can you a explian a bit more?
LikeLike
July 13, 2019 at 05:35
I’ve been exploring for a bit for any high quality articles or weblog posts in this kind of house . Exploring in Yahoo I finally stumbled upon this site. Studying this information So i am happy to express that I’ve a very just right uncanny feeling I came upon just what I needed. I such a lot indisputably will make certain to do not forget this website and provides it a look regularly.|
LikeLike
July 13, 2019 at 05:15
Highly energetic article, I loved that bit. Will there be a part 2?|
LikeLike
July 12, 2019 at 16:15
Assisted me a lot, just what I was looking for : D.
LikeLike