Setup CI/CD pipeline with VSTS & Azure Stack

We all know that DevOps brings together people, processes, and technology. In the Microsoft DevOps world A large part of the technology piece is utilizing Visual Studio Team Services (VSTS) for continuous deployment of workloads to Azure.

Microsoft launched their Hybrid Cloud on July 10th 2017. Azure Stack is the secret sauce of Microsoft’s the Hybrid Cloud. Microsoft’s offering is the only one true Hybrid Cloud in the market bringing Azure to on-premises data centers.

As Microsoft continues to move their Hybrid Cloud forward the DevOps integration and capabilities we have for Azure extend to Azure Stack. Again I was fortunate to participate in a preview of the VSTS integration with Azure Stack. I was happy to see Microsoft putting a priority on this functionality because DevOps on Azure Stack is a HUGE need. Cloud is often the catalyst to helping organizations adopt a DevOps culture fostering digital transformation. Some organizations not being able to put all workloads in public cloud Azure Stack is a good way for them to get the same cloud capabilities on-premises DevOps integration being one of them. The setup and integration between VSTS and Azure Stack is working nicely. The team at Microsoft has given me permission to share about this topic via my blog.

In this blog post I am going to cover setting up VSTS to work with Azure and setting up a continuous-integration and-continuous deployment (CI/CD) pipeline to Azure Stack. With Microsoft DevOps you can utilize the pieces of VSTS that make sense for you to use leaving the control up to you. Through VSTS you can use many other DevOps tools such as Jenkins, Octopus deploy, GitHub, Bitbucket etc into your pipeline making Azure Stack just as flexible as Azure is. Let’s Jump in!

Steps to prep Azure Stack for Visual Studio Team Services (VSTS)

#1 Ensure you have installed the Azure Stack PowerShell and Azure PowerShell modules.

Details can be found here:

https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-powershell-install

#2 Add the Azure Stack environment using the following syntax

# Navigate to the downloaded folder and import the **Connect** PowerShell module

Set-ExecutionPolicy RemoteSigned

Import-Module PATH\AzureStack.Connect.psm1

# Register an AzureRM environment that targets your Azure Stack instance

Add-AzureRMEnvironment `

-Name “AzureStackAdmin” `

-ArmEndpoint “https://adminmanagement.local.azurestack.external

# Set the GraphEndpointResourceId value

Set-AzureRmEnvironment `

-Name “AzureStackAdmin” `

-GraphAudience “https://graph.windows.net/

# Get the Active Directory tenantId that is used to deploy Azure Stack

$TenantID = Get-AzsDirectoryTenantId `

-AADTenantName “YOURDOMAIN.onmicrosoft.com” `

-EnvironmentName “AzureStackAdmin”

# Sign in to your environment

Login-AzureRmAccount `

-EnvironmentName “AzureStackAdmin” `

-TenantId $TenantID

NOTE: You will need the environment name and the tenant ID for the next script.

#3 Create SPN

Original SPN creation script can be found here:

https://github.com/Microsoft/vsts-rm-documentation/blob/master/Azure/SPNCreation.ps1

Documentation on creating an SPN can be found here:

https://www.visualstudio.com/en-us/docs/build/concepts/library/service-endpoints#sep-azure-rm

Below I will display the script I used. Note that you will need the following parameters for the script:

$subscriptionName

“Enter Azure Stack Subscription name. You need to be Subscription Admin to execute the script”)]

$password

“Provide a password for SPN application that you would create”

$environmentName

“Provide Azure Stack environment name for your subscription”

$AzureStackTenantID

“Provide tenant ID from when Azure Stack enviroment was added”

EXAMPLE:

.\CreateSPN.ps1 -subscriptionName “Default Provider Subscription” -password PASSWORDHERE -environmentName AzureStackAdmin -AzureStackTenantID ID HERE

Here is the script I used that you can run:

param

(

[Parameter(Mandatory=$true, HelpMessage=”Enter Azure Stack Subscription name. You need to be Subscription Admin to execute the script”)]

[string] $subscriptionName,

[Parameter(Mandatory=$true, HelpMessage=”Provide a password for SPN application that you would create”)]

[string] $password,

[Parameter(Mandatory=$false, HelpMessage=”Provide a SPN role assignment”)]

[string] $spnRole = “owner”,

[Parameter(Mandatory=$false, HelpMessage=”Provide Azure Stack environment name for your subscription”)]

[string] $environmentName,

[Parameter(Mandatory=$false, HelpMessage=”Provide tenant ID from when Azure Stack enviroment was added”)]

[string] $AzureStackTenantID

)

#Initialize

$ErrorActionPreference = “Stop”

$VerbosePreference = “SilentlyContinue”

$userName = $env:USERNAME

$newguid = [guid]::NewGuid()

$displayName = [String]::Format(“VSO.{0}.{1}”, $userName, $newguid)

$homePage = “http://” + $displayName

$identifierUri = $homePage

#Initialize subscription

$isAzureModulePresent = Get-Module -Name AzureRM* -ListAvailable

if ([String]::IsNullOrEmpty($isAzureModulePresent) -eq $true)

{

Write-Output “Script requires AzureRM modules to be present. Obtain AzureRM from https://github.com/Azure/azure-powershell/releases. Please refer https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/DeployAzureResourceGroup/README.md for recommended AzureRM versions.” -Verbose

return

}

Import-Module -Name AzureRM.Profile

Write-Output “Provide your credentials to access Azure subscription $subscriptionName” -Verbose

Login-AzureRmAccount -SubscriptionName $subscriptionName -EnvironmentName $environmentName -TenantId $AzureStackTenantID

$azureSubscription = Get-AzureRmSubscription -SubscriptionName $subscriptionName

$connectionName = $azureSubscription.SubscriptionName

$tenantId = $azureSubscription.TenantId

$id = $azureSubscription.SubscriptionId

#Create a new AD Application

Write-Output “Creating a new Application in AAD (App URI – $identifierUri)” -Verbose

$azureAdApplication = New-AzureRmADApplication -DisplayName $displayName -HomePage $homePage -IdentifierUris $identifierUri -Password $password -Verbose

$appId = $azureAdApplication.ApplicationId

Write-Output “Azure AAD Application creation completed successfully (Application Id: $appId)” -Verbose

#Create new SPN

Write-Output “Creating a new SPN” -Verbose

$spn = New-AzureRmADServicePrincipal -ApplicationId $appId

$spnName = $spn.ServicePrincipalName

Write-Output “SPN creation completed successfully (SPN Name: $spnName)” -Verbose

#Assign role to SPN

Write-Output “Waiting for SPN creation to reflect in Directory before Role assignment”

Start-Sleep 20

Write-Output “Assigning role ($spnRole) to SPN App ($appId)” -Verbose

New-AzureRmRoleAssignment -RoleDefinitionName $spnRole -ServicePrincipalName $appId

Write-Output “SPN role assignment completed successfully” -Verbose

#Print the values

Write-Output “`nCopy and Paste below values for Service Connection” -Verbose

Write-Output “***************************************************************************”

Write-Output “Connection Name: $connectionName(SPN)”

Write-Output “Subscription Id: $id”

Write-Output “Subscription Name: $connectionName”

Write-Output “Service Principal Id: $appId”

Write-Output “Service Principal key: <Password that you typed in>”

Write-Output “Tenant Id: $tenantId”

Write-Output “***************************************************************************”

Output should be similar to this:

You will use information from the Service Connection output in the next step.

Steps to configure Azure Stack as a Service Endpoint in VSTS

Log into your VSTS account at visalstudio.com

Navigate to one of your projects.

Go into Settings.

Click on Services.

Click on New Service Endpoint

A window will pop up. Click on “use full version of the endpoint dialog.”

Next input the needed data. This data comes from the Service Connection info that you copied.

You can put whatever you want in the Connection name and the Subscription Name. Note do not verify the connection. It will not succeed as VSTS cannot access your private Azure Stack yet. Click OK when done.

Setup build agent on Azure Stack host

Next you need to setup the build agent on the Azure Stack host. (Note: In this post I am using the ASDK.) From within VSTS download the Windows agent. Extract the download to a local folder.

Go to Security under your profile in VSTS.

Next add a Personal access token (PAT) for Azure Stack.

Copy the token. Note it will not be shown again ever after you leave this screen.

In the folder with the extracted build agent you will see the following. We need to run the run.cmd file from an elevated command prompt.

Here is a screenshot of running the run.cmd. I recommend deploying the build agent as a service. You will use your personal access token (PAT) here and the azure stack admin account.

After the run.cmd finished the folder with the extracted contents should look like the following:

You can now see the agent in VSTS.

That’s it for the setup for connecting VSTS to Azure Stack. Next let’s look at setting up a continuous-integration and-continuous deployment (CI/CD) pipeline for VM-deployment to Azure Stack.

 

THE BUILD

What I cover here is focused on infrastructure as code (IaC) using ARM templates. If you need to set up CI/CD to Azure Stack for Web Apps, Mobile Apps, Containers, etc the process is the same as it is on Azure with the only difference being that you point to Azure Stack. Also note that in this post I am using the ASDK not multi-node.

Within VSTS create a new repository and place your ARM template in it.

Next click on Build and Release. Create a new Build Definition.

In the build definition. Point the Get sources to the repository you just created. Add 2 tasks under Phase 1. The first task will copy the ARM template to the build staging directory. The second task will publish the ARM template so that a release definition can pick it up. Both tasks are shown in the following screenshots.

Copy Files to task

Publish Artifact task

OPTIONAL: To setup continuous integration click on Triggers. Here you can set a schedule to run the builds or you can click on the repository as shown in the screenshot and then check Enable continuous integration. By checking the box next to Enable continuous integration it tells VSTS that anytime content in the repo is changed to run a build.

Click on Save & queue. This will start the build.

The build will start. As long as everything is setup properly within your build it will succeed as shown in the following Screenshot.

That’s all for our build. Next up we need to create a release definition (RD) pipeline. The RD will take the build artifacts and deploy to an environment/s you specify.

Read More

Azure or Azure Stack “Write Once, Deploy Anywhere” Update

A while back I wrote a blog post about being able to take one IaaS VM Azure Resource Manager (ARM) template and deploy it to both Azure or Azure Stack. This blog post included a JSON file and the PowerShell to do this. The idea for that came from needing to set up a cool and working demo for MMS 2016 and the need to showcase the power of Microsoft’s HybridCloud. Here is a link to that original blog post:

Write once, deploy anywhere (Azure or Azure Stack)

Today I have finished updating the PowerShell and ARM template/JSON file to be more streamlined and to work with TP2. Here is the link to download these:Here are the updates:

https://gallery.technet.microsoft.com/Create-VM-on-Azure-or-3c6d0420

Here are the updates:

  • The JSON and PowerShell script have been modified to work with Azure Stack TP2.
  • This script now utilizes the connection PowerShell module AzureStack.Connect.psm1 from Azure Stack tools.
  • This is included with the download of this script and JSON file on TechNet Gallery.
  • The script is hard coded to look locally to import the AzureStack.Connect.psm1 module.
  • Streamlined the JSON file and PowerShell script.
  • The script no longer prompts for the publicDNS name. It is now automatically set to the same as the vmname.
  • The script no longer prompts for the storage account name. It is automatically set to vmnamestorage.
  • The script no longer prompts for the resourcegroup name. This is now automatically set to vmname-RG.
  • By default this script now uses a JSON file hosted on Github. This is set in the $templateFilePath variable as shown on the next line.
  • To keep it to the local directory just use the JSON file name.

GITHUB: $templateFilePath = “https://raw.githubusercontent.com/Buchatech/Azure-AzureStackVM/master/AzureandAzureStack.json”
LOCAL: $templateFilePath = “AzureandAzureStack.json

This will be my last blog post of 2016. See you next year folks…..

Happy Stacking!

Read More

Resource Group Clean-up in Azure Stack

If you are like me, you end up creating a ton of resource groups in Azure Stack when testing things out. I needed a way to delete them without having to click one each one via the portal. The best option of course is to leverage PowerShell. I threw together some PowerShell to handle this. I came up with two options #1 can be used to delete a bunch of RG’s that have a common name. For example, I had a bunch of VM00* resource groups. I use the script to go loop through and delete all resource groups with VMO in the name. Option #2 pop’s up a GUI window so I could select the RG’s I wanted to delete. It put them in an array and then looped through to delete them in one shot.

This is great because I can kick this off and go do something else. I will share both below in this blog post along with some screenshots. I won’t have a download for the PowerShell syntax so just copy from this post if you want to use it. Be sure to use AzureStack.Connect.psm1 for connecting to your Azure Stack environment before running any of the following code.

Code:
#1

#Create Variable of RG’s with common name
$Resourcegroups = Get-AzureRmResourceGroup | where {$_.ResourceGroupName -like (‘*VM0*’)}

#Create array of RG’s
$RGLIST = $Resourcegroups.ResourceGroupName

#Loop to remove each resource group in the array
ForEach(
$rg in $RGLIST
)
{
Get-AzureRmResourceGroup -Name $rg -ErrorAction SilentlyContinue | Remove-AzureRmResourceGroup -Force -Verbose
}

This image shows the array of RG’s that will be looped through. I highlighted vm003rg in the array and in the PowerShell status message.

rgcleanup-1

The following screenshot shows VM003RG being deleted in the Azure Stack portal.

rgcleanup-2

#2

#Create Variable of RG’s from GUI selection
$selectedrgs = (Get-AzureRmResourceGroup | Out-GridView ` -Title “Select ResouceGroups you want to remove.”` -PassThru).ResourceGroupName

#Loop to remove each resource group in the array
ForEach(
$rg in $selectedrgs
)
{
Get-AzureRmResourceGroup -Name $rg -ErrorAction SilentlyContinue | Remove-AzureRmResourceGroup -Force -Verbose
}

After running the Create Variable of RG’s from GUI selection part of the code a window as shown in the following screenshot will pop up. Select the RG’s you want to remove, click Ok and they will be placed into an array.

rgcleanup-3

Below if the output of the array. Run the Loop to remove each resource group in the array part of the code and each of the RG’s will be removed.

rgcleanup-4

I have also used this when a resource group would not delete from the portal. On some stubborn resource groups I have had to run this a couple of times. This is a short post. I hope this helps someone out!

Read More

Azure & RRAS Site to Site VPN Setup (Azure Resource Manager)

Background

I have not seen a lot of posts out there on setting up Azure Site to Site VPN Setup Azure Resource Manager. The ones I did follow had some missing steps. Setting up the AS2S is a pretty detailed process so I wanted to put the steps I had to follow in a blog post.

What you will need:

On-Premises

  • RRAS installed on an on-premises server.
  • At least 1 NIC on your RRAS server.
  • Your public IP. This will not work with dynamic DNS. You have to have a public IP. If you don’t own a public IP you can still use the dynamic public IP your ISP has assigned to you. You will need to just watch it and manually update it in Azure when it changes.
  • UDP Ports 500, 4500, and 1701 forwarded to your RRAS server. This is if your RRAS server is behind a NAT device.
  • Either your servers pointed to RRAS for their gateway or static routes setup on the VM’s you want to be able to communicate with Azure.

Azure:

Addressing:

For on-premises you can use whatever you want. I use 192.168.0.x/24.

For Azure we will use 10.121.0.x/16 and 10.121.0.0/24 and 10.121.1.0/24.

Steps:

~ AZURE CONFIGURATION FIRST! ~

First we will start off with configuring the network components we need up in Azure. Start by going to the Azure portal at https://portal.azure.com.

STEP 1: I would setup a resource group. You will put all of your resources for the site to site VPN in here for better tracking and management. As an example I named mine: “S2SVPN-Buchatech-LabRG“.

as2svpn-29

NOTE: Our resource group is empty at this point.

STEP 2:  Let’s start off by creating a virtual network. Go to:

Virtual networks>Create virtual network and click on Add. Let’s name this “S2SVPN-RRAS-Vnet“.

Give it “10.121.0.0/16” for the address space. This has to have enough room to place two subnets into it.

For the first subnet make the Subnet Name “Azure-VMs“. Make the Subnet address range “10.121.0.0/24“. Set it to the resource group you created in the previous step.

NOTE: I recommend placing all resources that will be a part of your site to site VPN in the same region. I used North Central US.

as2svpn-1

STEP 3:

We will be creating a virtual network gateway. This network gateway will contain the second subnet.

Go to Virtual networks>S2SVPN-RRAS-Vnet>Settings>Subnets.

Click on + Gateway subnet. For the Address Range use “10.121.1.0/24“. This address range is the IP range for your RRAS server to use.

So in your virtual network you should now have the two following Subnets:

as2svpn-2

STEP 4:

Now navigate to Virtual network gateways and click on Add. Name the gateway “S2SVPN-RRAS-VnetGW“. For the virtual network select our existing one named S2SVPN-RRAS-Vnet. Leave the gateway type to VPN, and leave VPN type to Route-based. For the public IP we don’t have one so we will need to create one here. Click on Choose a public IP address and a blade will fly out. Click on Create New.

as2svpn-3

I give it a name of “S2SVPN-RRAS-VnetGW-IP“. Your settings should look like this:

as2svpn-4

After the Virtual network gateways is created go and get the public IP addresses. We will need to plug this into RRAS later. You can get this by going here: Virtual network gateways>S2SVPN-RRAS-VnetGW>S2SVPN-RRAS-VnetGW-IP>Settings.

as2svpn-5

NOTE: It may take some time to provision the public IP so be patient here.

STEP 5:

Next up we need to configure a Local network gateway. Go to:

Local network gateways and click on +Add.  On Create local network gateway name it

S2SVPN-RRAS-LocalNetGW“, enter the public IP of your RRAS server, In the address space enter an IP range or ranges for your on-premises network , and select your RG.

as2svpn-6

NOTE: If you do not know what the public IP is on your RRAS server’s network just visit http://ipchicken.com and it will display it.

Now we need to create a connection in our local gateway. To do this navigate to the settings>connections and click on + Add. Name this “S2SVPN-RRAS-LocalNetGW-Connection“.

The Connection type will default to Site-to-site (IPsec). Leave this. Set the Virtual network gateway to “S2SVPN-RRAS-VnetGW“. Set a Shared key (PSK) to be used and remember this will also be used on the RRAS server so document this somewhere.

as2svpn-7

That’s it for the network configuration up in Azure. As long as everything was followed in these steps you should now have the following in your resource group.

as2svpn-8

~ RRAS CONFIGURATION! ~

Read More