Azure & Azure Stack Resource Group Cleanup Script

When building things in Azure & Azure Stack I tend to create a lot of temporary resources groups. I like to remove these when I am done. I have been using a PowerShell script for a while to help make this easier. I have decided to upload this script hoping others will find it useful as well. The script is named CleanupResourceGroups.ps1 and can be downloaded here:
https://gallery.technet.microsoft.com/Cleanup-Azure-Resource-d95fc34e

The script can be used two ways:

#1 the script can be run using -Like with an expression like where {$_.ResourceGroupName -like (‘*MySQL*’) in which the script would remove any resource group with MySQL in it. To use this option just un-comment the code in SECTION 1- Uses -Like, change MySQL to whatever you want, comment SECTION 2- Interactive RG selection code, and then run the script.

#2 the script can be run interactively allowing you to select multiple resource groups you want to remove. By default the SECTION 2- Interactive RG selection code is un-commented. If you run the script it will run interactively as shown in the following steps/screenshots.

After running the script it will prompt you to select an Azure subscription.

Next the script will give you a list of resource groups in the subscription you selected. Select the resource groups you want to remove and click Ok.

The script will loop through and remove the resource groups you selected. Note that script is using -Force so it will not prompt to ensure you intend to remove the resource groups. Make sure you want to remove the resource groups before running this script.

NOTE: When running this for Azure Stack ensure you are logged into the Azure Stack environment. For info on how to do this visit: https://bit.ly/2LkvddG

That is it. It is a simple script to make removing many resource groups easier. I hope you find this script useful as I have!

Read More

The “argument is null or empty” error in Azure Automation Runbook

I was recently working on an Azure Automation runbook that provisions an empty resource group in Azure. I was running into an issue when the runbook ran that the variable being used with New-AzureRmRoleAssignment was null. The errors I was receiving are:

New-AzureRmRoleAssignment : Cannot validate argument on parameter ‘SignInName’. The argument is null or empty. Provide
an argument that is not null or empty, and then try the command again.
At line:96 char:39
+ New-AzureRmRoleAssignment -SignInName $RequesterSignIn -RoleDefinitio …
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [New-AzureRmRoleAssignment], ParameterBindingValidationException
+ FullyQualifiedErrorId :
ParameterArgumentValidationError,Microsoft.Azure.Commands.Resources.NewAzureRoleAssignmentCommand

and

New-AzureRmRoleAssignment : Cannot validate argument on parameter ‘ObjectId’. Specify a parameter of type ‘System.Guid’
and try again.
At line:97 char:37
+ New-AzureRmRoleAssignment -ObjectID $RequesterID -RoleDefinitionName  …
+                                     ~~~~~~~~~~~~
+ CategoryInfo          : InvalidData: (:) [New-AzureRmRoleAssignment], ParameterBindingValidationException
+ FullyQualifiedErrorId :
ParameterArgumentValidationError,Microsoft.Azure.Commands.Resources.NewAzureRoleAssignmentCommand

It turned out to be a permission issue with AzureRM.Resources CMDLETS not being able to talk to AAD specifically Get-AzureRmADUser that I was using for a variable.

To fix this I had to give the following permissions for the AAD directory to the AzureServicePrincipal Run As Account:

Windows Azure Active Directory (AAD)
Application Permissions

·       Read/Write directory data
·       Read directory data

Delegated Permissions
·       Read directory data
·       Read all users’ full profiles
·       Read all users’ basic profiles

Microsoft Graph
App Permissions
·       Read directory data

In your runbook code you will typically have

# Authenticate to Azure resources
$connectionName = “AzureRunAsConnection”

# Get the connection “AzureRunAsConnection “
$servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
“Logging in to Azure…”
Login-AzureRmAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint

You may have a some differences like the connection variable and the name of the runasconnection. The point here is that the runas connection is what needs to have the proper permissions. You can find this account here to get the name and ApplicationID:

To give the permissions go to Azure Active Directory>the directory you are using in this automation>App registrations>and search based on the ApplicationID. Don’t forget to select All apps in the drop down.

Click on Add first and add the AAD and then Microsoft Graph permissions.

After you add the proper permissions make sure you click on Grant Permissions. The permissions are not actually applied until you do this. Once you click on Grant permissions you will see the prompt shown in the screenshot. Click Yes.

Verify the permissions have been added properly. In AAD go to All applications>select All applications. Find your service principle application.

Click on the service principle applications permissions.

Verify the AAD and graph permissions are listed. If the AAD and graph permissions are listed then the runbook should be good to go.

Read More

Unlink yourself from unused AAD directories

Working in the world of consulting I am often added to other Azure Active Directories that are managed by someone else. After a while these can pile up like in the following screenshot.

I like to clean these up as the inviting organizations typically don’t remove you. Here is a quick way to do this. In a browser go to https://myapps.microsoft.com. It will look like this:

Click on your name and select the directory you want to remove. Click on your name again and click on the cog for the settings.

You should then see the option to “Leave the organization”. Click the link.

You will see the following pop-up. Click on Leave.

That’s it. You will no longer see the directory you just removed listed in the Azure portal under your directory list.

Read More

Exploring the UniFi – Wifi & Software Defined Networking Solution

It has been a while since I have blogged about non-Microsoft technology. Well I recently moved to a new house and figured this was a good reason to upgrade my network and wifi equipment. I decided to go with Ubiquiti Networks – UniFi line. They have a physical hardware and Software Defined Networking (SDN) combo that I deployed. After deploying Unifi I realized how bad the previous wifi solutions I have used are and wanted to blog about Unifi’s solution. Lets jump in. Here is a list and pictures of the gear for my setup:

1 x UniFi® Security Gateway (Enterprise Gateway Router & Firewall with Gigabit Ethernet)

https://www.ubnt.com/unifi-routing/usg

1 x UniFi® Switch PoE 24 (24 port Managed PoE+ Gigabit Switch with SFP)

https://www.ubnt.com/unifi-switching/unifi-switch-poe

2 x UniFi® AP AC LITE (802.11ac Dual Radio Access Point)

https://www.ubnt.com/unifi/unifi-ap-ac-lite

This gear is enterprise level stuff at consumer level pricing!

NOTE: I originally also bought the UniFi® Cloud Key. This is basically an embedded server that runs the UniFi Controller software for managing all the network gear. It kept rebooting every 5 minutes and was super-hot. I ended up returning it after talking to tech support. I will either buy one in the future when they fix it or I will just run the UniFi Controller software on my own server.

I decided to go with all Unifi gear because it works seamlessly together. The gear overall has great designs especially the AP’s. The AP’s mount to a wall or ceiling and blend in like smoke detectors. The real star in the Unifi solution though is the UniFi Controller software. The UniFi Controller software gives you centralized management of all of your network gear. With the controller software you can Visualize the network in maps, get performance charts with real-time graphs, receive outage notifications and custom alerts, manage updates and schedule tasks, set up alerts, apply mass-configuration changes, get deep insights into metrics, setup VLANs, multiple wifi networks, access schedules, setup guest networks and more. I know this is just for my home network but I am a technical geek and am super excited to have this level of networking in my home. Now let’s explorer the UniFi Controller software on my setup.

In the UniFi Controller software you can add all of your devices. The following screenshot shows this. You can manage the devices from here such as rebooting, upgrading firmware, locating them and more. Something cool about locating the devices is that when you click on locate it makes the blue light the device has flash.

One of my favorite features of the UniFi Controller software is the ability to have network maps. You can upload custom floor plans into the UniFi Controller software and then you can place your devices on the map. In my scenario I uploaded maps for 3 floors. This screenshot shows the lower floor with the gateway and switch.

I have a main level map that has one of the AP’s.

I then have an upper map with the second AP. Something else to note about these maps when you have an AP shown is that you can display wifi coverage. You can should 2G or 5G coverage.

From the maps section of the UniFi Controller software you can also switch to the topology view. The topology view gives you a tree view of your devices and clients that are connected to devices. In the following screenshot you can see clients that are connected via CAT6 to the 24 port switch and you can see what clients are connected to each wifi AP. Something else shown in the screenshot is properties of a client. You can get device info, stats, and even deep packet inspection.

Read More

Made the cut. Microsoft MVP for 7th year!

Today was a very happy day for me and a sad day. It was a happy day because I once again have been renewed as a Microsoft MVP! It was sad because many MVP’s did not get renewed this go round and many of them are personal friends of mine. Shout out to all of you that did not get renewed. You folks are still community MVP’s. Keep doing all the great things you do and I will see you out in the tech community. Also congrats to all the new and renewed MVP’s!

Well, I made the cut. I am a Microsoft MVP for the 7th year! Here is the email and MVP site confirming my renewal:

This July is extra special. In fact, this award cycle ranks up there with the very first time I was awarded. I rank this 7th award so high because it was not easy to stay an MVP with so many not being re-awarded. I am one of the lucky ones that made it back in. Last year I made a conscious decision to shift my focus completely to Azure, Azure Stack, DevOps, and CloudOps. I like to think this shift of focus helped me get back in during this humbling award cycle.

Again this year I feel blessed to still be in the MVP program. I hope to continue to add value and remain an MVP. As always a huge thanks goes out to everyone in the community and Microsoft. Special thanks to Betsy Weber, David Armour, Joseph Chan, Ricardo Mendes, Tim Benjamin, Daniel Savage, and many other folks at Microsoft.

I will continue to do all that I can in the Azure, Azure Stack, CloudOps/DevOps communities this year.

My Microsoft MVP Profile: http://mvp.microsoft.com/en-us/mvp/Steve%20Buchanan-4039736

Read More

Microsoft Professional Program for DevOps Finished!

I am a firm believer that no matter how old you are, how far along you are in your career, and regardless of the industry you are in it is important to continue educating yourself. This helps you expand your skillset, stay relevant, and sets you up for new opportunities as they come along. My field of information technology has been changing at a rapid pace and so for a while, I wanted a good way to ramp up on DevOps as a whole. A while back I found out that Microsoft added a new track to their Professional Program for DevOps. When I checked it out I found it to be very thorough and it was not just focused on Microsoft’s DevOps tooling but included non-Microsoft as well. I jumped in without hesitation and started learning.

I finally completed the program last week. Here is my certificate https://academy.microsoft.com/en-us/certificates/67284e84-8afe-4f13-b477-d7620949fb18. I am planning to dive into the program they have for cloud next. If you have not heard of Microsoft’s Professional Program DevOps before here more information about it:

“DevOps is the union of people, process, and products to enable continuous delivery of value to end users. This program helps the student learn about continuous integration and deployment, infrastructure as code, testing, databases, containers, and application monitoring: skills necessary for a DevOps culture in today’s workplace. This program focuses on Microsoft DevOps technologies as well as some OSS (Open Source Software) DevOps tools. Some of the Microsoft DevOps technologies covered in this course consist of Azure, Azure Resource Manager, IaaS, PaaS, IIS, Azure App Service, DevTest labs, Desired State Configuration (DSC), Azure Automation, OMS, Application Insights, SQL, Nuget, TFS, VSTS, and Visual Studio. Some of the OSS DevOps tools covered in this course consist of Jenkins, Git, Github, New Relic, Nagios, Chef, Docker, DC/OS, swarm, and Kubernetes.”

Here is a link to it:  https://academy.microsoft.com/en-us/tracks/devops

This program consists of 8 required courses. Each course runs for three months and starts at the beginning of a quarter. In the end, there is a capstone that has to be completed. This capstone course is the 8th one. You have four weeks to complete the capstone. The capstone is a bunch of hands-on stuff you have to do. Courses average 16-32 hours per course to complete and are taken via the edX.org platform https://www.edx.org/microsoft-professional-program-devops.

Here is a list of all of the DevOps program courses:

      • Introduction to Dev Ops Practices
      • Infrastructure as Code
      • Continuous Integration and Continuous Deployment
      • Configuration Management for Containerized Delivery
      • DevOps Testing
      • DevOps for Databases
      • Application Monitoring and Feedback Loops
      • Microsoft Professional DevOps Capstone Project
      • The DevOps Capstone Project contains:
        • Automation
          • Use ARM templates to deploy and configure Infrastructure in Azure
        • Continuous Integration
          • Implement Continuous Integration solution using Visual Studio Team services (VSTS)
        • Continuous deployment
          • Implement Continuous Deployment solution using Visual Studio Team Services (VSTS)
        • Testing
          • Implement Unit tests
          • Implement Testing in Production
        • Application Monitoring
          • Implement application monitoring solution using Application Insights

As you can see from that list this program is not just all about VSTS. There is a lot of Azure baked in as well as other non-Microsoft DevOps tooling. I highly recommend this course for anyone jumping into DevOps, or CloudOps and especially for folks with an IT pro background. If CloudOps is foreign to you here are a couple of blogs related to this topic: Sys Admin to Cloud Admin…ITSM to CloudOps…On-Prem to Azure Stack/Azure and Native Cloud Management in Azure.

My personal opinion is that Microsoft should move away from the certifications as they are and to this format. This format combines training and testing. When Microsoft first started the Professional Program for they only had a track for data scientists. They have added more and more tracks over time. Today there are tracks also for Big Data, Web Development, Software Development, AI, IT Support, and Cloud Administration.

Here is a link for all the tracks so you can check them out: https://academy.microsoft.com/en-us/professional-program/tracks. These programs are a great way to expand your learning. Check them out!

Read More

Native Cloud Management in Azure

For those that know me know that I have been a System Center expert for some time focused on helping organizations manage their IT along with their ITSM needs. I have been working with Azure since it was released off and on but started to get serious about Azure after Microsoft’s move to resource manager. And even more recently I have re-focused completely to Azure and DevOps along with ITSM in the context of the cloud. I consider this combination CloudOps.

CloudOps is important when it comes to cloud and supporting DevOps. A part of CloudOps is cloud management. More specifically the tooling name for cloud management is often referred to as Cloud Management Platform (CMP).  CMP’s can be a CloudOps architect and engineers best friend or worst nightmare. There are many CMP solutions out there in the market that can be used to manage Azure and other clouds as well. Microsoft has done a nice job building and bringing in native solutions that can be used to manage Azure. The following image depicts the areas of cloud management that are in focus for Microsoft.

I am sure the plan for native cloud management will change and expand over time as Azure and its management needs continue to grow. The native set of cloud management tools in Azure can be viewed as a CMP. I am going to put together a group of blogs that at a high level cover the native solutions that exist for managing and securing Azure. There are so many areas in this topic that it has to be broken out into a blog series. This is the first time I am doing a blog series. It will cover the following:

Check back on this post soon. As I create more blog posts in this series they will be linked on the list above.

Read More

Azure Cost Management (Cloudyn)

IT financial management (ITFM) is an important part of IT operations as business dependency on IT continues to grow in the age of digital transformation. ITFM is a part of ITIL as a Service Strategy element in the framework. ITFM is a key part of CloudOps as well because spending in the cloud is based on an OPEX model and every single cost is tracked. ITFM and cost management in the cloud should be used to effectively and concisely connect the dollars spent on IT to the value delivered to the business. We can do this with Azure Cost Management. In this post, I am going to give an overview of Azure cost management highlighting many of the things you can do with it. Let’s dive into the solution now.

Overview

In June of 2017, Microsoft acquired Cloudyn a startup that had tooling for cloud monitoring and analytics tools focused on cloud financial management. Cloudyn’s solution is multi-cloud covering Azure, Azure Stack, AWS, and GCP. Through the acquisition of Cloudyn Microsoft was able to bring the tooling into the Azure ecosystem giving Azure customers an enhanced way to track and control cloud spend improving the improving the Azure cloud governance story.  As of right now, there is a free level and a paid level for Azure cost management. The following table lists what features are available with each level.

FREE capabilities:
Reporting Report on cost and usage
Data enrichment Categorize by resource tags
Budgets Create and manage cost and usage budgets
Alerting Create alerts on cost and usage budgets
Recommendations Eliminate idle cloud resources

Right-size cloud resources

PAID capabilities:
Chargeback features including cost markup, redistribution, and custom charges
Import external budgets
Customize recommendation thresholds
Categorize costs with custom meta-tags

Since the acquisition, Microsoft has added a link to the Cloudyn portal directly in Azure and integration with your Azure subscriptions giving you the ability to launch a new Cloudyn account that is tied to your subscription. Microsoft added Cost Management in Azure and this is where you will find Cloudyn and sign up. As shown in the following screenshot you can see the “Go to Cost Management” button. After clicking on that you will go the Cloudyn portal and will be able to add your various cloud accounts.  The thing that I really like about Azure cost management is that there is a ton of data and dashboards that are available right out of the box after adding a cloud account. There is not a bunch of configuration that you need to do to get the default dashboards and optimization tools.

After you are all signed up and have your cloud accounts added your dashboards will start to show data. The next two screenshots show a couple of the default dashboards.

The management dashboard gives a good summary of your cloud financials on one pane of glass.

 

The cost controller dashboard shows cost trends, some forecasting info, a breakdown of costs and more.

As you can see from the previous screenshots there are several other dashboards with other content. You can modify any of these dashboards adding or removing widgets. You also can create your own dashboard adding whatever widgets you want to it.

In Azure cost management, you can add cost centers known as Cost Entities. Entities are intended to mirror your organization’s hierarchical structure such as business units, divisions, departments, or teams within your organization some examples are engineering, R&D, development, marketing etc. The goal of the entities is to give you a way to track cloud spend by the entities. Keep in mind the cost entities can be anything that fits the way you want to structure and track cloud costs. You also can leverage tags, add budgets, and then associate costs and or budgets to the cost entities into cost models. Cost models give you a way to distribute and allocate costs. You can track costs back to these cost entities and you can track costs against budgets for showback or chargeback scenarios. Below is a screenshot of the cost entities screen. Keep an eye out for a detailed blog from me walking through how to structure and set up this part of Azure cost management. This area of Azure cost management warrants its own dedicated blog.

Here is an example of a budget set on a cost entity.

Read More

Azure Policy

A key component of cloud governance in Azure is being able to apply policies across cloud resources. In Azure, there is a  service called Azure Policy that can be used to define policies and enforce them across your cloud resources. Azure Policy can be used to create, assign and, manage, and apply policy definitions. Azure Policy can be set to just evaluate when resources are out of compliance or remediate when resources are out of compliance. These two modes are known as audit effect and deny effect.

Azure policies can be applied to Management Groups, subscriptions, or resources.

Azure Policy has been around for a while but recently it has revamped to make it enterprise ready. Azure Policy is in preview but it won’t be long before it will go GA and can be used to help manage your Azure. There is no pricing yet while Policy is in preview.

Azure Policy is not RBAC. RBAC deals with user access and user actions such as what users can access what resources and what they can do with them. Azure Policy deals with existing resources and resource properties during the deployment of them.

In Azure Policy you have something known as definitions. Definitions are essentially compliance rules that can be assigned to Azure resources. These definitions can just check to see if items are compliant or not and can enforce compliance. Definitions can be used to set conventions for resources, for example, all resources in a subscription should have a certain tag when created. Definitions are also used to evaluate something and take an action based on the result of the evaluation. A good example of this is that you could use a policy definition to evaluate if virtual machines are using managed disks or not. Azure Policies are used to help control costs and manage resources across your Azure subscriptions.

There are two types of definitions called Policy and Initiative. A Policy definition is a single definition. An Initiative definition is a group of Policy definitions. Initiative definitions are used to help achieve larger compliance need. To gain a better understanding of Initiative definitions you can look at Security Center as it leverages Initiative definitions. Security Center has a built-in Initiative definition named [Preview]: Enable Monitoring in Azure Security Center. This built-in Initiative definition for Security Center contains 13 Policy definitions related to security as shown in the following screenshot.

In Azure policy there are built-in and custom definitions. The built-in definitions have been created by Microsoft and are ready to be used to help with common needs in cloud. There are 36 built-in policy definitions today. Custom definitions are built by you. All Azure policies are JSON so writing custom polices is similar to writing ARM templates. Templates for Azure policies can be found in the Repository for Azure Resource Policy samples here: https://github.com/Azure/azure-policy. You can use these samples as a starting point when building your own. Here is an example of an Azure policies JSON:

Read More

Azure Management Groups

If your company is like most organizations that are using the cloud, then you have many subscriptions floating around. This is often due to “shadow IT”. However, sometimes organizations simply use many subscriptions as a way to put boundaries around cloud services for departments, teams or other reasons.

Microsoft has built a new service in Azure to help with the governance of your cloud. This new service is called Management Groups. Management Groups is still in preview but it is something I highly recommend you start trying out or using now as it is going to be as big for cloud as group policy was for on-premises AD based environments.

Management Groups sit above subscriptions. This allows Management Groups to be at the highest level in the chain so they can be used to effectively manage access, policies, and compliance for any subscriptions that belong to your organization. Within Management Groups you can set access controls (RBAC) and Azure policy to be applied to subscriptions. Subscriptions are organized in logical containers and the containers are the “management groups”. Your governance conditions are then applied to the management groups. This is the much-needed enterprise level type of management that has been needed in Azure for a while.

Management Groups will eventually become the starting point of governance when organizations embark on the cloud. Management Groups also can be used for organizations that are already in the cloud. I am going to dive into Management Groups giving you a high-level tour but first I need to give some more background on the components of Management Groups.

Each directory has a “root management group”. This root management group is at the top level of the management group hierarchy. All other management groups and subscriptions fold up to the root management group. Access and policies can be applied at the directory level via this root management group.

A couple of other things to note about management groups are that you can only have up to 10,000 management groups in a single directory, a management group tree can go six levels deep not including the root management group, and each management group can have multiple children management groups but only one parent management group.

Now let’s explore how I have structured my management groups to give some examples of how this works. Note that all the examples I show in this blog post are for my Azure environments but yours will be different based on many factors such as your organizational structure of departments, teams, etc.

You can find management groups under All Services>>Management Groups.

When you first access Management Groups you will need to create a root MG. Note that the root MG cant deleted or moved. You can rename the root MG. In the following screenshot, I am showing the creation of a sub MG in my root MG. Also, notice on the left-hand side you can set Access controls (RBAC) on this MG.

In order to configure Azure Policies and apply it to a management group, you do that within the Azure Policy itself. You can see in the following screenshot that I have an Azure policy and I am scoping it to the Prod01 MG. Whatever subscription/s and resources in those subscriptions will inherit the policy unless an exclusion is set in the policy or I am breaking inheritance at the resource group level.

In the following screenshot, I am showing the addition of an existing resource. The resources you can add are other MG’s or subscriptions.

In the following screenshot, you can see that I am going to add one of my subscriptions to my Dev01 management group. After doing this I can configure development related access and development related policies to this subscription. I also can do the same thing with my production environments/subscriptions.

Here is what my Management Groups hierarchy looks like:

In my hierarchy I have 3 subscriptions I split into two for production and 1 for development. I have created a root management group and placed all other management groups in it. I created a parent management group for my prod subscriptions and 1 for my development subscriptions in case I add more in the future. I then created a prod01 and prod02 pulling a subscription into each one. Doing this allows me to have separate access and policies per subscription. One thing you could do is pull multiple subscriptions into a single management group.

Note that I also could apply access and policies at the root level or at one of my environment management groups i.e. Prod_Env/Dev_Env and the sub-management groups would inherit the access and policies that are set at the environment management group level.

Also if you need to you can move management groups to a new parent management groups.

Thanks for reading this post. As I mentioned at the beginning of this post Azure Management Groups are currently in preview but they are worth checking out and potentially using now as these are going to become a critical part of the Azure governance story.

Read More