Azure

Monitoring Azure PaaS

by

I recently had the opportunity to present at the annual SCOM/OMS Day held by the MN System Center user group. Here is a link to the past event https://mnscug.org/meetings/499-october-2017-mnscug-meeting. Other presenters during this event included Microsoft MVP Cameron Fuller, Microsoft MVP Bob Corenelissen, and Nathan Foreman, another Minnesota local. I chose to present on Monitoring Azure PaaS. In this blog post I will cover the information from my presentation and dive deeper into the topic.

Defining PaaS

Before you can monitor something you need a full understanding of what it is that you will be monitoring. Let’s start out by clarifying what PaaS is. There are many facets to cloud and the services that are available in cloud. You also can utilize public cloud, run your own private cloud or utilize a combination of the two known as hybrid cloud. Regardless if you have public, private, or hybrid cloud you can leverage Infrastructure as a Service (IaaS), Platform as a Service, and Software as a Service.  Below is an image that has been around for a while that visually explains the main differences between running your own data centers and utilizing cloud.

After viewing the previous image lets dive a little bit deeper into what it is explaining. When you run your own data center/s you are responsible for EVERYTHING all the way down to the networking and storage including monitoring all of that. As you move to the cloud you reduce your administrative overhead releasing that to the cloud vendor.

Most organizations first foray into cloud is to utilize IaaS. With IaaS you take a lift and shift approach of essentially running your existing servers and or new servers in cloud as virtual machines. At this layer you no longer have to worry about and manage the hypervisor, servers, physical storage, and physical networking. At the IaaS layer you still need to manage and monitor what is running on the servers that power workload and applications consisting of things like the OS, middleware, data and the applications. You also manage and monitor software defined storage and networking.

As organizations move to PaaS in cloud you release even more to the cloud vendor reducing even more administrative overhead. Also with PaaS the cost of the cloud services decreases. With PaaS you are responsible for the applications and data. You no longer need to worry about maintaining the administrative tasks of the applications, middleware or the OS.

Examples of some Azure PaaS services are Web Apps, Mobile Apps, API Apps, Media Services, CDN, Search, Event Hubs, Notification Hubs, Service Bus, Batch service, Azure AD, B2B/B2C, Azure DNS, Storage, SQL/MySQL/Postgres databases, CosmosDB, Service Fabric, IoT, Azure Functions, Logic Apps, Azure Container Service, Redis Cache, HD Insight, Key Vault, Azure Bot service, and much more.

Let’s zero in on SQL as a service in the cloud. With traditional SQL you had to properly scope and size the server properly, ensure you have enough storage space, split data, logs etc. After that you would need to plan and make SQL highly available, tune a SQL server for performance, maintain it and more. With PaaS the majority of this goes away. In fact with PaaS there is no SQL server/s to manage anymore. With PaaS when developers or anyone in IT need a SQL database they simply go spin it up. IT can still put controls in place such as policy and governance standards that are essentially boundaries that the consumer of the service needs to stay within however it is all self-service.

Now even though SQL databases can be spun up by consumers on their own and the SQL servers are managed by the cloud vendor (Microsoft). Now you would think in a cloud PaaS model you no longer need to monitor as there is no SQL server/s to administer. This is simply not true and we will get more into the monitoring aspect more later on in this post.

Applications running in Azure are typically made up of multiple PaaS services and sometimes a PaaS service itself will have dependencies on other PaaS services. An example of this can be seen in the following Application Map.  This shows that PaaS services have many moving parts across multiple parts and can be complex.

With PaaS components that make up applications it is important not to just monitor the components but also the application itself.

Why Monitor PaaS?

Most folks automatically think that they don’t need monitoring of PaaS because they assume without servers and high availability they don’t need to. This simply is not true. Below is a list of reasons of why it is important to monitor PaaS.

Overall when it comes to PaaS best practice is to move away from the old ways of thinking and methods for monitoring servers and on-premises infrastructure and move to a focus of monitoring the business applications.

Understanding the monitoring framework in Azure

Next up let’s take a look at the framework of monitoring in Azure. This will help you to better understand what is possible and how the monitoring tools plug into this framework. There are three main areas of data that is generated by Azure services that can be leveraged in monitoring. These sit across IaaS and PaaS services. These areas are:

  • Diagnostic
  • Logs emitted by an Azure resource that provide rich, frequent data about the operation of that resource.
  • Resource-level diagnostic logs require no agent and capture resource-specific data from the Azure platform itself.
  • Can send these to OMS Log Analytics, Event Hubs, or an Azure Storage account.

_______________________________

  • Metrics
  • Gain near real-time visibility into the performance and health of Azure workloads.
  • Performance counters are emitted by most Azure resources.

_______________________________

  • Activity Log
  • Insight into subscription-level events that have occurred in Azure.
  • Determine the ‘what, who, and when’ for any write operations (PUT, POST, DELETE) taken on an Azure resource in a subscription.
  • Categories of data: Administrative, Service Health, Alert, Autoscale, and Recommendation. (Policy, Security, and Resource Health coming…)

The types of monitoring data sit at different layers on IaaS and PaaS. On IaaS the application logs and metrics come directly out of the application. Diagnostic logging sits across the application and OS layer while metrics sit across the OS layer and VM layer. The activity logging sits at the Azure infrastructure layer.

On PaaS both the diagnostic logging and metrics come from the Azure resources directly. The activity logs again are at the Azure infrastructure layer.

With the diagnostic logs and metrics you can access and configure these via the Azure portal, PowerShell, Azure CLI and many have API.

Diagnostic logs can be sent to OMS log analytics, Event Hubs or Azure storage for other consumption. Metrics can also be sent to OMS log analytics, Event Hubs, Azure storage, and Application Insights. With Metrics you can also fire off alerts and autoscale a service. Alerts can kick off emails, webhooks, and Azure Automation runbooks. The following diagrams visually breakdown what can be done with metric and diagnostic log data.

Options for monitoring Azure PaaS

When it comes to monitoring PaaS Microsoft has many options available. There also are options available from a ton of 3rd party vendors. In this blog post I will only talk about the Microsoft options. Majority of the monitoring tools from Microsoft that can monitor PaaS are cloud based but you also can do some PaaS monitoring via System Center Operations Manager. The cloud options are much faster, easier to onboard and have been built from the ground up with cloud in mind. With Azure you also have out of the box monitoring capabilities on most of the Azure services. For example with a web app in Azure on the overview blade you can see things like data in and out and the Azure Response Time as shown in the following screenshot.

It is great that we get some monitoring out of the box for PaaS services, however this does not help when you are running hundreds+ of services. To handle enterprise scale monitoring of PaaS services you need to centralize the monitoring and that is where the monitoring solutions come in. Microsoft has 4 cloud based monitoring tools to help centralize your Azure monitoring. These tools are able to scale as needed without any hard limits. SCOM is a 5th monitoring tool that can monitor Azure. SCOM is on-premises only though. Here is a screenshot of the various tools minus SCOM:

Here is an example custom PaaS monitoring dashboard in Azure combining widgets from the various monitoring tools:

Now let’s dive into what each tool is and an example of when and how you would use them to help monitor Azure PaaS services.

Application Insights is a Application Performance Monitoring (APM) solution used to monitor applications all the way down to the code. Application Insights is typically used for web apps and other Azure PaaS services to detect, triage, and diagnose the root cause of issues. Application Insights gives you the ability to monitor many things about your applications such as availability, metrics like data coming in and out, dependency mappings through application map, performance data, and even live streams of data points. The following screenshot is an example of a web app in Application Insights.

The following screenshot is an example of an availability test summary chart in Application Insights. It is a ping test pointed to a URL. It gives you the % of the apps availability, the successful tests and failures.

With the availability ping test you have control over a bunch of options such as the frequency, success criteria, any needed alerts upon failures, and the ability to select the locations the test runs from.

Here is an Example use case for Application Insights:

  • Debug a multi-tier Azure .NET web application for errors and performance issues.
  • Utilize Application Map in Application Insights to discover visually which parts of the application are unhealthy. For the parts that are not healthy drill down using Application Insights to pinpoint the root cause of the errors.

OMS stands for Operations Management Suite. OMS is goes beyond just a tool that can be used for monitoring. It is a suite that also provides, backup, DR, automation and security. It extends to on-premises and it can monitor both IaaS and PaaS. OMS is a platform and has something called solutions. Solutions are used to extend the functionality of OMS. The solutions are packaged management scenarios. I am not going to list out or dive into all of the solutions available for OMS here. Solutions can be found directly in OMS or from the Azure Marketplace. There are a bunch of OMS solutions that can be used to help monitor and gain insight into your Azure PaaS services. The following screenshot has some of the PaaS related solutions that are available for OMS.

In the previous screenshot the OMS solutions with the white background can be found in the Azure Marketplace while the other OMS solutions will be found directly in OMS. More and more solutions are being added to OMS and the Azure Marketplace all the time.

Below is a screenshot of the Azure Web Apps Analytics OMS solution used to gain insight into an Azure web app/s.

Below is a screenshot of Azure Storage Analytics OMS solution from the Azure Marketplace used to monitor and gain insight into Azure storage.

OMS example use case for monitoring Azure PaaS:

  • Front end application can sometimes connect to a SQL database; and sometimes it cannot. Suspected cause is SQL timeout.
  • Utilize the Azure SQL Analytics to drill-down into SQL timeouts that have occurred on databases.

Azure Monitor provides a consolidated place for monitoring data from Azure services and base-level infrastructure metrics/logs from Azure services. It is typically used to track performance, security, and identify trends on Azure services. Azure Monitor brings (OMS) log analytics, application insights, and even network watcher into one place. Azure Monitor is still a relatively new service in Azure and it is still taking shape. Azure Monitor does offer some data that (Application Insights and OMS do not). The data you cannot get in OMS and Application Insights includes the history of Azure service issues, planned maintenance, health advisories, health history, and Azure activity logs.

An example use case for using Azure Monitor to help monitor Azure PaaS is:

  • Need a report of all services issues for a specific region for the past 3 months.
  • Utilize health history in Azure Monitor to pull a list of all service issues for a specific region from the past 3 months. This example can be seen in the following screenshot.

The following screenshot shows the following areas in Azure Monitor that have important Azure monitoring data.

Azure Monitor also has the ability to integrate with many 3rd party solutions that are used by DevOps folks today. The following screenshot is a group of 3rd party integrations that are available for Azure Monitor.

SCOM can be utilized if you want to monitor Azure resources from on-premises you can utilize SCOM for this. There is a management for Azure. There also is a SCOM management pack for Azure Stack. The SCOM management pack for Azure Stack is used to monitor Azure Stack’s fabric. In order to monitor Azure Stack’s IaaS and PaaS you would use the Azure management pack pointing it to your Azure Stack enviroment. The Azure management pack can monitor the availability and performance of Azure resources that are running on Microsoft Azure via Azure REST APIs.

Azure services that can be discovered and monitored with the Azure SCOM management pack.

Below is a diagram of how the health rolls up in the Azure SCOM management pack.

Where to get the Azure Management Packs

Azure Management Pack:

https://www.microsoft.com/en-us/download/details.aspx?id=50013

Azure Stack Management Pack:

https://www.microsoft.com/en-us/download/details.aspx?id=55184

But what about security?

This is where Azure Security Center comes into play. Security Center is a unified security management and advanced threat protection for workloads running in Azure, on-premises, and in other clouds.

Thanks for reading and stay tuned for more blogs on Azure and Azure Stack.

Read more

Azure Stack SQL RP – Need Azure PowerShell with version 1.2.9 Error

by

I ran into this error when installing the Azure Stack SQL RP on the Azure Stack Development Kit:

“Azure Powershell Module with 1.2.10 version found. Need Azure Powershell with version 1.2.9. Please uninstall the “current version and rerun the RP setup”

If you look at the SQL RP doc here:

https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-sql-resource-provider-deploy

It says “If you have installed any versions of the AzureRm or AzureStack PowerShell modules other than 1.2.9 or 1.2.10, you will be prompted to remove them or the install will not proceed. This includes versions 1.3 or greater.” on step #6 under Deploy the resource provider.

 

On my ASDK host I had:

and

The funny part is that in the SQL RP deployment script titled has a line where it installs AzureStack 1.2.10 but this is the version that the SQL RP deployment script is complaining about. Here is the syntax from the SQL deployment script.

# Installs and imports the API Version Profile required by Azure Stack into the current PowerShell session.

Use-AzureRmProfile -Profile 2017-03-09-profile

Install-Module -Name AzureStack -RequiredVersion 1.2.10 -Force

So the next thing I tried to do was run:

Get-Module -ListAvailable | where-Object {$_.Name -like “Azure*”} | Uninstall-Module

It kept throwing these warnings and errors:

WARNING: The version ‘1.0.4.4’ of module ‘Azure.Storage’ is currently in use. Retry the operation after closing the applications.

PackageManagement\Uninstall-Package : Module ‘Azure.Storage’ is in currently in use.

At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:2157 char:21

+ …        $null = PackageManagement\Uninstall-Package @PSBoundParameters

+                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (Microsoft.Power…ninstallPackage:UninstallPackage) [Uninstall-Package], Exception

    + FullyQualifiedErrorId : ModuleIsInUse,Uninstall-Package,Microsoft.PowerShell.PackageManagement.Cmdlets.UninstallPackage

So now I was stuck in this endless loop of PowerShell module uninstall and install hell. For a moment I thought I went insane. After recovering from temporary insanity. I ran this:

Get-InstalledModule -Name “AzureStack” -RequiredVersion 1.2.10 | Uninstall-Module

No errors on this. I then ran:

Get-Module  -ListAvailable | where-Object {$_.Name -like “Azure*”}

to see if the module was gone. Boom it was!

I then kicked off the SQL RP deployment script again and this time it worked!

NOTE: If you somehow have AzureRM version 1.2.10 just run Get-InstalledModule -Name “AzureRM” -RequiredVersion 1.2.10 | Uninstall-Module to get rid of that guy.

Read more

Azure Stack Development Kit (ASDK) Deployment Step by Step

by

At Microsoft Inspire Microsoft announced the Azure Stack Development Kit (ASDK) as a replacement to the POC and the general availability of the production Azure Stack named Azure Stack Integrated Systems. The Azure Stack Development Kit is here to stay. This will remain single node and should be used for trying out Azure Stack. You can develop your ARM templates and or applications on it and they will work on a production Azure Stack. The Azure Stack Integrated Systems are the ones that you buy from the OEM partners HP, Lenovo, Dell and soon to be Cisco, Avanade, and Huawei.

The ASDK install has improved 1,000 times over the previous TP’s of Azure Stack. I am going to detail the steps in this blog post. The steps start after you have downloaded the Azure Stack cloudBuilder.vhdx. Here we go:

PREPARE AZURE STACK HOST SERVER:

First off download the Azure Stack tools onto your Azure Stack host server. Just download all the tools as you will need all of them at some point. They can be found here: https://github.com/Azure/AzureStack-Tools

I put these in a folder on the C: drive named ASTools. I extract them and place them in the root.

Open up an elevated PowerShell window, navigate to your Astools folder and run the asdk-installer.ps1 script. Next a GUI wizard will pop-up.

Click on Prepare Environment.

Point it to your cloudBuilder.vhdx and click Next.

Put in the host servers local admin password. Make sure this matches the Azure account you plan to use.

Select the other options as you see fit.

It will run for a while creating the unattended file for Windows Server 2016.

Once it is done click Reboot now.

DEPLOY AZURE STACK DEVELOPMENT KIT:

Next lets deploy Azure Stack. After the server has rebooted log onto your AS server. Use the localhost\administrator account and the password you set.

Once again from PowerShell run asdk-installer.ps1. A GUI wizard will come up. Click on Install.

Select Azure Cloud (Azure Active Directory) or ADFS. Put in your directory and password.

Verify and select the correct NIC.

Select DHCP or put in your static IP settings.

It will verify the network settings.

You will see the PowerShell deployment script that will be run. Click on Deploy!

The PowerShell deployment will kick off in a PowerShell window.

After a little bit (1-2 minutes) an Azure login window will ask for your Azure account creds. This is the account ASDK will be deployed under.

NOTE: We still have the log folder and files under CloudDeployment on the C drive.

A few hours later and there it is successfully!

Having been involved with Azure Stack since TP1 and losing about a week to deploying Azure Stack TP1 this is a much….much better deployment experience. Nice work Microsoft Azure Stack team!!!

Read more

OMS and Cherwell ITSM Integration

by

Microsoft recently released public preview of OMS and ITSM integration. Here is the link for that announcement:

https://blogs.technet.microsoft.com/msoms/2017/05/11/it-service-management-connector-public-preview

Microsoft has built an ITSM connector in OMS. This new ITSM connector can connect to many ITSM solutions out there. The ITSM solutions it can connect to are:

  • System Center Service Manager (SCSM)
  • Cherwell
  • ServiceNow
  • Provance

This new ITSM connector is bi-directional meaning work items can flow from the ITSM solution into OMS and OMS can create work items in the ITSM solution such as incidents, alerts, and events. Hopefully in the future OMS could be used to populate a CMDB and even create application maps from OMS’s Service Map.

I wanted to give this a test run with a test Cherwell instance that I have. There is official documentation for the integrations. The documentation is good however after setting this up I did find that there could be a few more steps spelled out as well as screenshots with the Cherwell piece.

Needed settings from Cherwell:

Before you set the connection in OMS go and get the information you will need. So you will need a username and password of an account that has access to Cherwell, the Cherwell URL, and a Cherwell Client ID.

If you don’t know your Cherwell URL you can get this from the Cherwell client console. Launch Cherwell.

Before you login you can edit the connection to see the URL as shown in the screenshot. You will want to copy this to use in the OMS ITSM connector setup.

Note that you do not want to copy the entire URL. Only copy to the .com like https://test.demo.cherwell.com.

Next we need to generate the Client ID. The Client ID is basically a generated string called the client key used for connecting to Cherwell’s API. To get this client ID Launch the Cherwell Administrator console.

Login and click on Security and then Edit REST API client settings.

A window will pop up and you will need to click on the green plus to create a new one. Give it a name and copy out the Client Key.

Copy this as you will need it later.

Setup in OMS:

Next log into OMS and add the ITSM Connector preview. It is shown in the screenshot below.

After this has been added go to your OMS settings screen click on Connected Sources>ITSM Connector and then click on Add New Connection.

Select Cherwell for the connection type add in your Cherwell settings and save it. If everything worked your OMS is now connected to Cherwell.

Exploring the ITSM Connector:

Next let’s check things out in OMS. Before I did that I first went and created a new incident so I could see this flow over into OMS. So I created the following over in Cherwell:

After doing that I went back into OMS and kicked off a sync because I did not want to wait.

The connector picked up my new incident right away. You can see the dashboard ITSM tile has 2 incidents.

After clicking into this I am brought to the full ITSM dashboard. I then clicked on the Incident tile.

I was then brought to the incident dashboard and I could see the new incident I created.

I clicked on the new incident and it brought me to the OMS search with the details of the incident. Very cool!

I am excited to see cool stuff like this in OMS and integration with many ITSM tools. Look out for more blog posts in the future about ITSM Integration in OMS as well as in Azure Stack.

Read more

Speaking at OSCON and MMS May 2017

by

May is a busy month for me with the opportunity to speak at both OSCON – Open Source Convention (https://conferences.oreilly.com/oscon/oscon-tx) and MMS – Midwest Management Summit (https://mmsmoa.com)! OSCON is May 8th through the 11th in Austin Texas. MMS is May 15th through the 18th in Minnesota.

At OSCON I will be presenting on “How to Motivate Technical Employees” with friend and fellow Microsoft MVP Samuel Erskine – @samerskine. This will be on Thursday, May 11, 2017 at 4:15pm–4:55pm. This session is for CIOs, CTOs, IT directors, and IT managers and will cover how to retain your top talent and give you five ways to motivate technical employees. Come to this session to learn the secret sauce for keeping employee’s engaged!  Here is a link to the session: https://conferences.oreilly.com/oscon/oscon-tx/public/schedule/detail/57374

At MMS I will be presenting three sessions! These sessions are:

“Awesomize your Azure Stack Deployments with Azure Stack Tools” with fellow MVP Mikael Nystrom -@mikael_nystrom.

Link: https://mms2017.sched.com/event/AUae/awesomize-your-azure-stack-deployments-with-azure-stack-tools

“Azure Operationalized” with fellow MVP Natascia Heil – @NatasciaHeil.

Link: https://mms2017.sched.com/event/AUbn/azure-operationalized

“Backup is Dead! Restore is Born in the Cloud!” with fellow MVP Robert Hedblom -@RobertandDPM.

Link: https://mms2017.sched.com/event/AUaR/backup-is-dead-restore-is-born-in-the-cloud

These conferences will be lots of great community fun! Hope to see you there.

Read more

Sys Admin to Cloud Admin…ITSM to CloudOps…On-Prem to Azure Stack/Azure

by

A while back I posted a blog titled “Surviving the future of IT as an IT pro”. In that blog post I set out to share my opinion on where IT is headed and what you should focus on as an IT pro going forward. I guess this post could be considered part 2 however in this post I will focus more on where things are heading as a whole.

So what is this blog really about? It is about “the Transition from ITOPS & ITSM to CloudOps via Azure Stack (Hybrid Cloud) powering DevOps and becoming core to the Digital Transformation of business” that is happening. Whew…..Ok, a lot was said in that previous sentence. J Let’s break it down.

Transition from ITOPS & ITSM to CloudOps

There has been this transition in IT for a while to increase the density in data centers. This was started with the wide adoption of the hypervisor (VMWare, Hyper-V, Citrix Xen etc…). The goal is to get more out of existing and less physical hardware. Think about 1 physical server hosting hundreds of virtual servers. Things have since accelerated at a fast pace. We now have containers, PaaS, and serverless. With these newer technologies, the density is even greater.

The real power behind cloud is software defined everything. With software, defined environments AKA cloud a new skillet and a different way of thinking about managing operations is needed. This new skillset and new way of thinking for the operationalization of cloud is known as CloudOps. IT Operations and IT Service Management do not go away with CloudOps. The evolution of ITOPS and ITSM become CloudOps. The best parts of ITOPS and ITSM (ITIL) funnel into CloudOps used for operating clouds.

Hybrid Cloud (Azure Stack)

Hybrid Cloud is going to be a huge part of cloud initiatives in many organizations for years to come. You can see this on the Gartner reports(http://www.gartner.com/newsroom/id/3354117), Right Scale reports (http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2017-state-cloud-survey) and based on the investments the major cloud players are making to build the best Hybrid Cloud solutions.

Hybrid Cloud Is the Preferred Enterprise Strategy, but Private Cloud Adoption Fell

From Rightscale “Cloud Computing Trends: 2017 State of the Cloud Survey” Report:

http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2017-state-cloud-survey#hybrid-cloud

Recently IBM and Red Hat announced their launch into the Hybrid Cloud space.

(http://www.networkworld.com/article/3182989/cloud-computing/ibm-red-hat-an-open-source-hybrid-cloud.html)

A while back Amazon and VMWare announced their launch into the Hybrid Cloud space.

(http://www.businesswire.com/news/home/20161013006574/en/VMware-AWS-Announce-Hybrid-Cloud-Service-%E2%80%9CVMware>)

Microsoft was the first to jump into the Hybrid Cloud space and is the only company that has a 100% true Hybrid Cloud solution. Both VMWare/Amazon and IBM/Red Hat have solutions that run private cloud on public cloud. The private cloud solutions are being retrofitted to run in public cloud as the framework for their Hybrid Cloud solutions. These are not consistent cloud platforms running the same exact bits on bare metal on-premises and in the cloud like Microsoft’s Azure Stack solution. Azure Stack is the same bits in the public cloud and on-premises down to the bare metal.

IBM and Amazon jumping into the Hybrid Cloud space is more proof this will be a large area of growth in IT for years to come. I wonder if Google will decide to jump into the Hybrid Cloud space at some point and what their strategy will be.

DevOps powered by Azure Stack and CloudOps

Azure Stack serves as a catalyst to help move DevOps initiatives forward within organizations. With Azure Stack’s comes the native ability to run the environment using Infrastructure as code, continuous integration, continuous delivery, microservices, integration with source control systems, and more. All of the aforementioned are a part of DevOps.

Along with Azure Stack is the need to run the environment using a CloudOps model. Here is a list of concepts that drive CloudOps:

  • Extreme Hardware Standardization
  • Software Defined Everything
  • Extreme Automation
  • Focus on Zero Downtime
  • Self Service
  • Measured Service
  • Multitenancy

CloudOps is overall focused on business applications critical for running the business through the continuous operations of clouds. CloudOps leaves business unit projects to DevOps. CloudOps instead focuses on the delivery of the the cloud infrastructure to support self-service leveraged by DevOps teams.

David Armour of Microsoft often shares great information on CloudOps and what it means. You can follow him on twitter here: https://twitter.com/Darmour_MSFT

CloudOps supports DevOps and DevOps is core to Digital Transformation

Digital Transformation is the accelerating transformation of the way businesses do business from traditional ways often brick and mortar to the digital front through the use of digital technologies. Businesses are shifting to meet their customers and employees where they are today on digital platforms. In the business world, today it is well known that you must innovate and grow through the use of technology or become obsolete and left in the wake of disruptive companies that are leveraging technology to meet their customers on the digital front.

Examples of digitally transformed company’s vs non-digitally transformed companies are:

  • Netflix vs Blockbuster
  • Amazon vs Target, Best Buy, Macy’s
  • Airbnb vs Wyndom hotels
  • Uber vs Taxi Companies

Digital Transformation is critical to business and IT departments need to be a core driver to help organizations move forward on the digital transformation front. Digital Transformation is the new Industrial Revolution of business today with CloudOps/DevOps being the Assembly line that will bring innovation to the business.

Through DevOps businesses can bring digital services to the market at very fast rates and can pivot quickly as needed to beat and stay ahead of the competition meeting the customers’ demands in an agile way. CloudOps allows the scale and another point to pivot on at any time to redirect in a new direction as needed by the business in an agile manor.

Through a Hybrid Cloud solution like Azure Stack things IoT, Microservices, extreme automation, hyper-scale, and agility can be realized for the business empowering Digital Transformation from the core.

The transition of the IT Pro to Cloud Pro

Ok. That was a lot of information and background on CloudOps, DevOps, Digital Transformation and Hybrid Cloud. You may be asking yourself at this point where does the IT Pro fit into the picture? Let me answer that for you and take you on a tour of Azure Stack to prove why as an IT Pro you should start working with it today!

The path for an IT Professional when moving from traditional IT into a Hybrid Cloud world consists of:

  A cloud administrator can configure and manage resource providers, tenant offers, plans, services, quotas, and pricing.
A tenant purchases (or acquires) services that the service administrator offers. Tenants can provision, monitor, and manage services that they have subscribed to, such as Web Apps, Storage, and Virtual Machines.

Those cloud roles fit in a new world of CloudOps including Cloud architect, engineer, and administrator. Being a part of CloudOps requires a different mindset. Think about dynamic shifts such as software defined everything and extreme standardization. More concepts and technologies that a cloud role requires an understanding of are:

  • IaaS
  • PaaS
  • Software Defined Data Center technologies
  • Automation
  • Source Control Systems
  • Business Intelligence (Showback/Chargeback)
  • High Availability technologies
  • Backup and Disaster Recovery
  • Scaling technologies
  • Containerization
  • Server less technologies
  • Cloud Security
  • Both Linux and Windows
  • Self-Service (Service Catalog)
  • Multitenancy technologies
  • Tenant administration
  • And more

Ok. Now let’s jump into some example of CloudOps tooling in Azure Stack. First off, we as a cloud admin you need to know how to perform management of tenants (customers). Here is an example of a dashboard for doing this in Azure Stack:

In Azure Stack, you will need to know and understand the administration of managing the cloud itself. This includes many things some of them being management of a region/s, resource providers that contain the services you can offer up to tenants, along with monitoring, high availability, and backup of the cloud. Below is an example of administration in Azure Stack at the cloud model layer of CloudOps.

We already mentioned monitoring. There is monitoring of the cloud environment itself but there also is a need to monitor the resources being consumed by the tenants. One of the great things about Azure and Azure Stack is the out of the box monitoring and health diagnostics of IaaS virtual machines. I am a SCOM guy and have done a lot of SCOM projects. SCOM works well and serves a purpose but the out of the box monitoring in Azure and Azure Stack is amazing in the ease of turning it on. Once turned on it just works and has very nice visuals to see and work with as shown in the following screenshot. As a cloud administrator, you need technology to be easy so that you can move away from complex setups and troubleshooting the monitoring solution and move to monitoring the resources.

One of the best benefits about Hybrid Cloud is the consistency between public and on-premises cloud. In the following screenshot news updates on Azure and Azure stack are the same. 🙂 Another huge point of consistency between Azure and Azure Stack is the ability to view, deploy and run items from the Azure marketplace in Azure Stack. This is called marketplace syndication.

 

Azure

  

Azure Stack

Azure Stack is set to release in 2017. I want to highlight some of the services already in Azure Stack and more coming to Azure Stack that can be offered in your Service Catalog to tenants.

Already in Azure Stack as of TP3:

  • SQL PaaS
  • MySQL PaaS
  • Web Apps PaaS
  • Computer IaaS
  • Virtual Machines (Linux or Windows)
  • VM Scale Sets
  • Storage
  • Networking
  • PaaS: Storage
  • Key Vault
  • Management of Azure Pack virtual machines
  • Marketplace Syndication

Coming to Azure Stack at some point:

  • Microservices
  • Service Fabric
  • Cloud Foundry
  • Blockchain
  • Container Service
  • IoT

Another big part of CloudOps is being able to measured services that are being consumed. Measured Service can translate to show back or charge back. Measured Service is the ability to track the usage of resources down to the individual resource level. With Azure and Azure Stack resource management (ARM) model resources are carved out and placed into resource groups. In ARM, each resource has an associated cost that is tracked via the usage. There is full role based access around resources and resource groups. Resources and resource groups can be tagged and each resource or resource group’s usage can be tracked and displayed on business intelligence reporting or a dashboard like shown in the following screenshot.

That concludes this blog post. I hope I was able to shed some light on the transition from IT Pro to cloud pro, from IT Ops/ITSM to CloudOps and showcase the power of Hybrid Cloud via Azure Stack. Stay tuned for more exciting stuff coming from Azure Stack.

Read more

Azure Stack POC Hardware

by

I have been asked several times what I use for my Azure Stack rig and where I got the hardware from. I am going to share in this post what I use to run my single node Azure Stack POC. I bought all parts from newegg.com. Here is a list of the parts:

  • Motherboard: MSI X99A SLI KRAIT EDITION LGA 2011-v3 Intel X99 SATA 6Gb/s USB 3.1 USB 3.0 ATX Intel
  • Video Card: EVGA GeForce 210 DirectX 10.1 512-P3-1310-LR 512MB 32-Bit DDR3 PCI Express 2.0 x16 HDCP Ready Low Profile
  • Power Supply: EVGA 750 BQ 110-BQ-0750-V1 80+ BRONZE 750W Semi Modular Includes Power On Self Tester
  • Processor: Intel Core i7-5820K Haswell-E 6-Core 3.3 GHz LGA 2011-v3 140W BX80648I75820K Desktop
    NOTE: I was not paying attention when I bought this. Azure Stack needs 12 cores. I am able to work around this and have not run into problems yet. When I get a chance and $$$ I will upgrade this.
  • 3 SSD Hard Drives: PNY CS1311 2.5″ 960GB SATA-III (6 Gb/s) TLC Internal Solid State Drive (SSD) SSD7CS1311-960-RB
    NOTE: I bought a couple of more Kingston brand SSD’s. I use these for the OS and general storage.
  • Memory: G.SKILL Ripjaws 4 Series 128GB (8 x 16GB) 288-Pin DDR4 SDRAM DDR4 2800 (PC4 22400) Intel X99 Platform Extreme Performance Memory Model F4-2800C15Q2-128GRKD
  • Case: Corsair Carbide Series Air 540 CC-9011034-WLED Silver Steel ATX Cube Computer Case

As you can see this is generic hardware. The cost of this hardware was just over $2k USD. I have been running Azure Stack since TP1 on this hardware and I am currently running TP3. This is a personal lab for just me and Azure Stack runs well on my hardware. Don’t let a lack of hardware stop you from diving into Azure Stack. As you can see from this post it does not take much to pick up some parts and get going.

I do also run another Azure Stack POC on much better hardware at work. I can’t wait to get a multi-node environment on one of the hardware providers (Cisco, Dell, Lenovo, or HP) platform.

Here is what my rig looks like complete with Azure Stack and other stickers :-).

Before it was built:

After it was built and running:

Read more

Backup Strategy should include Security

by

Planning for protection as a part of an IT Service Continuity plan often takes into consideration backup of applications and data as well as restore. But what about security?

When planning for protection of applications and data in your environment security should right up there in the forefront. “Backup Security” should be a key part of the plan.

Security in the context of backup can be thought of #1 as securing the backups, and #2 backups being used as an added measure for security breach mitigation. Let me break this down further.

In regards to securing backups you want to do things like encrypt backup data as it travels offsite, encrypting backup data at rest, being able to protect encrypted data, requiring security pins or further authentication of admins and more.

In regards to backup as an added measure for security backup becomes a direct part of Security planning in organizations. Sometimes when security measures fail backups are the only thing that can save you as a last resort. Backups are commonly becoming a way to recover from ransomware attacks as an alternative to paying the hackers. Here is a real world example.

Recently an unnamed hosting providers entire data center became hostage to a ransomware attack. This hacker got in due to a mistake of one of the system admins (more on how to protect at this level later) and basically had full domain admin rights to everything. Keep in mind majority of the servers in this scenario are for customers.

In this case the hosting provider had two choices. Option #1 go to the dark web via a tor network and pay a ton of money in bitcoin for the decryption key. Option #2 Restore everything from offsite backups and pray.

This hosting provider went for option #2 and thank goodness it worked. In this case if it weren’t for a solid offsite backup solution this hosting provider would have been up a creek without a paddle.

It is becoming more common that ransomeware will actually target backups because these are a high target and hackers understand this is a last resort for companies to save themselves. If the backups are deleted there is no other choice but to pay the ransom. This raises the security level of the backups. Administrative actions on backups need an extra layer of security.

Microsoft Business Continuity products help with not only protection but also security. These products consist of System Centers Data Protection Manager (DPM) and Operations Management Suites Azure Backup (AB) and Azure Site Recovery (ASR). In this post I am only going to touch on DPM and AB.

Some exciting things have been happening with Azure Backup and Data Protection Manager to ensure security is front and center as a part of your enterprise backup solution. Microsoft’s goal with the backup security is to provide prevention, alerting, and recovery.

More about this including a video can be found here:
https://azure.microsoft.com/en-us/blog/azure-backup-security-feature

Just yesterday DPM update rollup 12 for 2012 and update rollup 2 for 2016 was announced. Along with UR2 comes some enhanced security features for DPM. These will be called out later in this blog post. Microsoft has rolled out some great security features to both across hybrid clouds. I will go ahead and break these down.

– Azure Backup –

Encrypted backup data at rest
Described in DPM section.

Security PIN
With Azure Backup you can require a security pin for sensitive operations such as removing protection, deleting data, or changing other settings in Azure Backup itself such as changing a Passphrase.

Azure Backup also has some other security measures in place like a minimum retention range to ensure a certain amount of backup data is always available and notifications upon critical operations to subscription admins or others as specified.

NOTE: These security features are now also available in DPM with the UR’s (UR 12 for 2012 and UR2 for 2016) announced yesterday. When an administrator changes the passphrase, or delete backup data, you need to enter the PIN if you have Enhanced Security Enabled. Also, there is a minimum retention range of 14 days for cloud protected data that is deleted.

MFA
MFA is Multi-Factor Authentication. Microsoft has MFA available as a part of Azure Active Directory. Within Azure Backup you can configure it to require MFA of admins when performing critical operations. By enabling MFA you would then ensure via authentication from a second device usually physical to the user that they are who they say they are.

NOTE: When you enable security settings they cannot be disabled.

Ransomware attacks
Described in DPM section.

– Data Protection Manager –

Backup data encrypted during offsite transfer
When data is sent from DPM to Azure Backup it is encrypted before it even leaves your four walls. Data is encrypted on the on-premises server/client/SCDPM machine using AES256 and the data is sent over a secure HTTPS link.

Encrypted backup data at rest
Once backup data is on Azure it is encrypted at rest. Microsoft does not decrypt the backup data at any point. The customer is the only one with the encryption key that can decrypt the backup data. If this key is lost not even Microsoft can decrypt your backup data. This is very secure.

Protection and recovery of encrypted computers
The release of Hyper-V on Windows Server 2016 included a new feature known as Shielded virtual machines (VM’s). This feature essentially utilizes Virtual Trusted Platform Module (vTPM) technology and BitLocker to encrypt a VM to encrypt virtual machines at the virtual layer. This means if a VM is physically copied off a Hyper-V host whoever has the VM will not be able to get to the data on the virtual hard drive.

With the release of DPM 2016 it supports protecting Shielded VM’s. DPM can protect Shielded VM’s regardless if they are VHD or VHDX. This is great news because as a secure organization you should want to encrypt your virtual machines and DPM can protect them. This gives you an added layer of security on top of having backups.

Ransomware attacks
In today’s world ransomware attacks are a common thing. These type of attacks are targeted at small, medium, and large enterprise businesses. No company is too small or too big to be put in the crosshairs of ransomware attacks. A well-known attack is Cryptolocker.

As mentioned before in this blog post backups are an alternative to paying the ransom of a ransomware attack. They key here is to ensure you have a solid offsite backup in place such as Azure Backup. Having that offsite backup will ensure you can get your data back even if the ransomware attack get ahold of your onsite backup data.

I even go as far as to recommend sticking to the 3-2-1 rule (3 copies of backup data 2 offsite and 1 onsite). This way if something happens to one of your offsite copies of data you have another one. It may seem overkill to have 2 offsite copies but you would be surprised how often offsite backup data is accidently destroyed.

So there you have it. Security is a critical part of any backup solution. It is clear that Microsoft realizes this based on the security enhancements they have made to both Azure Backup and Data Protection Manager 2016. Their goal is to ensure both backup solutions are enterprise ready. I have been working with DPM for years and Azure Backup as soon as it came out. I know the team behind these products have a lot of new features and functionality planned for the future of these products and I am looking forward to it.

Read more

Monitor Azure Stack Fabric with OMS

by

I wanted to monitor my Azure Stack environment with OMS. This would include only the Azure Stack fabric servers and the host. I did not want to manually install the OMS agent on all of these servers especially since the Azure Stack fabric is a set of known servers. So I decided to put together a quick PowerShell script to handle the install of the OMS agents including the workspace ID and key. Here are details for the script:

<#

.SYNOPSIS
This script can be used to install OMS agents on all of the Azure Stack Fabric servers. This has been tested with TP2.

.DESCRIPTION
This script can be used to install OMS agents on all of the Azure Stack Fabric servers. This has been tested with TP2. This script can be run from PowerShell ISE or a PowerShell console. It is recommended to run this from an elevated window. This script should be run from the Azure Stack host. Ensure you are logged onto the Azure Stack host as azurestack\azurestackadmin. This script allows you to input your OMS workspace ID and key. The Azure Stack Fabric servers that this script will attempt to install on is:

“MAS-Con01”,

“MAS-WAS01”,

“MAS-Xrp01”,

“MAS-SUS01”,

“MAS-ACS01”,

“MAS-CA01”,

“MAS-ADFS01”,

“MAS-ASql01”,

“MAS-Gwy01”,

“MAS-SLB01”,

“MAS-NC01”,

“MAS-BGPNAT01”

Fabric servers can be added or removed from the array list if desired. The script will look for the OMS agent (MMASetup-AMD64.exe) in C:\OMS\ on the Azure Stack host. Ensure you create an OMS folder on your Azure Stack host and download the OMS agent to it. This script also copies the OMS agent to C:\Windows\Temp on each Fabric server. Ensure there is enough free space on the C drive on all of your fabric servers.

.PARAMETER OMSWorkSpaceID
This is Guid ID for your OMS workspace, it can be found in the OMS portal at: https://mms.microsoft.com >> Overview >> Settings >> Connected Sources >> Windows Servers

.PARAMETER OMSKey
This is the OMS API key for your OMS workspace. You can use the primary or secondary key. These keys can be found in the OMS portal at:
https://mms.microsoft.com >> Overview >> Settings >> Connected Sources >> Windows Servers

.INPUTS
None

.OUTPUTS
None

.NOTES
Script Name: AzureStackFabrickOMSAgentInstall.ps1
Version: 1.0
Author: Cloud and Data Center Management MVP – Steve Buchanan
Website: www.buchatech.com
Creation Date: 1-1-2017
Purpose/Change: Install OMS agents on Azure Stack Fabric servers.
Updates: None

.EXAMPLE
.\AzureStackFabricOMSAgentInstall.ps1 -OMSWorkSpaceID “20d4dd92-53cf-41ff-99b0-7acb6c84beedsr” -OMSKey “aazedscsjwh52834u510350423tjjwgogh9w34thg2ui==”
#>

The script can be downloaded here:
https://gallery.technet.microsoft.com/Azure-Stack-Fabric-OMS-3dac666c

To kick off the script run from PowerShell ISE or a PowerShell console. If you run from ISE you will be prompted for the workspace ID and the key. If you run from a PowerShell console run this syntax to kick it off:

.\AzureStackFabricOMSAgentInstall.ps1 -OMSWorkSpaceID “YOURWORKSPACEID” -OMSKey “YOUROMSKEY”

The script will kick off, building an array of the Azure Stack VM’s, looping through each of them to copy over the OMS agent, and then install the OMS agent setting the OMS workspace ID and key.

The script will detect if an OMS agent is already installed and will skip that server as shown in the following screenshot.

Otherwise the script will install the OMS agent as shown in the following screenshot.

The following screenshot shows the script running in a PowerShell console vs ISE.

You will be prompted when running the script for credentials. Use Azurestack\azurestackadmin as shown in the following screenshot.

After the OMS agent is installed you should be able to log onto any of the Azure Stack VM’s and see the OMS agent in control panel as shown in the following screenshots.

You can also log onto OMS and see your Azure Stack servers listed under connected computers.

Azure Stack fabric servers wire data:

My Azure Stack host in OMS Service Map:

Happy Stacking and OMS’ing!

Read more

Azure or Azure Stack “Write Once, Deploy Anywhere” Update

by

A while back I wrote a blog post about being able to take one IaaS VM Azure Resource Manager (ARM) template and deploy it to both Azure or Azure Stack. This blog post included a JSON file and the PowerShell to do this. The idea for that came from needing to set up a cool and working demo for MMS 2016 and the need to showcase the power of Microsoft’s HybridCloud. Here is a link to that original blog post:

Write once, deploy anywhere (Azure or Azure Stack)

Today I have finished updating the PowerShell and ARM template/JSON file to be more streamlined and to work with TP2. Here is the link to download these:Here are the updates:

https://gallery.technet.microsoft.com/Create-VM-on-Azure-or-3c6d0420

Here are the updates:

  • The JSON and PowerShell script have been modified to work with Azure Stack TP2.
  • This script now utilizes the connection PowerShell module AzureStack.Connect.psm1 from Azure Stack tools.
  • This is included with the download of this script and JSON file on TechNet Gallery.
  • The script is hard coded to look locally to import the AzureStack.Connect.psm1 module.
  • Streamlined the JSON file and PowerShell script.
  • The script no longer prompts for the publicDNS name. It is now automatically set to the same as the vmname.
  • The script no longer prompts for the storage account name. It is automatically set to vmnamestorage.
  • The script no longer prompts for the resourcegroup name. This is now automatically set to vmname-RG.
  • By default this script now uses a JSON file hosted on Github. This is set in the $templateFilePath variable as shown on the next line.
  • To keep it to the local directory just use the JSON file name.

GITHUB: $templateFilePath = “https://raw.githubusercontent.com/Buchatech/Azure-AzureStackVM/master/AzureandAzureStack.json”
LOCAL: $templateFilePath = “AzureandAzureStack.json“

This will be my last blog post of 2016. See you next year folks…..

Happy Stacking!

Read more