Monitoring your Windows Azure Web application with the NEW Application Insights Tools for Visual Studio: Where to Start?

 

1 Introduction

Once your Web application is deployed into the Windows Azure platform, how would you do to monitor it? Do you have a Support and Operational team monitoring it 24/7 ??

Well, if you think about it 2 weeks before the market of your application, I am sorry this is too late!
ather, in a DevOps manner, you’ll have to consider it at the beginning in a project that runs in parallel of your development.

This document presents the NEW tools available, as per today, and available to you to monitor your application with a 360° vision:

- Visual Studio Online – Application Insights
- Windows Azure web portal – http://manage.WindowsAzure.com

It is a very long topic. Hence I decomposed it into smaller sub topics.

• PART 1:
• The prerequisites
• Once installed, what does it change inside Visual Studio 2013 ?
• PART 2: The configurations
• PART 3: The monitoring

 

Note: As early adopter of Application Insights I had the opportunity to present Application Insights within Azure back in November 2013 (http://blog.thavo.com/2013/11/i-have-not-put-here-all-my-talks-and-i.html). The official announcement of Application Insights tools for Visual Studio (preview), made by Brian Harry, is here: http://blogs.msdn.com/b/bharry/archive/2014/02/06/application-insights-visual-studio-add-in-preview.aspx

 

 

2 Pre-requisites before we start

2.1 Install “Application Insights Tools for Visual Studio”

2.1.1 Solution 1: Download From Visual Studio Online portal

2.1.2 Solution 2: From VS2013 “Extension and updates”

2.1.3 Solution 3: From the Web “Visual Studio Gallery”

http://visualstudiogallery.msdn.microsoft.com/82367b81-3f97-4de1-bbf1-eaf52ddc635a

2.1.4 Solution 4: Directly From the within “VSO - Application insights”

Depending on the workflow you choose (see next), you’ll have the opportunity to download it from within VSO Application Insights.

2.2 Installl confirmation

2.3 The Documentation of Application Insights tools for Visual Studio (Preview)

Here is a documentation that may or may not help (still in preview as we speak):

http://appanacdn.blob.core.windows.net/cdn/readme.htm

 

3 Once installed, what does it change inside Visual Studio 2013?

3.1 New option: when creating a new ASP.Net Web application

You could tick the check box “Add Application Insights to Project”

3.2 New option: when adding to an existing ASP.Net application

On an existing Web Application, you will have a new entry in the menu.

It’s going to look for your VSO credentials to see which projects are you allow to monitor.

It is really important to choose the proper Web project name, as the indicators will be collected for that specific project. It will create the mess if you don’t choose the proper one.

Then, VS is adding the necessary to your project.

3.3 New Item type in visual studio: ApplicationInsights.config

In your web application, you’ll have a new item called ApplicationInsights.config and few changes in your configuration files.

Each one of them is “just” a pointer to the web portal of VSO-ApplicationInsights.

 

(To be Continued !)

SaaS multi-tenant application on PaaS Windows Azure : Where to Start?

The PaaS, Platform as a Service, allows to you develop and deploy rapidly applications in an Agile manner. Compares to IaaS (Infrastructure as a Service) it is far cheaper to operate once in production, since most of the complexity “behind the hood” is taken care of by Microsoft.

 

ASSUMPTION: This Post deals with multi-tenant Web applications. However, to ease the understanding and go straight to the essential point, I do not enter on purpose into the details of the multi-tenancy on the database side (well not too much). The iterative approach below will alow to consider Database aspects at the end. 

 

For having done multiple migrations and worked on Windows Azure PaaS projects, I wanted to present a rule of thumb when starting implementing the Windows Azure PaaS in a multi-tenancy web application.

The strength of the PaaS Windows Azure is that one could take an existing Web application (ASP.Net, but other technologies are supported) and literally migrate and publish it into the PaaS within 1 or 2 hours.

However, I often remind people that this is ONLY the beginning of the adventure and that as long as the web application is not available in Production environment with happy customers paying for it, then the job is not DONE YET!!

So where to start, when implementing (or migrating) a multi-tenant Web application for the PaaS Windows Azure?

My approach is based in DevOps, and I always “start by the end” where the most difficult and riskier elements are, which is the Production environment. Yes it is going to increase the bill sooner than expected but you will find quicker bugs that way.

So let’s stick to Lean principles “Think Big, act small, fail fast; learn rapidly”.

THE BIG picture is here and also below, where you don’t want to just deploy Web Roles within a Cloud Service, but in production you will consider all the Windows Azure “ingredients”, and start implementing them like pealing an onion: from the outside up until the inside (i.e. create a Subscription, a Region, a storage account, … in this EXCACT order).

Source of the document on the left: Microsoft.

 

Now let’s “act small” in the steps below.

 

OBJECTIVE 1 : MAKE YOUR APPLICATION AVAILABLE in the Cloud

1. In Visual Studio, open the solution containing your Web Application, and add a new project. Choose Cloud > Windows Azure Cloud Service (install any Azure SDK if needed when prompted)
   
   Either Add e new WebRole (and Edit the name as shown in the image below) or leave empty if you already have a WebRole (i.e. in our case an ASP.Net csproj).
   
2. Now you have a Cloud project suffixed by “.Azure” to distinguish it from the Web project, as highlighted in blue on the top of the image below.
   
3. NOW the FIRST thing to configure in this Cloud project is to protect your Azure Package from any missing files (DLL, stylesheet, …). Edit the property of the Cloud project, and choose “Treat warnings as error: True”
   It is VERY VERY VERY (did I say VERY ??) important and will spare you many hours of trouble shooting.
   
4. Still on the Cloud project, right click to create an Azure deployment Package “Package…” (you could tick the “Enable Remote Desktop for all roles”). If your application is great!! CONGRATULATION a BIG STEP is already crossed!
   The package will be composed of 2 files:
   * Cloud Service Configuration File (it is a Zip file that you can unzip, which contains other files, including another Zip)
   * Service Package file
   
   
   TIP: to prevent the problem “Yes, but it works on my machine”, create those 2 files from time to time. Indeed, even if your application builds and works on your machine, the generation of an Azure package (particularly with getting those 2 files with the option “Treat warnings as error: True”) adds extra checks to prevent you unnecessary upload files, start provisioning, and wait for a crash after 15 min! (remember, fail fast! Will cost you less in time and frustrations)
5. Now use Visual Studio 2013 to Publish your Cloud application.
   Note : Il it IMPORTANT to let VS doing so when you begin, because VS will be doing A LOT for you behind the scene (including the handling of any required certificates such as the RDP, the SSL, the Azure Management Certificate, … that is to be considered next). Then, later, we will see that you’d better off doing it more manually or with Powershell to better understand the working principles.

 

OBJECTIVE 2 : ENSURE YOUR ALLICATION IS ROBUST in the Cloud

1. To enforce the massive elasticity of the PaaS, Windows Azure PaaS does not authorize Load Balancing with “Sticky Sessions”. Now to ensure you have a true “Stateless” application, you will need to configure at least 2 instances in your Web Role (meaning at least 2 VMs in your web farm, but Azure uses the terms instances and Web Role).
   To do so, it is on the properties of the WebRole within the Roles of the Cloud project.
   
   
2. Start requesting SSL certificates (you’ll need *.cer and *.pfx), since it usually takes a long time to do so
3. Start requesting DNS usually a wildcard *.myGreatSaaSapplication.net so that each customer could have his own one, and all pointing to your same WebRole:
   - Client1.myGreatSaaSapplication.net
   - Client2.myGreatSaaSapplication.net
   - Client3.myGreatSaaSapplication.net
   - FreeTrial.myGreatSaaSapplication.net
4. Manually test your application and play long enough to identify:
   * performance issues (particularly true when considering PaaS Database, called Azure SQL Database, since it is a mutually shared resource, which enables far cheaper costs)
   * Sessions issues
   * Security issues (can Client1 see information of Client 2 ?)
   * Cost (particularly Outband, since VM is a predictable resource)
   * Is the Error handling working fine ?
   * (in the future with your database : is the EntLib Retrial Policy doing his job ?)
5. With Tools: Identify with tools flows in your application
   1. Performance: Load testing the Web application (eg. VS 2013 Ultimate)
   2. “Volume” tests : Use a Tool to populate massive data (VS 2013 Ultimate or Red Gate)
      Used to test performance, and see how robust your application can be once you have insert random data (i.e. your application should handle properly exceptions, since you will certainly perform import / export into your database one day or another !)
   3. Security : Perform intrusion tests (eg. Qualys free scan)
6. Add a DEBUG page on your web application to check:
   * which instance of the WebRole is hit
   * (and later the connection string)

OBJECTIVE 3 : ENSURE YOUR ALLICATION CAN OPERATE in the Cloud

1. Since you only have ONE codebase to lower the cost, you’ll need to know which client is using which feature and can access to which option.
   Start implementing ASAP tools that allows to do so. For the solution InishTech is based on tags that you add in your code (on methods, web service, …) and once in production, your Product Team and Marketing Team could define at the very last moment, what package should a customer access.
2. Start configuring and implementing operational indicators present natively in Windows Azure and through Application Insights for Visual Studio Online.

   

 

This article will evolve continuously et iteratively, so please provide feedback

Create manually your own custom VM image for Windows Azure

1 Introduction

In order to create from scratch and have you own custom VM image available in Windows Azure, either :

- you are using Microsoft System Center and could perform the below tasks easily,

- or you could do it manually, i.e. go through a long process, yet relatively simple and intuitive.

This document presents the second alternative. By doing it manually, you will understand under the hood how Windows Azure IaaS is working.

 

Table of content:

1 Introduction
2 Creating a Hyper-V virtual machine
3 Converting the virtual machine for Azure
3.1 Snapshot your VM!!
3.2 Convert YOUR VHDX into vhd
3.3 Step 1: Prepare the image to be uploaded
3.4 Step 2: Create a storage account in Windows Azure and prepare WIndows Azure
3.4.1 Installing AND CONFIGURE Powershell for Windows Azure
4 UPLOADING the VHD into Windows Azure
5 Create your Windows Azure Image from the vhd your uploaded
6 Conclusion

 

2 Creating a Hyper-V virtual machine

Pre-requisites: In my case, I am using Windows 8.1, with Hyper-V activated.

Open your Hyper-V management console :

ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr

For the training VM ideally 8000 MB (8GB) : (from my computer, I can only put 3000 MB)

Choose Intel(R) 82579LM … so that Internet could be activated

You could also choose a smaller size such as 50 GB to speed up upload time.

Choose the ISO of the OS that is automatically be installed on the VM. In ou example, let consider Windows 2008 R2 SP1 Enterprise French

fr_windows_server_2008_r2_with_sp1_x64_dvd_617591.iso

Start the VM

The OS starts is installed automatically:

Click “Installez maintenant” > “Win Serv 2008 R2 version complète!“

Choose « Personnalisé (option avancée) »

Install on « HD 127 Go »

Set the password of the Admin

You could either disable all IE security rules or install Firefox. See appendices a direct link to get Firefox

 

3 Converting the virtual machine for Azure

The virtual Machine under Hyper-V (Windows 8.1) is composed of 3 files

ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr.vhdx (7 GB)

ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr_BB96011B-1119-462D-8EB1-6880C2EBF187.avhdx (1 GB)

ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr_CEE7D61D-AA09-4D55-A6CE-B9E2DE8BFB9A.avhdx (12 MB)

As per today, the format to be uploaded into Azure is *.vhd (and not *.vhdx). Hence a conversion is needed, on the top of sysprep.

Documentation here: http://www.windowsazure.com/en-us/documentation/articles/virtual-machines-create-upload-vhd-windows-server/

Once this vhd is prepared, it is called “image”

3.1 Snapshot your VM!!

In order to back it up, in case the SystPrep fails!

3.2 Convert YOUR VHDX into vhd

As we speak, Windows Azure IaaS only supports VHD (i.e. legacy and fixed size hard disk). You will want to convert your VHDX into VHD.

This could be done easily with Powershell: PS C:\> Convert-VHD –Path c:\test\MY-VM.vhdx –DestinationPath c:\test\MY-NEW-VM.vhd

C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks> Convert-VHD .\ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr.vhdx vmForAzure.vhd

Conversion in progress

It is only a matter of a few minutes job

Then renamed into ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr_ReadyForAzureImage.vhd

More information : https://blogs.technet.com/b/cbernier/archive/2013/08/29/converting-hyper-v-vhdx-to-vhd-file-formats-for-use-in-windows-azure.aspx

3.3 Step 1: Prepare the image to be uploaded

In order to declare your VHD as a Azure Drive, you will have to SysPrep it, with “Generalize” option.

 

3.4 Step 2: Create a storage account in Windows Azure and prepare Windows Azure

Prerequisites:

3.4.1 Installing AND CONFIGURE Powershell for Windows Azure

See here: http://www.windowsazure.com/en-us/documentation/articles/install-configure-powershell
Within a minute the install is complete.

Now you have Powershell for Azure installed, you will have to provide the proper authorization to your Powershell CommandLet to access your subscription.

To do so, just carry on following the steps documented in the previous URL,

which will open a web browser to ask you your Windows Azure credentials (your usual Microsoft Account Login and Password).

If you have many subscriptions, choose the proper one, which concerns the VHD you want to upload:

Now your Publish Setting file is being prepared for you and it is downloaded (bottom left : *.publishsettings):

Now import your Publish Settings credentials on your computer, so that the CmdLets will have the proper authorization to publish into your Azure Subscription.

 

4 UPLOADING the VHD into Windows Azure

Now you have your PublishSettings configured on your computer, you could start to upload your VHD.

PS C:\> Add-AzureVhd -Destination http://visbisrvsapnoeustoacc.blob.core.windows.net/vhds/VISBT-WinSrv2008R2Fr-Base001.vhd -LocalFilePath 'C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr_ReadyForAzureImage.vhd'

/!\ Consider the option “NumberOfUploaderThreads” for Add-AzureVhd with the 32 threads

PS C:\> Add-AzureVhd -Destination http://visbisrvsapnoeustoacc.blob.core.windows.net/vhdstrainingsapbo/VISBT-WinSrv2008R2Fr-Base002.vhd -LocalFilePath 'C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\ViseoBI-FormationSAP-BO-WinSrv2008R2SP1-fr_ReadyForAzureImage.vhd' -NumberOfUploaderThreads 32 But not necessarily faster (2 days !):

(PS: An alternative of the Powershell “Add-AzureVhd” is the CSUpload.exe. Doc: http://msdn.microsoft.com/en-us/library/windowsazure/gg466228.aspx )

Before the upload, the Powershell starts by calculating MD5 Hash:

(note the if you have optical fiber the Throughput could be 1535 Mbps, or even faster)

Within 3 min the MD5 Hash calculation is over.

Don’t’ worry if you don’t see manything going on in this process, the Powershell is doing its job behind the sceen, as we can see with Fiddler:

Now, after the message “creating new page blob of size”, it is time for THE upload, which is the most time consuming part of this process. i.e. you can relax and wait few hours (3 to 6 hours, depending on your bandwidth). Strangly enough, after about 10 min my Throughput drops to 0.1Mbps.

 

AFTER FEW HOURS, you can now see your VHD in Windows Azure portal, but still not usable yet, because “just” a VHD, not yet an image (because a VHD could either be an “Azure VM Image” or an “Azure VM disk”, depend on how you prepared it)

 

5 Create your Windows Azure Image from the vhd your uploaded

Time to convert your *.vhd into a Windows Azure Image!

Under “All Items”, choose “Vistual Machine”

NOW your image is re-usable a multiple time, such as any ‘regular’ Windows Azure IaaS VMs that is provided by Microsoft.

DONE!!!

 

6 Conclusion

Don’t forget, this is relatively a long process since we are doing it manually. Use tools such as System Center, or CmdLets of Redgate, or wait for the fortnight Windows Azure portal imrpovements so that this could ease your Admin life!