More details on Windows Azure deployment lifecycle ??

Beyond having this tiny window in Visual Studio 2010 (with Windows Azure SDK 1.6)

Anyone knows where we could get a detailed information on the Windows Azure deployment Lifecycle ??
Indeed, we have nice articles, such as:
http://msdn.microsoft.com/en-us/library/hh127476.aspx
http://msdn.microsoft.com/en-us/library/gg433030.aspx
but it there any diagram such as :


Note: In this example, the deployment time I provided are real, but based on a slow Internet connection (using a USB Dungle Internet)

Lesson learned: ALM with SQL Azure and new tools (SQL Server Data Tools and latest DACPAC Data-Tier Application project)

The number of tools, either in CodePlex, in CTP, or MS released, or Third-party to manage the Windows Azure are numerous. To add some more complexity, the very same tool (e.g. SQL Management Studio), allows you to perform the same action but using many possible ways. Thus the degree of freedom is huge. However, not ALL combination works or sometimes not documented at the moment, or you have to find a KB to understand why it does not work the way it should.
This post provides an easy way of migrating your on-premise database into SQL Azure, and use the latest Microsoft tools to have a better SQL ALM than ever before.
After having migrated your on-premise database, by completing the first 4 steps from my previous post (the other steps just provide a ways to check the integrity of your migration), you’ll need to create a DACPAC project. However, this time you’ll be using the following latest Microsoft tools:
* SQL Server Data Tools [free], that extend your Visual Studio 2010 capabilities  http://msdn.microsoft.com/en-us/data/gg427686 (check all pre-requisites, which could take up to 1 hour of install). You could consider using Visual Studio 11 Beta… but only for SQL Azure / DACPAC projects (Indeed, Windows Azure bit is another story with VS11 Beta).
This extension allows to work with *.sql file in a better way, as we will see later.

* SQL Server 2012 Management Studio [currently, get it SQL Server 2012 and uncheck everything, except Management Studio].
I would recommend not using SQL Management 2008 (unless you want to have some frustrations). indeed, with the latest version of Data-Tier Application projects it presents some minor limitations/ (bugs ? It’s a feature ? ) and its working principles has been ‘improved’ in “SQL Server 2012 Management Studio”.
To create a new type of Data-tier Application project, called “SQL Server Database project” within Visual Studio 2010, follow step 5 to 7 of my previous post mentioned above (as a reminder, you need to install SQL Server Data Tools).
You could also let “VS2010 with SQL Server Data Tools” migrate an existing VS2008 DB project:

In the new type of DB project’s property, you should target the “SQL Azure” platform, and set the Data-tier Application properties to a suitable version number (this number is used by the “intelligence” of DACPAC that is now handed by SQL Server 2008 R2 or 2012. We don’t deploy any longer “stupid” *.sql files).

It could be noted that by using this new type of DB project, you could decrease your amount of *.sql files considerably. For instance, going from 800 files to 100 files, depending on the complexity of your database. This is due to the fact that the new one is organized differently:
* No more huge amount of crazy sub-sub-sub-sub-folder and numerous files for every single index, constraints, etc… . Here is an example of a old VS2008 DB project applied to a very basic and small project (ASP.Net membership provider):

* Now, everything is nicely grouped into a more “SQL-server-like” fashion way. On the top panel below (1), you’ll find a visual SQL designer of a Table with its associated constraints and properties, and on the bottom pan (2), you’ll find the plain-text version like the old fashion way… except that your constraints, indexes, … are now part of the same *.sql file.
The gain in the number of files also means less files to get/check-in and track in terms of source control.

You could notice also few variations in the “Build” options on the *.sql file’s properties (3). However, unless setting additional deployment files to Build=”None” or “Build Extension Configuration” I did not manage to make it work properly (as in VS2008) when using related *.sql files called from a master *.sql file using the “:r” option.
With this type of project, by targeting “SQL Azure”, which represents the most constraining scenario (because your T-SQL has to be cleaned-up), you can publish either On-Premise or on SQL Azure using the very same T-SQL (you might add some IFs if you really need specific SQL instructions on SQL Azure).
As far as Publishing is concerned, you could define a configuration file per environment (e.g. click (1) to load various profiles, such as on-premise staging,on-premise validation, localhost, SQL Azure staging,  SQL Azure  PROD, …). To get the new publish window (notice it is not called “deploy” anymore), right click on your VS DB project > “Publish…”, somehow wait for 2 to 10 seconds before the modal pop-up appears:

Ensure you have checked the 2 options (2) to benefits from the latest DACPAC type of deployment, and simply click (3) [which also means you could deploy the old fashion way by configuring the numerous options on Advanced…].

During the publishing, you’ll see a new output window that keeps it nice and tidy as opposed to the previous plain text version. It reminds me the Publish for Windows Azure, … but in a more usable way ! (i.e. no tiny reports with a tiny scrollbar and when a bugs occur, you don’t know what happens). I like this new “Data Tools Operations” output window :
          - it has many “view details” links to open a full-size window when needed
          - it could be collapsed (such as Win Azure deployment)
          -…
. It also seams to deploy/publish and generate data a lot faster that its previous version (may be due to better caching).

Now if you want to deploys into SQL Azure, simply load the corresponding config file, then here you go, it’s on SQL Azure in 1 click ! Now you could use a command line to perform that so that a TFS-Build could automatically publish that for you (I would not recommend, since it will start to impact considerably you Windows Azure bill if you publish continuously 10 or 20 times a day for instance).
Example of deployment time a database with less that 100 MB data to be populated
- on local DB : 5 sec
- on SQL Azure (broadband): 4 min
- on SQL Azure (USB Dungle Internet, and no broadband): 15 min
The last new feature that I am going to present to really ease your ALM is the “Snapshots” possibilities. Before any Staging or Prod deployment, keep track of the exact deployment package that you delivered and stores it as a Snapshot. It will store you project as a DACPAC file within seconds (recall: a DACPAC file is a Zip file of you DB schema stored as XML files), which should then be stored into you favorite code repository. In TFS, for example, it could be a TFS-Release-Branch.

The new possibilities allows a huge gain in productivity !!!
Although I have been using those tools on large projects, I would be really interested in anyone having further practical feedbacks, or may be different usage.

[Azure] Migrating your SQL Server 2008 Database to SQL Azure

They are many ways you could do to migrate your SQL Server 2008 database (SSIS bulk copy, Data Sync, dedicated tools, …). Here is one of them, that was quite painless, and free of charge (at opposed of using commercial tools).
Because the Windows Azure world is moving really fast, this method might be already out of date as you are reading !

Pre-requisites:

• Get a Windows Azure account (currently you have 90 days free http://www.microsoft.com/france/windows-azure/Offres/Essai-90-jours.aspx)
• Get “Free Download Manger” application http://www.freedownloadmanager.org/ (if you have a proxy at work, you could use a direct link from sourceforge: http://sourceforge.net/projects/freedownload/files/Free%20Download%20Manager/FDM%203.8.1173/fdminst.exe/download)
• Download the TFS11 and VS11 (i.e. what will probably be VS2012 and TFS2012): http://blogs.msdn.com/b/briankel/archive/2011/09/16/visual-studio-11-application-lifecycle-management-virtual-machine-and-hands-on-labs-demo-scripts.aspx
  from Brian Keller’s blog, using Free Download Manger
• Install the SQL Azure Migration Wizard: http://sqlazuremw.codeplex.com/

How-to?

1. Let us assume you have VS 2010 and SQL Server 2008 R2, and you have an existing DB that you are happy with,
2. Install and use “SQL Azure Migration Wizard” tool to connect to your existing database. It will generate you as T-SQL Script will errors that would occur in SQL Azure (you could find all the infringement rules alongside this CodePlex project),
   
3. Modify your T-SQL accordingly to fix highlighted errors and generate a new database. Let call it DB_SQL_AZURE_COMPLIANT,
   
4. Migration DONE !! (either let the tool correct for you or do it your self). This tool could even push your database (without its data) directly into SQL Azure.
   The steps after are to enable you to work in a secure manner and ensure your project is still compatible with SQL Azure, and provide a link to show how to also push your data (using BACPAC),
   
   
5. With VS 2010, create (or use an existing) a SQL Server Data-tier Application
6. Reverse Engineer you SQL database into Visual Studio Database DAC project using "Import Data-tier Application...", and connect to your new DB_SQL_AZURE_COMPLIANT database
   
7. You will have a VS 2010 DB project that is able to generate DACPAC v2.0 deployment file (basically, it is a ZIP file that contains schema of you database, without any data. As opposed to BACPAC that is a extension of DACPAC, where a BACPAC also contains you data as JSON format). More info on DACPAC format: http://msdn.microsoft.com/en-us/library/ff719373.aspx
   
8. Now you have a DB project that is most likely be able to be deployed into SQL Azure (you can be certain only once you have migrated everything). You could verify that by running again the “SQL Azure Migration Wizard” tool and deploy into SQL Azure. You now should have no errors.
   
9. To double check it is really compliant, we are going to use our VM,
10. Mount your VM and configure your network card (so you could access it vis MSTSC)
11. Make a Snap shot of your VM (in case you want to rollback)
12. Copy/Paste you DAC project (created with VS2010) into the VM and open it with VS11, and confirm the you want your project to be migrated into a VS11 DB project,
13. Because you’ve done a great job correcting the T-SQL Errors, and you have great Stored procedures, views, … and no errors, you could activate ALL rules CodeAnalysis (below) with confidence !!
    Also, ensure you have a Data-tier Application (.dacpac file) checkbox is selected, and that the “Target platform” is “SQL Azure”.
    
14. Install Silverlight 5 add on the VM. This is because the existing one on the VM is old and will not support new SQL Azure web portal
    (generally, unless you know what you do, do not install SL5 on your DEV machine if you are working on SL4, else you would spend hours uninstalling. Indeed, it will break your SL4 dev environment)
15. Connect to your Windows Azure web portal, and go to “Database” (this is in Silverlight 4)
    
16. Click “Manage” this will open the Web portal to manage SQL Azure
    
17. The opens a new Web portal to manage SQL Azure (in Silverlight 5)
18. Now you could play with your newly database migrated into SQL Azure (type in plain T-SQL and run it). This web portal contains MANY MANY MANY great functionalities that you should discover ONLY progressively (because it looks nice, but a bit messy !!)
    
19. The final step is to populate your database with data (simply use a T-SQL script as a quick hack). Indeed, again DACPAC only deals with schema and for data, this is done using BACPAC: http://www.4tecture.ch/blog/sql-azure-import-export-service-is-now-in-production
    But this is another story, based on other Windows Azure platform components and further CodePlex projects ! 

By the way, with a large database, I never managed to make the Microsoft “SQL Azure Compatibility Assessment” tool working. http://www.microsoft.com/en-us/sqlazurelabs/labs/sqlassessment.aspx.It seams really promising, but it always tells me that my DACPAC is of an incorrect format, and that I should use a SQL Server Data Tools (CTP4) to generate it.
But how come ?? unless my version of SQL Server Data Tools installed in my VS11 is out of date ! In which case, I must again run after the latest tool and latest news concerning Windows Azure.

[TFS11 & VS11] NDepend to reverse Engineer Microsoft Fabrikam Fiber

1. Introduction
In order to practice our skills with the latest version of TFS 11 and VS 11, I decided to reverse engineer rapidly the standard Microsoft Fabrikam Fiber project.
Because I am a big fan of NDepend (often used as quality and technical audit tool), I will use this tool to understand more about Fabrikam Fiber project.

Usually, I use NDepend along a Software Factory as a quality gate on the top of FxCop and StyleCop (Vincent Labatut has got a CodePlex project to integrate easily NDepend to TFS 2010's Build definition).

2. Presentation of Fabrikam Fiber
In the context of TFS 11 and VS 11 usage, Fabrikam Fiber is a Microsoft Web Project that to be a pseudo real case projec. As such it allows us to understand new concepts and practice ALM 11 tutorials. The source code is split accross 3 branches in TFS 11 : Main / Dev / Release1
 

3. How is it organized ?
It's where NDepend comes into play. It will help us not only reverse engineer quickly the solution, but it will also present Microsoft's Best Practices coding conventions.

To enable NDepend analysing your VS11 Solution just edit the version inside the *.SLN:

3.1 Overview of the solution
You'll be able to zoom in / out with NDepend, as well as move or edit the Dependency Graph below:
 

3.2 Investigating on specific elements
Moving the mouse over the main web project FabrikamFiber.Web,

tells us with a color coded conventions how many DLLs this projects are connected to. In the same time, it also presents a property window with loads of information (on the right). Moving the mouse to another assembly will dynamically change those info. The test project (below) is using NUnit framework with mocking. Which means, we will be able to play with the new VS 11 to use many type of tests (and not only MSTests).

Let’s have a look at the NUnit tests. To do so, I can use another view called “Dependency Matrix”, where I can see that there’s 2 members coupled between nunit.framework and the test project:

Then, I can drill down to have more details, and get another Dependency Diagram that explains me the various relationships. And finally drill down to the source code.

4. Code Quality of the source code
Out of the 82 Code Metrix provided by default, I am presenting you one of them  below. It concerns performance issue of type “Boxing/unboxing methods that should be avoided” (you could create as many metrics as you want to suit your needs, using CQL language). It shows potentially 27 Performance issues.

• Pan (1) : The CQL query along with some advices
• Pan (2) : Where this query is classified into the SQL Query Explorer (you could create your own entry)
• Pan (3) : The number of quality infringements that satisfies the query
• Pan (4) : The 27 quality infringements are graphically filled in blue. The entire fractal representing your entire solution (decomposed graphically into Assemblies, namespace, … and methods). On mouse over the blue rectangles, you have more details.
• Pan (5) : The 27 quality infringements  listed
• Pan (6) : Details property
  

5. HTML report
Here is an overview of the HTML report. Similarly to the “heavy” client, those reports are filled with plenty of help you understand various quality concepts.

I don’t know why, but the HTML report is the only way to visualize an interesting diagram “Abscractness versus Instability”.

This graph places all your assemblies into coloured zones and along 2 axis. From my experience, anything that does not fal into the green circled area should be considered as a pain to maintain. If an assembly falls into a red zone, you should fire the developer straight away !

Which mean this code’s implementation seams good from this diagram. Then we should double check with other indicators.

Since it is unfortunately a static PNG image, in large projects all assemblies are superimposed in the circled zone. Hence it becomes hard / impossible to read. As a result, I just use this graph as a trend indicator. Assembly in Orange or Red zone should be seldom, therefore you’ll be able to read its name.

 

Now I got a better idea of what this project is about, I can deep dive into the source code, and coming back and forth to NDepend, then drill down to the code again.

[SQL Azure] Overview of tools for using SQL Azure {plenty of Silverlight Line Of Business}

This post is about presenting rapidly tools for creating and managing SQL Azure.

I am delighted to see how much effort Microsoft put into Silverlight LOB tools {cf. Silverlight is not dead}.

has invested a lot of effort into those tools. put into their tools for managing their Windows AZURE platform.

STEP 1. First of all let`s login to the Windows AZURE platform Web portal: https://windows.azure.com/

The entire interface is a Silverlight LOB application to manage a large amount of information and management operations  {notice that HTML has not been forgotten and identified in Green above when resizable Rich Text is required. See note at the end}.

STEP 2. Go to Database Tab and instantiate a new SQL AZURE server, and choose one of the 6 Datacenter the MS `offers` you:

North Central US – Chicago, Illinois South Central US – San Antonio, Texas North Europe – Amsterdam, Netherlands, West Europe – Dublin, Ireland East Asia – Hong Kong Southeast Asia – Singapore

If you are concerned with data regulation when storing your database somewhere in the Cloud, here is a list of the countries and cities` location.

STEP 3. Create a database with Admin accounts. In this example, I just create a small database {1 GB max}.

STEP 4. Administering tool. To administer your database, use another SaaS tool  provided by Microsoft: SQL Azure Management Portal.

Knowing that dhaigdshn8 is the name of my server, the address to administer it is located here: https://dhaigdshn8.database.windows.net

It is again a Silverlight LOB application:

Once logged in, you have here is the home page:

Create any databases that is required

STEP 5. Administer your database {Create tables, views, stored procedures, run commands…}. Notice the nice “Office ribbon”

STEP 6. Use your database.

Once setup, just use tools such as

. SQL Server Management Studio
. or Visual Studio {here under VS 2011}

STEP 7. That`s all, just enjoy your cloud database create in few seconds. How long would it take if you would have asked you IT service to create such database in production environment ?

=================

NOTE 1: For information, you could apply to be part of Beta Programs on AZURE as shown:

NOTE 2: We can notice that MS used the Blob storage to store html pages {just change fr by en for English version}.
https://azureportalprod.blob.core.windows.net/dashboard-fr/commontasks.html

Free Load test on AZURE to test the scalability of your Website

When seeking for information on AZURE, I found out another free tool on its basic version for testing Web scalability: CloudNetCare

However, if you want to test Silverlight applications, I suspect that he may need to combine other tools such as Selenium Silverlight {see below} and customize Scripts.

Super quick and easy to configure, it has a double feature:

· It is based on AZURE to increase its ability to inject load,

· It creates not HTTP requests to simulate the load {Virtual Users}, but with the ability to scale-out on Azure, it instantiates many and true web browsers.

 

 

As it`s quite tedious to copy / paste many URLs, which does not allow to simulate some GUI behaviors, an interesting variation is based on Selenium Scripts that are recorded via the GUI of Firefox. Note that there are many helps and videos, that are hidden by default, to accompany us all along the process.

Then, a very `cloud ` approach when setting up the tests is to choose for example the DataCenter that will be used, and at what time you want to start the load.

then the characteristics of the tests: 

`Pour un test de plus de 100 utilisateurs… = `For a test that has got more than 100 users…`
=> Since the purpose of such application is to load up until the maximum limit for your Web servers and determine the breaking point, CloudNetCare includes a protection:

As a result and unfortunately you will not be not possible to assign 5000 servers in the cloud and break down your favorite e-Commerce web site and thus be on the front page of every newspapers.

Finally, you can share the results either by CSV export, or by creating a Web link. Here is a link to my test report {well, I just changed the GUID of my real one}:  http://www.cloudnetcare.com/WF_ShareResult.aspx?id=7fc52303d17f465e8dc7195502c3238c&Language=FR .

 

Really interesting concept, and big thank you CloudNetCare to have given us the ability to use free of charge the basic version.

The only problem with the public cloud, it that it is more difficult to test my Intranet applications... I cannot complain, I cannot have everything ... free and as easily. Anyway, I am gladly adding this tool my ALM item list to make life easier and more agile.

So have a good Agility, and a great Scrum hopefully.

Technorati Tags: Azure Load Tests ALM