Display List of Uploaded Azure VM Role vhd images in a list

Thursday, May 19, 2011

I just posted a sample on MSDN for displaying the list of base vhd images uploaded to Azure Portal, for Virtual Machine (VM) Role.

You can get the list of uploaded vhds simply by running the following command in command prompt

csupload.exe Get-VMImage -Connection "SubscriptionId=xxxxxx-xxxxxx-xxxxxx-xxxxxx;CertificateThumbprint=xxxxxxxxxxxxxxxxxxx"

Thus making use of Get-VMImage switch of csupload you can get the complete details but the output is not presentable. It looks like something as below:

csupload get-vmimage output

To make this presentable we just simple call this process form our c# code, parse the output, save properties to a list of object and bind it to a gridview, to make it look like as follow:

csupload get-vmimage formatted output

Follow this link for details on how to achieve the same: http://code.msdn.microsoft.com/Displaying-list-of-upload-4957b5c8

csupload issue on 32 bit machines : Azure SDK 1.4 Fix

Thursday, March 10, 2011

Many of you might have encountered an issue while uploading your VM Role base image via csupload from a 32 bit machine.

The issue has now been resolved with the new release of Azure SDK 1.4. Now csupload can be used from x86 platforms too.

There are many more additions to the Azure SDK 1.4, mainly including Windows Azure COnnect and Content Delivery Netwrok (CDN).

Installing Tomcat in Windows Azure

Wednesday, March 2, 2011

There are few options already available to install Tomcat on Windows Azure, which involve running some scripts that create a package and definition file for you that you can deploy to Windows Azure. However, i personally feel that we have a much easier solution for installing Tomcat.

The solution that i am discussing here makes use of the startup tasks in elevated privileges, which were introduced in Azure SDK 1.3.
  1. Download and install jre on your local system. Zip the jre folder and upload it blob.
  2. Download tomcat on your local system.
  3. Edit tomcat’s server.xml to configure specific ports and enable SSL
  4. If you need to deploy any java .war file in your tomcat, then copy this war file in tomcat’s webapps folder.
  5. Zip the tomcat folder an upload it to blob.
  6. Now create a worker role and enable TCP ports configured for tomcat.
  7. Add startup task in this worker role that does the following tasks:
    • Unpack jre zip file to local drive on azure.
    • Unpack our customized tomcat zip file to local drive on azure.
    • Set environment variable for JRE_HOME to the path where JRE was unpacked.
    • Set environment variable for CATALINA_HOME to the path where tomcat was unpacked.
    • Start tomcat.

It shouldn’t be difficult for you to implement the above steps yourself, however, i seem to have plenty of time today so let me explain these steps too.

Prepare Java
It shouldn’t be difficult for you to download java from oracle/sun site. Just download it and install it. It will create a folder on your local machine e.g. jre6. You need to zip this folder and upload it to your blob storage. You can obviously upload it to any other host too, but since we are discussing about Azure, so let’s keep it that way only.

Prepare and configure Tomcat
Now download your desired version of tomcat from http://tomcat.apache.org/ and unzip this folder on your local machine.
  • Setup Ports in Server.xml
    You will have to select the ports you want your tomcat to run on. Say for http you want port 80 and for https you need port 443.

    Go to the conf folder inside your tomcat folder and open file server.xml

    Search for the following line and replace 8080 with 80 and 8443 with 443:

     8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" />

    After changing the ports this line will look like

    80" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="443" />

  • Enable SSL
    Again open the server.xml file and search for the following line:

    This will be commented by default. Uncomment it and change the port to 443. This line will look like as bellow:

    443" protocol="HTTP/1.1" SSLEnabled="true" maxThreads="150" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS"/>

    For SSL we use a .pfx file with tomcat instead of using normal keystore file. I will assume here that you already have .pfx file for PKCS12 certificate. This certificate needs to be added to your Cloud Service in the azure portal. Copy this .pfx file to a folder inside your tomcat folder, say under webapps. And make the following changes to the above line in server.xml:

    443" protocol="HTTP/1.1" SSLEnabled="true" maxThreads="150" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" keystoreFile="\webapps\myservice.cloudapp.net.pfx" keystorePass="" keystoreType="PKCS12"/>

  • Deploy war files if required
    If you have some war files that need to be deployed with tomcat, then jut copy them under the webapps folder of tomcat. Now when tomcat would be started it will install these applications.
  • Upload tomcat
    Once done zip your customised tomcat and upload it to blob.

Worker Role 
Now create a new Cloud Service Project in Visual Studio and add a new Worker role to it.

  • Add Certificate
    Upload the certificate that you used for tomcat’s SSL to the portal or include it in your worker project.   
  • Enable TCP Ports
    You need to enable the TCP ports in your worker role that you configured for your tomcat. In our case these are port 80 and 443.

    public override bool OnStart()
                // Set the maximum number of concurrent connections
                ServicePointManager.DefaultConnectionLimit = 12;

                // For information on handling configuration changes
                // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

                TcpListener port80Listener = new TcpListener(RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["Tcp80"].IPEndpoint);
                TcpListener sslListener = new TcpListener(RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["TcpSSL"].IPEndpoint);


                return base.OnStart();

  • Startup Tasks
    Then as we do for startup tasks create a startup.cmd file in your worker role, mark it’s Build Action Property as Content and Copy to Output Directory to Copy Always.

    Your startup.cmd file will have the following commands:
    • Download tomcat.zip and unzip
      We will use GetFiles.exe developed by us for downloading tomcat.zip. It is just a console app that takes first argument a blob url and second argument is the path to save file. Then using another customised utility ZipUtility.exe we will unzip tomcat.zip in the C: drive to a folder tomcat.

      start /w ZipUtility.exe C:\tomcat.zip C:\

    • Download jre.zip and unzip
      Similarly download and unzip jre.

      start /w ZipUtility.exe C:\jre6.zip C:\

    • Set Environment variables
      We need to setup two environment variables.
      JRE_HOME needs to be set to your jre folder, which in our case is C:\jre6

      SET JRE_HOME=C:\jre6

      CATALINA_HOME needs to be set to your tomcat folder, which in our case is C:\tomcat

      SET CATALINA_HOME=C:\tomcat

      Note: Please note that you can set environment variables in your service definition file also using the Runtime section as below:

              <Variable name="CATALINA_HOME" value="C:\tomcat"/>

    • Start tomcat


So startup.cmd will look like below:

GetFiles.exe and ZipUtility.exe are custom console apps. They are also added to the worker role project with their  Build Action Property as Content and Copy to Output Directory to Copy Always.

Final Step
That’s it just deploy your package to the cloud. Make sure your hosted service has the certificate that you used for tomcat’s SSL.


Changing Drive Letter of an Azure Drive (aka X-drive)

Monday, February 7, 2011

Sometimes it might be necessary that you want your Azure drive to be always mounted on a fixed drive letter. Consider a scenario of an Azure VM Role where you need to mount an azure drive for data persistance and your VM demands the same letter for you azure drive, e.q. you installed SQL Server on your VM Role and for mdf files you specified azure drive as path so as to make the data persist.

But now, we know that Azure drives are mounted on random-drive letters. To always have a fixed letter what you can do is that after your drive is mounted, you can change the drive letter to a fixed value using diskpart from within the windows service you use to mount the drive in VM Role, or from other part of code if you are not working with VM Role. Check this post to know how to mount Azure Drive in VM Role.

To get a basic idea on how to change drive letter using diskpart visit this Microsoft support link : http://support.microsoft.com/kb/928543

To change the drive letter of the mounted Azure Drive using diskpart, we will create a temporary file in local resource storage. This temp file will be used to store the current and target drive letters, and using this we can construct diskpart commands. Following code can be used to achieve the same:


//create temporary diskpart file
string diskpartFile = drive.CachePath + "\\diskpart.txt";

if (File.Exists(diskpartFile))

string driveLetter = drive.DriveLetter;
//start the process
using (Process changeletter = new Process())
changeletter.StartInfo.Arguments = "/s" + " " + diskpartFile;
changeletter.StartInfo.FileName = System.Environment.GetEnvironmentVariable("WINDIR") + "\\System32\\diskpart.exe";


Mounting Azure Drive in Azure Virtual Machine (VM) Role

Mounting an Azure Drive in Azure VM Role can be beneficial in many scenarios. As we all know that Azure VM Role is not persistent, so once you deploy a VM Role and it is restarted, all the data that was not the part of the base image is gone.

As we already know from a previous post that in a VM Role data is not persistent between restarts or hardware-failures, we need to identify a way to provide data persistence. As a solution we will keep the data that needs to be persisted on a mounted vhd, and then this vhd will be uploaded to a page blob. Once VM Role is deployed to Azure this page blob will be mounted as Azure Drive. I already talked about the need of Azure Drive in VM Role for data persistence in my other post on - "Data Persistance in Azure VM Role". Therefore, in this post we will focus on how to mount azure drive in VM Role.

As azure drives un-mount themselves in 30 minutes if the process that mounted is no longer alive, we will create a windows service that will mount the azure drive containing SQL data. This windows service would be setup to start automatically, and thus it will behave as a VM Role Adaptor. See this post to learn how to create VM Role Adaptor for startup tasks.

The windows service that we use will mount azure drive in it's OnStart method and would unmount it in its OnStop method. The code to mount and unmount is same as normal. To initialize cache you can specify any local path while mounting drive. The cache path and size can be configured in app.config of the service.

string cachePath = "C:\Resources\AzureDriveCache";
int cacheSize = 500;
CloudDrive.InitializeCache(cachePath, cacheSize + 20);    

Also, you must keep the following things in mind:

1. Make the Windows service to start by itself once installed, as described in a previous post regarding startup tasks in VM Role.
2. Set the Windows Service to Automatic(Delayed) start. The delayed start would ensure that VM Role has been done starting and loading all the changes it requires.
3. In the Recovery tab of the service set the service to Restart after all the failures.

Data Persitence in Azure VM Role

Azure VM Role involves creating a base image, uploading it to Azure using csupload, and then creating a servic model in Visual Studio to point to the uploaded base image.

Once Azure VM Role is deployed, it creates a new drive with letter D: and name it Resources. A shortcut to this drive is added to your C: drive (Windows Drive). Once you restart the machine the data on drive D: would be persited, however all the data or any software installed after VM ROle is deployed would be gone. Also, in case of system/hardware failures, even the data in drive D: will not persist.

But, what if you need to have data persistance in Azure VM Role. For example, you are installing SQL Server or may be Sharepoint on your VM Role. In this case you want your SQL Server's data files to persist. The option here would be to mount azure drive and this azure drive will hold your data files. Again, mounting an Azure drive is not a plain simple scenairo and it desribed here - Mounting Azure drive in VM Role.

Unable to connect to Azure VM Role for a long time after it is Deployed or Restarted

Saturday, January 22, 2011

I have been really frustrated in waiting for days to log in to my VM Role instance via RDP. I have encountered this issue multiple times. Once a VM Role is deployed or it is restarted, we can not login via RDP for a long period, and it takes a lot of time, sometimes 2-3 (random) days once it is back and allows us to connect via RDP.


This issue occurs due to a problem with the Azure Remote Forwarder Service, which runs in your VM Role instance. Due to this service sometimes RDP packets get distorted which cresults in connection timing out or closing it.


The solution I describe here is not really a solution, rather it is a workaround. The workaround is to set the Windows Azure Remote Forwarder Service from Automatic to Automatic – Delayed, by going to Service Management Console (Services.msc)

Startup Tasks in Azure Virtual Machine (VM ) Role

Friday, January 21, 2011

Since the release of Windows Azure SDK 1.3 we can now have startup tasks defined in the ServiceDefinition file for web/worker roles. However, this is not allowed for a Virtual Machine role. So what to do in case you need to have some tasks/scripts to be run on your VM Role’s startup? For example, your VM Role runs some software applications that require some configuration information that can be available only at run time.

So, how will you have startup tasks in a Virtual Machine Role? If you think that it’s just an easy task that involves adding you tasks in the Windows Startup folder (C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup), then you are mistaken. This is not going to work as the tasks in the startup folders are executed only when a user logs in to the machine, but this is not what you want. Your tasks should run whenever your role is started.

The solution for this is to use VM Role Adapter. This basically involves creating a Windows Service that is configured to Start Automatically when windows start. This is really simple and involves following steps:

1. Create a WindowsService project in Visual Studio, let’s call it VMRoleStartup and write the operation you want to be executed at VM Role’s startup in the OnStart() method.

2. Add the installer class to the service by opening service’s class in designer mode and right-clicking. Select “Add Installer” from the menu as shown below. This class is used to install the service.

Add Project installer to Windows Service
Add Project installer to Windows Service

3. This will create a new class ProjectInstaller.cs with serviceInstaller and serviceProcessInstaller.

Project Installer Class
Project Installer Class

4. Go the properties of serviceProcessInstaller and select Account as LocalSystem. This is required so that the service will run as an administrator account.

Service Process Installer Properties
Service Process Installer Properties

5. Go to the properties of the serviceInstaller class and make following changes:

    • Make sure that the Service name is same as that of your Service name
    • Add Description and DisplayName as you want to be displayed in Services Management Console.
    • You can set DelayedAutoStart to True, to make sure that your VM role comes into a stable state before your tasks are started.
    • Set StartType to Automatic.

Service Installer Properties
Service Installer Properties

6. In the ProjectInstaller.cs class add the following code to start the service after it is installed.

Code to start windows service after installation
Code to start windows service after installation

Please note that MountXService here refers to your service class name.

7. Build the application under Any CPU or x64 platform (because VM Role runs under x64 platform).

8. Copy the .exe file (along with dlls if required) to VM Role and install it using installutil by running it from command prompt as an administrator user.
installutil” is available in “C:\Windows\Microsoft.Net\Framework64\v4.0.30319” path.

Install windows service using installutil
Install windows service using installutil

9. Upload your image to Azure and create a service model for VM Role to connect to this image.

This is it, now you are done and when your VM Role starts your windows service will be started and will execute the code you put up in it's OnStart method.

Azure VM Role CSUPLOAD Error 0x80070070 Cannot prepare VHD

Wednesday, January 5, 2011

Today I was trying to upload my base image to Azure via csupload so that i can attach this vhd to one of my virtual machine (VM) role. However, I got a strange error with no details. I kept on looking for the reason and finally I found the solution.

The error was as below:
Unexpected Error 0x80070070
Cannot prepare VHD C:\Users\user1\Desktop\baseimage.vhd

The reason for this error as i discovered was that the disk from where it was uploaded was full. When you use csupload it uses temporary directory for preparing image and by default it uses the location from where you are uploading your base image. So if this disk is full you get this error.

Delete some files from your drive to have more available space or use a location on another drive for temporary purpose using the switch TempLocation 



2009 ·Techy Freak by TNB