Cloud Service to Automate Windows Desktop Applications

Preface

When we need to automate some tasks which need desktop applications such as Microsoft Office, web browser, and other legacy software that requires a desktop environment, we're used to working with various types of techniques such as VBA, Macro, command script, power shell, and message handling with Win32, which can automate users' inputs.

Why Desktop Application Can't be Integrated in Server-side?

However, what if we need the same thing, but as a web service running in server-side for massive users, it would be tricky and even impossible to implement, because desktop applications are not suitable to run in server environment for two reasons.

Desktop Environment vs. Server Environment

First, desktop applications run only in desktop environment, mostly in Microsoft Windows, which uses native Win32 and GDI APIs. Most of the web servers including IIS don't support native APIs. It also requires a user credential, registry and storage to run properly. Web server can't provide such a broad but isolated environment for each client.

Memory Leaks, and Blue Screen

Next, even if web servers could offer desktop environment for each client, it's highly possible that desktop applications couldn't get 100% of the resource back to the system. We call it, "Memory Leak". It's due to the difference between desktop and server environment. Desktop application can be used if it has a small memory leak, because a user will finally shut down the computer. In server-side, however, application should be run repetitively for a long period of time. Applications won't return the same resource that they got from OS, and eventually OS will be out of resource for memory leak. For instance, if one application has 0.01% of memory leak not returning 0.01% of system resource every run, it will cause out of resource when it runs 10,000 times. (We used to have "blue screen" when we had such a situation in Windows.) In server environment, memory leak is critical for stability and durability of the service.

Main Idea

The main idea to establish a stable desktop automation web service first came up in my head when virtualization came up, which gives us crystal-clear virtual environment that could be rolled back, clone-able, and perfectly isolated. However, a scalability was still an issue how a service can run technically unlimited instances based on prospect requests. An isolated VM needs quite a lot of time to start and shutdown to return resources back to main server, and we can have only 8 active VMs as a maximum for each high-end profile server. If there are just maximum 200 active users requesting isolated automations at a time, it requires 25 high-end servers plus one main server, which require too much money to purchase and maintain 26 servers; even most of servers will not be active during service.

Service Structure

It needs to be isolated, and expandable

In proportion to a long introduction, to implement desktop automation service using cloud is pretty simple. It has mainly 2 layers.

Web Service Area

The first layer is a Web Service Area including Web Server, Communication Server, and Database. I use IIS 7.0 and MVC2 for Web Server, WCF (Windows Communication Foundation) for Communication Server, and SQL Server 2008 R2 for Database. It could be replaced by any kind of substitutions such as Apache, and MySQL. In this layer, every component is in the same network layer, so each can call each other. Web Server is, of cause, open to public network, and Communication Server is open only to cloud network using VPN.

Cloud Area

Another layer is a Cloud Area storing files and run desktop applications. It consists of Cloud Storage, Cloud Instance Controller, and Cloud Instance. In Cloud Instance, there is an application, Instance Runner, which is launched at startup and performs automation. I've used Amazon S3 for Cloud Storage and Amazon EC2 for Cloud Instance service which are parts of Amazon Cloud Service.

Network Structure

Cloud Storage is permitted to access only a web service area and in-network servers for security purpose. Cloud instances are not allowed to be accessed from an outbound network, but are permitted to access an internet from inbound using common TCP ports. Cloud Instances access Communication Server via VPN where only Cloud Instances are allowed to connect. In conclusion, Cloud Area is encapsulated and accessed only by Web Service Area, which is the only layer exposed to users.

Implementation

How It Works

Major procedures to run an automation service in cloud are four actions. The scenario is like this. A user visits a service website, registers as a new user, and then creates a project. Next, he uploads input files to Web Server. When he uploads all ingredients, he commands to run automation. A website keeps showing a status of instances that are reserved for his job. Once the job has been completed, instances will be terminated and return updated files to storage. Finally, a user can download updated files which were opened and modified from desktop application in desktop environment.

In scenario, it's pretty much simple and easy for a user, but a bit complicated for developers to develop, debug, and test the service, because it has multiple layers and deployments; developers should oversee every aspect that happens in different places communicating with different layers.

Excel Cloud Service

For more detail, I've implemented a prototype service that automates Microsoft Excel calculation in cloud desktop environment, and will share several challenges and tricks to develop such pattern of the service. Let's call this service as "ExcelCloudService(ECS)" for expedient.

Action I. Automation Design

In this action, a user creates a project for automation, and uploads files to be opened in automation. Most of the work is web front-end programming with membership service. I used built-in ASP.NET membership for authentication in MVC2 environment. It could be replaceable with PHP, JSP or other equivalents which are supported in AWS (Amazon Web Service) SDK. In ExcelCloudService, file security is top priority, so files are uploaded to web service via SSL and then uploaded to S3 bucket by web server. S3 bucket is closed to public and its location isn't exposed to any users, thus files can be accessible only though web server.

public static void SendFileToS3(string filename, Stream ImgStream)
       {              
            string accessKey = ConfigurationManager.AppSettings["AWSAccessKey"];
            string secretAccessKey = ConfigurationManager.AppSettings["AWSSecretKey"];
            string bucketName = ConfigurationManager.AppSettings["AWSS3OutputBucket"];
            string keyName = filename;
            AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client
				(accessKey, secretAccessKey);
            PutObjectRequest request = new PutObjectRequest();
            request.WithInputStream(ImgStream);
            request.WithBucketName(bucketName);
            request.WithKey(keyName);
            request.StorageClass = 
		S3StorageClass.ReducedRedundancy; //set storage to reduced redundancy
            try
            {
                client.PutObject(request);
            }
            catch (AmazonS3Exception ex)
            {
                throw ex;
                return;
            }
        }

Code 1. SendFileToS3

// GET: /ExcelCloud/Details/5
[Authorize]
public ActionResult Details(int id)
{
    if (Request.Files.Count > 0) // If files are uploaded
    {
        String savedFileName = "";
        var r = new List<ViewDataUploadFilesResult>();
        foreach (string file in Request.Files)
        {
            HttpPostedFileBase hpf = Request.Files[file] as HttpPostedFileBase;
            if (hpf.ContentLength == 0) // If file is empty
                continue;
            //Put file into directory named by id
            savedFileName = Convert.ToString(id) + "/" + Path.GetFileName(hpf.FileName); 
            try
            {
                SendFileToS3(savedFileName, hpf.InputStream);
            }
            catch(Exception e)
            {
                Debug.Fail(e.Message);
            }
            r.Add(new ViewDataUploadFilesResult()
            {
                Name = savedFileName,
                Length = hpf.ContentLength
            });
        }
        // Insert file info to database
        try
        {
            // Create entity class
            var entities = new EXCELCLOUDDBEntities();
            // Add ExcelCloudFile class into entity
            entities.ExcelCloudFiles.AddObject(new ExcelCloudFile() { 
                filename = Request.Files[0].FileName,
                projectid = id,
                description = Request["excelcloudform_description"], 
                S3path = savedFileName });
            entities.SaveChanges();
            // If status of selected project is zero (ready to run),
            if ((from files in entities.ExcelCloudFiles
                 where files.projectid == id
                 select files.status).FirstOrDefault() == 0)
            {
                // Run button info
                ViewData["RunURL"] = "/ExcelCloud/Run/" + id;
                // Project status
                ViewData["Status"] = 
			(int)(from projects in entities.ExcelCloudProjectDetails
                                  where projects.id == id
                                  select projects.status).FirstOrDefault();
            }
            // Query selected project and send it to view
            return View(from files in entities.ExcelCloudFiles
                        where files.projectid == id
                        orderby files.executionorder
                        select files);
        }
        catch
        {
            //Exception view here
            return View();
        }
    }
    else // If files are not uploaded, view a project detail
    {
        var entities = new EXCELCLOUDDBEntities();
        // If status of selected project is zero (ready to run),
        if ((from files in entities.ExcelCloudFiles
             where files.projectid == id
             select files.status).FirstOrDefault() == 0)
        {
            // Run button info
            ViewData["RunURL"] = "/ExcelCloud/Run/" + id;
            // Project status
            ViewData["Status"] = (int)(from projects in entities.ExcelCloudProjectDetails
                                       where projects.id == id
                                       select projects.status).FirstOrDefault();
        }
        // Query selected project and send it to view
        return View(from files in entities.ExcelCloudFiles
                    where files.projectid == id
                    orderby files.executionorder
                    select files);
    }
} 

Code 2. Upload Action

Action II. Request to run automation

When a user requests to run automation, Web Server requests new instance(s) on Amazon EC2.

// Create AWS EC2 Client object
AmazonEC2 ec2 = AWSClientFactory.CreateAmazonEC2Client(
        ConfigurationManager.AppSettings["AWSAccessKey"],
        ConfigurationManager.AppSettings["AWSSecretKey"]
        );
RunInstancesRequest ec2RIRequest = new RunInstancesRequest();
// Set instance type
ec2RIRequest.InstanceType = "t1.micro";
// Get AMI id
ec2RIRequest.ImageId = ConfigurationManager.AppSettings["AWSImageId"];
ec2RIRequest.MinCount = 1;
ec2RIRequest.MaxCount = 1;
// Set security group
ec2RIRequest.SecurityGroup.Add("DomusEC2Windows");
ec2RIRequest.KeyName = "domus";
RunningStatus model = new RunningStatus();
model.Messages = new List<string>();
RunInstancesResponse ec2RIResponse = new RunInstancesResponse();
try
{
    ec2RIResponse = ec2.RunInstances(ec2RIRequest);
}
catch (AmazonEC2Exception ex)
{
    // Put exception
    Debug.Fail(ex.Message);
}

Code 3. Request a new instance

Once a new instance finishes booting and is ready for automation, it should automatically pull files from S3 storage and run automation. This process needs several prerequisites accomplished on following states:

AMI(Amazon Machine Image) for instance should be designed and customized for automation.

Although Amazon EC2 provides various startup images, it needs to be customized for automation process.

For ECS, Microsoft Excel and COM automation objects are installed to automate Excel process. AWS SDK for .NET is also installed to access S3 storage. DotNet 4.0 Framework client profile is needed for Instance Runner to communicate with WCF Server.

Automation should access GUI¹

If you would consider a windows service as Instance Runner, you will have trouble getting access to GUI. There was an option, "Allow service to interact with desktop" to enable GUI access from service before Windows Vista, but it's blocked for recent Windows including Windows 2008 & R2. Also, some legacy applications would require a fully logon desktop environment providing user directories such as My Documents, temp and AppData. Finally, I decided to use a console application for Instance Runner, which is set as a startup app in Windows.

Instance should be set to login automatically as a user to get desktop environment²

Mostly, every server OS is set to wait for login on startup, but we need to customize AMI to logon automatically. To set it, open registry and locate the following subkey:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon

And add DefaultUserName string entry with your AMI user name, DefaultPassword string entry with password, and AutoAdminLogon string entry with value "1". In order to keep safe, you should input encrypted password in DefaultPassword³. There is a utility that set up auto logon with encrypted password⁴.

Instance should not be exposed to public

If EC2 instance is accessible to the public, it's critical for service, because it has AWS Access Key and Secret Key. When AMI for automation is ready to deploy, you need to block all outbound connections in firewall including RPC (Remote Procedure Call) and RDP (Remote Desktop Protocol).

Automation should get tasks from Communication Server identified by instance id

Once a new instance is created, we can get its instance id. ECS stores it into instance table with project id, so instance could get tasks of the right project. To get instance id in Instance Runner, send HTTP GET request as follows⁵:

WebClient wget = new WebClient();
byte[] instanceid = null;
try
{
    instanceid = wget.DownloadData("http://169.254.169.254/latest/meta-data/instance-id");
}
catch (Exception e)
{
    evt.WriteEntry("Error opening socket: "+e.Message, EventLogEntryType.Error);
}
if (instanceid == null)
{
    evt.WriteEntry("No instance id", EventLogEntryType.Error);
    return;
} 

Code 4. To get instance id

Finally, you will have two versions of AMIs; for development and for deployment. You can develop, debug, and upgrade legacy software or components in development AMI. Once everything is ready on AMI, you can build a deployment version by replicating and disabling outbound ports.

Action III. Execution

When an instance starts, it sends an initial request to Communication Server via WCF. Communication Server recognizes an instance by instance id, and then sends a list of files that should be calculated by Excel. To get a collection as DataMember, I created EntityCollection class which uses ICollection⁶. Recently, WCF has a feature having collection types as DataMember⁷.

[DataContract]
public class EntityCollection<EntityType> : ICollection<EntityType>
{
    #region Constructor
    public EntityCollection()
    {
        Entities = new List<EntityType>();
    }
    #endregion
    [DataMember]
    public int AdditionalProperty { get; set; }
    [DataMember]
    public List<EntityType> Entities { get; set; }
    #region ICollection<T> Members
    public void Add(EntityType item)
    {
        Entities.Add(item);
    }
    public void Clear()
    {
        this.Entities.Clear();
    }
    public bool Contains(EntityType item)
    {
        return Entities.Contains(item);
    }
    public void CopyTo(EntityType[] array, int arrayIndex)
    {
        this.Entities.CopyTo(array, arrayIndex);
    }
    public int Count
    {
        get
        {
            return this.Entities.Count;
        }
    }
    public bool IsReadOnly
    {
        get
        {
            return false;
        }
    }
    public bool Remove(EntityType item)
    {
        return this.Entities.Remove(item);
    }
    public EntityType this[int index]
    {
        get
        {
            return this.Entities[index];
        }
        set
        {
            this.Entities[index] = value;
        }
    }
    #endregion
    #region IEnumerable<T> Members
    public IEnumerator<EntityType> GetEnumerator()
    {
        return this.Entities.GetEnumerator();
    }
    #endregion
    #region IEnumerable Members
    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
        return this.Entities.GetEnumerator();
    }
    #endregion
} 

Code 5. EntityCollection for WCF response

Using collection type DataMember, PingHostResponse class returns a list of S3 paths with success value.

[DataContract]
public class PingHostResponse
{
    bool pResultSuccess;
    EntityCollection<string> pFileSequence;
    [DataMember]
    public bool ResultSuccess
    {
        get { return pResultSuccess; }
        set { pResultSuccess = value; }
    }
    [DataMember]
    public EntityCollection<string> FileSequence
    {
        get { return pFileSequence; }
        set { pFileSequence = value; }
    }
} 

Code 6. PingHostReponse

Instance Runner creates Office OLE automation object. You need to disable displaying dialogs that blocks automation processes by setting DisplayAlerts property.

Microsoft.Office.Interop.Excel.Application app;
try
{
    app = new Microsoft.Office.Interop.Excel.Application();
}
catch (Exception ex)
{
    evt.WriteEntry("Fail to start Excel object: " + ex.Message, EventLogEntryType.Error);
    return;
}
app.DisplayAlerts = false; 

Code 7. Create Office automation object

Files are stored into temporary directory. These files will be destroyed when an instance finishes its job and is terminated.

List<string> LocalFileSequence = new List<string>();
foreach (String filename in result.FileSequence.Entities)
{
    evt.WriteEntry("Start downloading a file : 
		" + filename, EventLogEntryType.Information);
    string tempfile = Path.Combine(Path.GetTempPath(), Path.GetFileName(filename));
    LocalFileSequence.Add(tempfile);
    try
    {
        GetFileFromS3(filename, tempfile);
    }
    catch (Exception ex)
    {
        evt.WriteEntry("Fail to download : " + ex.Message, EventLogEntryType.Error);
        return;
    }
} 

Code 8. Get files from S3

When an office automation COM object is closed, it is needed to clean up an interop object by calling ReleaseComObject method. Also, many articles recommend forcing garbage collecting in garbage collection even twice if you installed Visual Studio Tools for Office (VSTO).⁸

evt.WriteEntry("Closing excel object...", EventLogEntryType.Information);
try
{
    app.Quit();
}
catch (Exception ex)
{
    evt.WriteEntry("Fail to calculate : " + ex.Message, EventLogEntryType.Error);
    return;
}
Marshal.ReleaseComObject(app);
app = null;
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();

Code 9. Terminate automation

Instance Runner sends Complete message to Communication Server with its instance id, so Communication Server can mark the job completed and request a termination of the instance.

public void Complete(string value)
{
    var entity = new EXCELCLOUDDBEntities();
    EntityCollection<string> pFileSequence = new EntityCollection<string>();
    // If there is a record whose instance id is value,
    if (entity.ExcelCloudInstances.Any(i => i.instance == value))
    {
        var thisinstance = entity.ExcelCloudInstances.FirstOrDefault
				(i => i.instance == value);
        // Set status to complete
        thisinstance.status = 2;
        entity.SaveChanges();
    }
    AmazonEC2 ec2 = AWSClientFactory.CreateAmazonEC2Client(
            ConfigurationManager.AppSettings["AWSAccessKey"],
            ConfigurationManager.AppSettings["AWSSecretKey"]
            );
    // Create a terminate instance request
    TerminateInstancesRequest ec2TIRequest = new TerminateInstancesRequest();
    // Put instance id into request object
    ec2TIRequest.WithInstanceId(value);
    // Request termination
    TerminateInstancesResponse ec2TIResponse = ec2.TerminateInstances(ec2TIRequest);
} 

Code 10. Terminate instance

Action IV. To get results

Everything is ready to download, and a user can download calculated files on web browser. The only significance is that files that stored in S3 should not be directly accessible to a user for security reason. Web Server must grab files and toss it to the user. Finally, the only user who has sufficient permissions to download can get files from S3 storage. Any connection information of S3 should not be exposed to the public.

// GET: /ExcelCloud/Download/5
[Authorize]
public ActionResult Download(int id)
{
    var entities = new EXCELCLOUDDBEntities();
    var filerec = (from files in entities.ExcelCloudFileDetails
         where files.id == id
         select files).FirstOrDefault();
    if (filerec.owner == User.Identity.Name)
    {
        S3FileResult temp = new S3FileResult(filerec.S3path, filerec.filename);
        return temp.Result;
    }
    else return RedirectToAction("Details",filerec.projectid);
}  

Code 11. Download file

To return files from S3 to MVC FileStreamResult object, I create S3FileResult class wrapping S3 APIs and FileStreamResult class, so it downloads files from S3 bucket and store it into MemoryStream object (which doesn't require temporary files or directories) , and then return it as FileStreamResult object.

public class S3FileResult
{
    public FileStreamResult Result { get; set; }
    protected MemoryStream fs;
    public S3FileResult(string S3Path, string FileName)
    {
        string accessKey = ConfigurationManager.AppSettings["AWSAccessKey"];
        string secretAccessKey = ConfigurationManager.AppSettings["AWSSecretKey"];
        string bucketName = ConfigurationManager.AppSettings["AWSS3OutputBucket"];
        string keyName = S3Path;
        // Init Amazon S3 Client
        AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client
        (accessKey, secretAccessKey, new AmazonS3Config() 
        { CommunicationProtocol = Protocol.HTTP });
        GetObjectRequest request = new GetObjectRequest();
        request.WithBucketName(bucketName);
        request.WithKey(keyName);
        GetObjectResponse response;
        try
        {
            response = client.GetObject(request); // Request to get a file
        }
        catch (AmazonS3Exception ex)
        {
            // Exception here
            throw ex;
            return;
        }
        // Create a memory steram to route the file
        fs = new MemoryStream();
        byte[] data = new byte[32768];
        int byteread = 0;
        do
        {
            // Read stream
            byteread = response.ResponseStream.Read(data, 0, data.Length);
            // Write to memory stream
            fs.Write(data, 0, byteread);
        } while (byteread > 0);
        fs.Flush();
        // Init stream position
        fs.Position = 0;
        // Create a MVC FileStreamResult based on memory stream
        Result = new FileStreamResult(fs, "application/force-download") 
		{ FileDownloadName = FileName };
    }           
} 

Code 12. S3FileResult class for MVC2

Potential Exceptions

We've briefly overlooked a basic architecture of the cloud service to automate desktop application. It has a simple structure, but has relatively many layers, which could occur several of exceptions or security holes.

Interacting with Cloud Storage Service

When EC2 Instance Runner tries to get files from S3, it's needed to check if file downloads are successful. If it fails, it should return a fail sign to Communication Server, so Communication Server can kill the instance and show a fail message to a user. If result files are successfully uploaded to S3 but fail to download, it should say, "Cloud Storage is temporary unavailable".

EC2 Instance

Again, it is important that each instance should be isolated from public domain, because it has AWS Access Key and Secret Key. It is highly NOT recommended to let users upload their application onto instance, because it could be Trojan code that forces an instance be accessible to hackers. If you start a service that uses a 3^rd party application, you can install it on AMI. Even in Excel, macro could dominate a server and hack the system. ACL permission has to set up as strict as possible. Logon user should have limited access of administrative functions. It only needs credentials to run Instance Runner and automated application, and to access temporary work storage. However, it's inevitable that instance would expose its IP address with certain web-enabled application such as web browsers, so outbound connection should be strongly prohibited by firewall and network security. In addition to that, Instance Runner should monitor every action of automated application to see if it's freeze for some reasons. If so, Instance Runner needs to send a fail signal to Communication Server and start terminating the process. Thus, no matter what the process is terminated, Communication Server could terminate the mal-functioned instance and let a user and administrators know.

Conclusion

Finally, I'd like to mention things that I've not implemented in my prototype service. First, payment system is not yet implemented, because there is no decent API to figure out how much cost occurs per instance in Amazon AWS API. I can have working hours, so ballpark numbers, but it wouldn't be accurate. Devpay seems working only with AWS registered users. Next, I didn't implement variety of optional features; to select instance types, to monitor instance by screenshots and to broadcast a status of instance on Web Server via AJAX. I'll discuss in the next article if it's allowed. Also, new MVC3 has been announced, but I haven't got a chance to look through, yet. Maybe, I can publish MVC3 source code if it's possible. Our service is still in alpha phase, so I'll keep you posted in this article about any announcements. If you have any questions regarding cloud project, visit DomusDigital.com.

¹ http://msdn.microsoft.com/en-us/library/ms683502(v=vs.85).aspx
² http://support.microsoft.com/kb/324737
³ http://msdn.microsoft.com/en-us/library/ms995355.aspx
⁴ http://technet.microsoft.com/en-us/sysinternals/bb963905.aspx
⁵ https://forums.aws.amazon.com/thread.jspa?messageID=109571
⁶ http://social.msdn.microsoft.com/forums/en-US/wcf/thread/45dcd32f-605a-41df-ba63-8042b31a511d
⁷ http://msdn.microsoft.com/en-us/library/aa347850.aspx
⁸ http://stackoverflow.com/questions/158706/how-to-properly-clean-up-excel-interop-objects-in-c

History

• (03/01/2011) First version created
• (03/02/2011) Minor format fix