Elastic Beanstalk Apps v2

Introduction

This article describes the steps taken to configure my Elastic Beanstalk server deployment to completion with no manual intervention. This is yet one more step down the road to CI/CD.

Background

My latest assignment was to install several applications on a web server hosted on AWS Elastic Beanstalk. I create a zip file from my app in Visual Studio, and with some additions, install all other apps on the Elastic Beanstalk server cluster by just typing "deploy".

Using the Code

I had to host several apps on the same server, so I wrote another article here [which is possibly a shameless plug, but merely setting up prerequisites for my situation].

We know what happens when we deploy to EB, at a high level. We right click our web app in Visual Studio and click "Publish to Elastic Beanstalk". After entering some data, and click the final 'Publish' button, the app builds, gets packaged in a zip file, and is copied to c:\cfn\ebdata\source_bundle.zip on all instances in our cluster. The content is deployed in each of the instances.

Great. Now we have an app on the servers. My next step - deploy 1 to n additional apps on that server cluster. My primitive manner of doing it: I manually remote desktop to each instance, copy over the content of my other apps, and create the IIS applications and app pools from that content.

Hence the purpose of this article. I don't want to do anything to the individual instances. What if my cluster was 100 instances? I want my build server to run one command file which does it all, with no manual intervention. Using the Amazon tool "eb", we will automate those manual steps: copy content and run a command file to install that content.

Step: install ebtools, which involves installation of python, pip, and ebcli

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install-windows.html describes what to do. Eb tools is written in python so we first install python, and then use pip to download and install the package. After downloading and running the python installer, type python --version followed by pip --version to verify you have correctly installed. Then type pip install --user --upgrade awsebcli.

A little side trip I personally had to go through: after installing all this, I could not run eb. It turned out that my path variable did not get updated, so I ran control panel->system->environment variables, and added the directory eb resides in to the end of %path%.

C:\>dir eb.* /s/b
C:\Users\username\AppData\Roaming\Python\Python36\Scripts\eb
C:\Users\username\AppData\Roaming\Python\Python36\Scripts\eb.exe
set path=%path%;C:\Users\username\AppData\Roaming\Python\Python36\Scripts\

Now I can run eb, and it works.

Step: Configure aws. All aws commands are subject to your authorization. I can only update Amazon sites that I own.

C:\>aws configure
AWS Access Key ID [****************WR3W]:  I enter my big garbage string here
AWS Secret Access Key [****************96kh]: I enter my other big garbage string here
Default region name [us-east-2]:
Default output format []:

C:\>

As shown above, I interactively entered my menu choices. To be truly unattended, the input can be scripted as follows:

(echo big_garbage_string & echo big_garbage_string & 
echo us-east-2 &echo "" ) | "c:\Program Files\Amazon\AWSCLI\aws.exe" configure

Step: Configure eb tools. Suppose I'm a consultant, and I have 10 different clients, each having 10 different applications hosted on Elastic Beanstalk. Therefore I must tell eb what application to update, by doing eb init.

C:\eb init

Select a default region:

1. us-east-1 : US East (N. Virginia)
2. us-west-1 : US West (N. California)
3. us-west-2 : US West (Oregon)
4. eu-west-1 : EU (Ireland)
5. eu-central-1 : EU (Frankfurt)
6. ap-south-1 : Asia Pacific (Mumbai)
7. ap-southeast-1 : Asia Pacific (Singapore)
8. ap-southeast-2 : Asia Pacific (Sydney)
9. ap-northeast-1 : Asia Pacific (Tokyo)
10. ap-northeast-2 : Asia Pacific (Seoul)
11. sa-east-1 : South America (Sao Paulo)
12. cn-north-1 : China (Beijing)
13. us-east-2 : US East (Ohio)
14. ca-central-1 : Canada (Central)
15. eu-west-2 : EU (London)

(default is 3):1

Select an application to use:

1. Application_name_1
2. Application_name_2
3. Application_name_3
4. [ Create new Application ]

(default is 4): 1

Select the default environment.
You can change this later by typing "eb use [environment_name]".

1. Integration-Test-A
2. Applications-Dev

(default is 1): 2

Now any commands I issue with eb will be referring to the Applications-Dev environment in application Application_name_1, hosted in N Virginia. As shown above, I interactively entered my menu choices. To be truly unattended, the input can be scripted as follows. Note: If the menu choices change, like if a new application is added to the list, your numeric values may change. Perhaps a parsing utility will extract those menu choices in a later version. For now this works for me, with the clear expectation to improve in a later version.

C: \>(echo 1 & echo 1 & echo 2)|eb init

Step: In my web project, I use TFS for source control Not using git, I configure eb to use a local artifact [zip file]. When we completed eb init, we were immediately told the following.

Cannot setup CodeCommit because there is no Source Control setup, continuing with initialization

So we type the following command:

c:\>eb codesource local
Default set to use local sources

During the eb init process, a hidden directory ".elasticbeanstalk", containing a file named "config.yml" is created. It contains config choices we made. We will need to modify this file, appending 3 lines:

deploy:
  artifact: initial_deploy.zip

I said THREE lines. After the last line, [artifact: initial_deploy.zip], is a line containing a single space character; do not forget this important detail.
Reference: https://medium.com/@obezuk/deploying-git-submodules-to-elastic-beanstalk-981670817029

Step: Specify ebextensions

Amazon's documentation states that you can specify post install commands. Much experimentation and work with Amazon support has revealed this: make a directory in your project directory, as shown here in solution explorer, named ".ebextensions". It implies that it is hidden [normally, directories starting with a . are hidden; in this case, you do NOT need to have the directory hidden].

References

• [1] https://forums.aws.amazon.com/thread.jspa?threadID=137136
• [2] http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-windows-ec2.html
• [3] http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html

Example code in my file bootstrap.config:

container_commands:
  remove_hook:
    command:  powershell.exe -ExecutionPolicy Bypass -Command "C:\\cfn\\ebdata\\filename.ps1"
files:
  "C:\\cfn\\ebdata\\filename.ps1":
    content: |
      dir > C:\cfn\ebdata\filename.txt;

Upon completion of my app install, the code specified in the above block will execute. Surely this trivial example is not what you would want to do in your deployed app. What I did in my bootstrap config was to retrieve a large zip file from Amazon S3, unzip it into my target directories, and issue the IIS commands to create the web applications, referring to those target directories.

Step: Zip your deployable app

del initial_deploy.zip
cd initial_deploy
powershell.exe -nologo -noprofile -command "& 
{ Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]
::CreateFromDirectory('.', '..\initial_deploy.zip'); }"
copy /y c:\temp\initial_deploy.zip c:\that_dir_we_did_that_eb_codesource_local

What I do not show here is how I put the application into my directory "initial_deploy". Another future enhancement. Right now, I right click my web app in Visual Studio, click "publish to elastic beanstalk", and after it is complete, and copy that content that Amazon created in the temp directory. In time, I will be making this work in a single msbuild command.

Step: Deploy the zipped app

c:\>eb deploy Applications-Dev --staged --verbose

INFO: Deploying code to Applications-Dev in region us-east-1
INFO: Uploading archive to s3 location: AppName/app-170725_142636-stage-170725_142636.zip
Uploading: [##################################################] 100% Done...
INFO: Creating AppVersion app-170725_142636-stage-170725_142636
INFO: Environment update is starting.
INFO: Deploying new version to instance(s).
INFO: Waiting for current scaling activities to complete
INFO: Current scaling activities are complete
INFO: Instance 'i-0b99c77d19f9c1cfb' has been removed from your environment.
INFO: New instances found after commands have completed: i-0a962f04a2f1fc17f. 
      Re-running commands on these instances..
INFO: New application version was deployed to running EC2 instances.
INFO: Environment update completed successfully.

In the meantime, go to your Elastic Beanstalk console; you will see the state of your app turns from green to grey. As the install progresses, and finally completes on all 1 to N instances in your app cluster, the state turns back to green. Success.

Step: Visit your website

Step: What I did above was describe each step, and show what I typed. The NEXT step is to package all the above steps into a single deployer.cmd or deployer.ps1. After doing that, I can repeatedly update my content in the c:\temp\initial_deploy directory, and deploy it to my AWS cluster by simply typing "deployer.cmd".

Points of Interest

I look forward to evolving this into true ci / cd, and reporting my results. To describe scripted server installs, I'll invoke the catch phrase: "cattle; not pets".

History

• 2017/07/25: Initial revision