AWS: 6. Improving CloudFormation template

The previous post covered the AWS CloudFormation template that I developed to provision the environment to deploy the "ApiService" web service. The following diagram illustrates my journey so far.

AWS Services used and their interaction

Although it looks very simple I covered many AWS services including EC2 Config Service (the service to bootstrap a Windows instance).

In this post I plan to improve the AWS CloudFormation template by implementing the following features.

1. Further parameterisation 
2. Creating the requisite Identity and Access Management (IAM) role
3. Creating the Virtual Private Cloud (VPC) security group

Parameterisation

The new parameter section looks like the following.


I have parameterised the template so that the security key, availability zone, VPC and instance type can be determined at deployment time. I have also updated the template to resolve the Amazon Machine Image (AMI) through the "mapping" section. The mapping section follows a "dictionary" pattern where the key can be passed through using the intrinsic function "Fn::FindInMap".  See the following.

New IAM role

The purpose of the IAM role is to allow Elastic Compute (EC2) instance access to the Simple Storage Service (S3) bucket to download "ApiService" service binaries. In this particular case I am creating a role that has full access to S3 service. I have to admit that the syntax is not very intuitive.

The first step is to create the role. Thereafter an "Instance profile" resource needs to be created. From that I can gather, the Instance profile is an envelope that contains the role. This envelope is used to pass the role information to the EC2 instance.


Setting instance profile during EC2 provisioning.
The main benefit of the refined AWS CloudFormation template is that it creates the resources instead of using existing ones (e.g. security group and role). This can be very powerful because each stack can be created, and rolled back without leaving any residue.

The IAM role and Security group is created as part of the script and the only external resource I am depending on is the VPC. The VPC provides a personalised boundary over networking resources on the AWS platform and it is not something you should treat lightly. Normally there will be network engineers responsible for configuring and I doubt you will use an AWS CloudFormation template to provision a VPC (although it is totally possible, in fact I think we MUST to aid repeatability).

The updated AWS CloudFormation template is available here.

In the next post I plan to look at monitoring as it is something most developers leave to last. In my opinion monitoring must be a first class citizen of any solution design.

AWS: 5. Automating the deployment and provisioning using AWS CloudFormation Service

In the previous post I deployed the "ApiService" to the AWS Platform using the .NET SDK.
That was really powerful and a step towards automating the provisioning/deployment process.

If we take a step back and look at the deployment of an application to production, you do not see C# code being executed to provision the production environment. This begs the question whether there is another way to deploy our simple application. In this post I am going to attempt using the AWS CloudFormation service to provision the environment and deploy the application.

AWS CloudFormation service

The AWS CloudFormation uses a form of Domain Specific Language (DSL) to define an environment. The AWS CloudFormation service accepts a text file that defines the environment and provisions it as a "Stack".

The following points describes some of the key benefits of AWS CloudFormation that I see as extremely valuable.

• Automatic rollback when provisioning fails - my favorite!
• Developers are fully aware of the production environment. 
• Any change to the environment is managed through the CloudFormation template. (no random changes)
• Allows repeatability to provisioning; hence can move between regions.

There are tons more benefits of using AWS CloudFormation, and refer to the documentation to find out more information.

Provisioning and deploying using AWS CloudFormation

The first step is to create a CloudFormation template. The template uses the JavaScript Object Notation (JSON) format. You can use any text editor to create one and I used Visual Studio Code as it has fantastic support for JSON.

A CloudFormation template at a minimum must contain a "Resources" section. This section contains the resources that must be provisioned. The template I developed for the ApiService looks like the following.

I think the above section is pretty clear and can be read without knowing too many details of CloudFormation. The section describes an EC2 instance and sets some properties such as Amazon Machine Image (AMI), Security group etc. These properties can be mapped directly to the .NET SDK example that was in the previous post. You can even see the "UserData" (bootstrap script) being used to install the application.

I have used some functions such as "Fn::Base64", and in AWS lingo these are called intrinsic functions. The parameters to the functions are passed using the JavaScript array format ("[]").

Parameterisation

Although it is not necessary, I have parameterised the template so that some of the values are defined at deployment time. There is a special section for parameters which is called "Parameters" (surprise). The parameters section looks like the following:


I have allowed the AMI, availability zone and Key name to be defined at the deployment time. Normally parameters are used to define values that should not be stored in the template such as passwords.

There is another section called "Outputs", that can be used to display information such as service endpoint or anything else that is useful once the provisioning is complete. In this particular case I am displaying the service endpoint.

Using the template

I used the AWS Console to upload the CloudFormation template. Of course this can be done through the AWS CLI too. The Create Stack option needs to be selected from the AWS CloudFormation landing page.

Creating a New Stack using AWS CloudFormation service

The next step is to upload the CloudFormation template.

Uploading the CloudFormation template

The next screen brings the CloudFormation template into life! The values specified in the parameters section is set available. (See the following)

Setting Parameters in the template

At this point CloudFormation starts provisioning the environment.

Provisioning the ApiService environment

The "Events" tab contains a list of activities that is being performed by the AWS CloudFormation service. Once the provisioning and the application is deployed, the "Outputs" tabs is populated with the endpoint to the "ApiService".

"Outputs" tab with service endpoint

The "ApiService" is now fully operational.

Service fully operational

There is no doubt the AWS CouldFormation service is so powerful and I simply scratched the surface. In the next post, I am going to look at AWS CloudFormation in bit more detail and try to incorporate few more best practices. 

PS - The full template is available here.