ALTERthought Blogs

18 September 2009

Grails and Cloud Computing: Part 1 – Amazon EC2

Progress to Date

This entry is a part of the ‘What can you build in 40hrs‘ series– which, in a nutshell, involved an experiment to see what we could build in about typical weeks worth of work. In the last post we effectively completed the basic functional requirements of the application, and now have a working system that can be used to manage two sets of to-do lists as part of a task management application.

Cloud Computing

As part of this effort, we wanted to try something ‘new’ as part of the application initiative. We decided to make one of the system requirements of the application involve its deployment into a Cloud Computing infrastructure– the leading candidates at time of construction being Amazon’s Elastic Cloud Computing (EC2) and Google’s App Engine.

While both Amazon and Google offer similar infrastructure platforms, as it shall be seen, the level of effort necessary to deploy a stock Grails application into these infrastructures varies greatly.

This entry concerns itself with the deployment of the TwoDo application into Amazon’s EC2 infrastructure. A subsequent entry will detail the efforts necessary to get the same application running on Google’s AppEngine platform. (At the time I started this effort Google’s AppEngine had only recently released its Java API, and was dribbling out access to its infrastructure in a controlled fashion (FIFO?) As such, I did not initially have the rights to deploy an application to AppEngine– hence the decision to go with EC2 first.)

Cloud Tools Plugin

We are going to leverage the CloudTools Plugin for all the heavy lifting aspects of getting our application deployed into Amazon. Cloud Tools was developed by the founder of CloudFoundry– a service designed to deploy Java/Grails apps to Amazon’s EC2. I suppose I could have gone with the CloudFoundry as part of my deployment efforts (and indeed, will probably try it out) but for whatever reason chose to start at a lower level by using the plugin.

Getting Started

A prerequisite for using the CloudTools Plugin is to have an existing Amazon AWS account. The setup of which is easy, and well documented outside of this blog… so we aren’t going to concern ourselves with much of it here. If nothing else simply go to the main site and click the ‘Sign Up for Amazon EC2′ button

The first basic step was to install the plugin. I downloaded the plugin from the main CloudTools site and installed it from my local drive:

grails install-plugin /path/to/downloads/grails-cloud-tools-0.6.zip

The installation process creates a conf/CloudTools.groovy file which, in theory, could be simply added to the list of configuration files that Grails should slurp during boot time via an explicit specification in the conf/Config.groovy file like so:

...
grails.config.locations = ["classpath:CloudTools.groovy"]
..

However, I could never get this file to be located during the deployment process, so I eventually punted and copied the contents of the conf/CloudTools.groovy file directly into conf/Config.groovy

Once the contents of the conf/CloudTools.groovy file are pasted into the standard conf/Config.groovy file it needs to be edited with the specifics of our application.

Database Setup

The CloudTools plugin will create a MySql schema for us, as well as a user account used to access the DB as part of the deployment process. All we need to to is edit the parameters passed to the schema() method of the ClusterSpec() object that we pasted into our Config.groovy file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

cloudTools {
...
  clusterSpec = new ClusterSpec()
      .tomcats(1)
      .topology("SingleInstanceTopology")
       .instanceType(EC2InstanceType.SMALL)
       .slaves(0)
       .bucketPrefix(System.getProperty("grails.env"))
       .catalinaOptsBuilder("""{builder, databasePrivateDnsName, slaves ->
            builder.arg("-Xmx500m")
	    builder.prop("com.sun.management.jmxremote")
	    builder.prop("com.sun.management.jmxremote.port", 8091)
	    builder.prop("com.sun.management.jmxremote.authenticate", false)
	    builder.prop("com.sun.management.jmxremote.ssl", false)
	    builder.prop("ptrack.application.environment", "ec2")
	    builder.prop("log4j.configuration", "log4j-minimal.properties")
	    builder.prop("jdbc.db.server", databasePrivateDnsName)}""")
        .schema("blogtwodo", ["twodouser": "password"], [])

On line 18 above we configure the ClusterSpec to create the ‘blogtwo‘ database and the ‘twodouser‘, the specifications of which should match the contents of the conf/DataSource.groovy file.

CloudTools AWS Properties File

We need to indicate the location of our Amazon Web Service property file, which is set by editing the value for the awsPropertiesFile property within the conf/Config.groovy file.

...
cloudTools {
  awsPropertiesFile = "etc/ec2.properties"  ...
}

As you can see we have created a new grails-app/etc directory to hold this new file. The properties file is a CloudTools specific artifact that contains the relevant configuration information required by Amazon to host our application. In order to deploy into Amazon’s platform we are going to need provide several bits of information by way of this properties file. Let’s take a look at the contents of ours and dissect it line-by-line:

1
2
3
4
5
6
7
8
9

imageId.m1.small=ami-6f2cc906
imageId.m1.large=ami-0129cc68
imageId.m1.xlarge=ami-0129cc68
accountId=XXXX-XXXX-XXXX
accessKey=XXXXXXXXXXXXXXXXXXX
secretKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
keyName=blogtwodo
keyPairFile=/Users/don/DevelopmentWork/blogtwodo/etc/blogtwodo.pem
sshDir=/usr/bin

AMI Identification
An application deployed into the EC2 will reside in Amazon Machine Image (AMI). Think of this as a identifier for the base OS image/VPS. In our case (lines 1-3) we will simply use the ones specified by the plugin documentation:

1
2
3

imageId.m1.small=ami-6f2cc906
imageId.m1.large=ami-0129cc68
imageId.m1.xlarge=ami-0129cc68

Security Credentials
The next few lines contain part of the security credentials we need to provide to Amazon. From within the Amazon Web Services Console, we navigate to ‘Your Account -> Security Credentials‘. The accountId (line 4) is your 12-digit account displayed on this page under the ‘Your AWS Account ID, E-mail Address and Password‘ section.

While within the ‘Access Keys‘ tab, I clicked ‘Create a New access Key‘ and used the resulting generated information to fill in the accessKey (line 5) and secretKey (line 6) properties.

4
5
6

accountId=XXXX-XXXX-XXXX
accessKey=XXXXXXXXXXXXXXXXXXX
secretKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Amazon’s site does a good job describing the purpose of the above access credentials:

Access Key ID
Your Access Key ID identifies you as the party responsible for service requests. Use your Access Key ID as the value of the AWSAccessKeyId parameter in requests you send to Amazon Web Services (when required).

Secret Access Key
Each Access Key ID has a Secret Access Key associated with it. Use your Secret Access Key to calculate a signature to include in requests to web services that require authenticated requests. Your Secret Access Key is a secret, and should be known only by you and AWS. You should never include your Secret Access Key in your requests to AWS. You should never e-mail your Secret Access Key to anyone. It is important to keep your Secret Access Key confidential to protect your account.

Key Pair
In addition to the above credentials we need to provide a public/private key pair in the form of an PEM file. Keys are created from within the EC2 dashboard and I created a new keypair with the application name of ‘blogtwodo’ by logging into the ‘Amazon EC2 dashboard -> Key Pairs‘ and clicking the ‘Create Key Pair‘ link. The resulting blogtwodo.pem file was saved locally and moved into the grails-app/etc directory, where its configured by lines 7 and 8 below:

7
8

keyName=blogtwodo
keyPairFile=/Users/don/DevelopmentWork/blogtwodo/etc/blogtwodo.pem

Finally we need to indicate the path to an ssh client (line 9) to be used to securely upload our application to Amazon. I am using OSX on a Mac which includes this natively in the /usr/bin directory:

9

sshDir=/usr/bin

Deployment

OK. With our configuration out of the way lets try and deploy this thing. From the command line we invoke the gant deployment target added by the CloudTools plugin:

grails cloud-tools-deploy

… which promptly reports the following error:

Error executing script CloudToolsDeploy: No such property: DEFAULT_J5_DEPS for class: ExplodedWar_groovy

Whats the deal? Well, the CloudTools plugin was developed pre Grails v1.1.1, as it turns out there are a few incompatibilities that need to be addressed in order to get our application deployed.

This error is addressed by adding the following highlighted code (line 10) to the $GRAILS_HOME/scripts/_GrailsWar.groovy

1
2
3
4
5
6
7
8

...
DEFAULT_DEPS = [
    "ant-*.jar",
     ...
     ...
]
 
DEFAULT_J5_DEPS = DEFAULT_DEPS

OK. Lets try to deploy again….. this time, although we get farther, we eventually fail with the error:

ssh: connect to host ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com port 22: Operation timed out

Drat. Now what? After a little digging it’s determined that the problem is that we have not allowed the SSH protocol through the built in firewall associated with our application. This is fixed by navigating from within the EC Dashboard to ‘Security Groups -> default group’ and adding the SSH protocol to the table of connection methods. Let’s try our deployment again.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/Users/don/DevelopmentWork/blogtwodo/etc/blogtwodo.pem' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.

Sigh. OK. Trying again after chmod‘ing the PEM file to 400 (chmod 400 blogtwodo.pem).

After several minutes (and I do mean several.. deployment took close to 10 minutes) this time we get:

...
...
Location: http://localhost/blogtwodo/ [following]
--10:34:55--  http://localhost/blogtwodo/
Connecting to localhost|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 404 /blogtwodo/
10:34:55 ERROR 404: /blogtwodo/.
 
Finished deploying cluster

Hmm. The ERROR 404 does not look good, but it does look like the application technically deployed. A check on the EC2 console confirms that we have a single running instance, and clicking on it drills down to a panel that shows us the public DNS address we can use to test it out with a browser (running grails cloud-tools-describe gets us the same information from the command line.) When I visit the base address I get a generic Apache test page, which is encouraging:

But, sure enough, I am greeted with a 404 error when I try to hit our applications URL with a browser.

So now what? Well, when in doubt, go for the log files. We can ssh directly into our running instance using our PEM file for authentication:

ssh -i etc/blogtwodo.pem root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

A bit of poking around/exploring our VPS leads me to a promising log file: /var/log/tomcat5/catalina.out with a suspicious stacktrace:

...
java.lang.NoClassDefFoundError: net.chrisrichardson.ec2deployer.cluster.ClusterSpec
        at Config$_run_closure3.class$(Config.groovy)
        at Config$_run_closure3.$get$$class$net$chrisrichardson$ec2deployer$cluster$ClusterSpec(Config.groovy)
        at Config$_run_closure3.doCall(Config.groovy:87)
        at Config$_run_closure3.doCall(Config.groovy)
        at Config.run(Config.groovy:76)
...

It would seem that the CloudTools jars don’t appear to be packaged into our deployment war file. After a bit of investigation, this problem turns out to be another issue caused by behavior changes to Grails v1.1.1– notably in the management of plugins. The CloudTools plugin expects to find stuff located local to the grails project directory and the new Grails v1.1.1 paradigm moves all that stuff to centralized ~/.grails (on a Mac anyway) location. My hack to get things working involved creating a symbolic link so that CloudTools would deploy all the necessary bits (for all our other plugins as well):

ln -s ~/.grails/1.1.1/projects/blogtwodo/plugins ./plugins

OK. Surely that must be the last step. After deploying once again, we again get the 404 error. Now what? Well checking the logfile again reveals this:

org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Access denied for user 'twodouser'@'localhost' (using password: YES))

I don’t doubt that there are better solutuions, but simply I punted and simply forcibly granted our ‘twoduser’ access to the database via the mysql command line:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6 to server version: 5.0.22-log
 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 
mysql> use blogtwodo
Database changed
mysql> GRANT USAGE ON *.* TO 'twodouser'@'localhost' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)
 
mysql> GRANT ALL PRIVILEGES ON `blogtwodo`.* TO 'blogtwodo'@'localhost' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

This time we want to make sure that we redeploy the application to the same AMI where we just made our database changes, so we deploy the app using the cloud-tools-redeploy gant target:

grails cloud-tools-redeploy

At last we get something that appears to be a clean deployment:

Connecting to localhost|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://localhost:8080/blogtwodo/ [following]
--10:48:48--  http://localhost:8080/blogtwodo/
Connecting to localhost|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://localhost:8080/blogtwodo/public [following]
--10:48:49--  http://localhost:8080/blogtwodo/public
Connecting to localhost|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/html]
200 OK
 
Finished redeploying cluster

Pointing the browser at the site confirms it as working:

click to enlarge

Awesome.

Conclusion and Wrapup

All told the above took about 2 days (16 hours) to get working. Our total number of hours spent prior to this was about 30 hours, which brings our total to ~46 hours… just over our 40 hour goal. Still not too bad IMO. Let’s survey the pros and cons about the approach we used today:

Pros:

• Integration with EC2 is largely transparent to your underlying Grails application– once you get everything configured correctly that is… after which deployment is a snap
• This transparency is huge. I did nothing, repeat nothing, to alter the underlying application for suitability with EC2. This is NOT the case with Google’s AppEngine, as will be shown in the next post in this series.
• I don’t doubt that using the CloudFoundry management console makes deployment that much easier, without the need of a plugin. I plan to play with this more later, so expect a blog post on this as well.

Cons:

• It’s not free. We deployed the twodo application in an absolutely bare bones configuration– A single AMI which costs US$0.10 per/hour –> US$2.40 per/day –> ~US$72 per/month. While not catastrophically expensive, if you don’t need the ability to infinitely scale on a moments notice, you can certainly find cheaper hosting options.
• We currently have no persistent storage configured. Yes, we have a MySql instance running, but for all practical purposes it might as well be an in memory database. If the AMI ever goes away, our DB is slicked as well. A ‘real’ Cloud Computing solution would require the use of something like Amazon’s Elastic Block Storage Service (EBS) ( which is also not free)
• While it was not a _ton_ of work, it did take some sleuthing to get things finally working. I am assuming that this plugin is likely to no longer be actively supported, as (presumably) the CloudFoundry system effectively replaces it. Not a con exactly, but I think I would investigate what CloudFoundry has to offer before choosing this plugin as my definitive approach to getting an app into the Cloud.

That’s it for now. The next post in the series will document the deploying our application to Google’s AppEngine, so check back often/subscribe!

Technorati Tags: amazon ec2, cloud computing, grails, groovy, java, software development

8 Comments currently posted.

Christian Naeger says:

don says:

limcheekin says:

Marko says:

don says:

Chris says:

villalobos says:

Peter Braswell says:

Post a comment on this entry:

• Labs

• Categories

  • General
    • Articles
    • Budgeting
    • Business Planning
    • Estimation
    • Jobs
    • Marketing
    • Misc
    • News
    • Project Management
  • Technology/Technical Approaches
    • .NET
    • Agile Development
    • Groovy/Grails
    • How-Tos
    • Infrastructure
    • Java
    • Linux
    • Messaging
    • Persistence
    • Ruby/Rails
    • Terracotta
  • Uncategorized

• Meta

  • Log in
  • Valid XHTML
  • Valid CSS
  • WordPress
  -->
  • Gridlock 1.4 by hyalineskies