Sunjay will be presenting proven techniques that can take your team from vision to requirements in hours, not weeks.

Presentation Abstract

Markets, customers, and industries are moving quickly. Rapid application development is a strategic necessity. But, rapid application development techniques require rapid estimation and planning techniques. Business demands for improved time-to-market means your teams must estimate, plan, and budget application development projects with greater speed and accuracy.

Project Plan Ninja provides modern cues, clues, and tools for turning application development requirements into actionable budgets & work plans.

• Estimate application development projects in hours, not weeks
• Quickly and confidently enter feature negotiation with customers
• Envision a consistent estimation and budgeting process for your teams
• Bridge the gap between Agile-oriented and plan-driven estimation
• Understand how to build practical, repeatable work plans

There is no charge for attending this event, but preregistration is required. Please visit www.pmiwdc.org/Fairview and click on “Register Online” on the April 21 event.

This entry is a part of the ‘What can you build in 40hrs‘ series– which, in a nutshell, involved an experiment to see what we could build in about typical weeks worth of work. In the last post we effectively completed the basic functional requirements of the application, and now have a working system that can be used to manage two sets of to-do lists as part of a task management application.

Cloud Computing

As part of this effort, we wanted to try something ‘new’ as part of the application initiative. We decided to make one of the system requirements of the application involve its deployment into a Cloud Computing infrastructure– the leading candidates at time of construction being Amazon’s Elastic Cloud Computing (EC2) and Google’s App Engine.

While both Amazon and Google offer similar infrastructure platforms, as it shall be seen, the level of effort necessary to deploy a stock Grails application into these infrastructures varies greatly.

This entry concerns itself with the deployment of the TwoDo application into Amazon’s EC2 infrastructure. A subsequent entry will detail the efforts necessary to get the same application running on Google’s AppEngine platform. (At the time I started this effort Google’s AppEngine had only recently released its Java API, and was dribbling out access to its infrastructure in a controlled fashion (FIFO?) As such, I did not initially have the rights to deploy an application to AppEngine– hence the decision to go with EC2 first.)

Cloud Tools Plugin

We are going to leverage the CloudTools Plugin for all the heavy lifting aspects of getting our application deployed into Amazon. Cloud Tools was developed by the founder of CloudFoundry– a service designed to deploy Java/Grails apps to Amazon’s EC2. I suppose I could have gone with the CloudFoundry as part of my deployment efforts (and indeed, will probably try it out) but for whatever reason chose to start at a lower level by using the plugin.

Getting Started

A prerequisite for using the CloudTools Plugin is to have an existing Amazon AWS account. The setup of which is easy, and well documented outside of this blog… so we aren’t going to concern ourselves with much of it here. If nothing else simply go to the main site and click the ‘Sign Up for Amazon EC2′ button

The first basic step was to install the plugin. I downloaded the plugin from the main CloudTools site and installed it from my local drive:

grails install-plugin /path/to/downloads/grails-cloud-tools-0.6.zip

The installation process creates a conf/CloudTools.groovy file which, in theory, could be simply added to the list of configuration files that Grails should slurp during boot time via an explicit specification in the conf/Config.groovy file like so:

...
grails.config.locations = ["classpath:CloudTools.groovy"]
..

However, I could never get this file to be located during the deployment process, so I eventually punted and copied the contents of the conf/CloudTools.groovy file directly into conf/Config.groovy

Once the contents of the conf/CloudTools.groovy file are pasted into the standard conf/Config.groovy file it needs to be edited with the specifics of our application.

Database Setup

The CloudTools plugin will create a MySql schema for us, as well as a user account used to access the DB as part of the deployment process. All we need to to is edit the parameters passed to the schema() method of the ClusterSpec() object that we pasted into our Config.groovy file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

cloudTools {
...
  clusterSpec = new ClusterSpec()
      .tomcats(1)
      .topology("SingleInstanceTopology")
       .instanceType(EC2InstanceType.SMALL)
       .slaves(0)
       .bucketPrefix(System.getProperty("grails.env"))
       .catalinaOptsBuilder("""{builder, databasePrivateDnsName, slaves ->
            builder.arg("-Xmx500m")
	    builder.prop("com.sun.management.jmxremote")
	    builder.prop("com.sun.management.jmxremote.port", 8091)
	    builder.prop("com.sun.management.jmxremote.authenticate", false)
	    builder.prop("com.sun.management.jmxremote.ssl", false)
	    builder.prop("ptrack.application.environment", "ec2")
	    builder.prop("log4j.configuration", "log4j-minimal.properties")
	    builder.prop("jdbc.db.server", databasePrivateDnsName)}""")
        .schema("blogtwodo", ["twodouser": "password"], [])

On line 18 above we configure the ClusterSpec to create the ‘blogtwo‘ database and the ‘twodouser‘, the specifications of which should match the contents of the conf/DataSource.groovy file.

CloudTools AWS Properties File

We need to indicate the location of our Amazon Web Service property file, which is set by editing the value for the awsPropertiesFile property within the conf/Config.groovy file.

...
cloudTools {
  awsPropertiesFile = "etc/ec2.properties"  ...
}

As you can see we have created a new grails-app/etc directory to hold this new file. The properties file is a CloudTools specific artifact that contains the relevant configuration information required by Amazon to host our application. In order to deploy into Amazon’s platform we are going to need provide several bits of information by way of this properties file. Let’s take a look at the contents of ours and dissect it line-by-line:

1
2
3
4
5
6
7
8
9

imageId.m1.small=ami-6f2cc906
imageId.m1.large=ami-0129cc68
imageId.m1.xlarge=ami-0129cc68
accountId=XXXX-XXXX-XXXX
accessKey=XXXXXXXXXXXXXXXXXXX
secretKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
keyName=blogtwodo
keyPairFile=/Users/don/DevelopmentWork/blogtwodo/etc/blogtwodo.pem
sshDir=/usr/bin

AMI Identification
An application deployed into the EC2 will reside in Amazon Machine Image (AMI). Think of this as a identifier for the base OS image/VPS. In our case (lines 1-3) we will simply use the ones specified by the plugin documentation:

1
2
3

imageId.m1.small=ami-6f2cc906
imageId.m1.large=ami-0129cc68
imageId.m1.xlarge=ami-0129cc68

Security Credentials
The next few lines contain part of the security credentials we need to provide to Amazon. From within the Amazon Web Services Console, we navigate to ‘Your Account -> Security Credentials‘. The accountId (line 4) is your 12-digit account displayed on this page under the ‘Your AWS Account ID, E-mail Address and Password‘ section.

While within the ‘Access Keys‘ tab, I clicked ‘Create a New access Key‘ and used the resulting generated information to fill in the accessKey (line 5) and secretKey (line 6) properties.

4
5
6

accountId=XXXX-XXXX-XXXX
accessKey=XXXXXXXXXXXXXXXXXXX
secretKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Amazon’s site does a good job describing the purpose of the above access credentials:

Access Key ID
Your Access Key ID identifies you as the party responsible for service requests. Use your Access Key ID as the value of the AWSAccessKeyId parameter in requests you send to Amazon Web Services (when required).

Secret Access Key
Each Access Key ID has a Secret Access Key associated with it. Use your Secret Access Key to calculate a signature to include in requests to web services that require authenticated requests. Your Secret Access Key is a secret, and should be known only by you and AWS. You should never include your Secret Access Key in your requests to AWS. You should never e-mail your Secret Access Key to anyone. It is important to keep your Secret Access Key confidential to protect your account.

Key Pair
In addition to the above credentials we need to provide a public/private key pair in the form of an PEM file. Keys are created from within the EC2 dashboard and I created a new keypair with the application name of ‘blogtwodo’ by logging into the ‘Amazon EC2 dashboard -> Key Pairs‘ and clicking the ‘Create Key Pair‘ link. The resulting blogtwodo.pem file was saved locally and moved into the grails-app/etc directory, where its configured by lines 7 and 8 below:

7
8

keyName=blogtwodo
keyPairFile=/Users/don/DevelopmentWork/blogtwodo/etc/blogtwodo.pem

Finally we need to indicate the path to an ssh client (line 9) to be used to securely upload our application to Amazon. I am using OSX on a Mac which includes this natively in the /usr/bin directory:

9

sshDir=/usr/bin

Deployment

OK. With our configuration out of the way lets try and deploy this thing. From the command line we invoke the gant deployment target added by the CloudTools plugin:

grails cloud-tools-deploy

… which promptly reports the following error:

Error executing script CloudToolsDeploy: No such property: DEFAULT_J5_DEPS for class: ExplodedWar_groovy

Whats the deal? Well, the CloudTools plugin was developed pre Grails v1.1.1, as it turns out there are a few incompatibilities that need to be addressed in order to get our application deployed.

This error is addressed by adding the following highlighted code (line 10) to the $GRAILS_HOME/scripts/_GrailsWar.groovy

1
2
3
4
5
6
7
8

...
DEFAULT_DEPS = [
    "ant-*.jar",
     ...
     ...
]
 
DEFAULT_J5_DEPS = DEFAULT_DEPS

OK. Lets try to deploy again….. this time, although we get farther, we eventually fail with the error:

ssh: connect to host ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com port 22: Operation timed out

Drat. Now what? After a little digging it’s determined that the problem is that we have not allowed the SSH protocol through the built in firewall associated with our application. This is fixed by navigating from within the EC Dashboard to ‘Security Groups -> default group’ and adding the SSH protocol to the table of connection methods. Let’s try our deployment again.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/Users/don/DevelopmentWork/blogtwodo/etc/blogtwodo.pem' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.

Sigh. OK. Trying again after chmod‘ing the PEM file to 400 (chmod 400 blogtwodo.pem).

After several minutes (and I do mean several.. deployment took close to 10 minutes) this time we get:

...
...
Location: http://localhost/blogtwodo/ [following]
--10:34:55--  http://localhost/blogtwodo/
Connecting to localhost|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 404 /blogtwodo/
10:34:55 ERROR 404: /blogtwodo/.
 
Finished deploying cluster

Hmm. The ERROR 404 does not look good, but it does look like the application technically deployed. A check on the EC2 console confirms that we have a single running instance, and clicking on it drills down to a panel that shows us the public DNS address we can use to test it out with a browser (running grails cloud-tools-describe gets us the same information from the command line.) When I visit the base address I get a generic Apache test page, which is encouraging:

But, sure enough, I am greeted with a 404 error when I try to hit our applications URL with a browser.

So now what? Well, when in doubt, go for the log files. We can ssh directly into our running instance using our PEM file for authentication:

ssh -i etc/blogtwodo.pem root@ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com

A bit of poking around/exploring our VPS leads me to a promising log file: /var/log/tomcat5/catalina.out with a suspicious stacktrace:

...
java.lang.NoClassDefFoundError: net.chrisrichardson.ec2deployer.cluster.ClusterSpec
        at Config$_run_closure3.class$(Config.groovy)
        at Config$_run_closure3.$get$$class$net$chrisrichardson$ec2deployer$cluster$ClusterSpec(Config.groovy)
        at Config$_run_closure3.doCall(Config.groovy:87)
        at Config$_run_closure3.doCall(Config.groovy)
        at Config.run(Config.groovy:76)
...

It would seem that the CloudTools jars don’t appear to be packaged into our deployment war file. After a bit of investigation, this problem turns out to be another issue caused by behavior changes to Grails v1.1.1– notably in the management of plugins. The CloudTools plugin expects to find stuff located local to the grails project directory and the new Grails v1.1.1 paradigm moves all that stuff to centralized ~/.grails (on a Mac anyway) location. My hack to get things working involved creating a symbolic link so that CloudTools would deploy all the necessary bits (for all our other plugins as well):

ln -s ~/.grails/1.1.1/projects/blogtwodo/plugins ./plugins

OK. Surely that must be the last step. After deploying once again, we again get the 404 error. Now what? Well checking the logfile again reveals this:

org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Access denied for user 'twodouser'@'localhost' (using password: YES))

I don’t doubt that there are better solutuions, but simply I punted and simply forcibly granted our ‘twoduser’ access to the database via the mysql command line:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6 to server version: 5.0.22-log
 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 
mysql> use blogtwodo
Database changed
mysql> GRANT USAGE ON *.* TO 'twodouser'@'localhost' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)
 
mysql> GRANT ALL PRIVILEGES ON `blogtwodo`.* TO 'blogtwodo'@'localhost' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

This time we want to make sure that we redeploy the application to the same AMI where we just made our database changes, so we deploy the app using the cloud-tools-redeploy gant target:

grails cloud-tools-redeploy

At last we get something that appears to be a clean deployment:

Connecting to localhost|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://localhost:8080/blogtwodo/ [following]
--10:48:48--  http://localhost:8080/blogtwodo/
Connecting to localhost|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://localhost:8080/blogtwodo/public [following]
--10:48:49--  http://localhost:8080/blogtwodo/public
Connecting to localhost|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/html]
200 OK
 
Finished redeploying cluster

Pointing the browser at the site confirms it as working:

click to enlarge

Awesome.

Conclusion and Wrapup

All told the above took about 2 days (16 hours) to get working. Our total number of hours spent prior to this was about 30 hours, which brings our total to ~46 hours… just over our 40 hour goal. Still not too bad IMO. Let’s survey the pros and cons about the approach we used today:

Pros:

• Integration with EC2 is largely transparent to your underlying Grails application– once you get everything configured correctly that is… after which deployment is a snap
• This transparency is huge. I did nothing, repeat nothing, to alter the underlying application for suitability with EC2. This is NOT the case with Google’s AppEngine, as will be shown in the next post in this series.
• I don’t doubt that using the CloudFoundry management console makes deployment that much easier, without the need of a plugin. I plan to play with this more later, so expect a blog post on this as well.

Cons:

• It’s not free. We deployed the twodo application in an absolutely bare bones configuration– A single AMI which costs US$0.10 per/hour –> US$2.40 per/day –> ~US$72 per/month. While not catastrophically expensive, if you don’t need the ability to infinitely scale on a moments notice, you can certainly find cheaper hosting options.
• We currently have no persistent storage configured. Yes, we have a MySql instance running, but for all practical purposes it might as well be an in memory database. If the AMI ever goes away, our DB is slicked as well. A ‘real’ Cloud Computing solution would require the use of something like Amazon’s Elastic Block Storage Service (EBS) ( which is also not free)
• While it was not a _ton_ of work, it did take some sleuthing to get things finally working. I am assuming that this plugin is likely to no longer be actively supported, as (presumably) the CloudFoundry system effectively replaces it. Not a con exactly, but I think I would investigate what CloudFoundry has to offer before choosing this plugin as my definitive approach to getting an app into the Cloud.

That’s it for now. The next post in the series will document the deploying our application to Google’s AppEngine, so check back often/subscribe!

Technorati Tags: amazon ec2, cloud computing, grails, groovy, java, software development

To recap briefly our progress to date:

• Part 1: We introduce the basic challenge: Build a non-trivial app in about 40hours.
• Part 2: We talk about the basic app concept.. a todo list manager called (cleverly) twodo. We have decided on the general technical framework (Grails, etc)
• Part 3: The first ‘technical’ post. JSecurity setup, initial Domain object created.
• Part 4: The UI is refined, and we start using the GrailsUI plugin to populate a YUI DataTable via AJAX.

So, although this post it titled #5, its really only the third technical post. In the last post we made our UI a bit more pleasing (or at least more intentional), and introduced the GrailsUI plugin into the mix for purposes of using AJAX to dynamically update a YUI DataTable. In this post we will put the finishing touches on the core application, which primarily involves getting the GrailsUI DataTable to work the way we want to.

This post will be a bit on the long side, because I want to finish up the core application so the next couple of posts can deal with deploying the twodo application into a Computing Cloud like Google’s AppEngine or Amazon’s EC2.

DataTable and Cell Editing

In the last post we populated a single simple YUI data table with test data using the GrailsUI, and it did exactly what we asked of it. However, our current implementation produced a simple read-only table. I envisioned using a DataTable construct to not only display the data, but to allow the user to dynamically update it via AJAX– sorta like an online spread sheet type application (in a similar fashion to the YUI example provided here.)

To this end, I initially installed the latest (v1.0.4) version of the plugin as per standard instructions, and it worked fine for our initial tests– however, it did not allow for the editing of the table cells. As it turns out this is a known limitation of the current stable version of the plugin. So now what? Well, working The Google On The Internet Machine revealed that the 1.1-SNAPSHOT has inline cell editing capabilities so I downloaded the 1.1-SNAPSHOT version from the plugin site and installed it from my local drive (grails install-plugin /path/to/download/grails-grails-ui-1.1-SNAPSHOT.zip)

Recall that in the last post we used the gui:datatable tag to construct a simple DataTable like so:

1
2
3
4
5
6
7
8
9

<gui:dataTable id="dt_1"
    columnDefs="[
        [id:'ID', formatter:'text', hidden:true],
        [status:'Status', formatter:'text'],
        [description:'Description', formatter:'text'],
        [est:'Est', formatter:'text'],
    ]"
...
/>

The snapshot version of the tag supports additional editor configuration parameter to be passed along as part of the columnDefs, that allow for the specification of an AJAX target to be called when the cell is edited. Lets take a few baby steps and modify the ’subscription’ column to allow for editing:

1
2
3
4
5
6
7
8
9
10

<gui:dataTable id="dt_1"
    columnDefs="[
      [id:'ID', formatter:'text', hidden:true],
      [status:'Status', formatter:'text'],
      [description:'Description', formatter:'text',
        editor:[controller:'home', action:'tableChange']],      [est:'Est', formatter:'text'],
    ]"
...
/>

Note the additional code on line 6 where we indicate that changes to a cell in the ‘description‘ column are to be handled by making an AJAX call to the ‘tableChange‘ action in the ‘home‘ controller. Let’s point our browser at the page (http://localhost:8080/blogtwodo/home/datatable), and see what happens when we try to edit a cell in the description column.

click to enlarge

OK. As the screen shot above indicates, when we click the cell to edit it, some JavaScript automagically renders a little dialog box for us with some basic editing options. Perfect.

But when we continue and try to actually change the value of the cell and save it by clicking on the ‘Save’ button, we get the following ugly error message:

click to enlarge

Hey! Whats with the error? Well, the DataTable is attempting to make an AJAX call to the tableChange action– which does not exist yet. So let’s remedy that right now:

package com.alterlabs.twodo
class HomeController {
  ...
  /**
    * AJAX Call
    */
  def tableChange = {
 
    // TODO: Verify that this task belong to the current user!
    def task = Task.get(params.id)
 
    if ( params.field == 'description') {
      task.description = params.newValue
      task.save()
    }
    render "table changed"
  }
}

Our initial cut at the tableChange method is about as straight forward as it gets. We load the Task by id (without even checking to see if it belongs to our user! Yikes) and save the new value. Simply reloading our home page after saving verifies that our changes to the description field were indeed saved.

Now that we have the basic approach down, we can use what we have learned as a basis for constructing the real code that will be used to manipulate the two task lists in the application.
We can now add the additional configuration to provide editing for the other columns in the DataTable, as well as corresponding logic in the controller to handle the updates, like so:

1
2
3
4
5
6
7
8
9
10
11
12
13

<gui:dataTable id="dt_1"
  columnDefs="[
    [id:'ID', formatter:'text', hidden:true],
    [status:'Status', formatter:'text',
      editor:[controller:'home', action:'tableChange', 
        type:'dropdown', config:[dropdownOptions:['open','complete','delete'],disableBtns:true]]],    [description:'Description', formatter:'text',
      editor:[controller:'home', action:'tableChange']],
    [est:'Est', formatter:'text',
      editor:[controller:'home', action:'tableChange']],
  ]"
...
/>

Notice that in addition to the basic cell editing, we also specify a bunch of other nifty options to the to the g:datatable control that help customize the behavior of the editor. On line 6 we indicate that the status column is to use a dropdown list for editing (with the list values of ‘open‘, ‘closed‘ and ‘delete‘– we will use the ‘delete’ option later), and that the default save/cancel buttons are to be hidden. In the tableChange action we make the obvious/appropriate corresponding changes to handle the edits to the additional fields.

Up and until now we have been fooling around with the DataTable in a test page. We want the contents of our two task lists to be rendered as the users home page after they successfully log in, so its about time we reincorporate what we have learned back into the main application (views/home/index.gsp) Taking the sample code above and embellishing it a bit with CSS gets us something like this:

click to enlarge

So we have basic editing, but its still lacking a few fundamentals– notably callback notification on success/failure of the edit. We will address that in a bit below…

Adding Rows

So far all of the data we have been manipulating had been manually added as fixture data in the last post. Obviously a task management application needs a way to add new tasks so lets tackle that now.

Normally adding a new Task in a conventional non-AJAX fashion is trivial. (Actually, come to think of it, the standard Grails taglibs make using AJAX pretty trivial as well.)

However, accomplishing the same thing using AJAX and our DataTable, however, proves to be a tad bit trickier. Lets start again with baby steps and (reasonably) add a g:remoteForm at the bottom of our Work tasks list that submits via AJAX to an action in the home controller (once we figure it out for the Work tasks, we can replicate it for the Personal)

<g:formRemote name="worktask_form" onSuccess="alert('Success!')" 
        url="[action:'addRow',params: [ taskType:'work']]">
  <input type="text" name="newTask" id="newTask"/>
  <button type="submit">Add</button>  
</g:formRemote>

We put a simple alert() in the onSuccess event to prove things are working. Exercising our code thru the browser gives us the alert message we expect, but our DataTable is not yet updated to show the new row. If we refresh the page, we see the new row, but that’s hardly sufficient.

The obvious solution is to register a callback function that in turn forcibly refreshes the DataTable. Surprisingly, (at least according to the hours I spent trying to get this aspect of the system to work,) the YUI DataTable possesses no such straightforward function. The GrailsUI DataTable Object does, however, include a requery() function, which gets us 99% of the way there, so as a final resort I coded my own refresh() function based on this and extended the YUI DataTable Object by adding it to the javascript prototype:

1
2
3
4
5
6
7
8
9
10
11
12
13

// Used to refesh our contents after a row change. 
// Basically a copy of requery() with a new Callback
GRAILSUI.DataTable.prototype.refresh =  function() {
...
  // NOTICE: This is what has changed from the requery() function
  var oCallback = {
    success: this.onDataReturnReplaceRows,  //<<- this line    failure: this.onDataReturnSetRows,        
    scope: this,
    argument: newState 
  };
...
};

We can then make our onSuccess callback invoke this new function, and test it out:

 <g:formRemote name="worktask_form" onSuccess="GRAILSUI.dt_1.refresh();" 
         url="[action:'addRow',params: [ taskType:'work']]">

This time, adding a new row refreshed our DataTable and the new row shows up in our list– which is awesome.

Deleting Rows

Now that we can add Tasks to our list, we need to also provide a mechanism for removing them. Recall that we configured our dropdown list for the ‘status‘ column to allow the for three values: ‘open‘, ‘closed‘ and ‘delete‘. The plan is to use a cell edit on the status column to trigger the deletion of the Task. With that in mind I added the following code to the tableChange action (which is the action to be called during our cell edits):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

package com.alterlabs.twodo
class HomeController {
  ...
 
 def tableChange = {
   def profileInstance = Profile.current()
   def task = Task.get(params.id)
   ...
   if ( params.field == 'status') {
     switch (params.newValue) {
       case 'open': 
         task.completionDate = null
         task.save()
         break
       case 'closed': 
         task.completionDate = new Date()
         task.save()
         break
       case 'delete':
         // no easy way to tell what list it was from so remove from both
         profileInstance.removeFromWorkTasks(task)
         profileInstance.removeFromPersonalTasks(task)
         profileInstance.save()
         println "*** Deleted task ${task}"
         break
     } 
   }
...
}

When we change our status column to ‘delete‘ we see a rewarding message in the console, but our DataTable does not update to show any change. We have to refresh the page for the DataTable to show an updated list. Why? For that we move to the next section….

Custom GrailsUI DataTable Changes

One thing that became quickly obvious was that as great as the GrailsUI plugin is, it still lacking in a few areas. Notably there is no mechanism for notifying the UI of a successful cell edit. This is not a big deal when we are simply editing the contents of an existing row (because our browser screen is naturally kept in sync)… but it becomes a definite issue when removing a row from the DataTable.

What we need is the ability to register a callback with underlying YUI structures of a function to be called after a DataTable cell edit. If we view the browser page source and look at the underlying javascript that the g:dataTable tag generates for us we can see what is up:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

... 
var registerEditorListener = function(editor, field, url) {
  GRAILSUI.dt_1.loadingDialog.show();
  var editorCallback = {
    failure: function(o) {      // revert the cell value
      GRAILSUI.dt_1.updateCell(oArgs.editor.getRecord(), field, oArgs.oldData);
      // alert user
      alert('Recieved an error during edit: ' + o.responseText);
     }
   };
   YAHOO.util.Connect.asyncRequest('POST', url, editorCallback, 'id=' + oArgs.editor.getRecord().getData('id') + '&field=' + field + '&newValue=' + oArgs.newData);
                    });
};

From the code above we can see that the taglib creates a registerEditorListener function, and does indeed define a callback to be registered with the AJAX call. BUT it only defines what to do in case of a failure (line 5) and not what to do in cases of success. Knowing this, its easy enough to tweak the grails-app/taglib/DataTableTagLib.groovy file included in the GrailsUI plugin project to allow for us to pass in as part of the g:dataTable configuration a callback to use for cell editing. After tweaking, our augmented taglib generates something like this:

1
2
3
4
5
6
7
8
9

...
var editorCallback = {
  success: function(o) {
    ${editorSuccessCallback};
  }, 
  failure: function(o) {
    ... 
  }
};

…and we simply add the editorSuccessCallback parameter to our g:dataTable configuration data:

1
2
3
4
5

<gui:dataTable id="dt_1"
  ...  
  editorSuccessCallback="GRAILSUI.dt_1.refresh();"
  ..
]"/>

Now anytime we perform any successful cell edit (including deletes) , the DataTable.refresh() function is invoked and our table is rendered accurately. Killer.

Conclusion and Next Steps

Fundamentally the application is now functionally complete. There was plenty of mundane polishing and such that was not documented in this series, but all of the big stuff was touched upon.

The work accomplished in todays posts took a little over 20 hours to complete… the majority of the time split evenly between exploring the GrailsUI TagLib code trying to make the refresh work and tweaking CSS (which always seems to take more time than expected.) That brings our total time spent todate to about 30 hours.

The next post will (finally!) talk about deploying the twodo application into Amazon’s Elastic Compute Cloud (EC2). See you soon…

Technorati Tags: agile, grails, groovy, grailsui, java, software development

In this post we will refine the general look and feel of the User Interface, as well as start building the core list management functionality via a UI Plugin.

Look and Feel

To this point we have done almost no UI creation, and have not customized the default out-of-the-box look and feel of the application. I knew from talking to the stakeholder (in this case my business partner) that he envisioned a simple/clean interface with little clutter. The general site template was envisioned to look something like this:

Under the assumption that the look and feel, would (like all other aspects of the app) continue to evolve over the course of the development iterations, I did not want to spend too much time on polishing the initial cut at the UI template. I started with a simple raw HTML/CSS template, a bit of time in The Gimp and some CSS tweaking via Firebug until I got something that seemed initially acceptable. I added in some greeting text and voila– comparing the original/old login screen:

…and the new one:

OK. Still not award winning perhaps, but at least it looks intentional.

I don’t profess to be much an artist when it comes to web pages, and don’t intend to turn this post into a treatise on web design, but I think its worth peeking at the main views/layout/main.gsp layout file used for this project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<html>
   <head>
        <title><g:layoutTitle default="TwoDo" /></title>
        <link rel="stylesheet" href="${createLinkTo(dir:'css',file:'template_css.css')}" type="text/css"/>
        <link rel="stylesheet" href="${createLinkTo(dir:'css',file:'main.css')}" type="text/css"/>
        <link rel="shortcut icon" href="${resource(dir:'images',file:'favicon.ico')}" type="image/x-icon" />
 
        <g:javascript library="application" />
        <g:layoutHead />
  </head>
  <body>
    <g:render template="/layouts/includes/logo"/>
    <div id="page">
      <div id="page-bg">
        <g:render template="/layouts/includes/messages"/>
        <g:render template="/layouts/includes/errors"/>
 
        <!-- start content -->
        <div id="content">
          <g:layoutBody/> 
        </div>
        <!-- end content -->
        <div style="clear:both;">&nbsp;</div>
      </div>
    </div>
    <g:render template="/layouts/includes/footer"/>
 </body>
</html>

The main thing that is worth noting is that for every major rectangular region of the main template (as indicated on the napkin) we have further subdivided the layout into smaller templates which are then rendered from within main layout via the g:render taglib. As an example, we examine a few of the templates below.

Here is the layouts/includes/_logo.gsp template, which has been configured to:

• Show a simple text logo. We will eventually replace this with a graphic
• Display a greeting for the logged in user.
• Provide a logout link

<div id="logo">
  <h1><a href="${resource(dir:'')}">TwoDo!</a></h1>
<jsec:isLoggedIn>
  <span class="greeting" >
    Hello,<jsec:principal/> (<g:link controller="auth" action="signOut">logout</g:link>)</span>
</jsec:isLoggedIn>
</div>

And finally, a look at the layouts/includes/_messages.gsp which includes logic for the rendering of flash messages in a consistent manner:

<div id="message_wrapper" <g:if test="${!flash.message}"> style="display:none;"</g:if>>
  <div id="messageBar" <g:if test="${!flash.message}">style="display:none;"</g:if>>${flash.message}</div>
</div>

This divide-and-conquer strategy makes sense in many things software related, and the case of layout templates is no exception.

Security and Profile Management

It quickly becomes apparent that we require a mechanism for retrieving the current users Profile after a successful login. JSecurity provides a SecurityUtil class with a method that allows us to get the currently logged in Subject. From there we can get the Principle, and from the Principle we can get the Profile. I found it convenient to implement this logic in a pseudo-GORM like fashion by simply adding a static method to the existing Profile class:

1
2
3
4
5
6
7
8
9
10
11
12
13

...
import org.jsecurity.SecurityUtils
 
class Profile extends Base {
  ...
  /**
    * Return the profile associated with the currently logged in user
    */
  static Profile current() {
    def principle = SecurityUtils.getSubject().getPrincipal()
    return Profile.findByUsername( principle )
  }
}

Our controllers/whatever now simply need to call Profile.current() to get the currently logged in user.

GrailsUI Plugin

As per our sitemap in the last post, we only have a handful of pages in the entire application. Once a user logs in, they will be at their home (views/home/index.gsp) page, which will display their current list of tasks side by side. The manipulations of these lists seemed a prime candidate for some fancy UI and AJAXy goodness. In particular I envisioned a dynamic table that would use AJAX to manipulate the contents of the task lists. Rather than roll it all from scratch we will enlist the help of a the Yahoo UI libraries by way of the GrailsUI plugin. I had used the YUI library on prior projects and, wanting to maximize my productivity, so chose it over other competing frameworks (jQuery, etc.)

The GrailsUI plugin basically provides us with a wrapper taglib that encapsulates the underlying YUI javascript. In particular we are interested in the gui:dataTable tag, which wraps a YUI DataTable Control. Installing the GrailsUI Plugin automagically installs its YUI dependencies.

Before we tackle using the taglib/control however, lets look ahead, and address our data testing needs.

From reading the associated documentation, I knew that our GrailsUI DataTable was eventually going be populated by an AJAX call that will return its data in JSON format– which is all well and good. So the first order of business was to create some fixture data with which to test. To this end I added a simple action to the default Task controller we generated in the last post to create a few tasks for purposes of testing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

package com.alterlabs.twodo
 
class TaskController {
...
  def data = {
    def p = Profile.current()
    for ( i in 0..5 ) {
      p.addToWorkTasks(new Task(description:"Work Task #${i}", estimate:60))
      p.addToPersonalTasks(new Task(description:"Personal Task #${i}", estimate:60))
    } 
    flash.message = p.save()?"Added fake tasks to profile":"Failed to add fake tasks"
    redirect(action:list)
  }
...
}

The code above grabs the Profile of the current user then creates and associates some dummy Tasks with it. Pointing our browser at http://localhost:8080/blogtwodo/task/data invokes the action and allows us to visually verify the data by listing all the Tasks in the system.

OK. Now that we have some data, lets work on making a DataTable retrieve it via AJAX. The plugin home page has some helpful information about using the DataTable, as does the authors blog. As per usual I started with a crude test to work the kinks out, with the assumption that it could be suitably refined later. By following the examples provided in the plugin documentation I quickly put together a simple test page that looked like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

<html>
  <head>
    <gui:resources components="['dataTable']" />
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <meta name="layout" content="main" />
    <title>DataTable Test</title>
  </head>
  <body>
    <div>  
      <h1>Work DataTable:</h1> 
      <gui:dataTable id="dt_1"
         columnDefs="[
            [id:'ID', formatter:'text', hidden:true],
            [status:'Status', formatter:'text'],
            [description:'Description', formatter:'text'],
            [est:'Est', formatter:'text'],
         ]"
         sortedBy="id"
         allowExclusiveSort='true'
         draggableColumns="false"
         controller="home" action="tasks" params="[taskType:'work']"
         rowsPerPage="10" 
         paginatorConfig="[
             template:'{PreviousPageLink} {NextPageLink} {CurrentPageReport}',
             pageReportTemplate:'{totalRecords} total records'
         ]"/>
    </div>
  </body>
</html>

In a nutshell, the above code:

1. Includes the resources for the GrailsUI DataTable (line:3)
2. Makes a DataTable with 4 columns (‘id’, ’status’, ‘description’ and ‘est’). (lines:11-17)
3. Note: The ‘id’ column is strictly for sorting purposes and we don’t want it shown on the screen, so we mark it as ‘hidden’. (line:13)
4. Wires the DataTable to populate itself from the action ‘tasks‘ on the ‘home‘ controller and also passes the parameter taskType set to ‘work‘ (line:21)

The other configuration attributes are reasonably self explanatory, or will get additional treatment in later posts. The g:dataTable tag does all the (considerable) heavy lifting for us and imports all the UI javascript files, configures the underlying YUI DataSource and associated response schema, as well as setting up various AJAX callbacks on our behalf. Great stuff.

We have configured our datatable tag to point at a ‘tasks‘ action, so lets take a look at that now. Its pretty basic:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

package com.alterlabs.twodo
import grails.converters.JSON
 
class HomeController {
 ... 
  def tasks = {
    def list = []
    //
    // Get the current user profile
    // 
    def profileInstance = Profile.current()
    def tasks = (params?.taskType == 'work')?profileInstance.workTasks:profileInstance.personalTasks
 
    tasks.each {
       list << [
           id: it.id,
           status: it.completionDate?"done":"front burner",
           est: it.estimate,
           description: it.description,
       ]
    }
    def data = [
      totalRecords: list.size(),
      results: list
    ]
    render data as JSON
  }
  ...
}

The GrailsUI DataTable expects its response schema to be fed to it in JSON format. Our tasks action basically:

1. Gets the current Profile the logged in user (line:11 )
2. Looks at the params to see which list of tasks to work on (line:12 )
3. Iterates over the task list, adding a map of the Task attributes to a list (lines:14-20
4. Create a Map that holds the new list, as well as the total number of records. This is required by the response schema (lines:22-25)
5. Render the Map as JSON back to our view (line:26)

Pointing our browser to http://localhost:8080/blogtwodo/home/datatable should get us the following:

OK, it may not look like much now, but it proves out the underpinnings. We will use it as a starting point, and continue to refine it in later posts.

Next Steps

Alright. That’s about it for this post. To recap we have:

• Established a candidate look and feel for our application, and have broken out layout in to modular templates
• Provided a means to conveniently get the Profile of the currently logged in user.
• Installed the GrailsUI plugin and used it to populate a crude DataTable via AJAX

The above took about 1/2 a day, or 5 hours or so to make happen (most of which was spent wrangling with HTML & CSS,) which puts our running total of ~7 hours spent so far with ~33 remaining in the kitty.

In the next post we will continue to refine the overall UI and get our DataTable control to allow cell editing of our tasks. See you then!

Technorati Tags: agile, grails, groovy, grailsui, java, software development

This longish post will primarily concern itself with some initial setup and design and the inclusion and integration of JSecurity into our application.

The Vision

We know that our application is going to be a simple task management system called ‘twodo‘– the intent of which is to allow the user to balance their work to-do’s with their personal to-do’s.

When my partner (aka Stakeholder) and I were discussing the application, he drew a simple diagram on a napkin that showed two lists presented side-by-side, the leftmost list for work tasks and the rightmost list for personal tasks. This seemed as good of a place to start with regard to general design, and this simple vision drove much of the basic site layout and internal constructs.

After a few minutes of noodling, the following simple high-level sitemap emerged and seemed reasonable:

Basic Setup

I created a new project (grails create-app blogtwodo) and configured my conf/DataSource.groovy to point at a locally running MySql instance with a newly created/empty ‘twodo’ database.

Basic Domain Model

As stated above, the basic function of the app is to manage two sets of task lists– one list for work, and one list for personal tasks. To this end I created a Profile domain class and a Task domain class.

The Profile will represent a registered user within the system, and will contain the account information and some basic configuration settings. The Profile will also hold a reference to the lists of Tasks, one list for each task type.

package com.alterlabs.twodo
 
class Profile extends Base {
  String username
  String email
  String firstName
  String lastName
 
  int  hoursInWorkWeek = 40
  int  hoursInPersonalWeek = 10  
 
  static hasMany = [workTasks:Task, personalTasks:Task]
 
  static constraints = {
    username(unique:true, size:5..15)
    email(email:true,nullable:false,blank:false,unique:true)
    firstName(nullable:true)
    lastName(nullable:true)
  }
}

Similarly the Task class holds the task details, as well as a reference back to its parent Profile:

package com.alterlabs.twodo
 
class Task extends Base {
  String description
  Date completionDate
  int  estimate = 0  // minutes to complete task
 
  static belongsTo = Profile
}

You may have noticed that the Task and Profile Domain classes inherit from a Base class. From past experience I have found it handy to have a simple Abstract Base Class from which all Domain classes inherit. For now we will use this Base class to enable auto-timestamping for our Domain classes:

package com.alterlabs.twodo
 
abstract class Base {
  Date dateCreated  // grails will auto timestamp
  Date lastUpdated  // grails will auto timestamp
 
  static constraints = {
     dateCreated(editable:false)
     lastUpdated(editable:false)
  }
}

I figured the above Domain objects were good enough to start with, and I could worry about constraints, etc. as the design progressed. I used Grails to generate the controllers and scaffolding (grails generate-all), fired up the dev server and verified my efforts via the browser. Navigating to http://localhost:8080/blogtwodo presented me the expected default Grails welcome screen and listed the Controllers for my Task and Profile classes. (These generated CRUD interfaces are handy for initial testing and once the app progresses in functionality we can remove them.)

So far so good.

Security Setup

Our application does not have any exceptional security requirements– we want users to be able to able to start using the system with a minimum of fuss immediately after a simple registration process. We won’t need any roles for this application– users are either anonymous, or registered. I have used the Grails JSecurity Plugin on past projects and knew it would be more than sufficient for our needs.

To get the ball rolling I followed the steps listed in the JSecurity Quick Start Guide to install the plugin and autogenerate some basic underlying constructs.

The quick start procedure creates a bunch of Jsec-prefixed Domain classes and an AuthController into the default package. I like things a bit more organized so I moved both the Domain classes and the Controller into a com.alterlabs.sec package. As a result of moving these classes we also need to add an import statement to the generated grails-app/realms/JsecDbRealm.groovy file (import com.alterlabs.sec.*) so that it can find the various needed JSecurity objects at runtime.

As per the quick start guide I created a simple conf/SecurityFilters.groovy file. The following code requires authorization to access anything other than the (yet to be created) public controller:

import com.at.sec.*
class SecurityFilters {
  def filters = {
  // Ensure that all controllers and actions require an authenticated user,
  // except for the "public" controller
 
    auth(controller: "*", action: "*") {
      before = {
        // Exclude the "public" controller.
        if (controllerName == "public" || controllerName == null) {return true}
 
        // This just means that the user must be authenticated. He does
        // not need any particular role or permission
        accessControl { true }
      }
    }
  }
}

For purposes of initial testing, I also added some temporary code to the BootStrap.groovy file that creates a hardcoded user (user ‘don’, password ‘twodo’). We will remove this code later as the application progresses.

import org.jsecurity.crypto.hash.Sha1Hash
import com.alterlabs.sec.*
 
class BootStrap {
  def init = { servletContext ->
     // TODO: Remove this stuff later
    def role = new JsecRole(name: "User").save()
    def user = new JsecUser(username: "don",
            passwordHash: new Sha1Hash("twodo").toHex()).save()
    new JsecUserRoleRel(user: user, role: role).save()
  }
 
  def destroy = {
   }
}

To test it I fired up the app and was presented the generated list of Controllers, including an additional entry for the newly added com.alterlabs.sec.AuthController. Clicking on the Profile controller presents the Quick Start generated login page, and I can successfully login with the ‘don’ account created during the BootStrapping. Awesome.

Login and Registration UI

Recall that we want to allow for a simple registration process. Once a user has registered he/she should be able to login and start using the application. Right or wrong, I find that many agile based development efforts derive requirements from the imagined user interface, and that is the direction we will take here. I envisioned another napkin with as screen that presents the user with two basic panels, one for new user registration, one for existing users to login:

Anonymous users will only have access to the public space, and once authenticated they will be sent to a home controller, where we eventually hope to add the meat of our application logic and content. With that in mind I next created both a public and home controller (grails create-controller)

I replaced the default view/index.gsp page that Grails provides with a few simple lines that uses the JSecurity taglibs to redirect to either the home controller or the public controller depending on whether the user is authenticted.

${ response.sendRedirect("/blogtwodo/home") }
 
${ response.sendRedirect("/blogtwodo/public") }

Lets create a crude page, as per our napkin above that will allow the user to either register or login. We want this page to be publicly accessible so this was created in views/public/index.gsp:

 
<!--
#login_or_register {margin:10px;width:500px;}
#login_box {border:#006DBA 1px solid;float:left;width:220px;padding: 10px;}
#register_box {border:#006DBA 1px solid;float:right;width:220px;padding:10px;}
-->
<div class="errors"></div>
<h1>Welcome to TwoDo!</h1>
Register now to get started, or login if you have an account
<div id="login_or_register">
<div id="login_box">
 
      <label for="username">Username:</label>
<input id="username" name="username" size="20" type="text" />
      <label for="password">Password:</label>
<input name="password" size="20" type="password" />
      <button>Login</button></div>
<div id="register_box">
 
      <label for="username">Username:</label>
      <span class="value ${hasErrors(bean:profileInstance,field:'username','errors')}">
<input id="username" name="username" size="20" type="text" value="${fieldValue(bean:profileInstance,field:'username')}" />
      </span>
      <label for="email">Email:</label>
      <span class="value ${hasErrors(bean:profileInstance,field:'email','errors')}">
<input id="email" name="email" size="20" type="text" value="${fieldValue(bean:profileInstance,field:'email')}" />
      </span>
      <label for="password">Password:</label>
<input name="password" size="20" type="password" />
      <label for="vpassword">Confirm:</label>
<input name="vpassword" size="20" type="password" />
      <button>Register</button></div>
</div>

Pointing our browser tohttp://localhost:8080/blogtwodo renders the above (thanks to our redirect) as shown below:

OK, so we won’t win any design awards. It’s ugly but it will serve for now– we will worry about the aesthetics later. Basically we have a page with two forms that submit to actions within two different controllers.

The login form is trivial– it’s simply configured to submit to the existing signIn action within the AuthController that was created as part of the JSecurity Quick Start. Once again easily tested via the browser.

The first thing we need to do is provide the association between the JSecurity constructs and our own Profile object. To do this we simply add an association to the JsecUserRoleRel object to our Profile. The JsecUserRoleRel holds the relationship between a JsecUser and a JsecRole and has references to both– which in turn gives our Profile object access to both. (Note: Yeah I know I said we had no need of formal roles in our application, but it seems silly to paint ourselves into a corner.)

After adding this association to the Profile class now looks like this:

package com.alterlabs.twodo
import com.alterlabs.sec.JsecUserRoleRel
 
class Profile extends Base {
  ...
  JsecUserRoleRel  principle
  ...
}

The second form in the above GSP submits to the save action of the public controller for registration. The registration process needs to take care of the following:

1. Create and persist a new JSecurity JsecUser object
2. Create a new Profile object
3. Associate the JSecUser with the Profile and persist
4. Log the new user in
5. Send the authenticated user on to the home page

Our public controller now looks like this:

package com.alterlabs.twodo
import com.alterlabs.sec.*
import org.jsecurity.crypto.hash.Sha1Hash
 
class PublicController {
  def securityService
 
  def index = { }
 
  def save = {
    def profileInstance = new Profile(params)
    def role = JsecRole.findByName("User")
    def ju = new JsecUser(username: profileInstance.username, passwordHash: new Sha1Hash(params.password).toHex()).save()
    profileInstance.principle = new JsecUserRoleRel(user: ju, role: role).save()
 
    if(!profileInstance.hasErrors() && profileInstance.save()) {
      flash.message = "Profile ${profileInstance.id} created"
      // OK, lets log them in and take them to their home page...
      try {
        securityService.authenticate(params.username, params.password)
        redirect(controller:'home',action:'index')
      }
      catch (Exception ex){
        // Authentication failed
        flash.message = message(code: "login.failed")
        def m = [ username: params.username ]
        redirect(action: 'index', params: m)
      }
    }
    else {
      render(view:'index',model:[profileInstance:profileInstance])
    }
  }
}

NOTE: I have encapsulated the authentication logic from the JSecurity Quick Start AuthController into a SecurityService, which is called on line 21 above. (This is left as an exercise to the reader :) )

Once again we can verify our progress by using a browser to explicitly try out the registration process (and in this case I also created a few unit tests.)

Next Steps

OK, not a bad start. At this stage I had spent perhaps 1-2 hours on the effort, with a decent amount to show for it. We have created our core Domain objects, wired in security and have created the self registration process. The next post we will look at designing the View layouts, refining the User Interface, and adding some of the core functionality into the mix.

Technorati Tags: agile, grails, groovy, jsecurity, software development

The high-level constraints governing the experiment were as follows:

• The application should be web-based and ‘modern’ (AJAX, friendly URL’s, etc)
• It should be reasonably non-trivial.
• It should be developed as it were going to someday become a Software As A Service type of application.
• It should involve some new/untried/unfamiliar technology– just to make things more realistic/interesting
• Where possible, we’d explore some geeky ‘bells and whistles’

My biggest concern was the time constraint. 40 hours is not a heck of a lot of time, and given the other requirement of having to deal with some other unfamiliar technology I needed something that would allow for RAD. The resulting architecture for the application ended up looking something like this:

• The application was designed as a Grails (v1.1.1) application. No surprises here. I have used Grails on a number of prior efforts, and its my framework of choice.
• The more sophisticated/AJAXy UI components were provided courtesy of the excellent GrailsUI Plugin- which in turn is based on the excellent Yahoo UI JavaScript library.
• We decided to play around with deploying the application into a Cloud Computing infrastructure like Google’s AppEngine or Amazon’s Elastic Compute Cloud (EC2) platform. I had no experience with either prior to this effort.

The Cloud Computing aspect was the real twist — In one fell swoop, we attempted to meet the requirements for SaaS, ‘new or unfamiliar technology’ and high geek factor. As a bonus it could provide an online presence so that readers could visit and play with the app.

This post and a series of subsequent ones to follow, attempt to document in reasonable chronological order, the details associated with realizing this challenge. That is, the information is presented in the blog posts in the approximate order it was experienced in ‘real-life’– with a couple of notable exceptions.

In broad brushstrokes the application development cycle looked something like this:

1. Build a standard Grails application locally that meets the basic functional requirements.
2. Take the prototype from Step#1 and port it to Amazon’s EC2
3. Take the prototype from Step#1 and port it to Google’s AppEngine

The above approach seemed reasonable, and at the time I started this endeavor, I had signed up for Google’s AppEngine Java SDK access, but had not yet been granted it– so implementing an EC2 version of the application first seemed only reasonable. More on this in later posts.

OK! this is the last of the non-technical intro posts. The next post in the series will talk about the design and construction of the basic/local application.

Technorati Tags: agile, grails, cloud computing, appengine, ec2, software development

An Act of Congress to get this info?

Most organizations we’ve worked with have some tool they track issues with: jira, bugzilla, trak, good-old-excel. You name it.  So, in general, the mechanism for collecting issues should not be a big stretch.

On the other hand, the process of being diligent enough to log issues is another matter. From our experience in the Agile setting, those issues which get ‘parked’  by project members during a stand-up/scrum for further discussion are obviously clear candidates for further analysis and tacking. Clearly, during the scrum is not the only time that issues arise. The key point here is to be diligent about tracking issues. That being said, one of the key roadblocks to being diligent is insisting on excessive details about the ‘attributes’ of an issue. Now, there is always the respectable goal of having super fine-grained data that you can report and track progress against, but we all  know this fine-grained reporting rarely happens. Which leads us to …
Issue “Priority”

Many teams  go overboard on trying to assign priority to an issue. ‘Low, Medium, Medium-High, High, Super Duper High, Nuclear’ and variants are not uncommon to see in a priority field. At the end of the day, these designations tell us little-to-nothing about the significance of the issue. In our opinion its more fruitful to use a few meaningful priorities that convey the impact of an issue. For instance: full stop (all work being blocked), progress slowed (work continues, but decisions are needed fast), work continues, impedes/complicates some other function outside of development. Again, from the Agile lens, we care about things that are going to affect progress (i.e. velocity). There’s no need to split hairs by having umpteen variants of “work is being impacted.”  If its in the sprint plan, its important; period.

Issue “Statuses”

Status is another area where teams tend to get lost in trying to convey information that is — well — better left for a status report. From an Agile perspective, an issue is either: impacting this sprint, will impact the next sprint, will impact some other future sprint, is resolved, is reopened. So if an issue has priority of ‘full stop’ with a status of ‘impacting this sprint,’ its logical that it should get a lot of attention because you can be guaranteed that velocity is being affected as you speak.

So, when its all said an done, the attributes associated with an issue should basically boil down to: who’s responsible, which sprint, the open date, the close date, the priority, the status, and description. Anything else that is made a requirement for entry is either ‘nice-to-have’ or is getting in the way of effective issue management.

So, what to measure?

Okay, “step down from the sopabox, Sunjay.” So, what is it that we’re analyzing. Well, there are fancy names (in bold) for these things, and there are common sense (in italics) descriptions that help understand why its important to track. We’ll give you both.

• Issue Resolution Velocity – how fast issues are being closed: what’s the sprint/project average, mean, moving average? What is the open count of high priority issues versus all logged to date?
• Issue Leakage – are issues like zombies — being re-opened?: what’s the sprint/project average, mean, moving average?
• Issue Growth Rate – how is the log growing/shrinking?: how many new? how many outstanding, what’s the ratio of new to open?

So at the end of the day, the ‘issue’ category and the data that comes from analyzing it provide important fodder for the agile retrospective. Its a leading indicator that can be used to assess why velocity is slipping or what might happen with velocity in the future.  If you can, its probably a good idea to add it to your toolkit.