I finished reading The Phoenix Project at the beginning of May and it has had a profound affect on my outlook towards work. Not only was this a great read, but while I was reading I kept getting ideas on how I could implement the three ways at work. This diary documents the transition to an Agile Writing workflow.
Why bother?
Before I start covering the how, I thought I would spend a little time on the why. Agile is something that is usually associated with Software Development and you might be thinking "What has this got to do with writing?". I write technical documentation for a living, but I've been a long time have-a-go programmer and I recently had the pleasure of working on a professional software project. One of the things that amazed me was how Agile mad them so much more efficient.
- Scope creep was eliminated through issue and requirements management
- Many application releases gets product in the customer's hands sooner
- Team maintains a constant velocity and is always striving to be more efficient
- Everything that can be automated, is.
It is actually quite easy to draw parallels between application development and writing technical documentation. A document's markup is its source code, it gets compiled when it is prepared for Web/Print, it has regular releases, I could go on…
Document Markup != Source Code. Enter Markdown
Most documentation is written in a word processor. This is an issue for a number of reasons (don't get me started on application stability) the paramount of which is that the document formatting is embedded with the text itself. If for whatever reason I need to change the style of the final product, a frequent occurrence in the marketing department of a blue chip, I need to physically edit every document. Enter Markdown.
Markdown started out as a text-to-HTML conversion tool for web writers. It uses simple syntax to express headings, hyperlinks, images et. al. and most importantly it is a plain text format. Since it's initial inception there have been many additions to the standard Markdown syntax to support tables, references, citations and a whole host of other features. The two candidates for use in my project where MultiMarkdown and Pandoc - I chose Pandoc in the end, but that is a blog post on its own.
Distributed Version Control Systems
Getting Started
The first thing I set about doing was setting up a Distributed Version Control System. Version Control fulfils a number of roles in my documentation process:
- It removes the need to use SharePoint to hold backups of the project
- It tracks changes across every file in the project, allowing me to diff between versions
- Multiple people can easily work on a document at once
Having used Git before and taken a liking to the notion that you have your own local repository from which you can push changes back to the master (very useful for prolonged periods of being offline) and the branching/merging capabilities I decided it would be a great choice!
Installation
Setting up a Git server isn't hard, especially with the comprehensive Git book being available for free under the Creative Commons licence, if you find this book useful I'd encourage you to buy a copy to show your appreciation to the author. The only issue is that it isn't necessarily easy for non-developers to use.
GitHub on the other hand is great! but it's outside the company firewall… Fortunately Atlassian have a product called Stash, which takes the edge of managing Git repos, and also integrates with their other products. At $10 for a 10 user licence it's great value for money.
Installing Stash was a little more challenging. Unfortunately it doesn't have a nice packaged installer like some of the other Atlassian products. Fortunately you can easily meet the prerequisites:
sudo apt-get install git-core perl
Follow the instructions here to install the latest Oracle JRE and JDK.
Then it is just a case of following the Stash install guide and then Running Stash as a Linux service.
Stash gives you the choice of using it's internal HSQL database or another supported SQL server. I strongly recommend the latter, after having to troubleshoot some JIRA integration issues.
Optional - Setting up a Postgres database server
I spun up another VM as my Postgres server on Ubuntu 12.04 LTS by following the guide here. I then created a "stash" database, a "jira" database and the appropriate users.
Configuration and Use
Once Stash is ready to go, you can just browse to http://hostname.yourdomain.com:7990/ and follow the on-screen guide. I added my users then set up a project for each of my current projects and created a repository for each.
On my Mac, I followed the guide here to get Git set up.
Stash provides instructions for how to commit code to your newly minted repo, but in a nutshell:
cd /my/project/directory
git init
git remote add origin
https://hostname.yourdomain:7999/project/project.git
You can check the status of your repo as follows
git status
This will tell you which files are tracked, and those that are not. To add all files and commit you do:
git commit -a -m "Initial Commit"
This updates your local repo. You can then push this to the remote:
git push origin master
If you don't enter a commit message when using "git commit" you will be placed in the message editor specified in your git configuration, which defaults to vi. If you wanted to use nano:
git config --global core.editor "nano"
If you prefer a more visual approach to working with Git, you can use SourceTree also from Atlassian which is free!
A quick note on folder structure
I like to keep my projects organised, so I use the following folder structure
project
|- assets
|- bin
|- img
The markdown file goes in the root directory as I am currently using a single file per project. In a more complex setup with multiple markdown files I'd create a "src" directory. "assets" is for everything related to the project that isn't code or images. OmniGraffle drawings, MindMaps, OPML outlines etc… "img" is for the images used in the document. "bin" is for the final product. I'm still working on my build process, but I would imagine that I would have a folder here for each build target "html", "epub", "pdf" and these would contain the final product.
JIRA and GreenHopper
So now I have my documentation under source control, I needed to pick a product for requirements and issues tracking. Having seen JIRA in the past it was an easy choice for me and the pricing of $10 for 10 users is also very cool. The GreenHopper plugin for JIRA adds Agile goodness and is an additional $10 for 10 users but well worth it in my opinion
Installation
JIRA was easy to install using the installation guide.
Configuration
Configuring JIRA was actually quite easy, other than the custom tweaks I made. The first thing I noticed when creating projects was that each had a 3 letter acronym or project key. This was an issue for me, as all my document types have 3 letter acronyms and using ABC1, ABC2, etc.. would make it hard for me to identify a project from it's key, which is used quite frequently in JIRA. I settled on using ABC_NAME as my project key. This was easy to change in JIRA using the document here and for those interested the RegExp I used was ([A-Z]{3}[_][A-Z0-9]+)
One thing of note is that Atlassian claims that GreenHopper and Bamboo only support it's default project key. So far I've found that my pattern works with GreenHopper without a problem. The only quirk I have noted is that the underscore is rendered as a space on my Agile board. I can live with that.
The JIRA 101 is a reference for getting you up and running.
Like most people, I work on multiple projects at once. To be able to effectively plan across projects I created a new Agile board, and adjusted the filter to encompass all project with a specific tag. This way I can plan for multiple projects of the same type on the one board - Awesome!
Integrating JIRA and Stash
Integrating JIRA and Stash was tricky as I had used a custom project key. I followed the document here and noticed some quirky behaviour. Essentially it was not recognising my Project Key.
I found a note here that was some help to resolving this issue.
I stopped stash and modified my configuration:
sudo nano bin/setenv
Find the line JVM_SUPPORT_RECOMMENDED_ARGS and modify as follows:
JVM_SUPPORT_RECOMMENDED_ARGS = "-Dstash.jira.key.pattern='([A-Z]{3}[_][A-Z0-9]+[-][0-9]+?'"
The thing to note here is that the RegExp should not match that in your JIRA configuration, as that matches the project key. Instead you need to modify this to match the issue key, so I added [-][0-9]+?
Before you restart Stash you should clear the cache
sudo rm -r stash-home/caches/idx-snapshots/*
And you may need to remove any lines from the Stash database that don't look like Issue Keys. Follow the instructions here. This is much easier with a real SQL server in the back-end as opposed to using the built-in HSQL database.
Usage
So now I have my projects set up in JIRA, I can start to adopt an Agile approach.
With my product owner hat on, I start writing Epics (big goal-oriented user stories) and then break these down in to smaller stories that can be transformed in to product increments.
When a document is in review, or release, I can raise Bugs against the current version to identify things that need to be fixed. Eventually I'd like to extend access to my JIRA to all of my peers and users so they can log bugs themselves!
When I commit code, I can include the issue number that I was working on. When I look at the issue in JIRA, I can see relevant commits under the "Source" tab.
On the Agile tab, I can use GreenHopper to plan my sprints from the issues/stories in the sprint Backlog. I can assign Story Points to stories in the Plan view, Start a sprint, track it progress and move tasks from pending to in-progress through to completion. Perhaps the best thing is that I can pull up a Burndown chart to track my momentum - and manager's love charts!
Conclusion
It's been a busy week getting things set up, but now I have 3 servers - 1 for JIRA, 1 for Stash and 1 for Postgres. I have just planned and started my first two week sprint and already I'm noticing that documentation is proceeding toward release milestones at a faster rate. This is mainly due approaching planning which a focus on hitting many smaller releases rather than the one final release that I was used to. What next for week two? Well I'll be progressing along this sprint, planning my next sprint and looking at how I can automate my build process and perhaps use Continuous Integration. Stay tuned.
@dave_tucker