Overseas...

So what happens to the agility when the project is distributed? Well, it suffers...

It's not that it doesn't work in a distributed context. In fact ThoughtWorks has been doing distributed agile for quite sometime now. We talked about some of the best practices of agile earlier. Let us see what problems distributed agile poses to these principles.

Time difference
When working across oceans, the first thing that hits the team is the time difference. The typical arrangement renders the team to be in a low cost country like India and the client in the US of A or Europe.

The time difference is the first major barrier to communication. Because even if we assume that the other barriers like distance, language, culture, etc can be handled, we just wake up and live at different times.


Communication
As much as agile believes in face-to-face communication, it suffers badly in a distributed context. Mitigation consists of:

1. Cross pollination
2. Using Voice communication rather than mails
   
3. Video Conferencing
4. Adjusting working schedules on both sides

These techniques are most effective with mirrored teams. Of course, that's the essence of a distributed project, but it gets missed quite often in that the client site is left with a couple of analysts and the development goes on offshore. The mirrored team aims at having all roles replicated on both the shores which encourages more communication amongst the team members.

Documentation
One more thing hampered is the amount of documentation, which increases almost exponentially...

As the customer is sitting elsewhere all the analysts can't talk to the customer face-to-face which adds to the documents and artifacts created. Also for transparency and quicker feedback sake, tools like online bugtracking systems are used.

The whole effort towards reducing the feedback time and working closely with the customer comes with major overheads in distributed contexts.

Cost and Benefits
From a customer perspective, distributed development reduces cost significantly. The organization seemingly doesn't reap the benefits of distributed development in terms of profit margin due to the increased cost of travel, communication, etc.

But the benefit of having a distributed team like:

1. Better team skillset (as compared to on-site)
2. Reduced costs (as compared to on-site)
3. Reduced culture shocks to customer (as compared to offshore)
4. Improved business context (as compared to offshore) because people living in the same country have definite advantage in understanding the business
5. General cross-pollination leading to better diversity and a more cohesive work force overall in the organization.

Selling Distributed Agile
There are some important points that we need to keep in mind when selling distributed agile (even agile) to clients. Setting these customer expectations goes a long way in favour of project success. Other than the cost / benefits decribed above:

• Expect lots of communication. Mostly in the form of voice communication rather than email
• Expect much more participation than any waterfall project your organization has been a part of
  
• Expect having to give feedback right from the beginning of the project.
• Expect software to grow (an agile team will rarely go for place holders)

A related and more detailed article by Martin Fowler can be found here

Agility...

The key practices of agile software development, known and followed are:
1. Face-to-face communication over documentation
2. Quick feedback from end users over project phases (as in waterfall)

Collective ownership, Pair Programming, use of whiteboard, paper, and most importantly the terrific visual representation of the project health and progress... the wall

For an agile team the wall is everything. The action of moving stories & bugs along the columns and the instant snapshot of the project health is fantastic. Here's how it works...

The process :

1. The Business Analyst analyzes the requirement and writes a story and acceptance criteria
   
2. The developers estimate a bunch of such stories in a planning game
3. The developers (in pairs) then sign-up for a story each morning
4. When the development is done, the story is tested for desired functionality by the analyst
   
5. After the analyst has signed the story off, the tester picks it up
   
6. The testers and devs play ping-pong for a while with the issues (bugs) raised by the former
7. When the tester signs off the story, it's ready for the client to have a look at.

This procedure takes ideally about 3 days per story. The client can then look at the story and sign it off or (ask for some changes) !!!

The documentation per say in this complete procedure is an index card for writing the story, some more for the acceptance criteria and a few more for the bugs.

• The client and the analyst discuss the requirements face to face probably using some paper to make rough screen sketches sometimes.
  
• The analyst talks to the dev pair that signed for the card and describes the functionality.
• The devs use whiteboards & pair rotation for design discussions, issues and knowledge sharing
• The devs keep bugging the analyst all the time for clarifications
• The tester talks to the devs about issues and only writes a bug on a card if it can't be resolved then and there.
  

For anyone roaming around, the color coded, columned wall with the stories and bugs gives a clear picture of the iteration health. And a team can easily setup the wall to show the project health coz it's the team that decides the color codes, the statuses (columns), etc on the wall.

It's the team that decides when to start the day, what coding conventions to follow, the core hours of work and it all works out so smoothly because the client is part of the team!!

Now how much better can it get?

Value..

Stories being testable and valuable is as important as them being small and independent. Perhaps more... because these are the two important characteristics which in fact drive the size of the the story.

Like we saw that an agile analyst can sometime fool himself by creating independent stories, one can also blunder quite easily when it comes to testability and value of stories. Let's look at an easy example:

Imagine a customer entity in a system with about 50 odd attributes. Creating one story capturing all these attributes will be a real mess. An analyst would (and should) readily want to break the story. What he/she might end up doing, however, is breaking the story thus...

• First story for creating the customer domain object, giving values for all the attributes (in code) and saving it to the database.
• Second story for creating the screen
• Third story for adding validations.

Task & sizewise, this will work fine. But where is the value? What are we actually delivering to the user that he/she can use?

Another problem is the testability. What an analyst should remember is that a story should be testable by the end user... not through code.

So what does one do in these kind of situations? Well, maybe (and I am quite sure) that there are different types of the customer. Breaking the stories so as to capture details for each type can be a great start. This is called a vertical split.

In situations where this is not possible the team might have to resort to internal breaking of stories. The catch is it can be shown to the customer only as a complete functionality, which lengthens the feedback cycle which defeats the whole purpose of breaking the story!!!

These are the situations to beware of... But then that's what we get paid for right??

The perfect split is somewhere out there and finding it gives an analyst immense happiness... and who could help you better at that than the client?

;-)

Chicken and Egg

Managing dependencies is another tricky issue that plagues agile business analysts. Fortunately it's not as bad as the chicken and egg problem. The secret to successfully solving dependency problems is to start at the beginning... Well maybe an example will help.

Let's consider a system that sells books online. The roles which will interact with this system, will be
- the buyer
- user registration
- the administrator
- the sales team and
- the warehouse manager.

The most important entity in this system in undoubtedly the book.

Starting to write stories for creating a book is an obvious start for anyone. Now this is where the situation gets tricky. If all your initial stories are going to be related to books, they are inevitably going to be dependent on each other. For example :
View book details depends on Listing books which depends on Create book

Writing such stories doesn't make sense from a tactical perspective when we are doing iterative development because we can't develop such stories in parallel.

Instead if we decide to write a Create Book and a Register User story, we are suddenly in business. What this means is that we can start separate areas in the application simultaneously.

Sometimes however things are not as simple as that. There are various restrictions which can lead an analyst to write stories in the same area. What will typically happen is the analyst will write Create Book, List Book and View Book Details stories and get them played with valid assumptions and dummy data. The analyst has thus created independent stories... The consequence is the cost of integrating these pieces together.

There are several points in the application where everything comes together to form a complete picture. Integration of separate stories into a logical area and separate areas into a seamless solution is inevitable. What we need to understand is that reducing these integration points is optimal in terms of overheads.

Thus an analyst needs to take a call in consultation with the developers about finding v/s creating independent stories. Finding them is however definitly advisable.

If I were write user stories for evolution I think I would have developed the Chicken and the Egg seperately and integrated them in the end. Probably that's how it was done anyways!!!

Small Stories, Large Stories

One question that keeps analysts busy for a most of the time is whether or not to split a story. Here's my philosophy behind my unending support for small stories.

Let's start with one huge story that covers everything that is required to be done in a project. It is estimated by the developers at say 120 complexity points which converts to about 300 days. Following are the consequences.

• A requirements document (the story) - coupled with the time taken to prepare such a document
  
• The techinical discussions and decisions for implementing these and of course the time required
• The implementation of this story
• The testing and
  
• The user acceptance (well usually non-acceptance and it takes very less time too :p )

This is what we do in a waterfall approach towards software development. We are looking a 300 days feedback loop because that's when the customer will actually see the application.

Agile works towards shortening each of these phases and doing them over and over again until all the requirements are fulfilled. Thus everything boils down to the feedback cycle you are looking at. Let's look at a shorter feedback cycle, say 100 days. Thus you end up with 3 stories with an estimate of 40 complexity points converting to 100 days. Which means you get feedback thrice during the project because the customer reviews the functionality delivered by each 100 day story as soon as it is completed. That's somewhat better.

Let's just try and make this a 10 day feedback cycle. We thus have 30 storys with an estimate of 4 complexity points converting to 10 days. What we are looking at is basically a 2 week iteration. Now that's really good. I'd however go for one more split so my stories have a complexity point estimate of not more than 2.

The upside
The upside other than what we established above is that developer estimates are generally exponential (although they are supposed to be linear). Which means that one 4-point is not equal to four 1-point stories. It is exponentially bigger. Thus the smaller stories you have the lesser will be the actual time taken to implement, test, get feedback, make necessary changes and deliver.

The downside
Well, there is no downside. The only thing required is constant interaction with the customer for analysis of new stories and review of completed stories.

Todo
An agile team should set customer expectations appropriately before starting the project. The customer participation is the heart of the project. Successfully establishing a short, reponsive feedback loop is the first and foremost thing to do when doing an agile project... and the stories, keep them short.

Small is beautiful...

Once upon a time...

User stories are the basis of business analysis in an agile project. A user story is an artifact which captures a small but valuable piece of business functionality which can be implemented and tested independently.

The philosophy behind agile methodologies is basically iterative development with short & fast feedback cycle which is used in improving the solution and accomodating changes (subject to priority) so that the end result is really what the customer wants. Inevitably this creates a unique position of a business analyst as a broker between the customer and the developer. Writing user stories is what this broker spends most of his time doing.

There are numerous ways of writing user stories but the goal is to capture a "role", an "action" and an immediate "goal".

Like:
As a Business Analyst
I would like to write my stories in "As a, I'd like to, so that" format
So that I can capture the role, action and goal easily

Ok, that's not an awfully great example... but the general idea goes thus.


User stories should be

1. Small
2. Independent
3. Testable and most of all
   
4. Valuable to business
   

and of course the next question is why.

This leads us to look at the meaning, purpose & priority of each of the above.

Valuable
The user story should deliver a valuable piece of business functionality. This is important so that the story can be released and used by the real users as soon as it is developed, thus giving the team a faster feedback.

Independent
Independence is a very important characteristic of a user story. When a story is independent, it can be picked up for development anytime during the iteration. It can deliver a valueable business functionality independent of the rest of the feature.

Testable
A story needs to be testable. Which means that the desired functionality that a story aims to deliver should be capable of being demonstrated, for all related business scenarios.

Small
A user story should be as small as possible... possible keeping all the above characteristics in mind. You can choose to split a functionality into shorter stories as long as it remains Valuable, Testable and Independant.

This is the primary job of the broker and what's challenging about it is what I'll be posting hereafter alongwith my version of the solution.