Features V/s Sophistication

Usefulness of software:
Letting the user do what s(he) wants easily and fast.

Features:
Number of activities a user can perform with the software.

Sophistication:
Software intelligence which simplifies user interactions.

Software, like anything else follows the laws of Diminishing Marginal Utility. Here are the graphs as per my experience with software (Not as a software professional but as a simple user)




Features
I think the marginal utility of adding more "features" to the software diminishes at a much higher rate.

• Firstly because individual users don't use all the features but have to pay the price, in terms of money as well as processing power, disk space, etc.
• Secondly because users have to deal with all the "genericness" of the software and hence cannot do what they want easily and fast

Best example of such software is (my beloved) Lotus Notes.

Sophistication
The marginal utility of making software intelligent also diminishes albeit at a much slower rate. Google Calendar Quick Entry is a good example of software intelligence. So is GMail's drunk email protection.

What keeps GMail, GMail is that everyone (creators and consumers) abides by the simple rule that GMail is for sending and receiving email. If you want to do something else, you will have to use something else.

There are a couple of software that come to mind which do have a huge feature set but still keep the crux simple enough. Microsoft Word for example has done it rather well in that I can go in and create a simple document as easily as I could when it ran on Windows 3.1

As a business analyst this translates into a simple rule:
Know the crux of your application and stick to it.

When you set out to start a new project, get your users to agree upon the problem to be solved and do whatever it takes to solve that problem; not a penny more, not a penny less.

Software for Everything is as enticing a concept as the Theory for everything. I think there's a reason we haven't found it :-)

Cumulative Flow Diagrams

Once you have your wall in place, it’s time to start monitoring your progress through the project. Best done on a daily basis and best done with a “Finger Chart”. Here’s an example.

A finger chart is basically your wall turned sideways so that your swim lanes are horizontal rather than vertical. Now what you track is the number of stories in each state (finger).

This chart may not make sense to you immediately but there’s a lot of useful data hidden in here. Here are the different inferences you can make by looking at a finger chart.



1) Bottlenecks
The first thing that you monitor on a finger chart is bottlenecks. This is easy. Just check for a band thickening consistently. A band thickening means that there is either a capacity problem or a capability problem in the next band. For example, the BA/QA band consistently thickens a suddenly rises at specific intervals. This is because the Client is signing off the stories only at the end of each week. This may be due to various reasons which need to be investigated to see if something is wrong OR can be made better. The ultimate goal is to achieve a smooth graph.



2) Work in progress
At any given point if you measure the total across all bands vertically, it will give you the total work in progress. This is a total of everything that needs attention from someone or the other on the team. Always try to keep this as low as possible. You should ignore the Celebration State for this calculation.

3) Average Turnaround Time
If you measure the time across all bands horizontally, you get the turn around time for a story. This is a good measure of your entire process. How lean your processes are, how good the team is, how involved and keen the client is, etc. This is what tells you how much time your team needs on an average to turn an idea into working software.

Remember that the finger chart is a cumulative of all stories. Therefore the top line of the graph should never dip down unless you have really reduced scope.

Now whether to track Number of Stories OR Story Points on a finger chart is an interesting debate. I will try to present my ideas on that in another post.

UPDATE : As David pointed out these are called Cumulative Flow Diagrams in the Lean world. Finger Charts is/was more of a ThoughtWorks term.

Choose your lanes

Starting a new project is always fun. For Agile/Lean teams setting up the story wall is a part of the fun.

Some would say you are defining the process that you want to follow. The life-cycle of the user stories from being analyzed to delivered. But there’s more to a wall than defining the story lifecycle. In its wall the team chooses what it wants to monitor.

The lanes on the story wall are statuses in which the story can be. An agile team wants to track these statuses to quickly find problems that are slowing the team down. These problems can broadly be classified into two categories.

Capacity Problems – Found by tracking Wait States

• There is a bottle neck because people have too much on their plate
  
• Can be solved by
  
  
  
  • Adding more people
    
  • Finding out better ways of doing the work
    
  • Prioritizing the work correctly
    
  • Making the process leaner by removing statuses
    
  
  

Capability Problems – Found by tracking Work In Progress States

• There is a slowdown the right people are not doing the right job
  
• There is are problems external to the team which are slowing the team down
  
• Can be solved by
  
  
  
  • Training people if there is enough time
    
  • Bringing in specialists who can help out
    
  • Addressing external problems
    
  
  

Monitoring every state that the story is in is ideal but not optimum.

So choose your lanes keeping in mind what you want to monitor.

When something is not happening on its own, make it a ritual by adding a lane.

Keep your process lean by removing unnecessary statuses from the wall as well as the process.

Can't negotiate size... and a good thing too

In many companies the Business Analyst or the Project Manager “estimate” the project and bind developers on the project to that estimate. This is an absolutely brutal thing to do to your project. But even within agile teams where the developers estimate, the Business Analysts or Project Managers sometimes tend to challenge and negotiate developer estimates. Not because they don’t trust the developers but merely out of selfishness. Because it gives a tremendous kick to deliver something that a client wants faster.

Now if the team is “estimating” in a “time based” unit such negotiations make some sense. Because a developer might say “Well I’ll take 3 days to do it but developer X is really good at XML so he can probably do it in 2 days”. And that might help given that you can assign the task to developer X (who in all probability is good a everything).

Another way of doing this is for the team to “size” user stories based on relative complexity. Here’s how it works.

As a developer on the team you are only classifying the stories into buckets of high, medium or low complexity. Nothing you can do is going to change the complexity of a given story.

As an analyst or project manager on the team please don’t ask developers questions like “Are you sure this is a 4-pointer?” The only one who can simplify stories is you.

Something Faster is better than Everything Later.
Everything Faster doesn’t exist

What are we doing?? - Part Three

This is what got me thinking about "What (the %*&$) are we really doing?". Are gazillions of dollars being invested in IT, by organizations just trying to maintain their status quo? As the study shows, this is partly true.

1. Businesses require to grow for survival.
2. Growth requires accommodating more and more people and systems into your business.
3. It is very natural (and useful) to be paranoid of these new people / systems.
4. As businesses grow they have to become more or less bureaucratic.
5. Bureaucracy requires additional processing power.
6. This power can either be man OR machine
7. If the business chooses to invest in machines, we, the IT guys, make some money.

So that is what we do.

We help businesses grow...
...grow by overcoming their paranoia by automating bureaucracy...
...bureaucracy which is only bound by the stakes...
...stakes which are ever growing.

To be honest, I am still not totally convinced that this is all what software is about. What do you think?

What are we doing?? - Part Two

Any system is inherently trust based but always tends towards being distrustful. Consider a prisoners' dilemma example.

(A) = A crook with a bag of jewels
(B) = Another with $100

(A) needs the money and (B) needs the jewels. Now they are both "if you see my face, I'll have to kill you" kind of guys. So they develop a simple system. (A) leaves the money a designated place in the forest and (B) leaves the bag of jewels at another designated place.

If they both co-operate, they leave happy. But life is not so simple, is it?

Once they start losing trust in each other. They will employ various methods, making the system more sophisticated in the process, so as to make sure that they are not tricked. Here's what they could do (this list is not exhaustive ;-) )

1. (A) decides to send a friend to the place where (B) is supposed to leave the bag to see that (B) does leave the bag
2. (A) realizes that even if (B) leaves the bag, how will (A) know? So he decides to send two friends, one of whom shall come back and inform (A)
3. (A) grows suspicious of his two friends, coz they might run away with the jewels, so he has to keep their children as collateral. He appoints a person to guard the children and has to feed them as well.
4. The friends say that if (B) leaves an empty bag, it's not their fault. Which is fair, so they end up carrying mobile video cameras that transmit back to (A) on his mobile receiver...

... and so on.

Well actually at step 4 (A) goes "All this for $100?? Not worth it at all". Wise decision but if the stakes go higher, (A) will end up investing in the hi-tech solution which only improves his chance of not getting cheated by a fraction. By the way all this while (B) is also employing similar tactics.

So here we have two parties willing to invest in more and more complex systems as long as the stakes seem high enough for them. And here are we, the IT people, who will help them do it.

To be continued...

What are we doing?? - Part One

I am doing what we call an Inception for a new project. The situation is not at all unusual.

The Constraints

• An Organization with a bunch of generic e-commerce platforms
• Live customer sites on each of these platforms
• One of these platforms particularly painful
• Integration with proprietary back-office operations software

The Requirements
Build a custom platform that will replace all the existing ones, support all the customer sites and have a bunch of unique features. And BTW we need to go live in 2 months.

How?
The only way to do this is build a bare bones application that can take a couple of simple customer sites live in 2 months. We can then keep adding to the application and making it richer and better and migrating more customer sites as and when possible.

Now whenever you try to skin an application the first ones to suffer (and rightly so) are the administrators of the system. I say "rightly so" because normal users can potentially be dumb, careless, in a hurry, etc. and the application has to make sure they are doing the right thing.

Administrators on the other hand are trained to maintain the application. They can do without a jazzy UI and help and pointers all over the place. They know how to do their job. We, as designers of the system, take the liberty of trusting them. And in that trust lies the topic of this series.

To be continued...

End of a good project

For the past 8 months I have been working on omio. We got it from being just a mobile deals aggregator (for the UK) to also being a social network around mobile phones with reviews, ratings and stuff. Ruby on Rails was fun too... but then, that's always true, isn't it. Today was the last day of the project.

I have to say it's been a blast all throughout. First in Chennai with a new office and an extremely smart and totally insane team. Then in Pune with Sid, Aman, Deep and Anand... lot of fun.

Sid is rolling onto a .Net project and I did not try to hide the devlish pleasure I got watching him install Windoze on his machine :P But well, it's time he got out of his comfort zone. These rubyists... I tell you.

Aman, Deep and I remain on the beach (or bench, if you will) for a while. Hopefully something good will come out of it. Aakraman anyone?

Business Analysis and Quantum Physics

I recently read In search of Shrödinger's cat. It's a wonderful introduction to quantum mechanics; The basic (and intriguing) problem of the elusive electron acting as a wave and a particle at the same time.

Analysis is very much like observing the electron in the classic two slit experiment. The business domain is like a wave traveling through the two slits. But the moment we analyze, we make it boil down to one reality by asking it How, What, When and Who.

Waves give you a probability distribution, while particles give you one of the possibilities as a reality. Looking at a business problem, in the wave form is better since it let's you choose a reality that solves the problem well.

For Example: Let's say the business problem is "I want to let my customers know of the new offers I have so that they can take advantage of the offers and I can make money"

This is the problem in it's wave form. The reality can be one or more of email, website updates, RSS feed of offers, people going door-to-door, mobile advertisement vans and a million other options including telepathy, albeit with low probability perhaps.

The problem is that clients come and say "I want to let my customers know of the new offers I have by email so that they can take advantage of the offers and I can make money" and it takes a while to get into a mode where you challenge every reality and go back to the wave form, by asking "Why"? Why is the only question that let's you go back to the wave form.

The more we procrastinate our observation, the better will the chosen reality suit our problem. Agile takes this approach with the concept of the last responsible moment for a decision. It let's future remain future and hence full of possibilities.

Smart Business Analysis

A bunch of business analysts have started this blog recently. I hope to contribute to it more regularly than I update this blog. I will keep cross-posting here and linking good articles.

I hope to save smart business analysis from some of my more weird philosophies and analogies in the future. I thought I'll pick up a good practical BA problem for the first post there but ended up writing about "Business Analysis and Quantum Physics" for some reason.

Size or Estimate

Estimation, I think, has been one of the biggest problems of mankind. The Software development industry is no exception to this sad reality. We almost never seem to get it right.

The problem with the word "estimate" is that it's personal. It's relative to the person who is estimating. Moreover, it's a unit of time. That is why I prefer "sizing" stories rather than estimating them. Size is a non disputed (standard) unit, which measures the level of complexity involved in developing a piece of functionality, a story. The size is standard across a particular project team.

Pizzas are a great example of standard size. They come in small, medium and large sizes and, given the context of the vendor, the whole world knows what a small pizza means.

Pizza Appetite is the capacity to consume pizzas. A 5 year old will find a small pizza to big to eat. A teenager will probably find it perfectly filling. At 25, I find the small pizza too small. I can comfortably eat a medium pizza (or 2 small pizzas). I am sure there are people I don't know who can eat nothing less than a large pizza (or 4 small pizzas)!!

Stories are like pizzas. Given the context of a project, they have fixed sizes. As a developer spends more time on a project, getting older (and wiser), his/her capacity to develop complex stories increases. And this is where the trouble starts. At the beginning of the project the developer could do 1 small story every day. But now, instead of realizing that he can do a medium story (or 2 small stories) in a day, he begins to think that the medium story is actually a small one. The thing to remember is that the size of the story remains the same. It's just that the developer can now do more complex stuff in a given time.

Complexity points are never to be estimated (in fact they can't be estimated because they are not relative to individuals). In the OLAP terminology, complexity points are the fact. An estimate is a calculation you (as an individual) do when you are looking at them from the Time dimension.

So if you are estimating units of time, it makes sense for you to re-estimate so that you take into account your improved understanding of the system.

When you are sizing stories in complexity points be very careful not to fall in this trap. Re-sizing even sounds wrong anyway.

Functional Goals

After this post, I have had a number of conversations with people about what I really meant by setting functional targets instead of story point (velocity) targets. In one such discussion, I was given an example where someone set a goal to run for 90 minutes!!

At first glance you might not find the last statement worth the exclamation marks at the end of it. But this is exactly the difference between functional targets and velocity targets. I think it's wrong to set a target of running for 90 minutes. The real goal is to burn X number of calories. Or to travel X number of kilometers.

Once you set your goal right, you are open to find better ways of achieving it. If your goal is burning calories, you'd probably do it better (faster) by skipping rope for 30 minutes than by running for 90 minutes.

Now all you have to worry about is

• the rate at which you are actually burning calories AND
  
• what can you do to improve this rate
  

So instead of planning to complete 30 points next iteration (going by yesterday's weather, etc), I'd plan to achieve a set of features and keep optimizing the rate at which I am able to produce features assuming that I never compromise quality in wake of going faster.

Iterationless...

After working in week-long iterations for 2 and a half years, going iterationless left me a little confused on my recent project. Of course when I say iterationless, I mean using extremely small iterations to bring in a flow to stories. Lean? Distributed Lean, maybe?

Firstly, I like the idea of trying something new. The team being small and the project being developed in ruby helps this experiment a lot. There are, however, somethings missing in this way of working. I'll list them down and give my view on each of them in this post.

To realize what we missed, let's look at what we get from iterations. The following are the main motives behind iterations.

1. Celebrate small successes
2. Get regular feedback from clients (showcases)
3. Let the whole team know what we will be focusing on in the next iteration, including the business context around it (Iteration Planning Meeting)
4. Re-estimate stories based on additional information gathered during the earlier development (Iteration Planning Meeting)

So let's see how we are trying to handle each of these situations.

1. Celebrate small successes.
This is very important. To look back and realize that we actually celebrated achieving a story-point targets each iteration, however, now sounds really lame. The moment you move your focus away from finishing a set of stories in a given time TO finishing a set of features and going to production, the world starts making more sense.

I think in this aspect chucking iterations totally makes sense. What this means is you should have a very strong release plan, with frequent (less than a month long) releases.

2. Getting regular feedback from clients
Showcases have nothing to do with iterations. Iterations only provide regularity to showcases. To say that we will do showcases once a week (and do it) is I think good enough.

One thing to beware of is that showcases-bound-to-timelines (once a week) and development-bound-to-releases (of irregular time intervals) can be quite tricky to handle.

3. Let the whole team know what the focus of the next iteration is
Ours being a small (ideal?) team, I don't see too many problems in this area.

4. Re-estimate stories based on new knowledge
This, I thought, was the most important aspect we were missing out. But once you realize that your story point sizing is only relative to any other story you estimated, the world seems much better. This means that as long as your triangulation is correct, you are ok.

One thing that IPMs give us though, is an opportunity for the whole team to estimate together. Right now I am unsure of how important this method is. I am posting a few questions related to this at the end of this post. It'll be great to get some feedback on these.


All in all I think going iterationless is beneficial in terms of:

• Shifting the team's focus from story points to features / business value.
• Letting the team focus on estimating complexity of stories much more clearly.
  

What I am unclear about is estimation.

• If we spend 2 hours every week to estimate together, are we going against lean philosophy?
• On the contrary, if we estimate just-in-time, in smaller groups (pair to work on the story) are we losing the "group touch" to estimation?
• Also if we estimate just-in-time (when we decide to play a story), the business doesn't get an opportunity to re-prioritize based on sizing.