When in a inception...

Everyone is an analyst
Don't get bound by role boundaries. Everyone needs to understand the system. Talk to the clients, ask questions, draw diagrams, make suggestions, understand problems and solve them.

Make sure everyone facilitates sessions at least once
Especially the BAs since they have to interact the most with the clients. Clients need to feel confident about the BAs on the team. Other members should also be actively involved. Don't let a single person be the scribe all the time. The client might ask "why am I paying for this guy" at some point.

Have an intent for each session
and strive to achieve it. Have back up plans ready in case you find that your planned tools / techniques are not working for the client. Fail fast, change your approach, take a break and regroup, but make sure you get the right information out of a session.

Educate the client about the way you work
They should know what it would be like to work with you day-to-day. Clarify time commitments and kind of inputs required from your clients (suggestions during analysis of stories, showcase feedback, testing). Have special sessions that cover methodology. Offline the subject if it comes up elsewhere. It can derail your session plans easily.

Be honest and upfront
About any shortcuts you have taken, any information that you don't have, any help that you need.

Understand each stakeholder
You should know the pressures that they are dealing with, individually. Know how you can help them.

Create. Shared. Vision.

Number Lust

I am an amateur photographer while I am not not hacking project teams and building custom software at ThoughtWorks. Photographers are known to have bouts of lens lust* time and again, especially at the beginning. I realize that some managers seem to have similar urges when it comes to numbers, metrics. They suffer from acute number lust.

In ThoughtWorks, we believe in and encourage self organizing teams. BAs gather requirements, developers write code, QAs test and automate and the customer signs stories off in a flow. The Project Manager role is therefore reduced to making sure that nothing obstructs this flow.

The way this would be done is to gather and analyze the right data and take actions based on this analysis. Examples being:

• Finding bottlenecks from the wall or a CFD and taking actions to fix the problems.
• Looking critically at retrospective action items and seeing that all those issues are fixed (especially if they are issues outside the teams control, like infrastructure).
• Making sure that the right capabilities exist in the team at the right time. If not, get people from outside, arrange training, etc.

Some PMs are content in gathering just enough data to spot problems. But then there are some who can't get enough data ever.

They want to organise the whole world into neat boxes and label them and track anything and everything possible. The walls soon fill up with useless charts with graphs and numbers that the team can't use in any way. The worst part is, there is no analysis done around the data even by the people who gather it. Having people in a 15 member team draw emotional seismographs every iteration and sticking them on the wall is of no use if you are not going to make any decisions based on them.

Now I agree that gathering all possible information can enlighten us to some extent. But its a matter of marginal utility. If I have to do 5 extra things to do my job, just because we want to gather data, I am not doing it.

So guys, here's a humble request. Figure out with your client what kind of data he would like to see. Figure out as a team what you would like to improve and what data needs to be gathered to help you get there. Don't burden your team with things that obstruct OR slow down the flow. Keep it simple and keep it lean.

Our job is to provide quality software not elaborate reports.

* lens lust - the dangerous urge to keep buying new, expensive lenses not realizing that its practice that will improve your photography, not lenses!!

Onsite Business Analyst

This has come up in various discussions recently and I want to put down my thoughts about the role and responsibilities.

For a ThoughtWorks team in India (or China) most of the work is offshore agile development. Clients are usually in the UK or the USA. The team is structured as follows:

1. Offshore PM
2. Offshore Devs
3. Offshore QA(s)
4. Offshore BA(s)
5. Onsite BA(s)

This is how the communication works.

Of course there are other exchanges that take place but the Business Q & A and the Technical Q & A are the most important pieces of concrete information exchanged.

Other companies, even traditional development outfits, have a role called the Onsite co-ordinator. I believe this role facilitates similar discussions (although I hear that they are more technical than business related)

The onsite business analyst's main role is to facilitate business and technical discussions between the team sitting offshore and the client. She is not restricted to be just an onsite co-ordinator. She can take up a new stream to analyze by herself. Even feed it to an onsite development team, if there is one. But all this keeping in mind that the main purpose is bridging the communication gap between the client and the development team.

At the same time the onsite business analyst shouldn't go overboard with this and become a single point of failure. The direct communication lines between the offshore team and client should always be open.

Metrics

I have recently come out of a consulting assignment which has given me loads of time to read and think about processes, improvements and effectiveness. (People who follow me on google reader must have noticed). It also had me thinking about introduction of agile into a traditional IT outfit and what would make it more effective.

Top -> Down

The Top -> Down approach is where someone in the top management realizes (or is convinced) that agile is the solution to all their problems and goes on to "mandate" agile. This is not necessarily bad. If the organization hires the right consultants to bring agility into the project teams at the ground level.

Bottom -> Up

The Bottom -> Up is not really an approach. Its just a team realizing that they can do their stuff better and try to bring in improvements in their team without affecting the organization. This is later, agility may spread into other teams virally and this might trigger an organizational transformation with eventual buy-in from the management.

Here's what I think works best. Like any win-win solution, its a mix of the two extreme approaches described above.

Top -> Down buy-in, Bottom-Up implementation

The management decides, that they require to be agile to meet the market conditions but instead of pushing "agile" down the throats of people below, they let the people on the ground level realize what agile means and how it changes the way the team works. Agile Coaches are useful in this transition.

Sounds simple enough? Well here's the catch. How does the management know that the new process is effective?

The sad part is in a traditional organization's hierarchy, the only interface between the management and the teams is "Reports". Reports on various parameters (metrics) that the management believe should be monitored to see how stuff is going on. Whether the teams embrace agile themselves OR whether management asks teams to be agile, the teams will need to report the right metrics to the management to gain their confidence. If the teams don't do this, the management might start tracking the wrong metrics and hence encourage wrong practices. What you measure is what you get.

I opened this topic to a bunch of people from the client organization and we came up with a bunch of metrics based on the principle that we should report consistency rather than hard numbers. Consistency gives a measure of effectiveness and also encourages the right activities.

Here's the list that we came up with (There can be many more. The basic principle is to use proportions / percentages instead of hard numbers):

1. Velocity Barometer without numbers (we mark completed stories in red on a barometer, but we take out the numbers. Consistent color means good, ups and downs means trouble)
2. Cumulative Flow Diagram (consistent thickness of bands v/s actual numbers)
3. Ratio of open V/s closed bugs (Percentage. Instead of bugs found and bugs fixed)
4. Test Coverage (Percentage)
5. Automated V/s Manual tests (Percentage)

Really speaking it all boils down to using just Control Charts for project reporting and management. What do you think?

Testing considered wasteful??

@silvercatalyst posted on twitter a few days back that one of the trainees in his session counted testing as waste. I retweeted with a #funny but @silvercatalyst said he actually agreed with it. So we twiscussed it for a while. (By the way twitter is just the wrong tool for discussing interesting things). Back to the story.

Here's what we ended with after a few emails had been exchanged:

• Testing is not wasteful. But testing as an activity after development (especially after a time gap) is wasteful
• Some types of testing can be done upfront but other types still have to be done after the story is complete
• There are ways to prevent bugs (rather than catch them) by Dev + QA and BA + QA pairing

Mixing this conversation with Feature Injection technique and Mike Cohn's post on removing Finish to Start activities, I think that BA(or Customer), Dev and QA pairing on a story will provide tremendous boost to cycle time and significant reduction in bugs. But to achieve this, you need certain pre-conditions to be true.

1. Co-location with customer. Else a great BA who is an excellent customer proxy
2. Poly-skilled team members (not just smart)
3. Team members (including the Customer) open to work towards a moving target (with negotiable stories)

My next project will hopefully see this implemented, at least in small steps. Something a step ahead of the Ménage à trois that's already been tried out successfully.

Thanks @silvercatalyst for an interesting discussion and helping me put my thoughts together on a bunch of stuff I had read recently.

Features V/s Sophistication

Usefulness of software:
Letting the user do what s(he) wants easily and fast.

Features:
Number of activities a user can perform with the software.

Sophistication:
Software intelligence which simplifies user interactions.

Software, like anything else follows the laws of Diminishing Marginal Utility. Here are the graphs as per my experience with software (Not as a software professional but as a simple user)




Features
I think the marginal utility of adding more "features" to the software diminishes at a much higher rate.

• Firstly because individual users don't use all the features but have to pay the price, in terms of money as well as processing power, disk space, etc.
• Secondly because users have to deal with all the "genericness" of the software and hence cannot do what they want easily and fast

Best example of such software is (my beloved) Lotus Notes.

Sophistication
The marginal utility of making software intelligent also diminishes albeit at a much slower rate. Google Calendar Quick Entry is a good example of software intelligence. So is GMail's drunk email protection.

What keeps GMail, GMail is that everyone (creators and consumers) abides by the simple rule that GMail is for sending and receiving email. If you want to do something else, you will have to use something else.

There are a couple of software that come to mind which do have a huge feature set but still keep the crux simple enough. Microsoft Word for example has done it rather well in that I can go in and create a simple document as easily as I could when it ran on Windows 3.1

As a business analyst this translates into a simple rule:
Know the crux of your application and stick to it.

When you set out to start a new project, get your users to agree upon the problem to be solved and do whatever it takes to solve that problem; not a penny more, not a penny less.

Software for Everything is as enticing a concept as the Theory for everything. I think there's a reason we haven't found it :-)

What are we doing?? - Part Three

This is what got me thinking about "What (the %*&$) are we really doing?". Are gazillions of dollars being invested in IT, by organizations just trying to maintain their status quo? As the study shows, this is partly true.

1. Businesses require to grow for survival.
2. Growth requires accommodating more and more people and systems into your business.
3. It is very natural (and useful) to be paranoid of these new people / systems.
4. As businesses grow they have to become more or less bureaucratic.
5. Bureaucracy requires additional processing power.
6. This power can either be man OR machine
7. If the business chooses to invest in machines, we, the IT guys, make some money.

So that is what we do.

We help businesses grow...
...grow by overcoming their paranoia by automating bureaucracy...
...bureaucracy which is only bound by the stakes...
...stakes which are ever growing.

To be honest, I am still not totally convinced that this is all what software is about. What do you think?

What are we doing?? - Part Two

Any system is inherently trust based but always tends towards being distrustful. Consider a prisoners' dilemma example.

(A) = A crook with a bag of jewels
(B) = Another with $100

(A) needs the money and (B) needs the jewels. Now they are both "if you see my face, I'll have to kill you" kind of guys. So they develop a simple system. (A) leaves the money a designated place in the forest and (B) leaves the bag of jewels at another designated place.

If they both co-operate, they leave happy. But life is not so simple, is it?

Once they start losing trust in each other. They will employ various methods, making the system more sophisticated in the process, so as to make sure that they are not tricked. Here's what they could do (this list is not exhaustive ;-) )

1. (A) decides to send a friend to the place where (B) is supposed to leave the bag to see that (B) does leave the bag
2. (A) realizes that even if (B) leaves the bag, how will (A) know? So he decides to send two friends, one of whom shall come back and inform (A)
3. (A) grows suspicious of his two friends, coz they might run away with the jewels, so he has to keep their children as collateral. He appoints a person to guard the children and has to feed them as well.
4. The friends say that if (B) leaves an empty bag, it's not their fault. Which is fair, so they end up carrying mobile video cameras that transmit back to (A) on his mobile receiver...

... and so on.

Well actually at step 4 (A) goes "All this for $100?? Not worth it at all". Wise decision but if the stakes go higher, (A) will end up investing in the hi-tech solution which only improves his chance of not getting cheated by a fraction. By the way all this while (B) is also employing similar tactics.

So here we have two parties willing to invest in more and more complex systems as long as the stakes seem high enough for them. And here are we, the IT people, who will help them do it.

To be continued...

What are we doing?? - Part One

I am doing what we call an Inception for a new project. The situation is not at all unusual.

The Constraints

• An Organization with a bunch of generic e-commerce platforms
• Live customer sites on each of these platforms
• One of these platforms particularly painful
• Integration with proprietary back-office operations software

The Requirements
Build a custom platform that will replace all the existing ones, support all the customer sites and have a bunch of unique features. And BTW we need to go live in 2 months.

How?
The only way to do this is build a bare bones application that can take a couple of simple customer sites live in 2 months. We can then keep adding to the application and making it richer and better and migrating more customer sites as and when possible.

Now whenever you try to skin an application the first ones to suffer (and rightly so) are the administrators of the system. I say "rightly so" because normal users can potentially be dumb, careless, in a hurry, etc. and the application has to make sure they are doing the right thing.

Administrators on the other hand are trained to maintain the application. They can do without a jazzy UI and help and pointers all over the place. They know how to do their job. We, as designers of the system, take the liberty of trusting them. And in that trust lies the topic of this series.

To be continued...

Business Analysis and Quantum Physics

I recently read In search of Shrödinger's cat. It's a wonderful introduction to quantum mechanics; The basic (and intriguing) problem of the elusive electron acting as a wave and a particle at the same time.

Analysis is very much like observing the electron in the classic two slit experiment. The business domain is like a wave traveling through the two slits. But the moment we analyze, we make it boil down to one reality by asking it How, What, When and Who.

Waves give you a probability distribution, while particles give you one of the possibilities as a reality. Looking at a business problem, in the wave form is better since it let's you choose a reality that solves the problem well.

For Example: Let's say the business problem is "I want to let my customers know of the new offers I have so that they can take advantage of the offers and I can make money"

This is the problem in it's wave form. The reality can be one or more of email, website updates, RSS feed of offers, people going door-to-door, mobile advertisement vans and a million other options including telepathy, albeit with low probability perhaps.

The problem is that clients come and say "I want to let my customers know of the new offers I have by email so that they can take advantage of the offers and I can make money" and it takes a while to get into a mode where you challenge every reality and go back to the wave form, by asking "Why"? Why is the only question that let's you go back to the wave form.

The more we procrastinate our observation, the better will the chosen reality suit our problem. Agile takes this approach with the concept of the last responsible moment for a decision. It let's future remain future and hence full of possibilities.

Functional Goals

After this post, I have had a number of conversations with people about what I really meant by setting functional targets instead of story point (velocity) targets. In one such discussion, I was given an example where someone set a goal to run for 90 minutes!!

At first glance you might not find the last statement worth the exclamation marks at the end of it. But this is exactly the difference between functional targets and velocity targets. I think it's wrong to set a target of running for 90 minutes. The real goal is to burn X number of calories. Or to travel X number of kilometers.

Once you set your goal right, you are open to find better ways of achieving it. If your goal is burning calories, you'd probably do it better (faster) by skipping rope for 30 minutes than by running for 90 minutes.

Now all you have to worry about is

• the rate at which you are actually burning calories AND
  
• what can you do to improve this rate
  

So instead of planning to complete 30 points next iteration (going by yesterday's weather, etc), I'd plan to achieve a set of features and keep optimizing the rate at which I am able to produce features assuming that I never compromise quality in wake of going faster.

Agile Business Analysts

Yesterday, Craig Brown asked Do we need Agile Business Analysts?. Two and a half years back when I joined ThoughtWorks, I would have said "No, we don't". Today I strongly disagree with the view that Business Analysts can be done away with on Agile projects.

The objective of an Agile project is to provide maximum value to the client as soon as possible.

Now I agree that the ideal situation would be when you have business users & developers who agree and abide by this sentiment. You can happily get rid of the BA in such a situation. Real life, for some reason, is never ideal.

Anyone who has interacted with business users on a medium to large project will have faced situations where it has been hard for the business users to come to a consensus on what they want. Business users are generally representatives of different functions of the organization. Each of them has expectations from the application which more often than not are conflicting. Keeping these users focused on what we set to achieve when we started building the software is what a BA does.

Being on an agile project gives the business users many chances to dream, many chances to make changes. Developers are human too. They dream about technology. While sometimes this can lead to some really cool/unique/usable feature in the product, it can also lead to "overcoolified unnecessities". Now in an ideal situation, all these dreams would come true. Unfortunately, dreams in the real world are bound by budgets & deadlines. Managing these dreams, ideas, changes so as to keep them aligned to the original business objective and deliver most value in the given time and budget is what a BA does.

What agile aims at, is to start minimal and change as required. When a small bug on a story turns out to be a big chunk of enhanced functionality, we create a new story. The tasks mentioned above, along with writing user stories, supporting implementation of the functionality, planning the iterations and releases, testing the developed functionality, etc. together makes for a considerable chunk of work. That's when you create a seperate role called Business Analysts.

Now this role can definitely be performed by developers. In my experience, though, good developers don't really want to do any of this. They want to write good code. That's what they are best at. That's where they will deliver maximum value.

Hence Business Analysts.