Testing considered wasteful??

@silvercatalyst posted on twitter a few days back that one of the trainees in his session counted testing as waste. I retweeted with a #funny but @silvercatalyst said he actually agreed with it. So we twiscussed it for a while. (By the way twitter is just the wrong tool for discussing interesting things). Back to the story.

Here's what we ended with after a few emails had been exchanged:

• Testing is not wasteful. But testing as an activity after development (especially after a time gap) is wasteful
• Some types of testing can be done upfront but other types still have to be done after the story is complete
• There are ways to prevent bugs (rather than catch them) by Dev + QA and BA + QA pairing

Mixing this conversation with Feature Injection technique and Mike Cohn's post on removing Finish to Start activities, I think that BA(or Customer), Dev and QA pairing on a story will provide tremendous boost to cycle time and significant reduction in bugs. But to achieve this, you need certain pre-conditions to be true.

1. Co-location with customer. Else a great BA who is an excellent customer proxy
2. Poly-skilled team members (not just smart)
3. Team members (including the Customer) open to work towards a moving target (with negotiable stories)

My next project will hopefully see this implemented, at least in small steps. Something a step ahead of the Ménage à trois that's already been tried out successfully.

Thanks @silvercatalyst for an interesting discussion and helping me put my thoughts together on a bunch of stuff I had read recently.

Back to the Basics - 1 - The problem

Reading Martin's ConversationalStories renewed my confidence in this draft post from about an year ago. I just couldn't put it in the right words and gave up on it. I keep talking about deterministic universes over randomness and stuff like that but the gist of the matter is simple.

Writing stories is not the JOB of a Business Analyst in agile development. Writing stories is a collaborative effort in which Customer, BA, Dev, QA should all take part. This is the N in the INVEST principle.

And here's the post

==============

I have been talking about tracking and trends and smells and quantum physics for sometime but here's a post that takes us back to the basics. This is about the very problems (with waterfall) that we are trying to solve using alternative approaches (agile, lean, hybrid, etc).

As everything in life should start, we start by defining the problem. Waterfall is an approach that borrows heavily from Einstein's idea of a deterministic universe. The idea is that

If you know the position and velocity of each atom in the universe, you can accurately determine the state of the universe at a given point in future or the past.

The point is that this is just a theory. I am not contradicting the theory but I am just pointing to the simple fact that gathering knowledge about the position and velocity of each atom takes too much time to be of any use.

Waterfall tries to do the same thing at a very small scale; at a project level. And the argument remains the same. It'll take you too much time to understand every aspect of a problem for you to be able to successfully solve it while it's worth solving.

Agile is about improvisation. You accept the inherent randomness of the universe. This might be genuine randomness OR it might just feel random because we are not able to understand it, but at any rate the universe is random to Human Beings.

So we say, given that things are going to change in ways that cannot be determined, let's do our best to adapt to those changes as quickly as possible. To adapt is to understand what has changed, how it affects us and what can we do to "maximize our happiness" in the given situation.

This is where you require the key component of anything worth calling a success. Collaboration.

In the deterministic world of waterfall, the Business Analyst is supposed to be this wizard, who understands the whole problem, whose affected and how and formulates a solution to it all by herself. Few have succeeded at meeting this unrealistic expectation.

Agile says let us all work together towards solving this problem and I think that's a more realistic way of getting an optimum solution. Now what this means is that nobody's word is final on any solution unless everyone is happy. Neither the client, nor the Business Analyst and nor the Developers. That is what it means when we say that a user story is Negotiable (i-N-v-e-s-t).

* Increasing happiness as in the sole purpose of life.

==========

The Revolution

Software Development - Art OR Science?

A seemingly clichéd question. Never passed my mind all these years. But let me tell you how I got thinking about this and maybe it'll interest you a bit.

Like I said earlier, I have been following a lot of Lean-Kanban discussions, articles, etc lately. Some such material is Little's Law & WIP limits. Now the moment I saw an equation, I couldn't resist the temptation of trying out some math to see if the size and composition of my current team is optimal. Furthermore, I thought, given a few specifications of the project like domain complexity and technology, could I find the optimum team size and composition?

I hit two forums with this idea and most of the feedback was that there are too many things to consider and difficult things as well like the skills and experience of the people on the team. And I agree that people make most if not all the difference. But that's the problem isn't it?

What's so special about our people? Skills. Experience. Talent even. It's a sign. It's a sign of our industry being immature. I think its in around the occupation stage on this scale.



Enough demand can trigger the Art - Occupation - Industry - Science transitions for any activity. Note that the transition is never 100%. There's Art and Science in everything. The question is whether something is "more of" art OR science.

People keep arguing that Software Development is different. We are not like construction, we are not like assembly line manufacturing, we are not like Product Development either. I used to believe it till a few days back. But the more I think about it the more I feel that these are arguments of a losing population of craftsmen who are finding it increasingly difficult to meet the demand for their craft (which has BTW risen at unusual rates).

Think about the guy somewhere around current Pakistan who created the first piece of leather clothing many hundred years ago. Think about the leather industry right now. Think about the transition. At some point he must be saying "This is different. This needs skills. This needs experience". It took centuries for the transition but it happened. I am sure it can be co-related to the rise in demand for leather products.

The funny thing is, if you look at the list of general characteristics of the Art and Science sides, the first three points on each side don't really fit in with the IT occupation do they? We have loads of unskilled people sitting around producing amazing amounts of useless code all over the world.

So I think the revolution is inevitable. At some point the people who pays us boatloads of money for bad software are going to revolt. We either have to change OR die.

Here's another article roughly talking about similar things. The author anticipated a code market to emerge where reusable components would be bought and sold (which didn't happen OR hasn't happened yet). But the rest of the content is around the same theme as this post.

Kanban

Although I am not in the thick of things right now (because I am onsite, alone, so far away from my team) I follow the very active kanbandev yahoo group. It's a great resource for thoughts and questions on lean and kanban software engineering.

David Joyce, a group member, posted about a presentation he gave and I found it really good. So here's the presentation for anyone else interested.

It's definitely a long one but take some time to go through it.

Good stuff David.

Story Dependencies

As I mentioned in this post, I am pretty annoyed with myself because of the way I structured stories in the first release of my current project. Here are a few things that I learnt.

First of all when we say that Story A is dependent on Story B, it means that Story B development cannot start until Story A development is complete

There are different situations where one story is dependent on another. Each situation needs to be dealt with in a different manner.

Types and Approaches

Situation 1 - The wrong split
This is the classic vertical split. Many have advised against it but many more fall into the same trap again and again. Myself included :)

Look out for the vertical split problem.
Remember,
If your body is horizontally split in half as in you have no legs, you can still do something with yourself.

If your body is vertically split in half as in your left is separate from your right...

The first place to look for the vertical split is Admin Functionality / User side of the application. If you see your stories divided like Admin Stories and User Stories make as big a noise as you can quickly.

Situation 2 - The application flow
This is where we just plainly don't think.

Example:
When you get a message on Yammer, Yammer sends you an email. You can click on the email to look at the message you have got. BUT you can obviously keep your Yammer client open and see the message instantaneously and thats the way it's supposed to be used.

Email notification is a nicety. Showing the message on Yammer client DOES NOT depend on sending an email notification to the user.

Such errors are much easier to spot. Beware of the Business Process Workflows that you refer to. The business process workflow DOES NOT represent development flow. Mark the niceties and essentials apart in your workflow.

Situation 3 - Data dependence
This is a little more tricky. But I have seen this a lot of times as well. This is closer to David's comment on my earlier post.

Surely to sell an item, you need to set the item up in the system. Doesn't this mean that you have to build the Item CRUD features into the system before you can sell it.
NO!!
Theoretically the items could be setup directly in the database. If the goal of the system is to sell an item, then that's what should be done first.

Practically you should be doing the absolute minimum "item setup" before you build the functionality to sell it.

The first thing is to concentrate on the crux of the system. Selling Stuff, Lending Books, Renting out DVDs, Playing Music, these are the things to achieve as early as possible in respective projects. Therefore you need to do whatever is needed to get to these areas as soon as possible.

Musings-at-the-end
I was mostly plagued by the admin V/s user situation throughout this release. We split stories that way because there was a tough deadline to meet and we thought it's better to split the work so that more pairs can work on the project. It worked out alright and we are in the last iteration and even able to provide a few more story points than promised. But we did face a lot of problems in scheduling because we split the stories in a wrong way. So that's definitely something to avoid.

Another thought that keeps popping in my head is the idea of Programming by Convention. Like RoR.

If you have a team of fairly experienced, "like minded" developers... can they agree upon a few conventions to follow so that two pairs can work on "Setting up the Item" (Admin part) and "Using the Item" (User part) simultaneously? I am still talking about evolutionary design (code as well as database). I seem to think that with good build processes, this should be possible.

What do you think?

Credit Cards, PCards and Level 3 data

Here's a post that talks about business more than business analysis.

I have been researching what it means to process level 3 data for credit cards. For quite a while now. Finally today I had that "Oh!" moment and I want to share it because I have had trouble finding data.

Basics
Credit Card processing is simple. I am assuming e-commerce here.

1. Customer enters credit card information (Number, expiry date, billing address, CVV)
2. Site (Merchant) sends the information to a CC processor.
3. CC processor talks to the issuing bank and authorizes the card
4. Site (Merchant) gets paid by issuing bank
5. Customer pays the issuing bank later

Not So Basics - PCards
Large Corporations and Government Agencies (LCGAs) realized one day that they could save a lot with electronic transactions and decided to use credit cards. But the employees went crazy with their personal credit cards. So Visa and MasterCard came out with special cards that were issued to employees specifically for official purchases. This is the Purchasing Card (PCard).

Level 3
However, the usual credit card transactions were difficult to settle. LCGAs realized that if the Merchant sent more data about the purchase to the CC processor while charging the card, LCGAs could get this data back from the CC Processor and thus could reconcile all transactions electronically and hence easily.

So the merchants started sending "Level 3" data to the CC processors. This is detailed data about the line items for which the card is being charged.

Benefit
The basic benefit of Level 3 data is that LCGAs can reconcile the purchases made by their employees easily. They can also keep a check on what the employees are buying.

Conclusion
Level 3 processing is only beneficial for PCards. Not for consumer Credit Cards. It's advisable for merchants to pass Level 3 data to CC processors because it increases their chance of working for LCGAs as well as gets them some discount on transaction processing fees.

Can you shuffle your stories?

No? Neither can I and I am not happy about it.

So I got my new iPod Touch. One thing I find slightly annoying is the fact that when you select an album it gives you the list of songs. But the first option is "Shuffle". Being used to LPs and Cassettes and CDs I don't really like to shuffle songs. I have listened to my favorite albums in a given order of songs for so long, that I want to listen to them in that order.

So I started wondering about the requirement for a shuffle option itself. And I conclude that it does nothing but brings a tiny bit of randomness in our lives. A little bit of surprise. That's all.

What is also important is that the songs that we listen to usually are shufflable :) Because I do listen to Indian classical music and you can't shuffle that. The khayal has to come before the drut rendition.

So what's this got to do with user stories? Well, I am currently plagued by dependant stories and I suddenly realized that they are like India Classical music. They can't be shuffled. Separate post about why I have too many dependant stories on the current project.

Dependant stories are not always avoidable. But it is ideal to have a lot of stories that can be shuffled.