Labouring: An Analogy

The project plan says you're going to design your unit tests next week. Your code is being dropped into system test tomorrow. The Functional Spec was conditionally signed off on Monday (the condition was the successful resolution of one or two queries about inconsistencies in the Business Requirements Document)

Does any of this sound familiar?

Background

I've been a freelance consultant (or contractor scum or disguised employee) for ten years, though it seems longer. I've worked with plenty of your major, run-of-the-mill financial institutions that we all know, love, and trust with our money.

I'd started working with a popular Internet Bank at the height of the dot-com bubble when rates went silly (that's the rates they paid consultants, not the ones they paid their customers). Since then the bubble had burst, rates had collapsed and accepted wisdom was that those rates has gone for good and a recovery would mean being able to find work again.

And so I found myself sitting at my desk looking at the terms of another 6 month contract renewal from the human resources (HR) department of the bank.

So, why was I thinking so hard about refusing this renewal? Was I looking for more money? Did I think that the wording might have got me in trouble with Hector (remember him?) and the IR35 posse? No, neither of these. I was seriously considering refusing the extension on the basis that I just couldn't face wasting another 6 months.

I began calculating 6 months as a percentage of my age; as a percentage of my adult life; as a percentage of my future life. The percentage kept getting higher. In fact the more I though about it, the more it seemed that if I renewed this contract 6 months might constitute 100% of my remaining time on earth. Or maybe even more.

I didn't sign. I politely declined. After almost two years of trying to get a large business to adopt sensible software development strategies (like having a development process - not a specific process, but any identifiable set of activities that might conceivably constitute a development process) I just walked away. I left my PL/SQL test scripts to posterity and headed for the hinterland.

I decided I'd go back to my bad old ways.

Allegedly I'm a designer and/or architect (that's not what it said on my contract, which was of the generic "Hot-Deployable Freely- Interchangeable Plug-And-Play Software-Resource"-style) so I thought I'd do some designing and architecting. In the real world. With a building.

Another World

Now, I have no formal training in construction techniques, but some pals and me did build a lovely straw house a few years ago, so I have had some experience. (Any of you going to OT2003 will have the opportunity to join me as I use straw bale building projects to discover software requirements elicitation techniques, which should be fun!). Also, my partner and I had a house built for us a while back and have plenty of experience with the Planning and Building Control departments of our local council. (Luckily agricultural buildings don't actually require Planning Permission or Building Warrants, but more of that later).

I ran the project in an informal sort of way, with the customers on site all the time. My customers were vegetable growers that needed a shed to pack their vegetables after their old one had ended up 30 foot up an ash tree during a particularly violent storm in February.

So, I asked them what their requirements were (well, we actually had a more protracted discussion that started off with "What's it for?" but you get the idea) and I ended up with this list:

• The shed should be large enough to pack boxes of vegetables.

• It would be nice to have a separate space to store potatoes (and such like) in.

• An office space would be quite useful, too.

• It shouldn't be too warm in summer or too cold in winter.

• No vermin allowed in.

• Easy access for loading/unloading the van.

• Oh, and please could I make sure that it didn't blow away this time?

Could it be done for £3000 (not including labour) and be finished before the rains started in the autumn? Well, maybe not. It was in use within the timescale, but like so many projects it may never be 'finished'. And like so many projects it went over budget (by maybe 25%).

Awkward Analogies?

Why am I writing about this in a journal for software developers? I guess my basic question is that if small software projects are easy and fun, why are large software projects so difficult and depressing? Or in general, do you always have a bad time when things get big? My gut feeling says "Yes", but let's see where it goes.

The good things about this project were:

• We knew what was wanted

• Within reason we could use any appropriate technology

• I knew everyone working on the project and what they could be relied on to do

• No one was making silly rules that just slowed us down

• The budget & timescale weren't restrictive

Before we get started, let's dispel one or two myths.

1. We did have fun putting up this building, but it wasn't because all the work was enjoyable (putting rock wool insulation into a roof space is not fun) or that we spent all our time sloping off to the pub (the nearest pub was 5 miles away). We had fun because we were working as a team toward a recognisable goal.

2. We weren't 'pushing the envelope'. All the technologies involved were completely mainstream, so that wasn't what kept us happy. None of us were full time builders, so you could reasonably argue that the fun would lessen with repetition (I'll let you know).

Now let's have a look at where the 'good things' go when a project goes from small to large.

We know what we want

How well do we ever understand the requirements of a project? Human systems are complex and the relationship between size and complexity is not linear.

As complexity grows the ability of any one human to grasp what's going on disappears and the likelihood that something has been missed grows. We can cope with this, but the personal cost is high.

The prize of component-based development is simply the management of complexity by dividing large projects into smaller ones. The problems is that specifying the interfaces to, and responsibilities of, a component is no easy matter.

We can use the most appropriate tools

Large organisations often have preferred suppliers. It may not be fit for your purpose, but the license has already been purchased, so you'll just have to use it. Even if the ideal product is free it hasn't been certified by the Platforms team, isn't supported by a team of highly motivated technicians, and hasn't got a brand name behind it. Anyway, if it's so good why are they giving it away?

You'll notice that I talked about large organisations rather than projects here. This sort of constraint is due to the context of the project rather than the project itself. You might be working on a small project for a small company, but if the major client is a large company with a 500 page procurement process you'll be doing things the way they want whether it's a good idea or not.

The project is manageable

Things can slip in any project.

There's an anti-pattern that I read about somewhere that describes how an item on the project plan can be 90% finished for weeks on end, because the 10% left isn't well understood.

Then there are the bugs, misunderstandings, design flaws and 3rd party product problems. Not to mention compiler bugs ("this should work, so it must be a bug in the compiler").

Without proper feedback processes you won't spot the slippage until it's too late. This is true for projects large and small. The thing is that it's easier to put effective feedback processes in to small projects: you're familiar with the overall vision, you understand the design, and you know the team. On large projects all these things work against you and there will likely be layers of management with an inflexible attitude to time boxes.

Equally, you need to understand enough about the problems and the people to match the person to the task. If you fail in this then your staff end up feeling less part of a team and more like a cog in a machine. I know which I prefer.

Rather than try to overcome this, most large organisations talk about people as resources and seem to think that people with the same keywords on their CV are freely interchangeable.

As an example, I overheard a discussion between one of the HR team and an IT program manager. They were trying to decide what to 'do' with a new-start that was arriving on Monday. They considered placing him as a mid-tier development team leader, a front-tier developer and a tester, eventually settling on the latter! They had hired a 'resource' without any idea of the role they were trying to fill, and with very little understanding of what skills he possessed.

Bureaucracy didn't stop us

How easy is it to fix a problem?

The answer depends on many factors. How fundamental is the problem? Who can decide on the appropriate solution? Who can do the fix? What processes need to be followed before any action can be taken?

I've found that it's the last question that seems to be the most limiting in large organisations.

When you have to coordinate large numbers of people you have to put processes in place to allow monitoring and control. The designing of appropriate processes - structured enough to deliver the management requirements, lightweight enough to not be a burden and flexible enough to cope with the whole spectrum of project problems - is a high art.

Many is the methodology that has been found wanting!

Project constraints were realistic

How long does the project plan say the bit you are working on is going to take? Where did that number come from? Did you provide the estimate? Did you provide the estimate, only to be asked to shave some time off it? Did you provide the estimate, only to have functionality added? Does the plan allow for testing or rework?

It is widely accepted that there are 3 variables in any project: scope, resources and time. They are related. You can't change one without a compensating change in one, or both, of the others.

As the project size grows so does the management layer, and for some reason the sort of manager that fills that layer just doesn't seem to buy this self-evident theorem.

It's not even that the theorem is too simple, because what they propose is often so simple as to be idiotic:

• "I can motivate the team to deliver more functionality in less time".

• "I can compensate by hiring more staff"

• "The team will pull together and work longer hours"

• "I've promised that we will deliver, and my word is my bond"

Again this really shouldn't have anything to do with project size, but the larger a project the less likely the grim reality will make it though all those middle management layers to someone who is prepared to make a tough decision.

Conclusion

The shed was a small project:

• The complexity was not overwhelming.

• We used aterials that we had to hand and sourced from local suppliers, without needing to get it approved by the client or building control.

• The people helping me were friends who I've known for years. One was time-served electrician, the others were just generally handy and reliable.

• We didn't need planning permission or a building warrant, because it was an agricultural building. Bureaucratic involvement was minimal.

• I had no unrealistic timescales and the budget was only slightly unrealistic.

We had a lot of fun. More fun than we would have had working on a building site.

Large projects could be fun if you could decompose them enough to understand what's going on and your organisation is helping rather than hindering. I'm just not sure that either of these things is achievable.