Where Egos Dare

Recently I have had reason to look through some old software engineering textbooks, the kind of thing I used to read as an undergraduate and junior programmer. This reminded me of a few concepts I haven't thought about in several years. One of these is egoless programming. As I recall my university lecturers were very keen on this concept, it seemed to be a good thing . Although, to be honest, I've always had my doubts...

What Exactly is Egoless Programming?

I guess it all depends on what you mean by ego, so I turned to my favourite text on management and psychology (and all that kind of stuff) where I found the following definition:

" ego has to make sense of the internal conflict in our mind between the id and superego and the external world. The ego is the decision-making part of personality and is engaged in rational and logical thinking. It is governed by the reality principle." (Mullins, 2002)

Now this looks a bit confusing. Does egoless programming mean programming without rational and logical thinking? Well, I've seen my fair share of programs and I must say a lot of them do seem to lack rational and logical thinking, but I really don't think anyone wants to advocate this as a design and programming technique. What would an irrational program look like? What would an illogical program look like?

I'll admit, the first half of the sentence "conflict in our minds" makes sense, I often experience conflict when I'm programming: do I use ++i or i++ , or even i:=i+1 ? And I'm often torn between doing something the "modern" way (say with a template function specialisation) or the "old fashioned way" (with lots of verbose code). So maybe conflict-less programming would be a good idea.

But then, if there was no conflict why do we need people? It is the human judgement, honed over years of programming that makes developers so valuable. If there is no conflict, if there is an obvious solution every time, we can automate it, bring on the Case tools. Or even just enhance the compiler. The fact is, resolving conflicts and balancing competing forces is a fundamental part of what we as software developers do.

So clearly this is not the right definition for ego.

Maybe what the books mean is superego-less programming, if egoless is good, then surely, super-egoless must be better?

" Super ego is the conscience of the self, the part of our personality which is influenced by significant others in our life. " (Mullins, 2002)

Well programming without consciousness, that doesn't sound good. Once or twice I've programmed into the wee small hours of the morning and come pretty close to coding in my sleep but I wouldn't recommend it.

I'm sure I've met managers who would prefer it if our significant others didn't influence our lives. A manager once asked me to cancel a holiday, however the thought of how my significant other would react prevented me from agreeing.

So, I don't think superego-less programming is a good thing either. That leaves us with id-less programming.

" id consists of the instinctive, hedonistic part self.." (Mullins, 2002)

This sounds more like it. Id-less programming - programming without fun. This is work, this is serious, fun has no place in code. (Erh, why did I get into this business?)

While I've known many managers who don't see fun as an essential part of the job, I can't say any have really objected to a bit of fun. After, a bit of fun, a few smiles in the office, makes the day much more, erh, fun.

So I'm not a lot closer to understanding what egoless programming is meant to give me. Maybe I'm reading too much of the definition, maybe what we want is a simpler definition of ego. So this time I turned my dictionary, this definition looks more hopeful:

"ego n. pl. egos 1. The part of a person's self that is able to recognise that person as being distinct from other people in things. 2. A person's opinion of his or her own worth: men with fragile egos." (Collins, 2001)

So maybe egoless means we can't tell ourselves from other people, we lose ourselves in some kind of great group. Let's forget that 20th century literature and popular philosophy emphasise the individual, we want to hire programmers who can't tell themselves from anybody else. That might get really confusing at times, and where is the difference of opinions that can lead to so many useful insights?

Or maybe you want people who think they are worthless, we want programmers who don't have very high opinion of themselves. I can see this be a great position for manager to be in when it comes to the annual pay reviews, or for contract renewal.

Manager:

Well then Bob, I'd like to renew your contract for another 3 months

Bob:

No, no, you don't want me, remember that bug in my code? John had to fix it last week?

Manager:

Yes, I see, well, I can't throw you out on the streets, so what say I keep you for another three months with a 25% reduction?

Bob:

I am not worthy

It seems to me that ego is an essential part of people, and even of software people. On the whole it's more fun to work with people who are confident - I'm sure most people would say no if invited to join a team of people racked by self-doubt. Actually, we want programmers with egos. We want programmers who care, we want people who say "I'm proud to have worked on this project."

It seems "egoless programming" doesn't really stand up to analysis. We want people who are rational, logical, proud of their work and bring a positive attitude to work.

Origins and Setting

The term egoless programming originated with Gerald M. Weinberg's "The Psychology of Programming" in 1971. In the Silver Anniversary edition (Weinberg, 1998) and in IEEE Software (Weinberg, 1999) Weinberg has reprised his original ideas and claimed he has been misunderstood and misinterpreted. To be fair, Weinberg was making an argument for teams, it just happened people remembered the sound bite, egoless programming , and forgot a lot of his other ideas.

For Weinberg egoless programming is about code reviews and letting others comment on your work. Although the benefits of code reviews are well known they are not always conducted routinely. There are a variety of problems, not least because it often takes longer to review code than it takes to write it in the first place.

In Weinberg's original essay he suggested programmers' egos lead them to hide their code, and protect it, they don't want other people passing comment on it. Does this really happen? There is no shortage of programmers posting their code on SourceForge for all to view.

I think the problem is more the social setting of the review. In reviewing your code, and giving feedback, there is a great capacity to hurt someone's feelings. Receiving feedback can be hard, and it can hurt. Simply being told "think of egoless programming" is like being told to keep "the British stiff upper lip."

Giving criticism so it doesn't hurt, and receiving criticism without feeling personally attacked, are skills themselves. One organisation I know did code reviews by e-mail, a day or two after your check-in you would receive an e-mail from your reviewer listing your mistakes. This may be efficient but it is also brutal. Some developers would actually hold off check-ins until the last minute then make a lot in a short space of time, this usually meant the reviewers could not get the reviews done before the release negating the whole point of a review. I think this kind of humanless code reviewing from 30,000 feet is a bit like carpet bombing, painless for the reviewers but indiscriminate and uncaring.

Neglected Teams

Programming teams are hardly new concepts, they have been knocking around software development books for many years. Ever since programming projects got beyond the abilities of one person we have had teams.

The problem is that our traditional text books devote hundreds of pages to technical issues and almost nothing to team work. A statement to the effect that "much software is developed in teams, good teams need egoless programming" is about as far as many go.

For example, to take a standard textbook; if Pressman (1994) even mentions teams in the text it doesn't make it into the index. His fourth edition (Pressman, 1997) edition is slightly better but with over 800 pages you could easily miss the few pages on teamwork. Other texts can be better, for example Somerville (2001), devotes a whole chapter (20 pages) to managing people, and six pages alone to team working, pretty good going until you notice it is an 800 page book!

So, while we can all agree that teamwork is important few of us actually devote time to thinking about how to make it work. There is an assumption somewhere that teams just work, once told we are a team then all is sweetness and light.

Personally I don't see this myself. Teamworking is a skill just as much as C++ or SQL is, and we need to learn it. In fact, each team needs to relearn the skill itself. This may be especially true of programmers, I know quite a few like myself who preferred the warm dry computer room at school to the cold, wet, muddy football pitch where our teamwork skills where supposed to be developed.

The point is teamwork doesn't just happen, we need to be encouraged to work in teams. It can be scary talking to new people, it can be terrifying trusting somebody else to do work, and it can be demoralising to see somebody else do a piece of work that you really want to do - and I haven't even mentioned fixing people's bugs!

Simply extolling the virtues of teamwork, asking people to practice "egoless programming" doesn't make it happen. If a company wants these objectives it has to work for them.

The Irony

Perhaps the greatest irony of all is that people are social animals, we actually like interacting with other people, working and playing with other people. In fact work can be so much more enjoyable when you work with a great team, people we trust, people we value. When we trust people work becomes so much easier, we don't need to keep an eye on them, we don't need to secretly double check their code, and we feel happy for them to the see our code.

There are times when competing is the right solution. Put two teams on a football field, two companies in the same market, any company and Microsoft! Competition is the bedrock of capitalism, competition drives us, we all want to win. But competition is not always the right answer, sometimes we get more by co-operating then competition.

This is why companies exist, because as a group of co-operating individuals we can achieve something that the same individuals can't achieve by competition. And that is why programming teams exist, because some programs are too big for one person to write.

It Isn't Easy

One of the most difficult things for a team to do is to overcome Conway's Law (Kelly, 2003, Conway, 1968, Coplien, 2003). This is usually stated as:

"if n developers work on a compiler, it will be an n pass compiler"

It is so easy on any project to count the number of developers and divide the project into that number of pieces, even if the project can divided into more, or less pieces. This way we can keep everyone busy, everyone can have their own space and get on with a piece of work.

But is this always sensible? If a project naturally has a front and a back-end why divide it into three pieces just because we have three programmers? The bigger question is: What are we trying to optimise here? Are we just trying to make three programmers look busy?

A common variation on Conway's Law states:

"if n developers work on a compiler, it will be an n-1 pass compiler, somebody has to manage "

This assumes that the role of managers is as police. They are there to command and control the workers (programmers) and ensure they get the work done.

Actually, I think we need to extend Conway's law, it's really Conway's Trap:

"The sub-optimal team structure and system design that occurs when n programmers divide any given piece of work into n pieces and allocate one piece per programmer."

This formula is easy to apply, has a superficial attractiveness and is easy to manage. It's particularly easy to manage if some of the n programmers don't particularly like working in a team or communicating with other programmers. Unfortunately, there is no real reason for dividing the project into n pieces, no reason to believe the n pieces are of equal size, or equal complexity and for the project to complete each n piece must be delivered.

If we want our teams to really work well and develop good software we need to move away from simple exhortations to "egoless programming." We need to break our projects into the optimal development pieces and work as teams, rather than slavishly divide by the number of programmers and work as a group of individuals.

This means we have to think seriously about making our teams work well together. Fortunately some of the more recent writings on software development have started to put a greater emphasis on teamwork (e.g. Eckstein, 2003, Cockburn, 2002) and the (in)famous pair programming (Beck, 2000, Coplien, 2003) is a good example of this. But learning to work as a team doesn't start and stop with pair programming.

It helps if a team actually knows one another. Do they lunch together? Do they socialise together? Some companies try to get teams to socialise by arranging a Friday afternoon "beer bash", these can have an artificial feel to them (especially if everyone is driving home) but is a start.

Other high profile "teambuilding activities" like white-water rafting or paintballing can also be the subject of jokes and mockery. It's important to match your team building efforts to the attitude of the team, maybe a trip to the pub or the cinema is more in keeping with your team. Even ad hoc communal gathering areas such as kitchens can be far more effective than one off events.

This requires ongoing expense and commitment from the company, after all that kitchen space is a continuing cost while a paintball day is a one-off expense. But this long-term commitment is what is required to build a good team, it doesn't happen overnight.

Companies need to look at their own mechanisms: do they reward people for teamwork? Or do they award annual bonuses based on individual heroic coding efforts? And are teams broken up once a project finishes, or can they move together onto the next project? Are people allowed to sit together? Or are people squeezed into whatever space can be found when they arrive on day one?

There is a lot individual managers can do here too. If they see their role as commanding and controlling the developers they aren't going to get the best from them. They need to learn to develop their teams, encourage people to work together and learn together.

Keep Your Ego, Make Teams Work

The fact is we want developers to have egos, we want them to be proud of their work, we want them to think logically and rationally. But we want to harness these egos within a team. We want the team to succeed. This can only happen if we are socially aware and build towards this goal.

We can't expect any of this to happen just because we say it should happen. We need to work hard to make these teams work, people need to learn how to work in teams, how to work with their colleagues, how to give constructive feedback and how to accept feedback.

Unfortunately, neither Microsoft nor Rational sells a tool to do this, its something you have to create yourselves. You can bring in outside help but this is a long process, the rewards are great but it won't happen overnight.

The Vision Thing

One way to get a team working together is through shared vision. While "vision" may seem a bit abstract, vague or ephemeral, it does actually have strong supporters who argue that creating a shared vision is a powerful tool for managers and teams:

"Where there is a genuine vision (as opposed to the all-toofamiliar 'vision statement'), people excel, and learn, not because they are told to, but because they want to." (Senge, 1990)

There is a brilliant example of the power of vision in an IT project by Conklin. He described the management of the Digital Alpha AXP project in the early 1990s. This project employed over 2,000 engineers both in hardware (chip design, machine design, integration) and software (at least two operating systems, compilers, editors, etc., etc.). He called this Enrolment Management and at the centre of it was vision.

This shared vision was not a weak, ephemeral thing but a strong substantial, lasting vision which moved people to produce seemingly extraordinary work:

"given the group's commitment to the larger result, we found more aggressive behaviour. For example, the OpenVMS AXP group publicly committed to their target schedule and stated, 'We don't know how to achieve this, but we commit to finding a way'." (Conklin, 1996)

Enrolment management used a simple four-point methodology:

1. Establish an appropriately large shared vision;

2. Delegate completely and elicit specific commitments;

3. Inspect rigorously, providing supportive feedback;

4. Acknowledge every advance, learning as the program progresses.

The case study describes how the project management admitted they had no project plan. Nor could they possibly draw one up in the time available. Instead they took the difficulties as challenges and used each new problem as an opportunity to enforce the vision and increase the speed of development.

Perhaps most interesting about this is the similarities between many of Conklin's ideas and those of the Agile Development proponents. Perhaps the biggest difference is that most Agile advocates duck the issue of large teams engaging in Agile Development, but Conklin, in 1990, was doing Agile Development with 2,000 engineers.

Conklin, P. F. (1996) "Enrolment Management: Managing the Alpha AXP Program", IEEE Software , 13, 53-64.

Senge, P. (1990) The Fifth Discipline , Random House Books.

Bibliography

Beck, K. (2000) Extreme Programming Explained , Addison- Wesley.

Cockburn, A. (2002) Agile Software Development , Addison- Wesley.

Collins (2001) Collins Paperback English Dictionary , Harper Collins, Glasgow.

Conway, M. E. (1968) How do committees invent? , Datamation.

Coplien, J., and Harrison, N. (2003) Organizational Process Patterns (forthcoming), http://www.easycomp.org/cgibin/OrgPatterns , Wiki web site for book

Eckstein, J. (2003) Scaling Agile Processes (forthcoming), Dorset House, New York.

Kelly, A. (2003) "The original Conways Law", Overload .

Mullins, L. J. (2002) Management and organisational behaviour , Prentice Hall.

Pressman, R. S. (1994) Software Engineering: A practictioner's approach (European Adaptation), McGraw-Hill Book Company.

Pressman, R. S. (1997) Software Engineering: a practioner's approach (European adaptation), McGraw-Hill.

Somerville, I. (2001) Software Engineering , Pearson Education, Harlow.

Weinberg, G. M. (1998) The Psychology of Computer Programming , Dorset House Publishing.

Weinberg, G. M. (1999) Egoless Programming , IEEE Software.