eXtreme Programming An interview with Kent Beck

Extreme Programming is a humanistic discipline of software development, based on principles of simplicity, communication, feedback, and courage. XP was developed by Kent Beck. Nicolai Josuttis interviewed Kent for the German magazine OBJEKTspektrum. With the kind permission of SIGS that interview is reprinted here.

Nicolai Josuttis:

At the OOP'99 in Munich your talk had the biggest audience and the most feedback. First, I'd like to know where the idea of XP comes from. Who are the people that "invented" XP?

Kent Beck:

I am the one who put all the pieces together, and the one who coined the name. However, the ideas in XP come from many sources. I'm lucky if an original thought happens through my head once in five years, so generally I have to make my living putting together ideas that other folks haven't thought to put together. So, for example:

• The testing strategy in XP comes from Christopher Glaser, a compiler writer I worked with at MasPar

• The iteration schedule comes from Jon Hopkins (in my opinion the most under-rated thinker in the world of objects)

• The strict separation of business decision making from technical decision making comes from the architect Christopher Alexander

• The evolutionary design philosophy comes from my long-time colleague Ward Cunningham

• The idea of making change in small steps where you always keep the system running comes from my dad, Doug Beck, who wrote process control software in assembly language for 8 bit micro processors

NJ:

In the talk at the OOP you mentioned 10 aspects of XP:

1. testing

2. planning game

3. small releases

4. simple design

5. refactoring

6. pair programming

7. coding standards

8. collective code ownership

9. contiguous integration

10. metaphors

This seemed to be THE list of THE 10 elements of XP. However, looking on different Web Sites (such as the Wiki page "Extreme Programming Roadmap") these aspects seem to be only a part of "practices" and "concepts" of XP. Is this true?

KB:

You can explain XP at many different levels of granularity. Teams will evolve hundreds of little rules over a couple of years. From 10000 meters, XP is about short cycles with concrete feedback. The list above is good for helping programmers understand basically how they will work. It is also good for explaining how XP, which eschews many of the traditional forms of project control, can still be a stable, predictable, and reliable way to produce software.

My forthcoming book "Extreme Programming Explained: Embracing Change" (Addison-Wesley) goes into more detail about the practices above. No one book can possibly cover them all, so I've chosen to publish an overview book as quickly as possible, then help the XP community follow up with more detailed books as we gain experience.

NJ:

However, some practices seem to be more important than others. For example, testing seems to be a core aspects. If I understand it right, testing seems to be a requirement for successful refactoring, collective code ownership, and contiguous integration. So, which are the "core" aspects or practices of extreme programming?

KB:

It's tough to say exactly. In a way it's like a spider's web. If you pick up one thread, the rest of them come, too. I always end up saying, "The one most important practice is testing. Testing and refactoring. The two most important practices are testing and refactoring. And the Planning Game. Okay, the three most..."

Certainly without the tests, you're dead. Fortunately, test-first-coding is also the practice that is most useful whatever project culture you have. But, as I said, from there I quickly pull in the rest of the practices listed above.

NJ:

The most often short description of XP I heard from other people before and after the talk was "hacking". My impression was that people used this word as a mixture of joke and truth to categorise XP. Could you explain why XP is not just "hacking" or where the difference is?

KB:

There is a taboo against "programming before thinking". I think every programmer has programmed themselves into a corner at some point in their career, often very early in their career. So it becomes an insult to say that someone "jumps right into coding". And XP certainly advocates coding throughout the life of the project. So I'm not surprised to hear people say that. One big difference between XP and "hacking" is that hacking to me implies that someone does only enough design to keep from stopping altogether, where XP has a strong culture of making things simpler whenever you see that they can be made simpler. The heartbeat of XP is the coding episode, which goes like this:

1. A programming pair write the next automated test case for an engineering task. Writing the test case forces them to make design decisions about the interface for the new logic. And that design happens independent of what they are about to implement to satisfy the test case.

2. They run the test case, just in case it already runs. If it runs, and they didn't expect it to, they find out why it works, then go on to the next test case.

3. If the implementation isn't clean and simple, by which I mean if it involves duplicate code, or techniques that seem to be too complicated for the task at hand, they search for a way to restructure the existing system so they could implement the test case cleanly and simply. This is the second place design happens in XP. One of the strengths of this "situated design" is that it happens in the context of real code, not speculation, so the programmers are unlikely to make big mistakes (if they did, the code would look ugly or the tests would break and they would back up), and they are unlikely to put too much into the design.

4. They make the test case work.

5. They look back over what they did, and see if they can see any new opportunities for simplification. Once again, they do this design situated right in the code. This results in designs that are very good at expressing what the system actually does, not a design that is good at what thought, in the days of their ignorance, the designers thought the system might do.

So, XP projects undergo intense scrutiny of every aspect of their design many times every hour by pairs of programmers who believe in squeezing every last possible drop of excess complexity out of their system. And they continually develop and execute a battery of tests to make sure that the resulting system does what it set out to do. This doesn't sound like hacking to me. You tell me, though, is this just hacking?

NJ:

How would you categorise XP in the area of techniques for software or system development? Is it a new paradigm? Is it a new process? Is it a new method? Or is it just new way of programming?

KB:

XP is a step towards a new paradigm of software development. In the older paradigms, software was compared to existing activities, like mathematics, civil engineering, poetry, or electrical engineering. But software development is really fundamentally different than anything humans have done before, so any argument by analogy about how to do it is bound to limit its potential. XP starts from the premise that programming is programming, but that you have to add some activities to sustain it over time- like testing and refactoring. It is also a new process (what Alistair Cockburn calls "big-M methodology"), because it covers the development of the system from birth to death.

It is most certainly not a new way of programming. All the techniques in XP are nearly as old as programming itself. They were abandoned in favour of more complicated techniques to make up for technical weaknesses, like programming languages that didn't support enough abstraction mechanisms and databases that didn't support structural change. XP revives these techniques, relying on their synergistic effects and advances in technology to make up for their weaknesses.

NJ:

So, if we talk about the usual (evolutionary) system development process, how does XP fit into it?

KB:

XP is a set of techniques for making evolutionary system development work well in the business world. And work indefinitely. And I must say that evolutionary development is far from usual in the clients I see. Of course, that could just be my luck, but...

NJ:

That brings me to the question whether XP is a technique only for object-oriented programming or software development? Did you ever try to use it under other circumstances?

KB:

XP relies on the ability to keep the software you are writing soft, for keeping the cost of change from ever shooting up. Objects make it possible (although not certain) that you can keep the cost of change reasonably low. Other software technologies have more trouble keeping the cost of change low, but it isn't impossible.

Martin Fowler, Ron Jeffries, Ken Auer, Ward Cunningham and I are writing a XP how-to book. We thrashed around for a couple of hours trying to figure out how to proceed when someone suggested we "do it extreme". So we wrote stories, cut scope to fit our time and page budget (the book focuses on planning and testing), accepted responsibility for tasks, continually integrated (using Ward's fabulous Wiki-Wiki Web server), constantly refactored... It has been a enjoyable and enlightening process so far- not least because no one can figure out who wrote what paragraphs.

From the customer's perspective, XP is about managing the situation where you have too much to do. You have to estimate, set priorities, track changes. I certainly do a better job of avoiding over commitment now than I did before I started with XP (it's still a problem from time to time, of course).

NJ:

If I understand it correctly, there are several aspects in extreme programming that are not "extreme". What I mean is the following:

The ideas of pair programming and collective code ownership lead to teams that don't have experts anymore. The team is a group of people who tend to have the same knowledge. This seems to imply that it is difficult to use special features that are only known by certain people. I wonder whether this approach isn't a problem in software development where special knowledge is necessary. If you try to get the best of new technologies it seems quite risky not to have experts in the team who program "extreme" close to the edge.

KB:

You could have stopped that last sentence right after "risky". I hate risk. I hate flushing good programming down to toilet because a technical or business risk became reality, a risk that could have been avoided.

That doesn't mean that you can't ever do interesting things on an XP project, but you would never try new technology for new technology's sake. You would figure out what was the least amount of the new technology you could get away with using, then you'd write tests for it, and then you'd use it.

Too often I've reviewed failing projects where they are using every last bit of CORBA, for example, and then the project fails because it doesn't work. When you look at what they needed to use, really, it was 10% of what they did use. That doesn't make sense to me.

NJ:

Martin Fowler and you seem to have different opinions about documentation. Martin said, documentation is necessary to get an introduction into the concept of a system. You said something like "documentation may be only necessary before I die and can't explain it personally". It might be better to document "in the brain" and to explaining a system personally and by using test cases. However, managers still have the idea to be able to change software developers like hardware. Especially, they don't want to get dependent from their developers. Otherwise, a company may become out of the business when the developers are not available anymore. What happens if the XP team leaves the company (whether or not it is because the project ends)? So, don't you always need conceptual documentation?

KB:

First, I hate this "plug compatible programming unit" idea. Different people are different. Managers have to get used to this, or they'll get into trouble. Second, managers are absolutely dependent on their developers already. Of course, the developers are dependent on managers, too. If the whole development team ever walks out, the company is in big trouble, no matter how many trees you have killed. With an XP project, at least you'll have the test suite to keep new people from causing damage.

As you can imagine, this debate on documentation causes at least as many fist fights as evolutionary design. Projects often start with high ideals for documentation, but they always fall by the wayside in the crush of business. I observed this and said to myself, "Perhaps people don't maintain detailed documentation because it isn't actually a good idea." If it hurts running your head into a brick wall over and over, perhaps you should figure out how to get along without running your head into the brick wall.

This isn't to say that there is no place for a printer on an XP project. I think a 5-10 page guided tour of a medium sized system is an excellent idea. Writing such a paper forces the team to think about what is really important in the system. If it just isn't possible to reduce the system that far, there is a problem with the design.

On this topic, as on many others, the final answer is that the team has to decide for itself. If you can't imagine doing without lots of written documentation, go ahead and write it. But measure, always measure. If no one ever reads the documents, stop writing them.

NJ:

The best size for a team of XP seems to be a group of 5 to 10 people. However, there is software that can't get developed by such a small team (despite from the fact that a small group of software developers might be faster than a large group that has all the organisation overhead). Do you have any experience with XP in huge projects? Would it be possible to use XP in a hierarchical project structure?

KB:

First, notice that you mention irrational constraints for me, here - "you have to use 100 people even though 10 people would suffice." That isn't a good place to begin. But I agree that business will often impose such irrational constraints. Or try to impose them.

XP hasn't been used by more than 10 developers. I don't think it scales up to 100 developers. There are techniques, like continuous integration, that would be hard to implement at that scale.

What I think is possible, and this is just speculation, is that you could develop the first release of a system with a single team. Then with that experience you could find subsystems that could be worked on separately. You could split the team and grow each team back up to 10 or so. At most, I think you could get 4-5 teams working simultaneously this way.

NJ:

Your favourite language is Smalltalk. I was very impressed about the refactoring tool you demonstrated. You said that Java might have such a tool in about 5 years. What is your opinion about Java? Do you think it's a step backward?

KB:

Java is a step forward from Smalltalk in some ways, such as built-in networking and interfaces. In other ways, when I program in Java I think I'm back 15 years ago. Imagine, putting source code in files! How quaint. Imagine, not having decent support for refactoring! How annoying. Imagine, a compiler that refuses to run my program if it isn't done! My programs are never done.

On the other hand, my favorite language has lost the big battle for developer mind share, so I have to learn how best to deal with the Java world if I want to talk to lots of developers.

NJ:

Any opinion about C++?

KB:

Sure. C++ is an excellent language when the shape of the hardware strongly affects the shape of the software. I worked on a debugger for a supercomputer, and the server side was written in C++. I wouldn't have wanted to try it in Smalltalk.

NJ:

You mentioned two books. Could you give me some details about titles, authors, and when they will be available?

KB:

The first book is called "Extreme Programming: Embracing Change" and it should be out in October. I wrote it myself as a manifesto. I expect readers to use it to decide whether XP is for them or not. But it isn't a tutorial. You can't read it and immediately begin working extreme.

There is a lot of material on the Web that helps with how-to kinds of questions. For example www.xprogramming.com and c2.com/cgi/wiki?ExtremeProgrammingRoadmap . I have talked to three or four projects that are trying to execute XP just from this material, but it suffers from the same problem that much of the Web suffers, it isn't terribly well organised.

So, the second book I mentioned above is written for someone who wants to execute XP. But it doesn't cover all the practices, only those related to planning and testing. It will be called something like "Extreme Programming: Playing to Win".

I know that it will be frustrating to publish in bits and pieces this way. People would like to have "The Big Book of XP". But it will take four or five years to write, and it will contain 700 pages. In fact, that's what we're writing, we're just publishing it a little at a time, so we have the chance to get feedback from readers about what is most important to cover.

NJ:

How would you start with XP in a company or a project? I saw a discussion regarding this at the wiki pages. And it seemed to be a common agreement that it is a good way to start with testing and pair programming. However, you also recommended to start with the planning game. So, how should people start with XP?

KB:

I came up with all sorts of complicated schemes. Don Wells, one of the original Chrysler payroll people who went on to rescue a project at Ford using XP, set me straight. If you want to do XP, but don't know where to start, figure out your worst problem- quality, predictability, communication, whatever. Then do what XP says to do- write tests, play the Planning Game, pair program. When that problem is solved, figure out your next most pressing problem and solve it the XP way. The beauty of this strategy is that it is always working on the problem with the biggest impact, so everyone has motivation to make it work. It is also incremental, so people don't have to try to learn too much at once. And it can be done very cheaply at first. Once you begin to show results, you can get resources for the more changes.

NJ:

Thank you for taking your time for this interview and good luck for the future.

Nicolai M. Josuttis

http://www.josuttis.de/

Nicolai M. Josuttis is an independent technical consultant and system architect who designs object-oriented software for the telecommunication, traffic, finance, and manufacturing industries. He is an active member of the C++ Standard Committee library working group and a partner at System Bauhaus, a German group of recognized object-oriented system development experts. Josuttis has written several books on object-oriented programming and C++, including the brand new "The C++ Standard Library - A Tutorial and Reference".