XP introduction
Un article de Agile-Swiss.
eXtreme Programming (XP) is a methodology for software developments that originated in the states about 5 years ago.
Originally born among Smalltalkers, it has gained wide acceptance among Java developers in the recent years.
Unlike other methodology such as RUP, XP is not a product that can be bought in a box or from a series of consultants. eXtreme Programming is foremost a philosophy, which is transmitted to a whole team of developers in order to increase its efficiency.
In many respects, a software development project is a risky endeavor. Software projects are notoriously expensive, and more than often exceed the estimated budgets and go beyond the promised deadlines. Studies have shown that as much as 80% of software projects fail and never make it to their intended customers. For those who succeed, delivery occurs at the end of a bumpy ride and the value of the end result is often tarnished by the last minute efforts and struggles.
These problems stem from two kinds of causes.
The first one—rather well known—is the lack of competent resources. This is no news. However, this never-ending struggle for competent software developers has worsened with object-oriented programming. A recent (2002, at the time the original article was written) study by the Gartner group shows that only 50% of the demand for java developers is satisfied.
The second cause for the difficulty of software developments is largely ignored: a significant part of the high costs and slipping delivery times stems from the fact that the duration of a software project cannot be predicted reliably. Thus, software contractors have the tendency to inflate estimated times by a factor more or less defined depending on the experience of the team. Such practice being known by the customers has led to the inverse situation: customers are imposing unrealistic delay and cost limitations, taking advantage of a competitive situation on the market.
So far, the delay slippages have been attributed solely to the developers. Under such premises the obvious solution has been to impose a strict methodology. In such methodologies, inspired mostly from civil engineering practices, a software project is considered as a construction site. In spite of the fact that using these methodologies has not cured the problem, most people keep that analogy in the back of their mind and use planning in a dictatorial way to direct software projects.
The error in such thinking is that a software application, as the name implies, is soft in nature. In other words, its shape can change. Nobody would think of adding a few stories to a building nearing completion since everyone knows that it would likely cause the building to collapse. However, people are quite used to request new functionalities to an application. And yet, those last minute additions can have the same devastating effect on the overall architecture of the system, especially when one uses methodology based around the civil engineering paradigm.
To be able to stick closer to that paradigm, some contractors thought of taming the customer by imposing a strict adherence to a list of requirements established before a single line of code is written. The net effect of such practice is that, when the customer receives the completed product—after several months or worse a couple of years—the delivered application does not correspond to the needs of the customer. This is called the tunnel effect.
Since the mid 90s several people have elaborated radically new ideas for the conduct of software projects. The resulting methodologies have been grouped lately under the term agile programming because they have in common the following principles.
Accept the reality of change and integrate that fact into software developments, Get rid of the tunnel effect by working at very short term. The most famous of these new methodologies is eXtreme Programming, XP[1] for short. eXtreme Programming is mainly based on common sense, team communication and uses many proven techniques. The originality of XP is to gather all these techniques within a coherent structure. The extreme qualifier is a reminder that XP aims at using all these techniques without any compromise. XP articulates itself around 8 tenets.
1. Clear separation between business and technology.
XP respects the competence of each. The customer, who is considered part of the team, is the one that knows best the business issues. Developers know the technology. The customer is the one to decide the priorities. Developers are the one to estimate the delays. For example, the customer shall decide which functionalities must be implemented first, the design of the screens and the workflow. However, the customer shall not impose the software architecture or the layout of the database.
2. Test driven design (TDD).
Each functionality requested by the customer must be tested. These tests are called acceptance tests. Similarly, the code produced by the developers must be tested. These tests are called unit tests.
Both acceptance and unit tests must be executed automatically. This is quite valuable for the developer to test his/her code during development. In addition, all tests must be run after each modification of the code so that potential side effects of any modification or addition can be checked and caught immediately. New code segments are only validated when they pass all tests.
Moreover, XP requires that developers write the unit tests before writing the code to be tested. This precept helps to clarify how to use the code. Finally, the acceptance and unit tests constitute the best documentation possible. A developer can look at the unit tests to see how a particular class and/or method is used. Since the tests are used constantly, a developer can be assured that the method is indeed working the way it is used (described) in the tests.
3. Continuous integration.
Code that works—that is, code that passes all the tests—is deployed immediately on a system accessible by everyone. In particular the customer can test the application from his/her point of view on that system. Each developer is responsible for integrating and deploying the code that (s)he has written. Thus, deployment problems, that are increasingly hard to master for distributed applications, are kept under control.
4. Small releases.
This is a direct consequence of continuous integration. As soon as a series of newly coded functionalities makes sense, the customer can decide to create a new release. This completely avoids the tunnel effect as very little time elapses between the specification of a given functionality and its release to the end users (typically one week).
For example, the application Lifeware (of the Credit Suisse group, Switzerland) has been developed at the eXtreme pace of one new version per day.
5. Simple design
XP banishes virtuosity. Every developer is requested to produce code that can be read by everyone. The belief is that readable code is better than any comment. At the end of the day, one cannot trust a comment because what gets executed is the code and not the comment.
In addition, code must be written for the current need of the customer. Developers have the tendency to bring a general solution to a particular problem, anticipating on future needs. In practice, the only effect of "programming for tomorrow" is to bring a loss of time twice. First, time is lost during development since creating a general functionality takes more time. Second, time will also be lost at the time the foreseen functionality will be used. The code written by anticipation is rarely well targeted and usually cost more when time comes to use it since it requires adaptations that have not been foreseen.
In short, XP vies to implement a maximum of functionalities with a minimum of coding.
6. Pair programming
The rational behind pair programming is that one thinks while the other types in the code. A lone developer could be tempted to sometimes drop one of the precepts, especially when (s)he is stuck. The other member of the pair is there to bring back sanity. A lone programmer can spend days struggling to find a stupid mistake, a pair is seldom caught by such a snag.
However, pair programming goes beyond this regulating role. By permuting the pairs frequently, one ensures a better communication between the team members than any meeting or documentation. Last but not least, junior developers and newcomers are quickly integrated in the team and can quickly produce quality work when working in pairs.
7. Collective code ownership
With XP there is no secret garden anymore. Each developer has the right to modify the code of others. This means that a project cannot get stuck when someone is missing. Moreover, everybody gets a chance to increase her/his competences. Not only this is quite an efficient way to work, but also it is highly motivating for the team members.
8. Refactoring
With XP, development does not stop when a functionality is implemented and passes the tests. The developers must make sure that the corresponding code fits within the current architecture of the application. Thus, during an XP project, the architecture of the application evolves gradually while adapting itself to the customer’s requirements.
The refactoring step is essential to ensure a long-term maintainability of the application. This is what permits the addition of "new stories to a building without causing it to collapse". The reader who wants to learn more about refactoring should read the book of Martin Fowler et al.[3].
All these points are not independent from each other. For example, collective code ownership would not be possible without simple design and becomes easy with pair programming and test driven design. The book of Kent Beck[1] explains these relationships in details.
As we can see XP brings a democratic spirit in the world of software methodologies that were mostly based on totalitarian management. Old methodologies were addressing the problem of constructing an application with poorly qualified manpower. XP paves the way for creativity while maintaining a high level of efficiency.
Democracy, but not anarchy! XP adepts follow the 8 tenets above without any compromise, hence the name eXtreme. Unlike what is claimed by its detractors, however, XP keeps a planning and an architecture. The significant difference is that XP acknowledges that those are not immutable. Planning[2] is modified accordingly with the change requests of the customer; the architecture is adapted while the application is growing.
Many projects that were in a state of dereliction using other methodologies have been saved thanks to XP. Interested readers can consult the list of URL links below to learn about them and to document themselves about the full content of XP. They can also contact me directly if they wish: I’ll be glad to continue this discussion by E-mail, phone, or even better, around a glass of wine. XP is also about enjoying been a developer.
Didier Besset, November 2002
Links : http://www.extremeprogramming.org/ http://www.xprogramming.com/ http://www.martinfowler.com/
Books :
Here is my personal collection of the books about XP and related subjects. To find more references, follow the bibliographies given on the links above.
[1] Kent Beck, Extreme Programming Explained: Embrace Change, Addison-Wesley Pub Co, 1999, ISBN: 0-201-61641-6.
[2] Kent Beck & Martin Fowler, Planning Extreme Programming, Addison-Wesley Pub Co, 2000, ISBN: 0-201-71091-9.
[3] Martin Fowler et al., Refactoring: Improving the Design of Existing Code, Addison-Wesley Pub Co, 1999, ISBN: 0-201-48567-2.
[1] Not to be confused with an OS from Microsoft
[2] Planning in XP projects is not covered in this introductory article, but interested readers can refer to the excellent book by Kent Beck and Martin Fowler [2].
