Recent discussions on the WG2 "core" reflector [ 1 ] have involved considerable speculation about the proportions of the C++ community that employ different memory management strategies. While the comparative sizes of their constituencies are unclear it was apparent that both "smart pointers" and "garbage collection" have adherents and also that users of each approach had limited experiences of designs that deploy the other strategy.
Over the last month or so there has also been a newsgroup thread on c.l.c++.m - "Smart Pointers" - where there are clearly parts of the community on both sides of the question "are reference counted smart pointers a useful design option"? (And at the time of writing "garbage collection" isn't one of the alternatives that has been presented.) There is a difference between these discussions - the protaganists on the "core" reflector appear to be intent on learning from each other's experience, those on the newsgroup in evangelising. (Guess which is the more interesting to follow.)
There are many problems that must be addressed by C++ developers and, while the issues faced will vary from developer to developer, memory management is an issue that must be addressed by almost all.
The example set by the "core" working group is too little and too late: that the C++ community is failing to share and to build upon the experience of employing different memory management strategies signifies that there is something wrong.
It Matters That Developers Can't Talk to Each Other
A passage from " The IT Team " [ Lees03 ], reproduced in Figure 1, illustrates why it matters that developers communicate ideas.
Figure 1 - A Passage from "The IT Team"
"Tuckman (1965) identified four developmental stages for a team to which Tuckman and Jensen (1977) added a final stage of adjourn. The figure provides an adaptation of that model which attempts to illustrate that:-
-
A team may spend an undetermined and variable amount of time at each stage
-
May not reach all stages
-
May regress to previous stages.
"It is estimated that three fifths of the length of any team project, from start to finish, is taken up with the first two stages, Forming and Storming." (Robbins, Finley 1996 p.191).
"As these stages are non-productive the ease with which a team moves through them will have significant impact on the team's productivity."
A project balkanised according to the dialects its codebase is being written in isn't going to run as smoothly or efficiently as one that is unified in its approach. The costs are often hidden - developers may be reluctant to touch "foreign" code, reuse opportunities may be missed, barriers are raised to refactoring - all things that can be attributed to other causes.
I've seen it - in a codebase of only 50KLOC I've seen four date+time classes, three classes for parsing XML and a mass of utilities for converting strings from one implementation to another.
C++ Dialects
The C++ language deliberately sets out to support a wide range of usage styles, and a number of usage styles are easily identifiable:
-
Procedural programming ("a better C")
-
Object based programming ("Data abstraction")
-
Object-Oriented programming
-
Generic programming
-
Embedding of domain specific languages
-
Template metaprogramming
Even aspects of functional or generative programming can be tackled, although these are generally deployed to support the above approaches. While this flexibility provides the opportunity to select the most appropriate and effective approach in any given context, it also provides plenty of scope for selecting an inappropriate approach for the problem in hand.
The downside to this wide ranging support for different styles of use is that the developer community has split into groups using distinct dialects. Does this matter? I think it does, because it impacts the ability of C++ users to communicate with one another. While dialects do develop in other languages, C++ seems to be unique in its propensity for them to wreck a project, dividing developers into groups that do not communicate well.
In "Is high-church C++ fair?"[ Kelly01 ] Allan Kelly describes his experience of multiple dialects in the organisation he was working in. Allan describes four dialects (I'm sure we've all seen them) as follows:
"High-church - "the kind of C++ you see in C++ Report" : as I said above, templates, standard library, interface inheritance more important than implementation inheritance, full use of language features and new keywords
Low-church - "the kind of C++, C programmers write up to the kind of C++ in the MFC" : few templates, roll your own data structures, lots of void casts, implementation inheritance more important than interface inheritance
C - "the kind of C++ which is just C with classes" : we can probably all recognise this, and probably wrote a bit of it ourselves in the early days. However, after a short while we are either high-church C++, low-church C++ or, last but not least.....
Bad C++ - "where the developer writes C and has picked up a few bits of C++ and doesn't realise what they are doing" : no understanding of copy constructors, inheritance (of all kinds!) used without a clear reason, destructors releasing resources before they are finished with, no standard library, lots of void casts, unnecessary and inappropriate file dependencies."
Allan goes on to discuss the impact these have on his work.
"Once again it has come time for me to hand over a project and move on. This is a fairly big project, very important to the company, and, as I'm leaving the company it's important that the hand-over goes smoothly.
The code should be readable to most Overload readers (sorry, no examples), and most would not look out of place in C++ Report, in other words, it's what I call "high-church C++", it uses the language to the full, is littered with design pattern references (indeed, the main loop is a command pattern), a few templates, plenty of standard library, and inheritance is used mainly for interface not implementation inheritance.
…
I have deliberately used the terms high-church and low-church, not only because of well know religious programming wars but because both sides have their arguments (high-church claims to be more object oriented, while low-church claims to be more understandable to average programmers) and because it is easy for high-church to look down on low-church as less than true, while low-church regards high-church as fanatical.
…
Returning to my own project, if I where to take over a similar one, or someone of my experience where to take over this project I do not think they would have a problem. Indeed, they would probably pick up the code and understand it quiet quickly, the abstractions should mean you can understand the high-level side without understanding the details. Only when the details change do you need to understand them."
While these comments focus on only two of the dialects they do illustrate the divisions that can occur between the corresponding congregations Not only does each group have difficulty understanding the other, each also believes that theirs is the correct way to use the language. What is also interesting from this description is the implication that the team in which Allan was working does not seem to have resolved these differences and remains in the "storm" phase of team development.
The Origin of Dialects
I hope that I've demonstrated that C++ dialects matter - because they can dramatically reduce the effectiveness of the team. But where do these dialects come from? And why is C++ particularly prone to these problems?
Developers frequently move to C++ from another language and bring with them an approach that fits that language well. By design C++ allows them to do this - and, in doing so, fails to encourage an evaluation of the appropriateness of their approach in C++. As a result the developer works in a creole [ 2 ] formed of these two languages. In the early days of C++ the C/C++ creole was commonplace, but Java/C++ has gained recently.
Even developers coming first to C++ are not immune to this effect - they can inherit it from authors, trainers and colleagues. Kevlin Henney in "The miseducation of C++" [ Henney01 ] identifies three families of C++ dialects and goes on to relate them to the way that C++ is taught.
"Early C++: The first public incarnation of the language formed part of a growing interest in the use of object-oriented development in industry. Early C++ was a minimal language, extending C with a stronger type system, and the notion of classes with public and private access, single inheritance and operator overloading.
Classic C++: This is the C++ that many long-time C++ programmers know, and many new C++ programmers are still effectively being taught. It brought abstract classes, multiple inheritance and further refinements into the language. Classic C++ also introduced templates and exceptions, although they took longer to mature than other, more obviously evolutionary features. Where templates are used, it's normally to articulate container classes and smart pointers. Classic C++ saw the greatest proliferation in dialects as it was in this period that the language underwent standardisation. Changes ranged from the trivial to the major, from bool to namespace, from the proper incorporation of the standard C library to the internationalisation overhaul of I/O. Many projects still bear the scars of this portability interregnum.
Modern C++: This is where we are now. The language was standardised in 1998, providing a fixed target for vendors and a consolidation in thinking and practice. Exception safety is now a well understood topic and the Standard Template Library (STL) part of the library has introduced the full power of generic programming to the language, raising templates from being merely a way to express containers to an integral part of design expression in C++."
Despite the implication of a timeline in this classification, the dialects of Early C++ have not been assigned to the dustbin of history. They are still living languages: I've recently had reports of a heavily MACRO based style of C++ in one organisation that sounds as though it belongs to this group.
And "Modern C++" isn't the final, definitive way to use the language: there are also dialects that move beyond the scope of Modern C++ and into generative techniques. These have their place but, like anything else, when seen as a "shiny new hammer" the effect is to introduce complexity that serves only to postpone necessary design decisions and introduce uncertainty.
What the "timeline" does reflect is that there has been an evolution in the thinking regarding effective ways of using C++: the "later" dialects are the result of learning from the use of the earlier ones. As a consequence they generally avoid problems that arise in using the "earlier" dialects and, in consequence, are more capable of handling complexity. Thus resource management is problematic in Early C++ dialects, requiring constant attention, while Classic and Modern dialects incorporate idioms that free the developer from this constant distraction. (While this evolution of the idioms in contemporary usage provides for better solutions to harder problems it also creates a difficulty - a style that was considered "best practice" a few years ago might now be dismissed as misguided or sub-optimal.)
C++ Accents
There are also some more superficial elements that contribute to the separation of dialects in C++. Unlike the factors that separate Early, Classic and Modern C++ (or High Church/Low Church C++) these don't have a direct impact on the technical effectiveness of the language - and they can be found in dialects of all eras. But they are highly visible, and present a barrier to communication between their adherents:
-
lower_case or CamelCase
-
brace placement
-
prefixes and suffixes
-
…
I'm sure we all have preferences - mine is for consistency. (As I like to use the standard library and boost, I like to be consistent with the style these adopt.)
Solutions
The range of C++ dialects is vast - Allan describes four major groupings and Kevlin three, but a closer examination would probably find at least one for every C++ project in the world. If different developers in a team are comfortable with different dialects then these incompatibilities between them can make it very hard for the team to reach the performance levels it would otherwise be capable of.
Later in "Is high-church C++ fair?" Allan addresses one "solution":
"So may be my question is should really be turned onto management, the developers they hire, appoint to projects and training courses they use. But, most manages [sic] (well those I have encountered) gave up programming when they got the keys to the company car. If I actually asked them what I should write, I think I would be told to write low-church C++, but this would take me longer (more code would need to be written and more testing) and I do think it would be less maintainable in the long run. But without two individuals writing the same project in low and high C++ how can we quantify this?"
Allan returns to the subject in Overload 65: his article "The Developers New Work" [ Kelly05 ] examines a number of non-solutions:
"We could "dumb down" our code, make it really simple. Trouble is, we have real problems and we need real solutions. To tackle the same problem with "low Church code" just moves the complexity from the context to an overly verbose code base.
We could just hire real top-gun programmers. This isn't really a solution; once again we're pushing the problem down in one place and seeing it come up in another. Since there aren't that many super-programmers in the world finding them is a problem, keeping them a problem, motivating them is a problem and even if we overcome these problems it's quite likely that within our group of super-programmers we would see an elite group emerge.
Hiring a group of super-programmers is in itself an admission of defeat, we're saying: We don't know how to create productive employees; we're going to poach people from companies who do. In doing so we move the problem from our code to recruitment.
So, maybe the solution is to get management to invest more in training. But this isn't always the solution. The mangers I had when I wrote High Church C++ tried to do the right thing. Is it not reasonable to assume that someone who has been on a C++ course can maintain a system written in C++? As Alan Griffiths pointed out, this is about as reasonable as expecting someone that has been on a car maintenance course to change the tires during an F1 pitstop.
The answer of course is: No, knowing C++ is a requirement for maintaining a C++ based system but it isn't sufficient of itself. One needs to understand the domain the system is in and the system architecture - this is why Coplien and Harrison say you need a whole year to come up to speed.
How do we communicate these things? The classical answer is "write it down" but written documentation has its own problems: accuracy, timeliness, readability, and memorability to name a few. In truth, understanding any modern software system is more about tacit knowledge than it is about explicit knowledge."
Well, that tells us a lot about what not to do, but in real projects we still need an answer.
Given my metaphor, dialects of spoken languages, I'm sure that my solution won't be a surprise to you. One can learn something of a language from dictionaries and grammars, but the only way to become fluent is to use it to communicate with others. That means reading and writing code - in the appropriate dialect - and discovering if it is understood by other developers.
Allan's answer [ Kelly05 ] is similar:
"It is no longer enough to just cut-code. Sure you may need to do this too, but if you want to use modern C++ (or modern Java, Python, or what ever) it is your job to lead others in a change. And change doesn't happen without learning. Indeed, learning isn't really happening if we don't change, we may be able to recite some piece of information but unless we act on it we haven't really learnt anything.
So, when it comes to improving your code it isn't enough to sit your colleagues down and tell them that a template-template function is the thing they need here and expect them to make it so. You've imparted information, you may even have ordered them to do it, but they haven't been led, they haven't learnt and they won't have changed - they'll do the same thing all over again.
Simply informing people "This is a better way" doesn't cut it. You can't lecture, you can't tell, you can't enforce conformance. You need to help others find their own way to learn. Helping them find that way goes beyond simply giving them the book, they need to be motivated, people who are told aren't motivated, people who are ordered aren't motivated; motivating people requires leadership.
…
Hope lies not in code, not in machines but in people. If we believe that Modern C++ is best - and I truly, rationally, believe it is - then I have no choice other than to develop the people around me - and that belief is rational too."
I've more experience with this approach than Allan appears to have, and there is a problem that he appears unaware of: ten years (or so) ago I introduced my colleagues to elements of the latest thinking on how to use C++ - they were new to the language and I followed the literature. They adopted many of these idioms - things like "smart pointers" to managed object persistence, even some bits of STL. Then I left the organisation.
I'm still in touch with the development group and some of the developers there. Ten years ago "Classic C++" was the best way we knew to use the language, but C++ is a living breathing language and new idioms are being formed all the time. When I left these developers were no longer exposed to the "latest thinking" - like most of the industry they didn't "do books" (DeMarco and Lister - in "Peopleware" - report that the typical developer reads fewer than one technical book a year). They were not motivated to seek out new solutions in a subject where they were not experiencing problems. Nowdays a dialect of Classic/Low Church C++ is deeply entrenched and Modern/High Church C++ has passed them by. By failing to engage them directly in the wider community I left a situation where a local dialect was going to develop over time.
The lesson I take from this is that the goal should be more ambitious than ensuring a lingua franca within the team, more than introducing a contemporary dialect of C++. Developers must be exposed to developing ideas and engaged in the community developing them.
So What Works?
I don't have a magic bullet - I doubt that one exists. The different dialects of C++ have developed in response to genuine circumstances.
However, in the context of an individual project or team there is a range of things that, if employed sensibly, can make things better rather than worse:
-
Coding guidelines? - These are safe if they deal with "accents", but can entrench obscure and obsolete dialects unless they are carefully reviewed, living documents that change as ideas are revised. See http://groups.yahoo.com/group/boost/files/coding_guidelines.html for a good example.
-
Training courses? - These can expose those that attend to new idioms, but they cannot ensure they are either internalised or adopted by others.
-
Code Reviews? - If properly conducted these can help the team settle on a common dialect, or at least become familiar with those in use. On the other hand they can be a cause of friction and entrenchment.
-
Pair programming? - I've heard good reports of this - and bad. My experience does not include it working as a routine practice but I've been convinced that it can produce good results "when done properly".
-
Coaching/mentoring? - Good coaches/mentors are hard to find. Bad ones are worse than useless.
-
Reading and writing about C++? - People that read books and other publications are aware of wider community, but only those that participate (in newsgroups, at conferences, or writing books or for magazines) are truly involved in the community.
Conclusion
Divisions in the C++ community can cost in a number of ways, the most obvious being the inability of team members to communicate in the language. Another cost, that is easier to overlook, is that as techniques are developed to address the problems that developers face, they are not known outside the part of the community that develops them. This can lead to the "re-inventing of wheels" - I've lost count of the different smart pointers I've encountered (most of them with avoidable bugs). Even worse, it can lead to "living with" avoidable problems - the typical C++ program leaks resources despite the fact that RAII (and the techniques for implementing it) have been known to some parts of the community for a long time.
Naturally, everyone that reads this is making an effort to be part of the solution instead of being part of the problem. However, I urge you to make an effort to be more involved and to seek ways to involve your colleagues in the wider community.
References
[Kelly01] "Is high-church C++ fair?", Allan Kelly, http://www.allankelly.net/writing/WebOnly/HighChurch.htm
[Lees03] The IT Team , Sarah Lees 2002
[Henney01] "The miseducation of C++", Kevlin Henney, Application Development Advisor , April 2001, http://www.two-sdg.demon.co.uk/curbralan/papers/TheMiseducationOfC++.pdf
[Kelly05] "The Developers New Work", Allan Kelly, Overload 65 : 1. Tuckman B (1965) in Rickards (2000). 'Development Sequence in Small Groups'. Psychological Bulletin Vol.63/6 pp.384-399. 2. Tuckman B and Jensen M (1977) in Rickards (2000). 'Stages of Small Group Development Revisited'. Group and Organizational Studies, Vol.2 pp.419-427.
[ 1 ] "Core" is the group working on the part of the C++ standard that defines the language.
[ 2 ] A creole is a language descended from a pidgin that has become the native language of a group of people. http://en.wikipedia.org/wiki/Creole_Language
