One of the most important documents a project can have is some kind of elaboration of what the maintainers want to see happen in the future. This is the concrete expression of the project vision – it allows people to adhere to the vision, and gives them the opportunity to contribute to its realisation. This is the document I’ll be calling a roadmap.

Sometimes the word “roadmap” is used to talk about other things, like branching strategies and release schedules. To me, a release schedule and a roadmap are related, but different documents. Releasing is about ensuring users get to use what you make. The roadmap is your guiding light, the beacon at the end of the road that lets you know what you’re making, and why.

Too many projects fall into the trap of having occasional roadmap planning processes, and then posting a mighty document which stays, unchanged, until the next time the planning process gets done. Roadmaps like these end up being historical documents – a shining example of how aspirations get lost along the way of product development.

Other projects are under-ambitious. Either there is no roadmap at all, in which case the business as usual of making software takes over – developers are interrupt-driven, fixing bugs, taking care of user requests, and never taking a step back to look at the bigger picture. Or your roadmap is something you use to track tasks which are already underway, a list of the features which developers are working on right now. It’s like walking in a forest at night with a head-light – you are always looking at your feet avoiding tree-roots, yet you have no idea where you’re going.

When we drew up the roadmap for the GIMP for versions 2.0 and 2.2 in 2003, we committed some of these mistakes. By observing some projects like Inkscape (which has a history of excellent roadmapping) and learning from our mistakes, I came up with a different method which we applied to the WengoPhone from OpenWengo in 2006, and which served us well (until the project became QuteCom, at least). Here are some of the techniques I learned, which I hope will be useful to others.

Time or features?

One question with roadmaps is whether hitting a date for release should be included as an objective. Even though I’ve said that release plans and roadmaps are different documents, I think it is important to set realistic target dates on way-points. Having a calendar in front of you allows you to keep people focussed on the path, and avoid falling into the trap of implementing one small feature that isn’t part of your release criteria. Pure time-based releases, with no features associated, don’t quite work either. The end result is often quite tepid, a product of the release process rather than any design by a core team.

I like Joel’s scheduling technique: “If you have a bunch of wood blocks, and you can’t fit them into a box, you have two choices: get a bigger box, or remove some blocks.” That is, you can mix a time-based and feature-based schedule. You plan features, giving each one a priority. You start at the top and work your way down the list. At the feature freeze date, you run a project review. If a feature is finished, or will be finished (at a sufficient quality level) in time for release, it’s in. If it won’t realistically be finished in time for the release date, it’s bumped. That way, you stick to your schedule (mostly), and there is a motivation to start working on the biggest wood blocks (the most important features) first.

A recent article on lessons learned over years of Bugzilla development by Max Kanat-Alexander made an interesting suggestion which makes a lot of sense to me – at the point you decide to feature freeze and bump features, it may be better to create a release branch for stabilisation work, and allow the trunk to continue in active development. The potential cost of this is a duplication of work merging unfinished features and bug fixes into both branches, the advantage is it allows someone to continue working on a bumped feature while the team as a whole works towards the stable release.

Near term, mid term, long term

The Inkscape roadmap from 2005 is a thing of beauty. The roadmap mixes beautifully long-term goals with short-term planning. Each release has a by-line, a set of one or two things which are the main focus of the release. Some releases are purely focussed on quality. Others include important features. The whole thing feels planned. There is a vision.

But as you come closer and closer to the current work, the plans get broken down, itemised further. The BHAGs of a release in 2 years gets turned into a list of sub-features when it’s one year away, and each of those features gets broken down further as a developer starts planning and working on it.

The fractal geometer in me identifies this as a scaling phenomenon – coding software is like zooming in to a coastline and measuring its length. The value you get when measuring with a 1km long ruler is not the same as with a 1m ruler. And as you get closer and closer to writing code, you also need to break down bigger tasks into smaller tasks, and smaller tasks into object design, then coding the actual objects and methods. Giving your roadmap this sense of scope allows you to look up and see in the distance every now and again.

Keep it accurate

A roadmap is a living document. The best reason to go into no detail at all for  future releases beyond specifying a theme is that you have no idea yet how long things will take to do when you get there. If you load up the next version with features, you’re probably aiming for a long death-march in the project team.

The inaccurate roadmap is an object of ridicule, and a motivation killer. If it becomes clear that you’re not going to make a date, change the date (and all the other dates in consequence). That might also be a sign that the team has over-committed for the release, and an opportunity to bump some features.

Leave some empty seats

In community projects, new contributors often arrive who would like to work on features, but they don’t know where to start. There is an in-place core team who are claiming features for the next release left & right, and the new guy doesn’t know what to do. “Fix some bugs” or “do some documentation” are common answers for many projects including GNOME (with the gnome-love keyword in Bugzilla) and LibreOffice (with the easy hacks list). Indeed, these do allow you to get to know the project.

But, as has often been said, developers like to develop features, and sometimes it can be really hard what features are important to the core team. This is especially true with commercial software developers. The roadmap can help.

In any given release, you can include some high priority features – stuff that you would love to see happen – and explicitly marked as “Not taken by the core team”. It should be clear that patches of a sufficiently high standard implementing the feature would be gratefully accepted. This won’t automatically change a new developer into a coding ninja, nor will it prevent an ambitious hacker from biting off more than he can chew, but it will give experienced developers an easy way to prove themselves and earn their place in the core team, and it will also provide some great opportunities for mentoring programs like the Google Summer of Code.

The Subversion roadmap, recently updated by the core team, is another example of best practice in this area. In addition to a mixed features & time based release cycle, they maintain a roadmap which has key goals for a release, but also includes a separate list of high priority features.

The end result: Visibility

The end result of a good roadmap process is that your users know where they stand, more or less, at any given time. Your developers know where you want to take the project, and can see opportunities to contribute. Your core team knows what the release criteria for the next release are, and you have agreed together mid-term and long-term goals for the project that express your common vision. As maintainer, you have a powerful tool to explain your decisions and align your community around your ideas. A good roadmap is the fertile soil on which your developer community will grow.

I’ve been thinking a lot recently about mentoring programs, what works, what doesn’t, and what the minimum amount of effort needed to bootstrap a program might be.

With the advent of Google Summer of Code and Google Code-In, more and more projects are formalising mentoring and thinking about what newcomers to the project might be able to do to learn the ropes and integrate themselves into the community. These programs led to other organised programs like GNOME’s Women Summer Outreach Program. Of course, these initiatives weren’t the first to encourage good mentoring, but they have helped make the idea of mentors taking new developers under their wing much more widespread.

In addition to these scheduled and time-constrained programs, many projects have more informal “always-on” mentoring programs – Drupal Dojo and GNOME Love come to mind. Others save smaller easier tasks for newcomers to cut their teeth on, like the “easy hacks” in LibreOffice. Esther Schindler wrote a fantastic article a few years ago documenting best and worst practices in mentoring among different projects.

Most mentoring programs I have seen and participated in don’t have a very good success rate, though. In this article, I look at why that is the case, and what can be put in place to increase the success rate when recruiting new developers.

Why most mentoring fails

Graham Percival, a GNU/LilyPond developer, decided in 2008 to run an experiment. At one point, Graham decided that he would quit the project, but felt guilty about doing so in one go. So he started the “Great Documentation Project” to recruit a replacement documentation team to follow on after his departure. He then spent 12 months doing nothing but mentoring newcomers to get them involved in the project, and documented his results. Over the course of a year, he estimates that he spent around 800 hours mentoring newcomers to the project.

His conclusions? The net result for the project was somewhere between 600-900 hours of productivity, and at the end of the year, 0 new documentation team members. In other words, Graham would have been better off doing everything himself.

Graham found that “Only 1 in 4 developers was a net gain for the project” – that is, for every 4 apprentices that Graham spent time mentoring, only 1 hung around long enough for the project to recoup the time investment he put into mentoring. A further 1 in 4 were neither gain or loss – their contribution roughly equalled the mentor time that they took up. And the remainder were a net loss for the project, with much more time spent mentoring than the results could justify.

The GNOME Women’s Summer Outreach Program in 2006 had 6 participants. In 2009, the GNOME Journal ran a “Where are they now?” follow-up article. Of the 6 original participants, only one is still involved in free software, and none are involved in GNOME. Murray Stokely did a follow-up in 2008 to track the 57 alumni of Summer of Code who had worked on FreeBSD. Of these, 10 students went on to get full commit access, and a further 4 students were still contributing to FreeBSD or OpenBSD after the project. Obey Arthur Liu also did a review of Debian participants in 2008. Of 11 students from 2008 who had no previous Debian developer experience, he found that 4 remained active in the project one year later.

From my own experience as a replacement mentor and administrator in the Summer of Code for the GIMP in 2006, we had 6 projects, most of which were considered a success by the end of the summer, yet of the participating students, none have made any meaningful contribution to the GIMP since.

I feel safe in saying that the majority of mentoring projects fail – and Graham’s 1 in 4 sounds about right as an overall average success/failure rate. This begs the question: why?

Most mentored projects take too long

What might take a mentor a couple of hours working on his own could well take an apprentice several days or weeks. All of the experience that allows you to hit the ground running isn’t there. The most important part of the mentoring experience is getting the student to the point where he can start working on the problem. To help address this point, many projects now require Summer of Code applicants to compile the project and propose a trivial patch before they are accepted for the program, but understanding the architecture of a project and reading enough code to get a handle on coding conventions will take time. It will also take mentor time. It takes longer to teach a newcomer to your project than to do the work yourself, as anyone who has ever had a Summer intern will attest.

When you set a trainee task which you estimate to be about 4 hours work, that will end up costing a few weeks of volunteer effort for your apprentice, and 8 to 10 hours mentoring time for you during that time. Obviously, this is a big investment on both sides, and can lead to the apprentice giving up, or the mentor running out of patience. I remember in the first year of Summer of Code, projects were taking features off their wishlists that had not been touched for years, and expecting students new to the project to come in and work full time implementing them perfectly over the course of 12 weeks. The reality that first year was that most of the time was spent getting a working environment set up, and getting started on their task.

Mentoring demands a lot of mentors

As a free software developer, you might not have a lot of time to work on your project. Josh Berkus, quoted in Schindler’s article, says “being a good mentor requires a lot of time, frequently more time than it would take you to do the development yourself”. According to the Google Summer of Code FAQ, “5 hours a week is a reasonable estimate” for the amount of time you would need to dedicate to mentoring. Federico Mena Quintero suggests that you will need to set aside “between 30 and 60 minutes a day“.

When you only have 10 hours a week to contribute to a project, giving up half of it to help someone else is a lot. It is easy to see how working on code can get a higher priority than checking in with your apprentice to make sure everything is on track.

Communication issues

More mentoring projects fail for lack of communication than for any other reason.

Apprentices may expect their mentors to check in on them, while mentors expect apprentices to ask questions if they have any. Perhaps newcomers to the project are not used to working on mailing lists, or are afraid of asking stupid questions, preferring to read lots of documentation or search Google for answers. In the absence of clear guidelines on when and how parties will talk to each other, communication will tend towards “not enough” rather than “too much”.

No follow through

Many mentoring programs stop when your first task is complete. The relationship between the mentor and the apprentice lasts until the end of the task, and then either the apprentice goes off and starts a new task, with a new mentor, or that is the end of their relationship with the project. I would be really interested to hear how many Summer of Code mentors maintained a relationship with their students after the end of the Summer, and helped them out with further projects. I suspect that many mentors invest a lot of time during the program, and then spend most of their time catching up with what they wanted to do.

Project culture

In her OSCON keynote in 2009, Skud talked about the creation of a welcoming and diverse community as a prerequisite for recruiting new developers. Sometimes, your project culture just doesn’t match newcomers to the project. If this happens regularly, then perhaps the project’s leaders need to work on changing the culture, but this is easier said than done. As Chris di Bona says in this video, “the brutality of open source is such that people will learn to work with others, or they will fail”. While many think that this kind of trial-by-fire is fine, the will not be the environment for everyone. It is really up to each project and its leaders to decide how “brutal” or forgiving they want to be. There is a trade-off: investing time in apprentices who will contribute little is a waste of time, but being too dismissive of a potential new developer could cost your project in the long run.

Mentoring best practices

Is all the effort worth it? If mentoring programs are so much hassle, why go to the bother?

Mentoring programs are needed to ensure that your project is long-term sustainable. As Graham says in his presentation: “Core developers do most of the work. Losing core developers is bad. Projects will lose core developers.” Do you need any other reason to start actively recruiting new blood?

There are a few simple things that you can put in place to give your mentoring program a better chance of success.

Small tasks

Mentored tasks should be small, bite-sized, and allow the apprentice to succeed or fail fast. This has a number of advantages: The apprentice who won’t stick around, or who will accomplish nothing, has not wasted a lot of your mentor’s time. The apprentice who will stay around gets a quick win, gets his name in the ChangeLog, and gains assurance in his ability to contribute. And the quick feedback loop is incredibly rewarding for the mentor, who sees his apprentice attack new tasks and increase his productivity in short order. Graham implies that a 10 minute task is the right size, with the expectation that the apprentice might take 1 hour to accomplish the task.

A ten minute task might even take longer to identify and list than it would to do. You can consider this cost the boot-strapping cost of the mentoring program. Some tasks that are well suited to this might include:

• Write user documentation for 1 feature
• Get the source code, compile it, remove a compiler warning, and submit a patch
• Critique 1 unreviewed patch in Bugzilla
• Fix a trivial bug (a one line/local change)

Of course, the types of tasks on your list will change from one project to the next.

Mentoring is management

Just as not everyone is suited to being a manager, not everyone is suited to being a mentor. The skills needed to be a good mentor are also those of a good manager – keeping track of someone else’s activity while making them feel responsible for what they’re working on, communicating well and frequently, and reading between the lines to diagnose problems before it’s too late to do anything about them.

When you think of it in this way, there is an opportunity for developers who would like to gain management experience to do so as a mentor in a free software project. Continually recruiting mentors is just as important as recruiting developers. Since mentoring takes a lot of time, it’s important that mentors get time off and new mentors are coming in in their place.

Pair apprentices with mentors, not tasks

An apprentice should have the same mentor from the day he enters the mentoring program until he no longer needs or wants the help. The relationship will ideally continue until the apprentice has himself become a mentor. Free software communities are built on relationships, and the key point to a mentoring program is to help the creation of a new relationship. Mentoring relationships can be limited in time also, 6 months or a year seem like good time limits. The time needed to mentor will, hopefully, go down over this period.

Regular meeting times

Mentors and apprentices should ensure that there is a time on their calendar for a “one on one” at regular times. How regularly will depend on the tasks, and the amount of time you can spend on it. Weekly, fortnightly or monthly are all reasonable in different situations. This meeting should be independent of any other communication you have with the person – it is too easy for the general business of a project to swallow up a newbie and prevent their voice from being heard. Rands said it well when he said “this chatter will bury the individual voice unless someone pays attention.”

Convert apprentices into mentors

Never do you understand the pain of the initial learning curve better than when you have just gone through it. The people best suited to helping out newcomers to the project are those who have just come through the mentoring program themselves.

This is a phenomenon that I have seen in the Summer of Code. Those students who succeed and stay with the project are often eager to become mentors the following year. And they will, in general, be among the best mentors in the project.

Keep track

For all involved, it’s useful to have some idea of the issues newcomers have – ensure that documenting solutions is part of what you ask. It’s also useful to know how successful your mentoring program is. Can you do better than the 1 in 4 success rate of LilyPond? Keeping track of successes and failures encourages new mentors, and gives you data to address any problems you run into.

Manage the mentors

All of this work has overhead. In a small project with 1 or 2 core developers, it’s easy enough to have each core developer take an apprentice under their wing, and co-ordinate on the mailing list. In bigger projects, keeping track of who is a mentor, and who is mentoring who, and inviting new mentors, and ensuring that no-one falls through the cracks when a mentor gets too busy, is a job of itself. If your mentoring program goes beyond more than ~5 mentors or so, you might want to consider nominating someone to lead the program (or see who steps up to do the job). This is the idea behind the Summer of Code administrator, and it’s a good one.

Go forth and multiply

Developer attrition is a problem in open source, and recruitment and training of new developers is the only solution. Any project which is not bringing new developers up to positions where they can take over maintainership is doomed to failure. A good mentoring program, however, with a retention rate around 25%, organised continuously, should ensure that your project continues to grow and attract new developers.

Replenishing your stock of mentor tasks and recruiting new mentors will take effort, and continual maintenance of someone putting in a few hours a week. If you execute well, then you will have helped contribute to the long term diversity and health of your project.

On gnome.org, Rodney Dawes argued that developers tend to stay away from mailing lists because the more public lists are very noisy:

For me, mailing lists are a huge risk vs. low return problem. They can become a time sink easily, and it’s quite often that pointless arguments get started on them, as offshoots of the original intent of the thread. Web Forums also have this problem. And, to really get much of anything out of a list, you must subscribe to it, as not everyone who replies, is going to put you specifically in the recipients headers. That means, you’re now suddenly going to get a lot more mail than you normally would for any highly active project. And for anyone trying to get involved in an open source community, 99% of the mail on that list is probably going to be totally irrelevant to them. It will just make tracking the conversation they are trying to have, much harder.

I agree with Rodney that dealing with a new level of volume of email is one of the trickiest things for new contributors. I still remember when I signed up to lkml for an afternoon in college, only to find 200 new emails 3 hours later. I panicked, unsubscribed, and gave up that day on being a Linux kernel hacker.

Since then, however, I have learned some email habits which are shared by other free software hackers I know. Everyone I know has their own tricks for working with medium or high volume mailing lists, and some combination of them may make things livable for you, allowing you to hear the signal without being drowned out by the noise. LifeHacker is a good source of tips.

Rob Staudinger says something similar, pointing the finger at bikeshed discussions as a big problem with many community lists:

Will the zealots go and suggest postgresql’s process model was poor, or samba’s memory allocator sucks? Unlikely, but they will tell you your GUI was bad or that you’re using a package format they don’t like, just because it’s so easy to engage on that superficial level.

Over at LWN, meanwhile, Ciaran O’Riordan makes a good point. Many developers working on free software want to separate their work and personal lives.

When I leave the office at 6pm, my work should have no more relevance until the following morning. Same when I quit a company. I might choose to tell people where I work/worked, but it should be a choice, and I should be able to choose how much I tell people about my work. Having mailing list posts and maybe even cvs commits might be too detailed. Maybe waaay too detailed.

Finally, here at neary-consulting.com, MJ Ray suggested that asking individuals to respond to a request can backfire:

Publicly referring to individuals on a mailing list is a double-edged sword. It might bolster the confidence of the named individual, but it also reduces the confidence of other people who might have answered the question. In general, I feel it’s best not to personalise comments on-list. Some e-democracy groups require all messages to be addressed to a (fictional or powerless) chair or editor, similar to the letters pages of The Times.

While I agree with MJ in situations where the answer is accessible to the wider community, but often only developers working for you, the manager, are in a position to reply – at that point, you have a choice: get the information off your developer and answer yourself, or ask him to answer the question. and I’ve found that asking on the list has the positive side-effects I mentioned.

In my experience, common reasons for this timidity are a lack of confidence in written English skills, or technical skills, nervousness related to public peer review, and seeing community interaction as “communication” or “marketing” (which are not part of their job), rather than just “getting stuff done” (which, of course, is part of their job).

Lack of confidence

Shy (Credits: maong@flickr, CC BY-SA)

For non-English speakers, this phenomenon is more prevalent, but in general, professional software developers have not developed writing skills during college, and often have not been required to learn them as part of their job. In addition, while most engineers appreciate good logic, they are never taught how to structure an argument. The end result is that when asked to write about their work, or to ask a question on a public forum, they hesitate.

One engineer I worked with would (eventually) write an email about a topic if I specifically asked him to, but would not react to responses or questions from community members. When I approached him, he would rationalise, say he hadn’t had time, or didn’t see the need to reply, or refer the questions to someone else. But as part of a pattern, I concluded that he was nervous about sending an email to a publicly archived mailing list. He was nervous about making a mistake, writing something he shouldn’t, overstepping some unwritten corporate policy, or felt uncomfortable expressing himself in writing. It doesn’t help that “bad netiquette” is often dealt with rather harshly, reinforcing the lack of confidence.

The only way to cure the problem is with baby steps.

A great first step is to get your engineers answering questions. One way I have found to help get this to happen is to publicly refer a question to an engineer on the mailing list, while affirming his expertise in the area. “Tommy knows the RLE algorithms inside out – Tommy, would you mind answering Franz’s question, please?”

This serves a triple purpose: an individual can easily ignore a question if it is addressed to a group, but it is a rare person who doesn’t answer when you ask them a question directly – it’s human nature; affirming your engineer’s skills makes him feel good about himself, and gives him more confidence; by identifying an individual engineer with a specific skillset inside your team, you are giving people outside your company a glimpse inside the walls – you are making your team human, a collection of individuals with different strengths and weaknesses, rather than an amorphous group – “the Acme Co developers”.

Another good step is to give a series of creative writing workshops to your developers. It is not enough to stand over people with a whip and ask them all to have a blog. First you need to get them in the habit of writing regularly. Since terse language, often in bullet form, is the norm for mailing lists, the goal is not to turn them all into novelists, writing perfectly crafted works of art. The goal is to get your team in the habit of writing.

Finally, you should train your engineers in basic netiquette and writing good email – there are lots of good resources out there, including Producing Open Source Software by Karl Fogel, The Open Source Way, the Linux Kernel Contributors Guide from the Linux Foundation, and ESR’s “How to ask questions the smart way“. Michael Meeks suggested to me that developers should treat writing email in a similar way to patches – when you generate a patch, typically the last thing you do before you send it is you check over it, to make sure nothing silly is included. Michael suggests that the same habit applied to email will identify any places where phrasing is awkward and ambiguous, resulting in better email.

Nervousness related to peer review

Tough crowd (Credits: seedingchaos@flickr, CC by-sa)

It is one thing to have engineers answer questions when they have the knowledge to do so. It is another thing to have them submit their plans and patches to a community forum and have them exposed under the harsh light of peer review.

On more than one occasion, I have heard a hiring manager say that he didn’t have time to have a developer go through peer review of specs or patches – after all, he was hired because he was competent to do the job, and what are we paying him for if he’s going to be second-guessed by “the community”? After a first job or internship, peer review is more an exception than the rule for professional software developers (regrettably, I might add).

In community projects, peer review is expected – in fact, it is a best practice, one of the things that separates successful community projects from the crowd. Developers expect to hear about features before they are developed, and have an opportunity to suggest better ways the feature can be implemented. They expect new contributors to submit patches that they can review – it is the way a new contributor builds trust before getting the keys to the house. It is such a recommended practice that the only treatment I can suggest is that you should help your developers to get over their nervousness & embrace peer review, by making it the norm in your team.

The best way I know to get people used to peer review is to have a company policy against the “day one commit bit” – the practice of getting commit access to the project repository on the day you start in the company. For corporate-sponsored projects, new developers should go through the same review process for their work that contributors outside your company have to go through.

For corporate contributions to community projects, that means discouraging internal branches and a “gatekeeper” project structure, where one or two developers commit the work of others in the team. Developers should submit their work upstream at the same time it is being submitted internally. For those changes which only apply to your internal branch, you should create a probation period when patches are submitted via an internal mailing list, and subjected to the same peer review you would expect outside the company. This is a hard change for many organisations, but one which is necessary to transition your team
to successful community contributors.

Having new developers go through this review period is important for a number of reasons – the most important is that you are demonstrating that employees have the same burden to prove themselves in the project as non-employee contributors, and it ensures that new employees have a period where they familiarise themselves with project coding & communication conventions & norms, and when they also introduce themselves to the community at large.

Communication & marketing are not my job

Hiding (Credits: stuartpilbrow@flickr, CC BY-SA)

The developer I mentioned before shied away from the mailing list because he set a very high bar for himself on any public communications. In his mind, posting project plans to the mailing list was an announcement, and needed substantial preparation. In reality, most communities have a very high tolerance for things like spelling errors, incomplete plans (as long as they are marked as such) and other things which would be unacceptable in a press release or an official announcement. This is very definitely a case where Voltaire was correct: the perfect is the enemy of the good.

The developer was framing “sending an email to a mailing list” as “making an announcement”, something which fit in the “marketing” box in his head. One other developer I worked with once told me that he didn’t have time to send email to the mailing list because he had real work to do – “dealing with the community” was my job as community manager, as far as he was concerned.

Part of this issue comes from a general discomfort with written communications. But another part of it comes from the culture which is pervasive in most companies – only a small number of people get to talk for the company, and everyone else should just keep quiet. The authors of “The Cluetrain Manifesto” claimed that in the modern connected world, there is no such thing as a marketing department, that every interaction between an employee of your company and someone outside your company is an opportunity to win or lose reputation.

In free software development teams, this is even more true. There is no marketing department for project communications, and nor should there be. To be productive you need to talk to your peers, so the institutional barrier to external communications *must* disappear for people dealing in community projects.

It is all fine & well to say that this should be the case, but even if the organisation is clear about giving engineers a free hand to talk about their work, the engineers may impose limits on themselves, requiring proposals to be perfect before sending them on (which, of course, they never are). In fact, waiting until a proposal is finished is usually a lost opportunity, since by the time it is finished, there may be so much emotional investment in the proposal that making changes to it can be difficult. It is better to release a rough early draft, giving the author an opportunity to integrate early feedback.

The best way to prove to your team the benefits of “release early, release often” is to do so by example. As a team lead or manager, you can lead by publishing team plans publicly and early, and pointing out the benefits which result to your team. Breaking down this barrier will take time, but by making it clear that perfection is not expected, and by rewarding early release of information and encouraging feedback, your team will soon learn that participants outside your company are peers, not an audience.

Another useful technique is to ask your engineers to break tasks into smaller parts, so that even if they do hold off until the first part is up to a high standard, information is still getting out there more quickly, allowing feedback to inform later stages of the process.

However, another phenomenon is fighting against this desire for openness.

It is not uncommon for companies to want to keep their plans secret, so as to have a Big Reveal announcement effect at a major trade show. This can lead companies to ask their engineers to work internally on significant features for fear that the big surprise will be ruined otherwise. The alternative seems to be to announce a project when you start, rather than when you have something to show – but this can result in a long wait before products get to market, and impatience and bad press from the mainstream press.

I would argue that having engineers talk about design decisions & implementation details of significant features in a mailing list will not result in significant attention outside of your community – and when the press release and announcement comes, the community who knew it was coming will feel better about having been in on the secret from the beginning, rather than feeling worse because they have to deal with a big code drop which no one person can review.

Bashing bashfulness

The key lesson here is that you want your developers to feel a connection to people working outside the company. That requires people outside your company to feel a connection with them, too. By drawing the curtain on your team, its members and their skills & priorities, you are creating the circumstances for people to come to appreciate each other as peers, and to feel comfortable discussing features & patches on their merits. When you get to that point, you have already won – it’s impossible to be shy when you’re among friends.

Hi Dawn,

I plan to put this live tomorrow, and I'd really appreciate getting any
thoughts & feedback you might have. It's still missing a re-read or two
& some links and maybe a photo or two but I think the content isn't bad.

Thanks!
Dave.

One of the most common issues I have seen with experienced professional
software developers who start to work on community software is a
reluctance to engage with public communication fora like mailing lists.
Understanding the reasons why, and helping your developers overcome
their timidity, is key to creating a successful and fruitful
relationship with the community you are working with.

In my experience, common reasons for this timidity are a lack of
confidence in written English skills, or technical skills, nervousness
related to public peer review, and seeing community interaction as
"communication" or "marketing" (which are not part of their job), rather
than just "getting stuff done" (which, of course, is part of their job).
Lack of confidence

For non-English speakers, this phenomenon is more prevalent, but in
general, professional software developers have not developed writing
skills during college, and often have not been required to learn them as
part of their job. In addition, the art of rhetoric, structuring an
argument or thought to make it convincing, is not taught to engineers.
The end result is that when asked to write about their work, or to ask a
question on a public forum, they hesitate.

One engineer I worked with would (eventually) write an email about a
topic if I specifically asked him to, but would not react to responses
or questions from community members. When I approached him, he would
rationalise, say he hadn't had time, or didn't see the need to reply, or
refer the questions to someone else. But as part of a pattern, I
concluded that he was nervous about sending an email to a publicly
archived mailing list. He was nervous about making a mistake, writing
something he shouldn't, overstepping some unwritten corporate policy, or
felt uncomfortable expressing himself in writing. It doesn't help that
"bad nettiquette" is often dealt with rather harshly, reinforcing the
lack of confidence.

The only way to cure the problem is with baby steps.

A great first step is to get your engineers answering questions. One way
I have found to help get this to happen is to publicly refer a question
to an engineer on the mailing list, while affirming his expertise in the
area. "Tommy knows the RLE algorithms inside out - Tommy, would you mind
answering Franz's question, please?"

This serves a triple purpose: an individual can easily ignore a question
if it is addressed to a group, but it is a rare person who doesn't
answer when you ask them a question directly; affirming your engineer's
skills makes him feel good about himself, and gives him more confidence;
by identifying an individual engineer with a specific skillset inside
your team, you are giving people outside your company a glimpse inside
the walls - you are making your team human, a collection of individuals
with different strengths and weaknesses, rather than an amorphous group
- "the Acme Co developers".

Another good step is to give a series of creative writing workshops to
your developers. It is not enough to stand over people with a whip and
ask them all to have a blog. First you need to get them in the habit of
writing regularly.

Finally,you should train your engineers in basic netiquette - there are
lots of good resources out there, including Producing Open Source
Software by Karl Fogel, The Open Source Way, the Linux Kernel
Contributors Guide from the Linux Foundation, and ESR's "Asking Smart
Questions".
Nervousness related to peer review

It is one thing to have engineers answer questions when they have the
knowledge to do so. It is another thing to have them submit their plans
and patches to a community forum and have them exposed under the harsh
light of peer review.

On more than one occasion, I have heard a hiring manager say that he
didn't have time to have a developer go through peer review of specs or
patches - after all, he was hired because he was competent to do the
job, and what are we paying him for if he's going to be second-guessed
by "the community"? After a first job or internship, peer review is more
an exception than the rule for professional software developers
(regrettably, I might add).

In community projects, peer review is expected - in fact, it is a best
practice, one of the things that separates successful community projects
from the crowd. Developers expect to hear about features before they are
developed, and have an opportunity to suggest better ways the feature
can be implemented. They expect new contributors to submit patches that
they can review - it is the way a new contributor builds trust before
getting the keys to the house. It is such a recommended practice that
the only treatment I can suggest is that you should help your developers
to get over their nervousness & embrace peer review, by making it the
norm in your team.

The best way I know to get people used to peer review is to have a
company policy against the "day one commit bit" - the practice of
getting commit access to the project repository on the day you start in
the company. For corporate-sponsored projects, new developers should go
through the same review process for their work that contributors outside
your company have to go through.

For corporate contributions to community projects, that means
discouraging internal branches and a "gatekeeper" project structure,
where one or two developers commit the work of others in the team.
Developers should submit their work upstream at the same time it is
being submitted internally. For those changes which only apply to your
internal branch, you should create a probation period when patches are
submitted via an internal mailing list, and subjected to the same peer
review you would expect outside the company. This is a hard change for
many organisations, but one which is necessary to transition your team
to successful community contributors.

Having new developers go through this review period is important for a
number of reasons - the most important is that you are demonstrating
that employees have the same burden to prove themselves in the project
as non-employee contributors, and it ensures that new employees have a
period where they familiarise themselves with project coding &
communication conventions & norms, and when they also introduce
themselves to the community at large.
Communication & marketing is not my job

The developer I mentioned before shied away from the mailing list
because he set a very high bar for himself on any public communications.
In his mind, posting project plans to the mailing list was an
announcement, and needed substantial preparation. In reality, community
fora have a very high tolerance for things like spelling errors,
incomplete plans (as long as they are marked as such) and other things
which would be unacceptable in a press release or an official announcement.

The developer was framing "sending an email to a mailing list" as
"making an announcement", something which fit in the "marketing" box in
his head. One other developer I worked with once told me that he didn't
have time to send email to the mailing list because he had real work to
do - "dealing with the community" was my job as far as he was concerned.

Part of this issue comes from a general discomfort with written
communications. But another part of it comes from the culture which is
pervasive in most companies - only a small number of people get to talk
for the company, and everyone else should just keep quiet. The authors
of "The Cluetrain Manifesto" claimed that in the modern connected world,
there is no such thing as a marketing department, that every interaction
between an employee of your company and someone outside your company is
an opportunity to win or lose reputation.

In free software development teams, this is even more true. There is no
marketing department for project communications, and nor should there
be. To be productive you need to talk to your peers, so the
institutional barrier to external communications *must* disappear for
people dealing in community projects.

It is all fine & well to say that this should be the case, but even if
the organisation is clear about giving engineers a free hand to talk
about their work, the engineers may impose limits on themselves,
requiring proposals to be perfect before sending them on (which, of
course, they never are). In fact, waiting until a proposal is finished
is usually a lost opportunity, since by the time it is finished, there
may be so much emotional investment in the proposal that making changes
to it can be difficult. It is better to release a rough early draft,
giving the author an opportunity to integrate early feedback.

The best way to prove to your team the benefits of "release early,
release often" is to do so by example. As a team lead or manager, you
can lead by publishing team plans publicly and early, and pointing out
the benefits which result to your team. Breaking down this barrier will
take time, but by making it clear that perfection is not expected, and
by rewarding early release of information and encouraging feedback, your
team will soon learn that participants outside your company are peers,
not an audience.

It is not uncommon for companies to want to keep their plans secret, so
as to have a Big Show announcement effect. This can lead companies to
ask their engineers to work internally on significant features for fear
that the big surprise will be ruined otherwise. I would argue that
having engineers talk about design decisions & implementation details of
significant features in a mailing list will not result in significant
attention outside of your community - and when the press release and
announcement comes, the community who knew it was coming will feel
better about having been in on the secret from the beginning, rather
than feeling worse because they have to deal with a big code drop which
no one person can review.
Bashing bashfulness

The key lesson here is that you want your developers to feel a
connection to people working outside the company. That requires people
outside your company to feel a connection with them, too. By drawing the
curtain on your team, its members and their skills & priorities, you are
creating the circumstances for people to come to appreciate each other
as peers, and to feel comfortable discussing features & patches on their
merits. When you get to that point, you have already won - it's
impossible to be shy when you're among friends.
--
maemo.org docsmaster
Email: dneary@maemo.org
Jabber: bolsh@jabber.org

The end of October saw the release of MeeGo 1.1, the second major milestone release of the platform since it burst onto the scenes in February 2010. The MeeGo project was born under the auspices of the Linux Foundation from a merging of Nokia’s Maemo platform, targeting smart phones, and Intel’s moblin platforms, aimed at netbooks.

[...]

This contrasts sharply with Android which is primarily developed behind closed doors by Google, and iOS which is a completely proprietary platform. If there is a key differentiator for MeeGo in the hand-held market, this is it. It remains to be seen whether the open development model will be a selling point which will tip the balance when manufacturers are choosing a platform for a device.

[...]

It does not feel fair at this point to compare MeeGo, a project which came into being 8 months ago, with iOS or Android, but this is the yardstick which will be used when the first MeeGo smartphone comes on the market. The project has come a long way since its inception, in particular in working towards an open and transparent development model. There is still some way to go but improvements have been happening daily.

Read more

Why the change of heart? My intention was never to make a fortune with the report, my main priority was covering my costs and time spent. And after 24 hours, I’ve achieved that. I have had several press requests for the full report, and requests from clients to be allowed to use the report both with press and with their clients.

This solution is the best for all involved, I think – I have covered my costs, the community (and everyone else) gets their hands on the report with analysis as soon as possible, and my clients are happy to have the report available under a license which allows them to use it freely.

You can download the full report now for free.

By looking at the modules in the GNOME 2.30 release, made last March, we aim to answer many of those questions, and give deeper insight into the motivations of participants in the project.

GNOME activity over time, horizontal bars are release dates

Here are our key findings:

• GNOME has a rhythm – there is a measurable increase in activity before release time, and after the annual GNOME conference GUADEC
• While over 70% of GNOME developers identify themselves as volunteers, over 70% of the commits to the GNOME releases are made by paid contributors
  

  GNOME committer self-identification - volunteer/professional

• Red Hat are the biggest contributor to the GNOME project and its core dependencies. Red Hat employees have made almost 17% of all commits we measured, and 11 of the top 20 GNOME committers of all time are current or past Red Hat employees. Novell and Collabora are also on the podium.
• A number of top company contributors are consultancy/services companies specialising in the GNOME platform – Collabora, CodeThink, Openismus, Lanedo and Fluendo are in the top 20 companies. As many of these companies grew initially through work on Maemo, this is a sign of the success of Nokia’s strategy around the GNOME stack.

One of the interesting things that we have done for the census is to look at who is maintaining modules by looking at commits over the past two years, and use this data to identify areas of the platform which see lots of collaboration, areas where the maintenance burden is left to volunteers, and areas where individual companies assume most of the maintenance burden.

There are a number of modules in the platform which see a considerable amount of co-opetition, including Evolution, Evolution Data Server, DBus and GStreamer. Most modules in the platform, however, are either maintained to a large extent by volunteer developers, or see the vast majority of their contributions from one company.

I see this information being useful for companies interested in using the GNOME platform for their products, companies seeking custom application development, potential large-scale customers of desktop Linux or customers buying high-level support who want to know who employs more module maintainers or committers to the project.

The GNOME maintenance map, with modules coloured according to the company maintaining them

Update: The release has now been published under a Creative Commons licence on October 1st 2010.

GNOME activity over time, horizontal bars are release dates

But that ship has long since sailed. No company has every described themselves as “an Open Core company” to anyone except VCs, as shorthand for their business model. In the software business, Open Core has no-one defending it, and it has no brand value. In fact, in free software circles, Open Core has been a pejorative phrase almost since it was coined – fauxpen source, popularised by Tarus Balog, cites Open Core as a synonym, and pretty much every mention of it which I have found has not been by a vendor referring to themselves, but by an analyst or commentator referring to a class of business models.

In one sense, that’s what is amusing about this – the valuable brand, Open Source, is the one everyone uses,  and vendors who sell Open Core products find themselves in the unenviable position of having to defend calling themselves “an Open Source company” to what they doubtless consider zealots, but there is no other brand they can use which gives the same marketing benefits while providing sufficient differentiation with Open Source.

I don’t consider the OSI or other people decrying this use of the Open Source brand as zealots, they are simply protecting their brand against dilution.

Matt argues that Open Core products don’t deliberately cripple their free software offering (at least, the good ones don’t), they merely “sell value-added features that are designed to be of value to paying customers”. I would argue that is simply a restatement. Regardless, I accept that there are some features which a proportion of the user base will be prepared to pay for, and which most of the rest of the user base won’t care about. If you can ensure that the proportion of your user base which cares about the feature but is not prepared to pay for it is small, then you have the potential for license revenues without damaging your reputation too much.

As you increase the number of features which users need to pay for, however, you will find that the number of people who care about at least one commercial feature who are not prepared to pay for it will become significant. You only have to look at the SugarForge (a nice initiative in community enablement, by the way) to see that the lack of reporting and sales forecasting in SugarCRM’s community edition is something which a lot of people are interested in – the most active & popular community projects are related to those features. Does the decision to omit those features from the core product represent hobbling? Does the presence of modules that implement those features in the Forge show the victory of the Open Source way? Both, perhaps?

SugarCRM is far from the worst Open Core vendor in this space, so I don’t want it to come across as though I’m picking on them. OpenBravo also does a good job of enabling its community to fend for itself. Others don’t fare so well, and in private will admit that their community edition is “not suitable for production environments”. While this may just be a sales tactic to get people to pay for support, the message that this sends to potential customers is “Open Source might be OK to test out & play around with, but when you want to get serious you’d better pony up or you might have trouble”. Not exactly inspiring confidence in the Open Source way there…

Open core, Open core,  more Open core… the debate goes on and on, with Monty the latest to weigh in.

When you get down to it this is a fight over branding – which is why the issue is so important to the OSI folks (who are all about the brand). I don’t actually care that much how SugarCRM, Jahia, Alfresco et al make the software they sell to their customers. As a customer I’m asking a whole different set of questions to “is this product open source?” I want to know how good the service and support is, how good the product is, and above all, does it solve the problem I have at a price point I’m comfortable with. The license doesn’t enter into consideration.

So if that’s the case (and I believe it is), why the fighting? Because of the Open Source brand, and all the warm-and-fuzzies that procures. “Open solutions” are the flavour of the decade, and as a small ISV building a global brand, being known as Open Source is a positive marketing attribute. The only problem is that the warm-and-fuzzies implied by Open source – freedom to change supplier or improve the software, freedom to try the software before purchasing, the existence of a diverse community of people with knowledge, skills and willingness to help a user in difficulty – don’t exist in the Open Core world. The problem is that for the most part, the Open Core which you can obtain under the OSI-approved license is not that useful.

Yesterday on Twitter, I said “Open Core is annoying because the “open core” bit is pretty much useless. It doesn’t do exactly what it says on the tin.”

Now, I wasn’t expecting that to be particularly controversial, but I got some push-back on this. Dan Fabulich replied “Ridiculous. Like the free version of MySQL is useless?” Which leads me to think of Inigo Montoya on the top of the Cliffs on MoherInsanity turning to Vizzini and saying “You keep using that word. I do not think it means what you think it means.”

With all this talk of Open Core, clearly some confusion has crept in. Perhaps it’s on my part. So allow me to elaborate what I understand by “Open Core”.

First, companies can’t be Open Core. Products are Open Core. So whereas Monty considers that from 2006 on, MySQL was not an “Open Source company”, I would contend that MySQL Server has always been, and continues to be, Free Software, and an Open Source product. That is, not Open Core.

Open Core for me means you provide a free software product, improve it, and don’t release the improvements under the free software license. In my mind, Mac OS X is not “Open Core” just because it’s based on the NetBSD kernel, it is proprietary software.

Perhaps it would be useful to give some examples of what is Open Core:

• Jahia is Open Core – significant features and stabilisation work are present in the Enterprise Edition are not available at all in the Community Edition
• SugarCRM is obviously Open Core. Key features related to reporting, workflow, administration and more are only present in the commercial editions
• JasperSoft BI Suite is Open Core. Lots of useful features are only available to people buying the product.

The key here is that support contracts and extra features are only available if you also pay licensing fees. To take the oft-cited example of InnoDB hot back-up tool for MySQL, you can purchase this and use it with the GPL licensed MySQL Server – supporting my claim that MySQL is not Open Core.

This is why I say that Open Core products “don’t do exactly what it says on the tin” – the features you see advertised on the project’s website are not available to you along with software freedom.

I have talked to companies who deliberately avoid adding “spit & polish” to the community edition to encourage people to trade up for things like better documentation, attractive templates and easy installation – and don’t provide an easy way for the community edition users to share their own work. Other products have an open source engine that doesn’t do much except sit there, and all useful functionality is available as paid modules. Yes, a persistent, skilled, patient developer can take the Open Source version of the product and make it do something useful. For the most part, however, if you want to actually use the software without becoming an expert in its internals, you’ll need some of the commercial upgrades.

There is another name for this which is even more pejorative, Crippleware. Deliberately hobbled software. And that’s what I think gets people riled up – if you’re releasing something as free software, then there should at least be the pretense that you are giving the community the opportunity to fend for itself – even if that is by providing an “unofficial” git tree where the community can code up GPL features competing with your commercial offering, or a nice forum for people to share templates, themes and extensions and fend for themselves. But what gets people riled is hearing a company call themselves “an Open Source company” when most of the users of their “open source” product do not have software freedom. It’s disingenuous, and it is indeed brand dilution.

That said, let me repeat – I have no problem with companies doing this. I have no problem with them advertising their GPL-licensed stuff as Open Source. I would just like to see more of these companies providing a little bit of independence and autonomy to their user community. But then, that’s potentially not in their long-term interest – even if it is difficult to imagine a situation where the community-maintained version outstrips the “Enterprise” edition in features and stability.

Who knew that educating people in simple sabotage (defined as sabotage not requiring in-depth training or materials) could have so much in common with communicating free software values? I read the OSS Simple Sabotage Field Manual (pdf) which has been doing the rounds of management and security blogs recently, and one article on “motivating saboteurs” caught my eye enough to share:

Personal Motives

• The ordinary citizen very probably has no immediate personal motive for committing simple sabotage. Instead, he must be made to anticipate indirect personal gain, such as might come with enemy evacuation or destruction of the ruling gov­ernment group. Gains should be stated as specifically as possible for the area addressed: simple sabotage will hasten the day when Commissioner X and his deputies Y and Z will be thrown out, when particu­larly obnoxious decrees and restrictions will be abolished, when food will arrive, and so on. Abstract verbalizations about personal liberty, freedom of the press, and so on, will not be convincing in most parts of the world. In many areas they will not even be comprehensible.
• Since the effect of his own acts is limited, the saboteur may become discouraged unless he feels that he is a member of a large, though unseen, group of saboteurs operating against the enemy or the government of his own country and elsewhere. This can be conveyed indirectly: suggestions which he reads and hears can include observations that a particular technique has been successful in this or that district. Even if the technique is not applicable to his surroundings, another’s success will encourage him to attempt similar acts. It also can be conveyed directly: statements praising the effectiveness of simple sabotage can be contrived which will be pub­lished by white radio, freedom stations, and the sub­versive press. Estimates of the proportion of the population engaged in sabotage can be disseminated. Instances of successful sabotage already are being broadcast by white radio and freedom stations, and this should be continued and expanded where com­patible with security.
• More important than (a) or (b) would be to create a situation in which the citizen-saboteur acquires a sense of responsibility and begins to educate others in simple sabotage.

Now doesn’t that sound familiar? Trying to convince people that free software is good for them because of the freedom doesn’t work directly – you need to tie the values of that freedom to something which is useful to them on a personal level.

“You get security fixes better because people can read the code”, “You have a wide range of support options for Linux because it’s free software and anyone can understand it”, “Sun may have been bought by Oracle, but you can continue to use the same products because anyone can modify the code, so others have taken up the maintenance, support and development burden”, and so on.

Providing (custom tailored) concrete benefits, which comes from freedom is the way to motivate people to value that freedom.

In addition, the point on motivation struck a cord – you need to make people feel like they belong, that their work means something, that they’re not alone and their effort counts, or they will become discouraged. A major job in any project is to make everyone feel like they’re driving towards a goal they have personally bought into.

Finally, you will only have succeeded when you have sufficiently empowered a saboteur to the point where they become an advocate themselves, and start training others in the fine arts – and this is a major challenge for free software projects too, where we often see people with willingness to do stuff, and have some difficulty getting them to the point where they have assimilated the project culture and are recruiting and empowering new contributors.

For those who haven’t read it yet, the document is well worth a look, especially the section on “General Interference with Organisations and Production”, which reads like a litany of common anti-patterns present in most large organisations; and if you never knew how to start a fire in a warehouse using a slow fuse made out of rope and grease, here’s your chance to find out.