Everyone is using open source software now but most people have not figured out how to interact effectively with the open source software community. Mike McQuaid explains how to improve your open source software interactions whether as a user, contributor or maintainer. This talk will teach how to get the open source community to work with and for you to succeed in your goals.
[00:00:05] So I managed to do the novice thing of putting my speaker notes on the screen before starting, so you may have seen that one of the first things in my speaking notes is telling myself not to say "um" or "like." So if you decide to count how many times I say those things, then I guess email me afterwards and I'll make sure to file it appropriately into my trash.
[00:00:24] Right, so there was great talk from Anna before, and — just as little side note — 10 years ago, or maybe when I was in uni, I used to rant to my friends at the pub that government should be open source and stuff like that. So it's nice to see that that's actually becoming a reality now with GOV.UK. I'm coming at this topic of "effective open source interactions" with two different hats on. So one hat I wear is that I'm the lead maintainer of Homebrew. How many people in here have used Homebrew or know what it is or anything like that? Okay. A reasonable number of people. Anyone who doesn't, it's like a package manager for Macs, so it's ways of installing all your development dependencies and stuff like that. I've been working on that for about eight years now. And that's taught me most of what I know about open source at serious scale, and we'll talk more about that. So the other angle I have is that I work for GitHub. So who uses GitHub here? Reasonable number of people. So I've been working there for about three and half years and I'm focusing on open source there. So my goal there is, without directly changing the product all the time, making GitHub better for open source, better people wanting to contribute to open source and better for people wanting to maintain open source and stuff like that. So if you have any ideas about that feel free to grab me at the conference, feel free to speak to me on Twitter or email or whatever.
[00:01:54] So I'm going to walk through today four different stages of thinking about effective open source interactions. So the first thing is, I think it's helpful to think about the different groups of people and how they get involved in open source. So we're going to group users into different categories of open source participants. The next is the funnel. This used to be the open source contributor funnel and I'll introduce that later on and we'll talk about how users move between different areas when they're participating in open source. Then upsell, how — if they don't just move by themselves — we can sometimes give them a delicate shove between areas they may or may not want to work on yet. And then finally how you can help out regardless of whether you're just using open source on a day to day basis or whether you're maintaining a large open source project. What are the things you can do to make your life a little bit easier.
[00:02:53] So let's have a look at the groups of people. And I think the most helpful way of thinking about this is asking yourself "okay, so for any given individual, how are they interacting with an open source project?" So the first group I think about are users. I'm going to do a lot of hand raising I'm afraid, and some of it is going to be obviously patronising and everyone's going to stick their hand up, but please indulge me. So, who here has used open source software at some point in their life? If you didn't put your hand up you're probably wrong. So most people nowadays, and particularly most developers have. I assume most people here are engineers or are least interested in engineering. I mean, open source is more or less one. We're all using primarily open source tools most of the time. Even things like C# and the Microsoft ecosystem are increasingly becoming open source now. So we're all familiar with using open source software and we're all familiar with going on GitHub and seeing how these projects are run, and then having problems or maybe filing problems with those projects.
[00:04:02] So when I'm talking about users here I'm talking about a subset of the people who use open source software who are just users. So, thinking about the type of people who maybe use open source software, they read the docs, if they have a problem at most they might submit an issue and say "okay here's the problem we're having." Those issues can pretty variable in quality. I got an email the other day saying "Help I no use tool anymore, help uninstall. Thanks bye." So I wasn't quite sure what to do with that. And then you get some people who are technical users and make really really great issues but aren't interested in resolving them themselves or kind of working on effects. So I'm copying those as users.
[00:04:47] Now let's have a little think about what users want for software in general, but open source in particular. So they want their software to be high quality. Obviously who doesn't. They want to be able to just go along and use the software and have it work the way they expect it to work. And that sort of segways into the next point, which is that they want the next versions of the software. They want to be able to upgrade their software via a Ruby Jam MPM package, something they install with Homebrew, something they manually download off the Internet or whatever. And they have the expectation that a new version is not going to break all of their workflows. And that's why I say no v.2.0. So I'm a little bit contentious with this. With Homebrew maintainers I often argue — because people have all these ideas for like "oh if we break backwards compatibility, here are all these cool things we can do, and we can do it all the next time" or whatever. I've seen so many open source projects make that type of mistake, they say "okay, well we made all these mistakes in the past, we're going to throw away all our code and rewrite it from scratch. Everything's going to be great and perfect and wonderful." And what ends up happening is they obviously have hundreds of times more bugs because all the nice bugs that they fixed over the last years, they maybe remain fixed in the new one but then they have a bunch of new bugs which are all broken now. And also if you force all your users to relearn how to use your software then it's just as easy for them to move to your competitor. And in the open source world, card a competitor can be another library that's doing things better, or often in open source, someone will go and just fork your old version and continue to run what you don't want to run anymore. And you will frustratedly look as that boat sails off into the distance with all your users. And the final thing is people want to work quietly. What I mean by that — again, relating to the two previous points — people want to use open source software and not have to interact with your project. Ideally people want to be able to just have everything work exactly how they expect, they go and look in the docs when they have a problem, and it's documented just there exactly what they need to do, and then all is wonderful.
[00:06:56] So moving on to the next group of people. So this is when people step up and maybe want to get a little bit more involved. So I'm going to talk here now about contributors. So who has ever contributed a commit, pull request, anything to an open source project? Cool, probably about half the room I reckon. Maybe a little bit more. So these are the people for whom open source is something they tend to be a little bit more passionate about, or it's just a transactional thing where I have a bug, I want it to get fixed, it's not getting fixed, I can fix this. So what I'm talking about with contributors is not everyone who commits to a project, not everyone who works on a project, but the subset of those people who do not have access to write to the project directly but they come along and they submit a fix or whatever. So these are the people who are a step above users — where they're probably using the software already — but then they've encountered a problem and rather than filing an issue, or finding that there is an existing issue that was filed four and a half years ago, they take it upon themselves to go on a noble quest to go and fix the bug, submit a pull request and then hopefully everything gets nicely merged. So contributors generally want good pull request review. I'm generally going to use GitHub parlance and assume everything's on GitHub because a lot of it is. But I think most of these principles apply elsewhere as well. So when they submit their pull request, they want good review, and by good review I mean they want the stuff that Anna highlighted earlier as well. They want it not to be pedantry about tabs versus spaces or whatever. Like that stuff should all be automated away, partly because humans respond better to bots being pedantic than other humans being pedantic, and partly because it's nice if I can go and check that all the pedantry is sorted on my local machine before I have to kind of push it to the project. And they want that review to be kind of nice and friendly and constructive and helpful and appreciative and not mean and pedantic and yuck.
[00:08:57] The next thing they want is timely review. It's tough for opensource maintainers, but in general we have really good data now. Mozilla did a study recently where the single biggest thing that impacts whether someone is going to stick around an open source project is how quickly they get a response to the review. If you submit a request and you get someone who comments, gives you feedback, you can then address that feedback and then you commit back. Then they address it again and say this is good to go. They manage it and they deploy to production within a day. In that case you have had a very positive experience. If that takes a year, or a year and a half, then chances are next time you think there's an issue with this project you're not going to be wanting to go through the same amount of process because it's not been as viable for you. So I think that's really important for projects to consider and also individuals to think about how you can ensure that you can get that timely review by checking as many boxes as you can. And the final thing they want is earned praise, and by earned praise I mean they don't want to just get congratulated like a puppy for every single time they don't poop on the floor. But they want to make sure that you know when they've done a good job. They want to be told "You've done a good job, well done." And to have appreciation for the work that they've done.
[00:10:17] So the final group of people I'm going to talk about is maintainers. These are probably people who use the software. They're people who contribute to the software and probably still control the software. But these are the people who's main thing is having commit access to the software. So their job is to kind of wrangle community contributions, deal with user support requests and basically just make sure the project keeps running. So it tends to be quite a multi-hack job, particularly in mid-sized projects where they're providing user support. They're writing new features, fixing bugs, prioritizing bugs and then providing decent code review and deciding whether or not to merge people's contributions.
[00:10:59] What maintainers tend to want in projects is no bike shedding. Who's familiar with the term 'bike shedding'? Most people? But if you're not, it's the idea that if you want someone to provide architectural view on your multi-agent distributed system, then you may not find many people in your organisation to do so. If you want to find people who will discuss with you what color the new office bike shed should be, everyone all of a sudden has an opinion because the barrier to entry and that discussion is incredibly low, and everyone can just say, "Well I really think it should be red." "No I think it should be Blue." Whatever. And in open source these types of conversations are sadly common, where it stops being about the code. A relatively common one is a discussion in which we all agree that something is bad and something needs to be done, and there's just comment after comment of people saying something needs to be done. I agree this is bad. Something needs to be done. And the thing with that stuff is that that time that the maintainer and contributors are spending reading and responding to those comments is time they can't spend actually working on the project.
[00:12:13] And ultimately, the maintainers — although it's fun sometimes to argue with people on the internet — ultimately they primarily want to be making the software better. That requires them to have that time to be able to spend sitting down, writint a fix, writing a new feature to address the problem. The next thing I think is really helpful — but can sometimes be a little contentious — is some sort of private chat between themselves. So if I disagree really, really strongly with another maintainer on my project, it's more helpful for me to be able to work through that strong disagreement in a private setting and then post the conclusion of our conversation. And it's basically just because, when stuff is in the public, people can sometimes be more touchy, more sensitive, and it's harder to phrase things in a way that is sometimes very clear, but maybe not too strong, but sufficiently strong et cetera, et cetera. So I think those types of private chats are really good at resolving conflict and also at building a sense of community. It's nice to have for maintainers to have their own sort of private safe space where they can kind of chat about things and sort of build up friendships and rapport. Without that, all they can to be on a bug tracker.
[00:13:30] And the final thing is — they want more help. Almost no maintainers in open source are going to stick around forever. In Homebrew — I've been working on it for eight years — I wasn't the first maintainer but am by far the longest one who's stuck around. You need to always be thinking about how you're going to get more people to help you build this project and how you're going to get someone to replace you. So when you want to leave you can do that, but also if you struggle with bus factor — that being how many people in your project can get hit by a bus before your project can no longer function. So trying to ensure that the bus factor on your project is larger than one is a good thing to do.
[00:14:17] So, the next thing is thinking about the funnel. So obviously you have users, contributors, maintainers who kind of escalate the level involvement, and you probably have enough experience yourself to know that in most communities they decrease in their numbers. So this is what I'm thinking about with the funnel. So a few statements to start with. No one ever became a contributor without being user first. I'm sure in some cases this has happened, but generally people are contributing to open source software because they're scratching their own itch. Are people familiar with scratching your own itch? A few people... Anna noted that too. Scratching your own itch is the idea that, in open source, the best way to get started is not to go and find a project which has an open issue that you can go and write some code for, because frankly that sounds a bit like homework and I intentionally left university a long time ago for a reason. What's more interesting generally is thinking "Okay, I have a particular problem. I use some open source software. This problem is in this open source software. I'm going to go and fix my own problem." And that's referred to as scratching your own itch.
[00:15:27] So in general, that's where most people end up contributing from. They're a user, they see a problem, they want to fix the problem, they scratch their own itch and then... ta-daa they're a contributor. Similarly, no one ever became a maintainer without being a user. In general, people don't just appear in a project and get access to maintaining the project. People generally have to contribute a bit first, they have to use the project a bit first. In general, it's not very common for people to say, "Please make me a maintainer, please make me a maintainer." And that's generally a bad sign.
[00:16:01] Generally, I've found that the people who become very good maintainers are the people with whom you have to continually battle their imposter syndrome for the first few months and say, "No, you are good enough. Yes we want you to help. Please help us." And then they kind of join the project and help you out. And then finally, no one excels as a maintainer without remaining a user. And that relates to what I said before: there may be a time when eventually it's time, if you were an open source maintainer, to step away from a project and go and move on. Speaking of which, how many people here would say they maintain any sort of open source project in any sort of fashion? OK, probably a quarter of the room.
[00:17:33] Is anyone here a sales person or has anyone ever done any sales? Almost no one. Great. So no one is going to call me on probably being wrong. So this is my understanding what a sales funnel is. You start with leads, and then you have prospects, and then sales. So leads are like people who are notionally interested in buying your thing, prospects are the people who've actually sort of engaged with a sales person or whatever and started a conversation, and then sales being the people who have actually handed you some sort of money. And my idea is that actually open source is kind of vaguely similar. If you're looking as maintainers as the goal, you have users being the people who have shown interest in your project but not really shown any interest in maintaining your project or contributing or anything like that. You have contributors who are the people who've been involved maybe intermittently, maybe just as a one off, but they're the next step to becoming maintainers, who are the people involved on a day-to-day basis.
[00:18:32] So to read out some numbers — because the funnel is fairly sharp in Homebrew's case — we have, I reckon, about 1.2 million Homebrew users. Google Analytics says about 4 million but there's like bots and CI machines and stuff like that included in that. We have 6,000-ish Homebrew contributors, so 0.05% of our users. And today we have 16 Homebrew maintainers, which is 0.0025%. I don't think all projects are going to have numbers that are quite that sharp. Obviously it would be kind of depressing if every project required you to have, you know, about 100,000 users for one maintainer. But its worth stating because it shows that this type of funnel and trying to push people into it is partly a numbers game. The more users you have I think the more contributors you're going to get, and the more contributors you get the more maintainers are — at least potentially — out there.
[00:19:38] So let's talk about upselling people. So in the sales funnel you're trying to convince people to hand over money to you, whereas in the escalating contributor funnel you're trying to convince people to hand over time and care to helping you out on this project. I guess it's technically a down-sell, but I can't find an emoji that's pointing down, so there you go. So let's start with a statement for that as well: Most maintainers get talked into being maintainer, as most don't want to do it. It's not something they really aspire towards doing, but they are just people who become helpful on a project over a reasonable period of time and then as a result they get asked, "Would you consider helping out more formally with the project and like joining us and helping us try things and merge PRs as well as reviewing or committing or whatever?" So this is an example of upselling a kind of — apologies to this person in particular — not great bug report into maybe a better one. So we've said to them, "Okay, rather than your 140 character issue on Twitter can you maybe consider opening an issue on our [inaudible] repositery?" And then walk them through the issue template, some guides as to how best to do that. This is one of my canned replies that I use to try and nudge people towards understanding exactly what they're doing, what went wrong and why. This is also — as long as none of you promise to tell my co-workers — useful as a canned reply with co-workers who don't adequately explain their problems.
[00:21:20] The next step beyond that is trying it upsell when people are submitting good issue reports and they seem fairly savvy and engaged. This is another one my canned replies to try and upsell them to a request. So we have a nice little guide for how to open up a Homebrew pull request and that sort of walks people through step by step. There's diminishing returns with stuff like this and the level you can go to. We try and help people who maybe aren't that familiar with GitHub to know what commands they should run. A co-worker sort of elaborated on this as trying to help people on how to GitHub, how to development, how to computer. I think "how do I computer" is a little bit beyond my realms of being willing to help with, so in general we assume that people who want to contribute to Homebrew or are contributing have a development background and are able to at least sort of know their way around editing some code, and then we just want to make docs to help them as much as possible.
[00:22:23] We also have — you can probably see in the middle here — a single command. If you're doing the most common pull request we have, which is upgrading the version of a package that we have — then you can literally do that with a single command that will open the program for you. And that's sometimes fun for people as their first contribution, it being something that was almost entirely automated for you, and then people can move to more complex contributions from there. We also have a new maintainer checklist. Again, this is sort of documenting the way that people go from being contributors to being maintainers, and this kind of walks through what we look out for, and then the emails that we kind of tend to send people to ask them to be maintainers, and what that part of the process looks like.
[00:23:05] So we've seen these different users and how we can kind of move between them. So wherever you might find yourself, let's see how you can help with open source and have better interactions with open source projects. So in general as an open source user I think the important things are, firstly, to have reasonable expectations of that project. I guess Anna touched on some of this before. I think projects are getting better now at setting expectations of how much support you can expect to receive. It's very important to remember that in developer tools you might use very often, like Homebrew for example, no one is paid to work, even part-time. Some people are able to kind of carve some little bits of time out from their employer to work on it but there's no organisation or corporation backing Homebrew. Homebrew has four digits of money in the bank from a Kickstarter we did a few years ago and stuff like that. So we just don't have the resources to be able to help every single person to the level that we would like to. So you need to have expectations when you use tools like that, that if the developers can't reproduce your issue on their machine, chances are they're not going to be able to invest a huge amount of time into fixing it.
[00:24:17] The next thing is help how you can. That doesn't mean everyone needs to be a maintainer or a contributor or submit huge amounts of pull requests or whatever, but it means that if you're filing an issue or whatever, try and take the time to really think about how to explain your problem clearly. Make sure you are on the side of providing too much information. And I guess this is my main minor rant is that the little issue template that GitHub issues, please always fill that out. A lot of people think they know better than filling it out, but chances are the maintainers have asked for that information because they need that information.
[00:24:48] And finally, which relates to that, prioritize maintainer time. As I said before, Homebrew has maybe 1 million users, and even if it were a factor of 10 less than that — like say it were 100,000 users, 10,000 users or whatever — there are still way way way more users than there are maintainers. So maintainers need to be able to prioritize what they work on and when. And as a result, we need the users to help us in that prioritization.
[00:25:16] Next, as a contributor you should always try and follow the project guidelines. By that I mean try and read the docs, try and read the contributing file, or try and read the code of conduct and stuff like that, and run any tests you have locally just to make sure that you're not having to go through a dance where people are telling you to read the output of the tests and so on. It's more helpful for everyone if you can try and do as much as you can up front. Obviously sometimes you'll miss that and that's no big deal, but putting in that effort is really helpful.
[00:25:48] The next thing is deferring to maintainers. This doesn't mean that maintainers are always right, but it does mean that in the context of a project like Homebrew for example, if it's your first pull request and if you end up in an argument with a maintainer or somebody who's been working on it for five years, chances are they're kind of winning that argument by default because they have the choice whether to commit your stuff or not, and you don't have the choice whether to or not. So although you may not like what they say, generally following their advice sooner rather than later makes all their communication a little smoother.
[00:26:20] And finally, being willing to go outside your comfort zone. The example I tend to use for that is my first contribution to the Homebrew. Homebrew was all written in Ruby from day one, and I had never written a single line of Ruby before touching Homebrew. So I start making like a couple of character line changes and then a couple of line changes and then editing Homebrew's code itself. And now for the last 5 years I've been writing Ruby for a living. I learned Ruby through Homebrew and if people hadn't nudged me to start using Ruby, to learn, to continue to improve and start learning more about Ruby, to be able to make less silly bugs in Ruby, then I wouldn't have. I probably wouldn't have the job I have today.
[00:27:02] So as a maintainer, again, three things. So the first is self-care. If the sky is falling over some really big bug that has caused a lot of users a lot of issues and a lot of people are very angry, it's important to prioritize looking after yourself, because whether people articulate it or not, they would rather have you continue to work on the project than necessarily fix their problems right now. So sometimes that means just walking away from the computer and going and seeing a friend or having a drink or whatever. Whatever helps you deal with the stress of open source sometimes. The next thing is acting as a force multiplier. Again as a maintainer, if there's lots and lots of users and lots and lots of contributors, what you want to try and do is spend your time on the stuff that people can't come along and trivially do for you. You want to spend your time trying to enable other people to work on whatever your open source project is. So I try and spend a lot of my time now working on tooling and working on the things that other people aren't going to do rather than doing too much in the way of fixing bugs that other people can fix. And finally, focusing on 90 percent of users. I guess this as a general Products comment anyway, but it's kind of nice to focus on the majority of people working on and using your software, rather than the tiny use case of a maybe a very vocal minority. This will also help you prioritize your time and prioritize what issues you're going to work on as well.
[00:28:30] Final plug. This was built by some people at GitHub, but it's all open source. The Open Source Guides. This is a really good resource for getting people involved with open source, learning some basics about maintenance and stuff like that. It elaborates on almost everything I've touched on in this talk but to a much higher degree. So I would recommend going and checking this out.
[00:28:52] So we walked through the different groups of open source users. We've got users, contributors, maintainers. We talked about how the open source contributor funnel means that they can work from top to bottom and come out the bottom as lovely maintainers. We talked about how to gently nudge people between them with issue templates, with documentation, with canned replies and stuff like that. And then finally all the different ways in which you can help out as you interact with open source projects. So thank you very much for having me. I'm Mike McQuaid. You can email me or tweet me or whatever if you have any questions or thoughts. Thank you very much.
Mike McQuaid is a senior engineer at GitHub where he works from home in Edinburgh. At GitHub he works on improving the quality of internal and external software whilst attempting to automate himself out of the job.
Outside of work, he is the lead maintainer of the Homebrew package manager for macOS, author of Git in Practice (published with Manning) and has contributed to a wide array of other open source projects including KDE and the Linux kernel.
Previously in his career he’s worked as a full stack engineer and built CI and CD at AllTrails, been an international engineering consultant and lead and trainer at KDAB, setup CI and CD as first employee and engineer at Mendeley and created high-performance network analysis tools at BT.