In the past few years much of the code written in government has been either written in the open or later published under open source licences. This has been a huge transformation from the previous government culture. Anna is Open Source Lead and will talk about how this transformation has happened, what it means in practice and the benefits and challenges of coding in the open.
[00:00:05] So it's 2012, and the way you find out information about anything to do with government is looking at this site Directgov. And here's a tax calculator on Directgov. So this is a tax credits calculator. This is where you put in some information about your income and other behaviours and it will give you an estimate of how much tax credit you might be due and if you as a member of the public found an error with this tax calculator this is what you'd have to do. First you'd have to get in touch with the Directgov team. So let's say that was straightforward - you filled in a form on the website and you got talking to the right person who explained what the error was. Then the Directgov team would have two options. If there's a content error that would be no problem. So you know it's a typo or something like that. Directgov know CMS so they could make that change straightaway. But if it was a code error, if you'd notice some kind of issue with the calculation, that was a different matter. The Directgov team would have to talk to their supplier because Directgov is outsourced. And in order to investigate whether the error that you spotted required a code fix - so this is just for the investigation - the cost would be tens of thousands of pounds. And if I did realise that the error did require a code fix, then the estimate for fixing that would be between 50 and 100 thousand pounds. And Directgov was um, Big Bang releases once a year, so if you timed it badly the error you raised you could maybe even be waiting 18 months for that to get into production. But realistically if it was a small error the chances are that wouldn't happen because that's a lot of money to spend on fixing a small code change. Later in 2012 we launched the GOV.UK website and that replaced Directgov.
[00:01:42] Let's fast forward now to 2016 and this is another tax calculator this time on GOV.UK. And this is a calculator that tells you how much tax you might have to pay on any child benefit you receive. And a member of the public did find an error in this calculator. So the error was that in the situation where you were claiming tax benefit, you were earning over 50,000 pounds a year and you also contributed to contributing to a post-tax pension, he found that one of the figures used was the net instead of the gross. So he went to the code on GitHub and he made a fix to the error and he raised the poor request. So he raised the request last year, the 24th of January, a Sunday. On Monday the 25th of January the team came in, they saw his poor request, they discussed it, they agreed that he'd found an error in the code and that a fix was required. On Wednesday the 27th of January they merged his code and on Friday the 29th of January the code was deployed to production. So that's quite a big change from how the Directgov website was run. And the reason that he could do that was because GOV.UK is coded in the open.
[00:02:55] So I'm going to talk to you today about what coding in the open is, why we do coding there, what the benefits are, and a bit about what the benefits are to government industry.
[00:03:07] My name's Anna Shipman. I am a technical architect. I used to be a technical architect on the GOV.UK website, I was a technical architect on Gov Uk PaaS, which is a platform hosting digital services, and am now the open source lead.
[00:03:20] When we talk about open source and government, we tend to be talking about three different things. Firstly using open source software. So GOV.UK runs on nginx... We use MySQL, we write it in Ruby on Rails, so we definitely use open source software in government. The second thing we tend to be talking about is contributing to open source software, so that's where you find a bug in nginx and you submit that patch back to nginx. And we do that as well. And the third thing we tend to be talking about is open sourcing our own software, and that's what I'm talking about today. Over the past six years a huge amount of government code has been published in the open - either coded in the open or published openly after it's finished. And again that's not, as Rachel mentioned, that's not what you might think of as government. So I'll explain how we got here.
[00:04:12] In 2011, Baroness Martha Lane Fox was asked to write a report on Directgov, the first site we saw, and she actually went wider with her report and looked at Digital across the whole of government. She made several recommendations, one of which was the formation of the government digital service and that's what I work for. It's a unit of government within the Cabinet Office. This is in the center of government and our job is to transform government into a more modern digital organisation. First thing we did was launch the GOV.UK website that I've already mentioned, and we're now working on helping government be digital by default. So a couple of examples of transformations we've done: You can now register to vote online. Before we launched this service, the way you would register to vote was to fill in the paper form and send it back to your local council. You can now do that online. Another example is you can now view information about your driving licence online. So you can go and see if you have any penalty points, what vehicles are entitled to drive that kind of thing. Previously you'd have to do that by looking at the paper version of your form. So another thing we have is a digital service standard on all new digital services, and government have to meet this digital service standard. And that's one thing that GDS help support. And the digital service standard's a set of principles that digital services have to meet. And point eight is make all new source code open. So all new government digital services have to make their source code open.
[00:05:47] So I said that GOV.UK is coded in the open and I'll just explain what I mean about coding in the open rather than open source, because coding in the open is technically open source. Open source means that the software where the code is freely available and it's published under a license that allows free use, modification and sharing. So the GDS code is in our GitHub organization Alphagov. So the code is freely available and we publish it on the MIT license, so it is technically open source. But the reason we refer to it as coding in the open rather than open source is because even though the MIT license sort of limits warranty, there's still some things that you expect about open source that we don't really support or offer. So we don't expect that our code will necessarily work in your environment. If it does, that's great, but we're not doing anything to support that. We don't make any guarantees that we're going to continue to support it at all. So for example if we realize there's a better way to write calculators, we'll just retire the calculator code and write new code and we won't think about who might be using that code. And we haven't worked to build a community around the code. There are some communities around the code but we haven't worked on maintaining that. Community is really important for open source and Mike's going to talk about a bit more about that next. But what we mean by coding in the open is that all our code goes to the open straightaway. So you make a commit, you push it to get harder and that that code can immediately be seen. You raise a poor request, that can be seen publicly. Our code review takes place in public, our code is merged in public. And the reason we do that is because one of our design principles is make things open. It makes things better.
[00:07:39] So I'm going to tell you now about how coding in the open makes things better. I'm going to talk about how it makes the code better, how it makes the organization better and how it makes government better.
[00:07:52] So the headline about how it makes the code better is, basically, it encourages good practice, because when you know that somebody could be looking at your code, you raise your game. Maybe you try harder to write good commit messages because you know that somebody might be looking to see what the intent of that code was. You're positive and constructive in code reviews because you know that anyone could be looking at that conversation, you pay attention to security, you think about making sure that you don't commit your AWS credentials for example, you make sure you separate your coded configuration and you make sure you follow good architectural principles, because the point is that even if nobody is looking at your code, people could be. And so everyone works hard to make it a bit better.
[00:08:43] Second way it makes the code better is by making it easier to work with other people. So I'll give you an example that I really like, something that happened recently. So all government services are mandated to look like GOV.UK, even if they're produced by a different department. So this is a service from DVLA allowing you to take the license plate off your vehicle and put it on a new vehicle. But as you can see it looks like GOV.UK, even though it's produced by different departments, produced by DVLA. And the easiest way for other departments to make their code look like the GOV.UK front end code is to use a GOV.UK front end code. So the team that maintains a GOV.UK front end code is working on making it easier for other departments to use that code. And earlier this year they tried an experiment to see if they could transpiral some of the code into different languages, and they weren't quite sure how to do it. They were having some problems and so they discussed it in an open design community slack channel. And somebody from 18F saw that conversation. So 18F is an organization in the US which is similar to GDS, does similar things. Anyway so this guy saw the conversation, he thought that it looked like an interesting problem. Talked to the team a bit about what exactly they needed to do and then wrote some code that solve their problem. And so the team were then able to use his code to finish their experiment and see if that would work. And so the reason that was able to happen was because they coded in the open, because they were open about the problems they were facing, because he coded in the open. And so those people, even though he's in the US and they're in the UK, they were able to work together. They've written a blog post about it. I'll publish these slides afterwards and I'll include links to all the blog posts and things that I mentioned so you can check that out later.
[00:10:42] Closer to home there's another example. So the Ministry of Justice have written a tool to allow you to find out the postcode if you put in an address and vice versa. And HMRC have also written a tool to do pretty much the same thing. And because both of these teams have written that code in the open, they became aware that they were both solving the same problem. And so now those two people in those two departments are working together to make sure that they both do the same things and then they're going to turn one off and use the other one. And again that was possible because they coded in the open and they can work together on it because it's in the open. They don't have to go walk from one department or the other or travel from one department to the other.
[00:11:27] And another way that coding in the open helps make the code better is that other people can do work for you and help make your code better. So I showed you the example at the beginning of somebody fixing an error in the calculator's code. We've got quite a few more examples of that so you might have seen this site. This is the petition site and this is where you can create a petition, and if it gets over 100,000 signatures then it will be debated in Parliament. And last summer there were some quite high profile petitions that got a lot of attention. And so it used to be the case that when you went to look at a petition the number at the bottom of people who'd signed it was the number of people who'd signed it when you went to look at that page, and it wouldn't change as you read about the petition, decided whether you wanted to sign it or not. That number wouldn't change. But last summer there were some high profile petitions that got lots and lots of spectators. And so somebody basically added auto updating in real time. So after he'd added that and they'd merged his change, when you were looking at those petitions you'd see the numbers go up as people were signing it. Another somebody else added. So I said that a debate would happen in Parliament if there's more than 100,000 signatures. That debate gets scheduled and somebody added to the view the scheduled date. So these are ways that people have helped make this code better.
[00:13:00] And then this is an example I really like because it's a really tiny example. This is somebody who works in a different part of government and was using some of our code, and he noticed that - so if you see the top one the left blob is like chopped off a tiny bit - so the offset was slightly wrong. So he fixed it to the bottom one, and this crown is used in one place. It's used in Safari pin tabs. So that is a tiny tiny little icon, there's just no way that you would have noticed that if you hadn't been looking at the code. And I really like this example because if you had to raise a bug report and explain that to a team it would take so long and then how would it gets set on the teams priorities? It probably wouldn't be that important. But if the code is open you can scratch those itches, you can make that fix, you can suggest to the team to take on.
[00:14:01] So I don't want to oversell this point, I don't want you to get the idea that, you know, you make your code open and suddenly there is a whole army of people who are willing to do all your work for you. It's not quite like that. These examples, we have a few of them, but we have a lot of code and they're few and far between. But the point is, if the code is open it's possible for people to make those changes. If it's closed that can't happen.
[00:14:24] Okay so those are some examples of how it makes the code better. I'm now going to talk about how it makes the organisation better. So coding in the open makes it easy to share your learning. We have a format on GOV.UK which is called a smart answer and that's a way of presenting complex information so that it's easier for you to find the information you need to know. So an example of that is this smart answer that allows you to calculate how much leave or pay you might be entitled to if you have a child. So that's maternity leave, paternity leave, shared parental leave and pay and the roles around that are extremely complex. So we could present just a wall of text - lots and lots of information - and you would have to go in and work out what's relevant to your particular situation. But with a smart answer what we do is we ask you a few questions: When's the baby due? How long have you been with your current employer? How much do you earn? And then at the end of it give you the information that's relevant to you.
[00:15:24] So another part of government needed to do something similar. The skills funding agency needed to present some quite complex information about how much of a lever you might be entitled to if you took on an apprentice, and it was quite complex because lots of different kinds of apprenticeships in their various different rules apply to them, and they were told they had to produce a tool to allow people to calculate that. And they started working out how they do that and they estimated that the work was going to take about two months. And then somebody in the team pointed out Smart Answers and they went to look at the Smart code on GitHub and it wasn't quite what they needed. It was in Ruby and the rest of their code was in C sharp and they wanted some features that the Smart Answers code didn't have. So what they did was they used the Smart code as a template and they basically copied the pattern to write their own tool. So it's not a perfect open source story - like a perfect open source story is they go and get the Smart Answers code, they that code, they make changes and improvements and they contribute them back so that other people can benefit. But it's a great story because they used the Smart Answers code and they built this tool in a week. So they were able to get our learning and save themselves a lot of time and money.
[00:16:43] So another way that it makes your organization better is that you can show off details about what you're working on. So next time if it's you up here on stage wouldn't it be really nice if you could like point at some of the code you've written? It helps build up pride in what you're working on with specific examples of stuff you've worked on, and that's good for the organization. And another sort of similar example of that is GOV.UK Puppet code is open. So Puppet is a configuration management tool, it's how we can figure our server in production and the GOV.UK Puppet code is open on GitHub. And now Puppet, the company, uses the GOV.UK Puppet code as a reference example. So that's really nice. It's really nice for us working in the organisation and it's good for the organization. So basically if you can share your code then you can get that positive effect.
[00:17:44] Open code can also advertise your interesting problems that you're working on. And code review, if you do constructive code review in public, that can act as a positive advert for your culture. So people can see that code review is constructive and realise that it's a nice place to work. So I mean another way of putting that is coding in the open is good for recruitment and it's good for retention, so there are some ways it makes your organization better.
[00:18:08] I'm now going to talk about how it makes government better. There are three ways that coding in the open makes government better. The three most important things.
[00:18:18] The first is, it means that you can see what's being done with public money, so it's good for transparency. Coding in the open supports more ways of open working. So this is a GOV.UK roadmap, and that's open. This is a Trello board of one of the teams on GOV.UK showing the tasks they're working on, that's open. And on GOV.UK when we have any kind of incident, like an outage, we have a post-mortem and then we write an incident report. And those are also published openly. So you could do all those things without coding in the open but when your code is open that kind of underpins a culture of openness across the organisation and that's good for taxpayers. You can see what public money is being spent on.
[00:19:04] The second thing is coding in the open forces clarity on the ownership of the code. So I mentioned we use the MIT license and the copyright of the MIT licence is Crown copyright. So the Queen owns the copyright for all the work that we do in government. And when you code in the open from the beginning of the project, and you release under a license, then it forces clarity around those issues at the beginning of the project. So you make sure the government owns the IP for the work that's done, the intellectual property. So the advantage of that is - let's say a department doesn't have developers and they need to recruit suppliers to work on something - they could recruit some suppliers to work on the first part of the code and then you know, they could maybe recruit some other suppliers to work on the second part of the project and there's no issue about working on the same code because government owns the IP for that code. Or you could build a team from multiple suppliers and you can all work on the same code. And the big advantage is it stops us being locked into sort of massive contracts. Because if government doesn't own the IP for the work that's done on a massive contract, when that contract comes to the end, we've got two choices. We can either renew the contract at whatever cost they say or we can start again from scratch with a new supplier because we don't own the IP for the code. Whereas if the product is coded in there from the beginning and those IP issues are sorted out then we're not locked in to that supplier.
[00:20:36] And finally it's cheaper. So access control is complex and so it's expensive. It's complex to work out who can have access to what things and so it costs money, and public code repositories are usually free so it saves money.
[00:20:52] So there are two benefits to open source that are usually given as the headlines. And I haven't mentioned them and I'm just going to talk about why I haven't briefly now. The first is reuse. The idea that you could not reinvent the wheel so you could write some code and then it could be easily reused by someone else and so that's a good thing for open source. And that is great but it's not always that straightforward. You know sometimes it takes a lot of work to use open code. Sometimes you have to make a lot of changes and so it's not quite as simple as "you can write it once and then reuse it and there's no problems there." It's great if reuse happens but I hope I've shown that there are lots of benefits to coding in the open even if there's no reuse happening on any significant scale. And the other thing that people often say is this quote "Given enough eyeballs, all bugs are shallow" And that's the idea that enough is enough. If the code is open enough people look at it, they can fix the bugs and make the code more robust. And we know that that's not true because of Heartbleed. So this was a massive bug that was in Open SSL. Pretty much everyone was using Open SSL for face GTPS, and the bug was in the open code for two years before anyone told the maintainers about it. So the fact is that just because the code is open doesn't mean that enough people are going to be looking at it and fixing bugs for you. Open doesn't mean that there will be enough eyeballs.
[00:22:28] But so the point I want to make is that even if those two things aren't happening - it's great if they are, they're nice to have - but coding in the open has a lot of benefit even if those things aren't happening.
[00:22:42] So there are some challenges to coding in the open. I'll talk about two kinds of talent. The first is cultural. It can be quite difficult to get started. It can be quite stressful if you come from an organisation where you don't code in the open to think that anyone could see your code. It can be stressful to think that you're going to have a code review in public and people are going to maybe be criticising you in public. So it's important to build a strong culture around that. So for example we have guidance around how to raise requests and how to review requests. We do a lot of automation in the poor requests. So for example we automate linting to check that you're meeting the house style so code reviews don't descend into like a political war about tabs versus spaces. That's all done automatically and the code review is just a constructive debate about the particular implementation. So the point is you have to make sure you have a strong culture around supporting people when you go out in the open.
[00:23:40] And then the other challenge is making sure that you don't publish the wrong thing. So that's kind of publishing credentials, as I mentioned and there's two ways to address that. Firstly you should have processes that make mistakes harder. So maybe you want to protect a master branch so you can't deploy it into production without realising. You store your secret separately and you inject them into the code. And I said at the beginning that coding in the open raises your game. It forces you to do things properly from the beginning. It forces you to think about keeping your secrets separately and injecting them in because the fact is if your code is closed and you sort of are lax about that and you might have secrets in the wrong place they will leak anyway, even if the code is not made open. So you should do things right and coding in the open forces you to do that.
[00:24:32] And the second thing is you need to be able to recover quickly. So even if it's really hard to make a mistake, mistakes will happen and the important thing is you need to be able to recover from any mistake quickly. So for example, you need to be able to rotate your credentials quickly. This is a nice blog post, it's worth checking out, it talks in more detail about this, about how the the important thing to optimise is the mean time to recovery not the mean time between failures.
[00:25:02] Okay so that's all very well for government. We don't make a profit from the stuff we're not in the business of selling software. So you might think that's nice but if you're not working for government you might think we can't make our code open. But there is value for commercial companies as well in making their code open. So to explain that I'm just going to give you some examples of some commercial companies that do code in the open or release open source. So the Guardian, they do a lot of coding in the open. You can see their code on GitHub. The Guardian is a print newspaper and their website is free. So maybe that's not a great example but the Financial Times also code in the open and FT.com makes more money than the print edition. And as of last year it makes more money than advertising. So FT.com is a major source of of their income. Another content provider that releases a lot of open is Netflix. So you can see a lot of their open source code on GitHub and they're pretty obvious about why they release open source. If you go and look at their website about it, the headline before you even see about the open source is: do you want to come work for Netflix? They use it as a recruiting tool.
[00:26:16] So those examples are content providers, they're not actually selling software. So let's talk about some examples of companies that are selling software. So GitHub. GitHub also release code in the open, and one of the things that GitHub has released is Libgit2 which is a tool that you can use in the engine that you can use to run GitHub. And Gitlab, who are a direct competitor of GitHub, use Libgit2 to run their platform. But you know, you don't see GitHub going out of business because of that.
[00:26:51] And then finally I want to give you the example of Feedbin. So Feedbin is an RSS reader that allows you to keep up with blogs or news. And Feedbin is a paid for service but is entirely open source. If you wanted to you could take the feed code and run your own Feedbin for free. But people don't. I mean some people do, but many people still pay for the Feedbin service. Feedbin has been entirely open source for four years and is still going strong, so it hasn't affected their business. And you should read this blog post because that's a good example of why he thinks it's positive.
[00:27:26] So hopefully you are all really enthused about the idea of coding in the open if you're not doing it already. You're thinking "how can I get involved?" So I've got some answers on that. Three things you can do. Firstly you can have a look at our code. So the Government Digital Service code is on Alphagov, is on GitHub in our Alphagov organization. GitHub have a page of other government organizations that code in the open so you can go and check out others. The Scottish Government Code is on GitHub and you can go and check out. You can make code from your company open and maybe you can use this talk as an example of what the benefits are if you need to persuade anyone. And finally you can come work for government. GDS are hiring, the Scottish government is hiring, so come work for us. Make things open and make them better. Thank you.
Anna Shipman is Open Source Lead at the Government Digital Service. She is a technical architect and software developer. Prior to her Open Source role, she led the Government PaaS team, building a platform to host government services, and was a technical architect on GOV.UK.
She is a big fan of Perl, Vagrant, DevOps and Open Source.