Rob is the UX Manager at Perform Group, a leading digital sports content and media group. Here, he manages the UX output and promotes a user-centered approach to developing their global network of media apps and websites, enjoyed by millions of sports fans on a monthly basis.
During his time at Perform, Rob has led several successful projects. Including the release of the Official Copa America App, which ranked as the number 1 free app in the App store in 8 different countries.
Rob likes to write about UX and Design, contributing to both uxdesign.cc and theuxblog.com. His articles have also featured on the renowned sidebar.io. He like to live by a simple mantra, which anyone can easily embrace: “Learn from the community, give back to the community.” Outside of work, he stays fit and healthy with his keen passion for climbing.
[00:00:06] Thanks, Rachel. Thank you all for attending. I'd just like to start by addressing the title of the talk because I know it's a little bit ambiguous: "Minor bug fixes." You might be thinking, "what the hell's all this about?" Today we're going to talk about release notes of minor bug fixes, normally accompanies the release of some sort of software, doesn't really matter what it is. And I've kinda picked up a few issues over the years. Companies tend to use this minor bug fixes regularly and they're actually doing a lot more work — and normally, the word 'minor', there's not a right lot of 'minor' about it. Actually, quite a lot of effort has gone into this next release.
[00:00:53] The TL;DR — "too long; didn't read", in case you were wondering what that means — for this talk, is the content's basically about apps, app stores and release notes, but I'm quite aware that there are quite a few web people in the room, so we're going to tie that back to communication, communication channels, and ways to improve that. And like Rachel just said, I work for a company called Perform Group, which probably no one's heard of in the room. I'd be very surprised if anyone has. And the reason for that is it's kind of like an umbrella for a lot of different sports brands, and the sports brands that we have are sports media websites and apps. Basically taking sports content that's written by ourselves or stats that are produced by ourselves and distributing these across the world.
[00:01:41] So if we take a look at Goal, which is on the left here, and this is one of our biggest websites. I've highlighted for you some of the countries that we have local editions in. We have 38 local editions, serving 20 languages and the reach for this kind of audience is basically around 57.4 million unique users per month, and this is taking numbers just from the website alone. So if we were to add the apps into here we're looking at over 1 billion unique page views every single month, which is a massive audience. And basically that kind of gives you a little understanding of the scale of my work on a daily basis. I am the UX Manager for this company. There's about five UX designers within my team and we're basically based in Leeds, and I've highlighted two other countries there, Poland and France, because for the apps, we work with a development team in Poland and we've also got another development house in France as well. So that comes with quite a few challenges because, obviously, we've got to cover quite a lot of different languages and different types of content, but all the designers and their product guys are based in either London or Leeds.
[00:03:07] So Perform comes and we often have these kind of issues, like the business suddenly changes direction, we've got a lot of products to serve so you're working on this one, but oh wait, we've got to work on that one. And ultimately the main problem is everything takes quite a long time to produce. It's not like a really nimble, agile way of working at times. But the benefit to this is that we are working on software at a huge scale, so we have access to a massive audience. Football and sports have a really passionate audience and they're really keen to always offer their opinions and so on, and so forth, and it's really great to get that impact, and have that kind of reach.
[00:03:51] So I wanted to start by taking you through one project that we worked on. As I said before, we kind of hop and change between different brands that we're working with. Back in December 2014, the goalnews.com app was feeling a little unloved — I've kind of used the little grandma and grandpa on there as like an indicator to say it's not been touched in a while, basically. Basically, it took about one year, three months and eight days, to be precise, to actually work on the redesign of this product. And this is not anything that's revolutionary. It's people digesting news content, article pages, category pages. We're not reinventing the world here. And we ended up with something that looks kinda like this - it's got nice new skin, some minor tweaks and improvements as we've gone along. And like I said, one year, three months and eight days. It's a lot of work that's gone into this. The design team, the product team and the development teams put in their blood, sweat and tears to get to this point.
[00:05:08] I'm sure that you've all been there and you've got to the release date and you have a release party, to start to celebrate. Unfortunately, there's one guy in the room - which is myself - looking more like this guy, and thinking "What the hell?" And the reason for that is this: the release note that accompanied all this work just simply read: "Thank you for using Goal. Your favourite football app just got better with a full redesign." And you know in my mind that doesn't really reflect the amount of work that's gone into that kind of project. The guy who wrote that, I'd have much preferred if he'd done something like this. That's not hanging himself, that's just making something that looks visually exciting and engages the audience a little bit better. As I said I'm the UX Manager so I come from quite a visual design background and type, typography and type hierarchy are all the kind of things that I always try to look to — including, like in my presentations in this case and work. Someone just quickly typing this up and chucking it out there to me felt a little bit misaligned to the amount of effort that we've actually gone into releasing this product. So I start asking around and basically trying to convince people that I've got this idea that we can do better than this. We can definitely improve our products through release notes. And I'm met with not the best kind of response. People kind of saying "Oh well, we haven't got the time for that," like I've already said we serve loads of languages so it's going to be a nightmare translating this, and you know essentially no one reads these things so why are you even bothering. And I'm thinking, well I read it and I saw it, so someone must read them somewhere else.
[00:07:00] So I thought I'd spend a bit of time looking for some evidence to convince people that there's a better way of doing this, and it turns out that no one does read release notes, as I'm sure some of you may be thinking. My job today is to convince here to actually spend a little bit more time and thinking about the end stage and actually what the benefit is to this.
[00:07:24] But first let's just touch on why people don't read release notes. There's two reasons for that. Oh well there's probably more, but these two things that stand out. One: Android and iOS both offer auto-update these days, so people just see apps automatically getting updated so there's absolutely no reason to read about what the new features are. And secondly: even if you did visit the "What's new, I've got some updates" section, you don't actually see the release notes; you've got to tap to expand those — which, again, isn't ideal when you try to tell users, "hey, we've got this nice new feature." And that's ultimately led to companies just being a bit lazy with release notes and generally in the App Store, just not bothering. We've got Facebook on the left that just says month after month "Hey, keep your app up to date and you'll get some cool features," which to be honest isn't that great. It's just as bad as the stuff that we churned out, or even worse — the title of the talk, just 'bug fixes, bug fixes, bug fixes'. And this is some of the biggest companies in the world that are doing this.
[00:08:30] So my job today is going to be to convince you that these guys are actually doing it wrong and there's a much better way, and I'm going to try to talk you through some benefits of what we've seen by taking a different approach. And it is self-explanatory, but I just want to make it clear that if you're releasing something and you're writing it, and the users are over here and they're reading it, it's very much a one-way conversation, you can just send something and hoping that they're going to kind of pick up on it — or not pick up on it in some cases. And we found, looking back at some examples of how we can do this better, we found one way which is to actually include an option for feedback, so include an email address, say "hey, we've got some new features but we're always working on improvements. Why don't you drop us a line?" And that's already better than just writing 'bug fixes' because you open up that conversation, so it's less one-way conversation street, and it's more, if they want to they can get in touch with you and you can respond to that and basically have that sharing of conversation. And I've just kind of touched on some people that do a really bad job with this, and I don't generally like slagging people off, so let's talk about some people who do a really good job of this, and in particular 1Password and Slack.
[00:09:52] These guys have documented better ways of actually writing release notes and how to communicate to your audience and actually actively engage them. And from a visual design background, I'm kind of dyslexic as well and my spelling is atrocious so I'm not going to patronise you on what you should write, but instead I'm going to try to talk about simple design tips that you can apply to just plain text release notes. In this case we're looking at using a very limited palette. We've haven't got bold, we haven't got colours, we've haven't got italics, we haven't got different sizes. We've got pretty much these characters that you see on screen right now, so how can we use this to actually create something that's actually engaging? And let's start by looking around at some examples of companies that do a really good job of this. And like I've already mentioned, 1Password on the left do a really good job. They use that TL;DR which is 21st century for just summary, like you don't need to read all this stuff that we're talking about but at least you get a little snippet of what's to come, and they use capital letters to create a title, space to break out this section. And Medium as well are using TL;DR, it seems to be like quite common practice in release notes in particular. We don't have access to bullets, but we've got the alt 8 if you're using Mac and you can still include bullets. And again you can get creative and use text. In ESPN's case they've used open brackets and pluses. And we've already talked about this, getting people's feedback - why not include that? It's a good idea. And you can always reference the major release. I'm not trying to say that writing 'minor bug fixes' is always bad; I think that there's definitely some cases when you are just quickly fixing something, but you've probably done some really good work recently, and you probably want to talk about that. So the point would be: don't just say the last thing.
[00:11:57] So to illustrate that point a little bit, I'm going to talk to you about the tipping point at which I started to get more buy-in for this kind of new way of thinking. And you guys have already seen this. This is the point at which we released the update to the Goal News app and put out that crappy release note that just said "it's got a brand new redesign." One year's worth of work down the drain. And we didn't learn from our mistakes at this point and obviously I wasn't able to get enough buy-in.
[00:12:28] The next time we released, it was coming up to Euro 2016, and that's a real key date for Perform, because obviously we're a sports media company. We make a lot of money at key sporting events such as World Cups and Euros and so on and so forth. And we spent a lot of time, like the next six months or so, really working on improving the app and getting a dedicated Euro Section for this time. Two days later we released minor bug fixes. We shipped it with something that was a bit of an issue. And the reason that that was bad is that if we just kind of map this out on a calendar year, we've got the Euro section getting released which is great. A lot of our competitors didn't have this. It's a really unique feature that our app has to offer, and four days later we've shipped a minor bug fix. We also had two of our releases that month, which aren't really that important, but I just put them on there for context. And the highlighted area there is the time that - in the App Store, if you look at your own app, the "What's New" Section is a really great space to kind of talk to users and say, "hey we've got this really good feature, you should check it out and so on and so forth - and during that time period, if we just look back to that calendar, the only thing that we're seeing there is minor bug fixes. So for a start we can use that method that Evernote used where you say "minor bug fixes" but you still reference your major release. And the reason that this got so much traction is if I map where the Euros started and finished, is only actually four days where, in the App Store, we're not talking about something like a really good feature release, and obviously it's a key time for us — we're getting a lot of traffic to that page, and we're not really capitalizing on it, on unique features that we've got.
[00:14:26] So at that point I kind of share in all these insights that I've been gathering with the team, particularly the product team which is traditionally the ones that are responsible for writing this kind of stuff. And I'm saying can we work together and try to work out a perfect release note? What should we put in there, what shouldn't we put in there? Can we have some sort of template to speed up this process a little bit? And we decided on this, which I'm going to quickly talk you through. You can always start with a nice little icebreaker. It's not essential. I think sometimes it's appropriate, sometimes it's not. Always include that summary section because there's a lot of people who don't have time to read or don't want to read. You know we've already touched on the elephant in the room, that no one reads release notes, so just at least offer that little snippet to users. And then just going down into different sections - so using capital letters to create a title for the content, breaking content down by something that's new, something that's improved, something that's fixed. You know you can talk about these in as much or as little detail as you want but at least you can cover the fact that you've done quite a lot of work and obviously have talked about this a few times already. Offer them the opportunity to get back in touch with you because that's important in creating that two way conversation.
[00:15:50] So life at Perform wasn't great before I kinda got this momentum behind this whole new way of thinking. We've got Olé on the left here, which is an Argentinean app: we're using English to release in that app store, which is not cool because it's not a native language. Same problem in Dutch, we're using bug fixes and then 'kleine bug fixes' — forgive my accent there. And it's just inconsistent — like, sometimes we're writing a lot, sometimes we're not writing a lot. It's generally a bit of a mess. And life after this template: we kinda saw a whole load of consistency, we're using native language; we're breaking down titles; sometimes there's not a lot to talk about, but we're still using that kind of structure; users can expect that kind of interaction with us. In fact, there's only one of these that's got a nice little icebreaking section, and again you know you don't need to use it. For any keen audience members today, I'm trying to break this talk down in a similar structure, so that's the reason for including TL;DR at the start.
[00:16:58] And at this point life is good because we've really turned a corner and that's pretty much been from me, the UX Manager, who is not the brand advocate or the release manager, but kinda made some positive change here, I'm feeling great. But unfortunately, we've got some issues to tackle, and one of those is that translations take a really long time. You know I've already talked about how many languages we cover. If just one person does not reply - the Arabic guy didn't get back to us - that delays the entire release, which is not cool. So we've started working on a new kind of collaborative way of working. It's nothing revolutionary; it's a Google spreadsheet. We write English on the left hand column, and we've got different columns for different languages and editions. And at this point we've made it part of our workflow.
[00:17:52] As a designer picking up a minor task I can write in here, "hey, I want to tell my users about this," or as the developer who's just spent two weeks refactoring the app for the next update, we want to make people aware of that, that we're still chipping away and improving that. And if I go back to that map that I showed you at the start, with the design and product teams based in Leeds but that's suddenly meant that we're getting involvement from Poland, and we're getting involvement from France, and we've got, obviously, daily standups and things like that, but it's like an active thing that everyone's kind of passionate about within the team, which is great to see.
[00:18:29] And then the next issue would be, like I've already said, sometimes 'minor bug fixes' are acceptable, but we make a lot of money through advertising. One example would be that we spend a lot of time adding a new additional ad network to our app, which isn't cool. And traditionally we'd have tackled that by just saying "Hey, minor bug fixes," sweep it under the carpet, the users don't need to know about this. And that led me to have a little bit of an investigation into minor bug fixes over the last 12 months. This graph here kind of shows the amount of effort that's gone in — I touched on it at the start; I don't actually believe that there's a thing called a 'minor bug fix', because no way is a bug fix a minor amount of effort. I've highlighted the kind of minimal ones, where maybe you could get away with saying 'minor bug fixes' at this point — but this piece here that nearly took 260 man hours to actually get out, is the release where we actually added the new ad network. And like I said, traditionally we're using 'minor bug fixes' for this. But from the design team's point of view we did something really simple and it's probably not even worth talking about. But before we add our cards, we have a title and a little snippet, and we decided, "hey, why don't we remove that snippet?" and all of a sudden users can see a lot more news within that view. And it's not a great thing to talk about, but it's much better than just saying 'minor bug fixes', so we can take that release note and make it look something more like this - where we say "hey, we've got a brand new card format. You guys can now read a lot more news than you could before." We could just mention the fact that we've optimized ads; I know that adding in a new ad network is not what the users want to hear, but it might be faster, there might be some sort of benefit to it; maybe we don't need to go into too much detail. And the other idea would be to reference the amount of bugs that you've actually fixed. So, that piece of 260 hours was spent fixing 24 bugs, which is quite a lot, and users might be getting frustrated by something and then try and check that out again and see if they fixed that particular one.
[00:20:47] The other way of doing it, is if you've not got anything like that to say, and you are just releasing a new ad network, which is not cool, you may have some new features in the backlog which you know are going to come up relatively soon. So if there are big improvements around the corner you can tell your users you're letting them know about that.
[00:21:07] And this is the point at which I want to kind of address that problem that no one reads release notes. You know, the product owners are saying to me, "it seems like a hell of a lot of effort for not a lot of reward." Like I've already said, no one does read release notes; auto-update is on traditionally for pretty much 90 percent of all Android and iOS users. But one of our competitors does a really good job of this. They signpost people who use the app to a running blog of new features and this is where they kind of reuse that copy and say "hey this is new, this is fixed, this is improved, this is what's coming up" and they're taking it away from the App Store. We're carrying it for some people who do use the app store - and not that many do - but we're also tackling the majority people by hitting them the first time that we see the app. And this kind of stuff's not something that we've got in our apps at the minute but is something I'm really keen to press for.
[00:22:12] And other ways that you can reuse that: I've just highlighted Nokia's Health Mate app — they pop up a modal the second that you land on the app, and it's the exact same copy word-for-word. So all that effort you've just put in you've translated it to 20 different languages, you can just reuse that. If you guys generally deal with the app store rather than web sites you can always do what Dropbox does: aty every major release, send your users an email if you've got access to their email addresses.
[00:22:43] So I just want to spend the last part of my talk just kind of proving to you guys that this works. I wrote an article about this and I'll share it on Twitter so you guys can read it if you're interested. But coming to Turing Fest I really wanted to come here with some more evidence and be like "look, we've done this and this is proper cool" and give you guys a little bit more insight. So going back to that point of speaking to users and allowing users to speak to you, like we've already said not a lot of people read these so it's really quality over quantity. And what we've seen in our inboxes is things like this, where someone has gone to the effort to download our app, to update it, to read our release notes and actually send us an email saying "hey I think you guys could do some better work." We've gone back saying "Hey, would you mind just like explaining a little bit more? We king of get your problem but we're not really seeing it within the office." This guy comes back and literally broke down why his notifications aren't working and so on and so forth, which is great. Some features are only there because users really bought into that process and put quite a lot of effort in to get to this point.
[00:23:52] And we've also picked up users for interviews. Someone again getting in touch with us through the release note, went back and forth just dead quick and then said "hey, would you mind just jumping on a call, it'd be nice to speak to a little bit more." And that's trying to get the point across about quality over quantity.
[00:24:11] Now the next slide really proves where quantity comes in place and where release notes might not be the place for quantity, but there are other drivers where you can get that real rapid increase. And this is something that we released off the back of that major release that we referenced earlier. "Thanks for using Goal, your favorite app has just got better." Oh crap, loads of people are slating us in the app store — as I'm sure you probably are familiar with, if you change everything, users tend to hate it immediately. And we did this dead simple thing where we just introduced an App Store rating block, it's quite common. You get the good people going to the App Store and you get the people who are a bit pissed off directing them to an email so that you can have that two-way conversation, and the benefit to that is twofold. One: you can address people who are pretty annoyed and give them some comfort or reassurance that you might work on their problems in future. But two: if you are familiar with the App Store, the way that ratings works is its quantity and quality. So if you can get the four and five star guys into the app store and you can get them there in abundance, your apps will suddenly start getting featured, especially on iOS. And this is the left during the Euros: we've got the Goal live app featured. We've been on the "hot this week" and on the picture fills right at the front, which is great exposure for the team and for the app.
[00:25:42] And I just wanted to show you like the effect of this in the App Store and the rating. So this is the Android app store. At this point we've got that release which is a brand new redesign. And like I said you can see the increase in the negative feedback. The next release we added the ratings block to kind of try and stop that negative feedback from sending us down the charts. And what we found is all of a sudden the volume just like doubled overnight. We're actually encouraging people to speak to us in the right channels. We're not using the App Store for people slagging us off. We use that for that two-way conversation. And up until recently you coulnd't actually reply in the App Store to people's complaints so we were getting them to go to the emails.
[00:26:29] And then touching on the Olé app, so this is the South American app. It does not get a lot of love and we did a release recently and with that ratings block implemented from the start. And it's like shot up overnight. I've quadrupled the amount of response that we got.
[00:26:48] So I just wanted to kind of touch on a little saying that my previous boss at my previous company drilled into my head, and it's kind of stuck with me a long time now. I think you guys can all take it today, apply it to your own work streams. It's the 2mm Rule by Tony Robbins. It's essentially the fact that to get from average to good takes a hell of a lot of effort, and I'm pretty sure that none of you guys in this room are going to settle at average. You all want to get to good. To get from good to excellent is actually only two millimeters more, but pretty much everyone just gives up when they get to good because they go "oh, yeah, it's good enough; ship it, it's gone." If you actually spent a little bit more time focusing on something like the real minor things, like release notes, then you can actually achieve something that's truly excellent and you actually realise how close you were to getting to that point.
[00:27:34] And then it wouldn't be a talk about release notes without finishing on that feedback we've talked about. Hit me up on Twitter it's @rob_gill_; I'll tweet out the article that I wrote about this so you guys can have access to all the assets, but feel free to let me know I did today bit more than welcome to engage in a conversation. Thanks very much.