
On the 9th of January 2022, _The Sunday Times_ published [a story](https://www.thetimes.co.uk/article/end-of-free-lateral-flow-tests-as-country-told-to-live-with-covid-3bpz8lnqf) that asserted that the UK Government were in the process of phasing out a policy that had arguably kept normal life running during many months of the Covid-19 pandemic.

Where other countries had made Covid testing something citizens had to seek out privately, the 'Test and trace' programme in the UK — aside from doing very little tracing — had made free lateral flow tests availble, delivered to your door, in packs of 7.[^1]

Given this policy was supposedly coming to an end whilst the Omicron wave was still out in force, and public life was still heavily disrupted for those looking to avoid catching the virus, there was a rush on Covid tests. The website crashed repeatedly, and frequently was returning a page stating that home delivery tests were out of stock.

Enter the Twitter bot, [@LFT_Alert](https://twitter.com/LFT_alert), the inspiration for this tool, which alerted people in their Twitter feed whenever tests became available. A fantastic service, and anonymously set up on New Years Eve 2021, this bot tweeted to its 9000 followers whenever tests went in and out of stock.

But there were two things I could see about this approach that could be improved.

Firstly, whoever was running LFT_Alert needed to keep a computer running themselves in order to repeatedly verify if tests were available. This effectively became a single point of failure to the system (and a point of maintenace and annoyance for the owner) as disruption to the machine would end the service.

And secondly, Twitter is a mess. For me, following over 3000 people, my Twitter feed is awash with hundreds of different faces. To pick LFT Alert's valuable messages amongst the noise of retweets and chatter would be difficult. Twitter offers notifications for any one account, but this had the secondary effect of notifying me whenever tests went in or out of stock. It only matters to get tests when you are running out. To be alerted every time would be a hassle.

So, far too late to the party, as I would soon find out, I picked up my keyboard and spent a few hours on the night of the 9th of January building an alternative to solve this.

<Tweet id="1480456256242339840" />

[lft.tk.gg](https://lft.tk.gg) was born.

## The app

Built on two principles: keep the work to the person searching for the test, and don't abuse the LFT portal.

When a user loads the page, they immediately start a timer that executes every minute[^2] which triggers a check to a shared serverless API endpoint, that is cached for 30 seconds. This endpoint goes to the NHS service and uses a private API to check if LFT orders are open or closed. The API returns the state of the ordering, and the last time the check ran.

```
{
"status": "OPEN",
"timeChecked": 1642919745205
}
```

The page then responds, updating the last time the check ran. If the ordering is open, it then uses the web notifications API to send a push notification to the user, triggering a link through to the order page.

I thought this was the simplest solution to the problem. A user needing tests could simply open the page in the morning, work through the day, and get the notification as soon as tests were available.

It's a little frustrating that the government service didn't offer something similar to this. The only major downside I had was learning that the `Notification` API was (of course) not available for mobile browsers (and full on crashes Safari when used natively). I thought about working out how to send notifications to a mobile browser. This would have required some extra work to set up a service worker, and then using the `Push` API instead. I planned to do this if the service got a lot of traffic (and may still), but the short term supply issue appears to have been resolved, and the status is now very rarely `CLOSED`.

Still, I enjoyed building the app and learning a few new things about notifications, and keeping browser tabs awake (it turns out you can play audio to keep something permanently awake, but Chrome _checks_ to see if the audio is audible!).

I hope it really won't be useful in the future! Stay safe.

[^1]: Just a aside on how infuriating it is that then they also enforced mandatory private lateral flow testing, at a cost of around £20 per flight, for international travellers, despite everyone keeping a half dozen identical tests at home, provisioned free at a massive discount by a public provider...
[^2]: [Chrome throttles background functions](https://developers.google.com/web/updates/2017/03/background_tabs) running more frequently than this)


LFT notifications in your browser


At WWDC yesterday Apple announced _Mail Privacy Protection_, an initiative seeking to bring blocking of _Superhuman_ style tracking pixels to Apple's Mail app.

In the video demo Katie Skinner, Manager of User Privacy Software, introduced marketing emails as using hidden pixels to collect your activity, such as your IP address, or when you open or take an action in these emails. Their solution is [_Mail Privacy Protection_](https://youtu.be/0TD96VTf0Xs?t=3143).

<Image
  src="https://img.tk.gg/WWDC+2021++June+7++Apple.mp4_snapshot_00.52.31.057.jpg"
  height="480"
  width="854"
/>

> It hides your IP address so senders can't link it to your online activity, or determine your location. And it prevents senders from seeing if and when you've opened their email. So now you can catch up on email with greater peace of mind.

That's it. That's all we know so far. And though focused in this context on marketing emails, it has significant implications for publishers and the newsletter boom.

Litmus, an email marketing consultancy, [estimated that the top email client globally is _Mail on iPhone_](https://www.litmus.com/blog/email-client-market-share-2021-q1/), with 38.9% of opens coming from the app in the first three months of this year. On desktop, if this feature launches on Apple Mail there, that's another 11.5% of sends. The current market share of both is expected [around 57%](https://emailclientmarketshare.com/).

This is probably a fuzzier statistic than I'd like it to be, but we can I think safely say that a lot of emails are going through this ecosystem.

As a publisher you probably have two statistics that you are using to determine performance via your <span style={{borderBottom: "1px dashed black", cursor: "help"}} title="Email Send Platform">ESP</span>: open rate (the number of people who opened your email for any amount of time, judged by a small invisible image that is loaded uniquely per subscriber), and click through rate (the number of people who clicked on one or more of the links in the email, judged by each link being tracked via a forwarding service). Apple have just attempted to wipe out one of those.

We don't know how Apple accomplish this, but I think it's safe to presume that it's similar to how [Hey.com announced their privacy push](https://www.hey.com/spy-trackers/) in their email client. Hey keep a denylist of tracker addresses, and scan your email for elements that look like they could hide a tracker (anything that's a 1x1 gif) to remove them.

Hey has a second line of defence also: if a tracker slips through via some more structural image, Hey proxies the request to the image via their servers, meaning your IP and approximate location are never leaked to the sender. Apple may or may not do this.

I'm broadly not against privacy online, and I think Hey.com's second step is especially good at protecting users from invasive tracking. In fact it's exactly what [Google Mail started doing back in 2013](https://gmail.googleblog.com/2013/12/images-now-showing.html). But, much like other generally-sensible-in-theory provision like the GDPR, moves like this from Apple are going to hit hardest the solo and freelance publishers with their Substack newsletters.

<Image
  src="https://img.tk.gg/WWDC+2021++June+7++Apple.mp4_snapshot_00.53.04.433.jpg"
  height="480"
  width="854"
/>

A major publisher will be hurt, sure. They'll lose a lot of data on which they sold their newsletter sponsorships. They'll be less able to confidently purge subscribers who haven't opened their newsletter in months (what if they're iPhone users?). They'll see their open rates drop 30%+ overnight.

But there will be ways around it. And suddenly, with over half your email data at stake, this is what both marketers and publishers will start working on in the coming months. Until emails are wholly plaintext there's always something you can make uniquely identifiable.

A smaller publisher, a local newspaper, a solo freelancer, a small blog; all these will lose data on a significant part of their audience. A likely valuable part of their audience. And it may stifle or slow their growth or opportunities.

Where previously you could unsubscribe readers who hadn't opened your newsletter to save money, now you don't know if they're loyal or not. You'll have to find other ways to entice them to let you know they are reading. A larger publisher can afford to keep 20,000 recipients on a list that never open an email. A smaller outfit cannot.

Apple's fight for privacy is really a fight against the web. In signing up for a newsletter, a publisher or marketer already has a more valuable piece of PII: your email address. By focusing on IP addresses, and blocking trackers rather than proxying them on a fuzzy delay[^1] (which would provide the same useful publisher data without any PII leak of location or time), Apple are not really fighting for their users so much as they are fighting against email.

Apple every year see that [the walls of their garden](https://www.wsj.com/video/series/joanna-stern-personal-technology/how-apples-iphone-and-apps-trap-you-in-a-walled-garden/) get higher. It's no doubt that whenever something like this happens, Apple have an in-ecosystem solution for you: launch an app on their store, post to their own publishing system Apple News, just don't use the open web.

[^1]: This really should be what Apple do instead: they know what the trackers are, so load them later, from a server rather than from the phone. Give the publisher the data that the email has been opened, and by whom (via the email address), but not where or when.


Apple's Mail Privacy Protection is lazy and hurts small publishers


The 30th of March 2021 marked [five years](https://www.theguardian.com/media/2016/mar/30/times-rolling-editions-website-sunday-times-apps) since the launch of _The Times and Sunday Times_ 'digital editions'. A project that took [two years](https://www.theguardian.com/media/2015/mar/24/times-website-editions-project-d) from the initial concepts, to a final launch.

Now, 1500 editions later, not much has changed of the original. Though it was initially received with skepticism in bundling publish times and generally bringing a newspaper's then-tablet workflow to the web, it has at least been influential on several other publishers since.

<Tweet id="715231553916239872" />

- In 2019, the _i_ launched ["three daily editions"](https://www.pressgazette.co.uk/the-i-launches-paid-for-news-app-with-three-daily-editions-and-live-breaking-news/) on its apps, alongside a livelier breaking news operation.
- The Guardian recently relaunched their paper tablet app as 'Guardian Daily[^1]', supporting readers wanting to read ["coverage at a slower pace"](https://www.theguardian.com/membership/2019/dec/07/guardian-daily-app-launch-new-edition).
- MailPlus, the slightly-weird paid newspaper-made-digital version of the Daily Mail (not to be confused with dailymail.co.uk 🤦‍♂️), have a homepage that looks remarkably similar to The Times, and has stories updated at set times of the day, with the update time called out specifically in the first paragraph.

But even looking past legacy publishers with the kind of content scale that would permit editions, the product themes (timeliness, bundles, and finishability) are behind the rise of many new faces.

Daily deliveries like [_The Morning Brew_](https://www.morningbrew.com/daily/subscribe) briefing, the bulleted 'why it matters' summaries of [_Axios_](https://www.axios.com/), the timely written prose of the [_Quartz_](https://qz.com/) homepage, and the growing array of newsletters on [_Substack_](https://substack.com) that cover news at a slower pace, in a more digestible manner, all exhibit parts of these three components.

Five years on though, the industry is still scratching the surface of what an edition product might look like. Instead of working on the job an edition does for a reader, we are once again focused on what job a digital offering does for the journalists and editors that produce it.

We are not our readers.

The story of editions outside of the digital-first startups can be told as an allegory of the fundamental disconnect between newsrooms and the concept of _news product_ that seems to constantly evade them.

In technology roles you are encouraged to take the time to look back, to retrospectively analyse your successes and failures to learn from them, something that there is rarely the opportunity to do in editorial given the relentless pace. Five years after I helped build it, and as many of that team depart _The Times and Sunday Times_ I think editions as a concept deserves a retro.

So I thought I would write about why I think editions done well are still a compelling product, and then how we continue to get them wrong.

## What do I mean by editions?

There are three concepts that I think go into building editions, and they align reasonably well with the concept of _RFV_ as a business metric. Recency, frequency, and value (as in recency of visit, regularity of visit, and length of visit) are common metrics that news providers might use to determine their performance amongst readers. Generally if you can bring people in, give them reasons to come back, and serve them well during the time they are with you, you have a happy reader.

One of the driving forces behind this behaviour, and something that editions explicitly encourage is creating habit. A bundle of high quality, curated, and timely content, produced regularly, are excellent foundations on which to build a habitual, valuable audience.

### Loops

This works best if you can provide a platform that marries this bundling with other features that help build further habit. As Nir Eyal describes in his book [_Hooked: How to Build Habit-Forming Products_](https://amzn.to/3c8ecmI), you want to build loops that the user is prompted to come back to, however you may create or deliver that prompt, and then provide value and opportunities to trigger further prompts in the future.

These loops are a part of why that [slim proportion](<https://en.wikipedia.org/wiki/1%25_rule_(Internet_culture)>) of readers that engage in comments are so much more likely to return: they typically read more content, and spend more time reading it, than those that do not engage. Leaving a comment is an 'investment' as Eyal describes it in the last part of the hook. A comment may also, depending on your capabilities, provide the first part of the hook, the trigger ("Matt liked your comment", "Matt replied to your comment" as an email notification or otherwise). Creating touchpoints for readers to engage in ways that permit you to bring them back are gold.

So how does an edition relate to a comment?

It's in the product's ability to create that reader investment, and initial trigger for them to return. With a live news site, you could allow readers to follow topics (like [My FT](https://www.ft.com/tour/myft)), or authors (like... er... My FT). Both of these up front investments likely result in notifications (a trigger), but one that could potentially come at any time, with an unknown volume.[^2]

An edition model, done well, enables the ultimate end-goal in planning: an open newsroom with your readers, telling them what you are producing and when they can expect it. They can sign up to a known and predictable quantity and timing of editions, with the potential for them to engage further in highlighting coverage they want more of in coming editions.

Digital technologies allow the same kinds of notification for individual stories or [storylines](https://www.bbc.co.uk/ontologies/storyline), and they provide regular touchpoints (the edition publish times) to re-engage readers to subscribe to the next touch point, or the one after that.

### An enforced limit

<Tweet id="1363123171562577921" />

<p style={{ textAlign: "center" }}>🤔</p>

The other aspect of editions I think a lot of is 'finishability', described as creating a valuable visit for the reader through enabling them to see—though perhaps not meet—the finish line.

It would be very difficult for you to 'complete' the digital product of _The Guardian_ today. You'd need to find every story _The Guardian_ has written, quickly check it out and determine whether you wanted to read it or not, then potentially read it. This is the power of the alternative in Guardian Daily. It's a front-to-back bundle of editorially curated items, and once you've finished browsing through them you're done. Your trust is in the publisher to curate what is important for you to see, which is a service you pay for just as much as the journalism itself.

This idea of 'finishability' is fundamental to the success of sites like Axios, which trades in both breaking news and detailed analysis, but lets no story lose the reader's attention through a succinct summary delivered in bullet points.[^3] Brevity is crucial.

_The Guardian_, years ago now, used audience data to cut their output by a third. As described by Chris Moran [in this exceedingly worthwhile interview with Emily Bell](https://pca.st/Be3s), the problem presented by the internet was a combination of the feeling of unlimited space, whilst still being limited by opportunities for promotion. A homepage, or a social feed, or a newsletter can only display so many possible items for consumption. Why not limit yourself to a much more succinct service through design once again?

Editions, at their best, should attempt to reintroduce the limitation of print space. They should make editors acutely aware of the attention commitment they are asking of the reader, and of the reader's potential need at the time. Journalists have a very odd fascination for the idea that more stories, and more choices, are _always_ better than less. We need to remember that we are not our audience, and their needs at any time are very different from ours.

Attention in an article is an important metric, but when applied to the reader's entire visit it is their quality of attention we should strive for: how quickly can you get a reader into a value-providing action such as reading an article or watching a video. Time spent hunting is time they could afford to save.

Of course there is a lot more that is possible: in learning more about habits and interests of readers you could deliver them partially or entirely custom bundles, or more effectively A:B test structure and content than it may be possible to do in a more real-time traditional digital news product.

### Timeliness

Editions rebundle articles. This is a step back from the course of the internet where the article has become the atomic unit of consumption. When it was impossible in a printed world to buy an individual article, in the digital domain you likely land on an article page directly when searching for news, and may never see the overall package.

This is fine for flyby consumers, but in publishers with membership models the homepage, and other elements of the digital offering, have increased value. Readers may visit directly and use it to skim your reporting, or to get a feel for what your editorial focus is.

Homepages today are (almost all) live news wires, with latest news interspersed with high-performing features and older articles. Repeat visitors over the course of the day (themselves rare, admittedly) may see articles they have already read sitting in the page, in a different position or with a different headline[^5].

Editions offer the opportunity in rebundling articles to create periodic updates to your homepage or homepage-like features (such as a specific 'edition' page). Your predictable updates provide opportunities for nudges and notifications at known times for your audience.

And, should it be useful to your content strategy, editions can help to move newsrooms away from the always-breaking news agenda that is endemic of the transition to the internet. A rush to publish first means a multitude of updates, work passing between hands, and readers not knowing the difference between your outlets and the dozens of others carrying the same stories at the same frenzied pace. National broadcasters and cable networks will always cover breaking news, so bring your expertise and angle when it's ready in the next edition.

## The state of editions

Moving on from the thematic elements of editions to the practicalities of introducing them to readers, it's no surprise to see that in practice little of the potential is realised for readers.

_The Times_, _MailPlus_ and the _i_ all intersperse new stories with old, they all have long scrolling homepages based on print sections rather than reader need. None meaningfully personalise or engage further than comments or social pushes, and none offer to notify me about editions, leaving me to learn for myself when the next update is by deciphering unloved page furniture.

_The Times_ has even recently started to completely break with the concept, starting rolling live news coverage on their homepage, something which has become increasingly common as time has gone on. Journalists are working rigorously to serve themselves, and the social and search giants, rather than their loyal paying customers, or carving any kind of niche provision.

The editions model is the clearest indication of the product/journalism divide I think there has been. In a world of print, publishers competed on content, geographical monopolies aside. In a global digital world, attention is more sought after than ever, and content is most commonly combined with some valued delivery mechanism to provide the best experience. This could be a movie streamed at adaptive resolutions straight to your smart TV app, or a song from a playlist constructed from your listening habits to introduce you to new artists.

In our world of journalism it could equally be an edition. Some form of the packaging of journalism that isn't atomic, isn't gamed for search, and doesn't purely rely on the levers of social networks. A product in its own right that delivers a unique service for readers[^4].

That is where we could be.

[^1]: The [GitHub code repository](https://github.com/guardian/editions) for this app is even called 'editions', although it was originally called Mallard, which is super cute 🦆.
[^2]: Speaking of volume of stories, Feedbin, Feedly and other RSS readers do this really nicely by showing you the number of stories you are likely to receive by subscribing to any one feed. eg. one article per month, or twelve articles per day. Don't tell me that information wouldn't help you make a better decision as to what to spend your time with.
[^3]: Axios also have this lovable, almost-pointless `Go deeper` clickthrough for stories on their list pages that calculates the minutes to read. More often that not, that time reads `(<3 minutes)`.
[^4]: I feel it's worth nothing that this might not, and probably should not be the only product that you offer to readers as part of your service (unless you are really committed to this specific branch of readers), but increasingly could be something that an editor produces from your coverage, or is almost entirely automated. Publishers that jump into editions one day, and out of them the other, do nobody a good service. How on earth are your readers supposed to know what's in your head?


A retrospective on five years of digital editions


_The New York Times_ recently produced an awesome article series called [Primal Scream](https://www.nytimes.com/interactive/2021/02/04/parenting/working-moms-coronavirus.html), where a published phone number allowed readers to call in and talk about their experience raising children during the pandemic.

Listening to your readers, however you can, is crucially important in evolving your delivery of digital journalism. Entire consultancies such as [Hearken](https://wearehearken.com/) are built on involving communities in business, especially journalism. [City Bureau](https://www.citybureau.org/) in Chicago bring people together in workshops to discuss the problems they are facing, ultimately producing media that is more honest, more involving, and crucially more desired.

But how do you do that when you're a national newsroom at scale? Or even at the scale of a single city? How can we use technology to make it easier than ever for readers to get in touch with journalists and becoming involved in our storytelling.

Whilst years ago running a phone in line like this was something that only the largest newsrooms with the deepest pockets could accomplish, advances in digital technologies have made creating the _Primal Scream_ phone line possible for even the smallest reporting outfits.

Recording hundreds of voicemails to a dedicated phone number could cost you as little as $20. It requires very little technical knowledge, and just a few short steps to get everything up and running.

We used the same technology when I worked for [The Times](https://www.thetimes.co.uk) for our [NHS memories project](https://www.thetimes.co.uk/article/nhs-at-70-your-memories-of-the-national-health-service-xg5rlh56n?shareToken=293212bec002bdb7752eb9ecdd3a5450). Hearing the voices of readers telling their own stories adds impact that a typical written interview wouldn't have. Primal Scream I think shows that impact in a whole new way.

## So how do you do it?

Basically we have a company called [Twilio](https://www.twilio.com/referral/1GyN4i) to thank for this. Twilio bring all that complexity in call-centres and telecoms networks and make it accessible and programmable over simple control panels. Fortunately what we'll be asking it to do is very easy to follow, and you don't need to know any code to get started.

There are three main parts to this project. You're going to need to set up two things: a phone number, and a handler function for calls to that number. And you will need to record one piece of audio to upload that will act as your voicemail greeting.

Let's get going:

1. Sign up to [Twilio](https://www.twilio.com/referral/1GyN4i) here: https://www.twilio.com/referral/1GyN4i [^1]
2. To start, let's get a phone number. From your control panel find 'Programmable Voice' from the overflow menu (pin it for easy use later), and then go to '[Numbers](https://www.twilio.com/console/voice/dashboard)' ![](https://img.tk.gg/chrome_8CETMoCvFK.png)
3. You'll need to upgrade your account by adding a small amount of cash at some point around about here. If you aren't prompted, or can't find out how, there's a guide on how to do that [here](https://support.twilio.com/hc/en-us/articles/223183208-Upgrading-to-a-paid-Twilio-Account). Unless you enable the auto-recharge feature, should you run out of credit, Twilio will simply cease to operate service to you.
4. Find '[Get a number](https://www.twilio.com/console/phone-numbers/search)' and search for a number you can use. Your number will need to support voice calls, so look for the 📞 phone icon.
5. Once you've purchased your number, we need to set up your voicemail. Let's start with the recording. Use any audio recording program, such as [Audacity](https://www.audacityteam.org/download/) or any online tool to get a file that you are happy with.
6. Now go to [Functions->Runtime](https://www.twilio.com/console/functions/overview) in the menu (pin this also) and click `Functions Classic`. The new `Services` feature doesn't support what we need to do yet. Click the red plus and give this any name you want `voicemail-recorder` for example.
7. To upload the file we're going to need to open the menu again and go to `Runtime->Assets`. Select `Assets (Classic)` in the sidebar, and upload your file. Make sure it's public. Once it's uploaded select the URL in the `PATH` field and save it somewhere, you'll need this in a moment.
8. Now back to the `Functions Classic` menu, and open your function. Let's give it a `PATH` like `/voicemail` (this must start with a slash). Underneath in the configuration field make sure `Access Control` is disabled, select the `Event` to be `Incoming Voice Calls`, and then in the `Code` box enter the following. Make sure to put your saved asset URL in!

   ```
   exports.handler = function(context, event, callback) {
       const twiml = new Twilio.twiml.VoiceResponse();
       twiml.play("YOUR FILE URL GOES HERE"); // Put your asset URL in this between the quote marks
       twiml.record({
           timeout: 10, // The number of seconds of silence to wait before hanging up
           transcribe: false, // Change this to true if you want Twilio to try to transcribe your recordings ($)
           maxLength: 300, // This is the maximum length that the audio recording can be.
           playBeep:true // This tells the function to play a bleep after your message or not.
       });
       twiml.hangup();
       callback(null, twiml);
   };
   ```

9. Hit save! Hopefully there are no issues. Now we need to make it so when someone calls your number, this code is run. So go back to the programmable voice menu for your phone number.
10. From here, scroll down to the `Voice & Fax` options. Select `Accept Incoming` and check it's set only to `Voice Calls`. Then where it reads `A call comes in` change the first option in the drop down menu to `Function`, and now in the `Service` menu that appears to the right, select `default`, and from the `Function path` menu, your path.
11. Everything's set! Call your number and check that it's all working. You can find recordings in the `Calls log->Incoming` menu of your phone number.

If you have questions, or can't get it working, give me a [tweet](https://twitter.com/MattieTK) and I'll see if I can help you out.

### Be aware!

It is possible for a hostile actor to attack you by repeatedly calling your number and leaving long messages. Though they will not be able to block legitimate contributors, they can cost you money, and they can cost you time in sifting through the results.

It is also possible for your fun experiment to turn into $1 a month forever unless you turn off your phone number at the end of the project. Make sure to remove any reference to the phone number from your reporting at this point, as otherwise it is possible for another actor to potentially take over the number.

[^1]: This is a referral link where we both receive $10 of credit once you top up


How to create a voicemail inbox for your journalism


In the last post I took you through a short summary of what it took to get Edition Builder live at The Times and Sunday Times. If you haven’t read it, you should, it’s pretty good background.

(In the last post I also mentioned I’d be writing this in a few weeks. That was pre-coronavirus, so these posts have taken a seat at the very back of a very long series of seats. Apologies for that!)

As one of the larger engineering projects we have undertaken, Edition Builder broke new ground in a bunch of areas. We tried things that we hadn’t attempted before, and pushed systems to their limits (and well past them in one incident).

There were a lot of challenges we encountered as a team that we had to work out how to navigate. Though its not likely that you’ll learn a lot about the specific issues we encountered, I hope there are patterns you can recognise that are best avoided, and things you can think about should you work on anything similar.

---

## Searching for articles

Search in Edition Builder was, for most of its life, unworkable. I don’t know why I persisted so long convincing myself that it would work, but it was only ever going to get worse from the moment we realised this problem. Not raising this earlier in the lifecycle of the project was a big mistake from me, I think because I thought it was unsolvable, but as with most things online, someone else has usually solved the problem you are facing, and often they’ve built a SaaS solution that will cost substantially less than your engineering time building and maintaining it.

On to how search was terrible. Our content API was never built for search. This is because of the link between our CMS and our content api was only one way: via publishing. Our readers used a managed SOLR cluster for site search. Editorial users only ever needed to work with content inside the CMS, which had reasonable search and categorisation functionality. In the world of Edition Builder existing, suddenly there was a need to find articles before they had gone through the publishing process.

<ImageCap
  image="https://img.tk.gg/1_EvqKLCBT8axTCVp7OYqgTw.png"
  height={916}
  width={1119}
  caption="All 776 of those articles were stored in full in your browser’s memory, because performance be damned, we needed somewhere to put them!"
/>

“No problem”, we thought, “we’ll set up another version of that API” (because building versioning in to the API was not something we wanted to be responsible for). (and it has ended up coming anyway as part of a later rewrite).

Except of course there was no reasonable way to perform a scan or search as the endpoints just didn’t exist. So in the end we settled for fetching a range of data between two dates, and storing that in the application state to then sift through locally in the browser.

Constructing that index took as long as three seconds per 24hr period searched, so we limited it to three days at a time. Worse: to get new articles, you had to refresh your local index, causing another delay of however long you had selected.

The moment this hit user testing it became clear to us it was a big problem. The search was just too slow. But was it even within our power to fix? We sat on the problem, hoping a solution would rear its head, as every fix we could think of involved work well outside the scope of the project (and adding months to its delivery).

Step forward a couple of months and a couple of members of the team had been tasked with something new. We’d just signed up for [**Algolia**](https://resources.algolia.com/customer-stories/the-times), a SaaS search provider, and needed to get it working for our readers as soon as possible, decommissioning the SOLR cluster as we went.

Algolia is a breeze to work with, and we quickly saw significant improvements on what SOLR had been returning. We feature flagged one engine over the other, and, with journalists being some of the world’s best critics on search engines to surface their own writing, I jerry-rigged [a glitch.com app](https://times-search-test.glitch.me/) that we sent round to find out which performed better.

A few tweaks to the ranking algorithm later, we were satisfied. And then it dawned on us that we could use exactly the same thing in Edition Builder.

Tearing out the old system and replacing it with a prototype took just a few days. Again we played around with ranking and faceting to get what we needed, and then we threw it in front of users to test.
Immediately we saw improvements. A local in-browser index was no-longer necessary. Every query went fresh to the remote index, resulting in more recent results. We could return results almost instantly, just as you typed into the box.

Even better, when users talked about issues they were having with search, they were relatively easy to diagnose. We could replicate the same searches, see how the ranking affected the results, and adjust accordingly, even on a per-query basis.

---

## How we structure edition updates

A second issue we looked to solve focused on how we structured editions. On reflection, I think we could have gone further, and we shouldn’t have shut conversations down so early without seeing them out. Again here there was too much of an assumption we had the best solution at the time, and not enough thought to restructure things early on.

In the previous system, editions over the course of the day were overwritten when the latest update was published. This meant throwing a lot of useful data away, a lot of which we were technically paying for in terms of journalistic decision making. Wouldn’t it potentially be useful to be able to learn about how stories move around the edition, and save different updates separately.

We initially explored a git-style branching system where root editions would have branches with changes on them that then potentially were merged back over time. This had the advantages of immediately making sense to developers, and the disadvantage of it not making any sense to any of our users 😭. Can you imagine playing with git-editions on deadline?!

<ImageCap
  image="https://img.tk.gg/0_fFwub9PiTsadeVGj.jfif"
  height={353}
  width={700}
  caption="How a branching model might have enabled multiple versions to work together with each other"
/>

The ‘git’ style feature would have brought us some level of historical and history data, which we could have used for scrubbing back and forth, but the complexity add at the time, along with the UI challenges of having this work seamlessly for users was problematic.

We set out to solve the problem actually in front of us, and built the simplest solution we could see that did that.

<ImageCap
  image="https://img.tk.gg/0_3blbvJMGpwdg_m0V.jfif"
  height={920}
  width={1920}
  caption="Our old system never stopped being edition A. In the new system we just create a new edition each time that is a copy of the previous one, whilst saving its parent."
/>

Simply enabling users to copy past published editions into future timeslots solved the data problem, and enabled users to work on future editions whilst still being able to make tweaks and changes to past ones.

In hindsight though, it didn’t solve enough of the problem. For example, in a short timeframe you might want to urgently prepare two options for a future version. A direct copy gives you very little flexibility as other stories that might change need to then be made in two places. We thought at the time about creating overlays you could flatten into an edition update, but again it was difficult to work out if the value would ever be delivered.

Looking back I wish we’d thought harder about how sections interacted with editions and made that less fixed. In reality an edition is just a bundle of sections, and making them updatable independently and then bundleable for an update might have both provided an easier structure to manage, and an easier product to break up should we plan to in the future.

The saving grace at least, given the changes made, is that it will be significantly easier to integrate our bespoke analytics service (INCA) into this system, so as to show journalists when they place stories what the story is expected to do based on its position in terms of our key metrics. This will help journalists understand the power of their decision making better (as stories towards the top of the edition have generally higher numbers than those lower, stories in larger slots likewise etc).

---

## Publishing and the initial breakdown

Finally we have the bug that held us up for four months. The lesson: never trust orphaned systems to do anything.

<ImageCap
  image="https://img.tk.gg/1_EvqKLCBT8axTCVp7OYqgTw.png"
  height={916}
  width={1119}
  caption=""
/>

This is the working version (we also added lazy loading… which would have been a really good idea at the start)

Edition Builder has a feature that shows the entire edition to you at once in a single screen. In order to do this we have to fetch all the articles that exist in this view from the API.

In addition, Edition Builder, for the first time, had to handle published and unpublished content (as mentioned in the search feature). This meant that two versions of the same article could exist at the same time, given editorially you might want to feature either (the unadjusted published version, or the perhaps more-up-to-date unpublished version).

The consequence being this view needed to switch from two sourcing content APIs, the in-production API, and our preview API, depending on the article.

And when we published articles, we switched any reference for this lookup from the preview version to the production version. Simple!
When it came to running our first live editions everything went really smoothly. We worked our way back from the 5pm edition to the 9am, in order to have the least impact in the most supportable working hours, and finally had the midnight edition in our sights.

<ImageCap
  image="https://img.tk.gg/1_uwg3a5UmyrFOBBWMD1stYw.png"
  height={226}
  width={564}
  caption="I don’t always @channel, but when I do it’s the best @channel in the world"
/>

Where the 9am, 12pm, and 5pm editions are relatively small updates on the main bulk, the midnight edition publishes several hundred articles all at once. For architecture that was not necessarily built for this in mind, bursts like this can be fatal… which we learned to our peril.

<ImageCap
  image="https://img.tk.gg/1_khy5fJZSN2geytRWwYSlEw.png"
  height={184}
  width={414}
  caption="😰 👉⏺"
/>

<ImageCap
  image="https://img.tk.gg/1_Ljto-7_OnPus-WRFdeJJjA.png"
  height={428}
  width={546}
  caption="The as-promised ‘crippling load’ 🔥"
/>

When Edition Builder publishes the midnight edition, all of these articles get posted into the content API in the same queue. This usually causes a little bit of a spike.

But Edition Builders incredibly smart and well-meaning feature that showed you the entire edition after publish, then immediately tried to load these hundreds of articles from the same system that was trying desperately to ingest them.

And what’s more, in order to check if there were any changes, it reloaded them every thirty seconds.

Cue heavy breathing from our operations team as our apps started to intermittently return 500 errors. And by _intermittently_, I mean _all the time for about fifteen minutes_. 😬

<ImageCap
  image="https://img.tk.gg/1_g7nffKQpmVrMsYqJmw7veg.png"
  height={1588}
  width={2048}
  caption="Oh that does not look good."
/>

...

Fixing this was no mean feat. The content API at this stage was practically orphaned, with a replacement on the way, but with no clear end in sight.

What’s more, this problem only occurred in production, and we couldn’t work out how to replicate it on our pre-production instances, which all handled the load fine (perhaps because it was the only load they were seeing 👀).

We, sensibly, were not allowed to continue our production rollout whilst this error still occurred, but we couldn’t replicate the error to show it was solved, so our catch-22 just kept enumerating.

After far too long, our solution came in the form of new talent. Stefano Berdusco joined me on the long road to salvation and together we wrote load tests (he wrote them), we built a perfect replica of production (he build it), and we flipped the sourcing to the new API, so as not to cripple the old one (maybe my idea, but given my track record probably also his).

😍 Our tests, with load applied, showed the new API (built with this burst in mind) was able to easily handle the preview state. And we were back on track.

So that’s that. Thanks for reading, I hope we all learned something.
(Also in going through our Slack logs for this post, perhaps my favourite short-lived bug below)

<ImageCap
  image="https://img.tk.gg/1_Rl8YBk2tT0ySbm0vTKMmlA.png"
  height={123}
  width={557}
  caption="❤️ this team"
/>


What we learned from rebuilding the editions CMS for The Times


Internal tools never get enough attention.

Building them can be a thankless, unseen task that’s rarely prioritised, as it is not immediately obvious how to tie investment in them back to the bottom line. It can be too easy to jump on the customer-facing speedboat and jet away from them, leaving a wake of stopgaps that can often leave you with tightly coupled intricate workarounds that are enormously difficult to pick apart.

This was the case at [_The Times_](https://www.thetimes.co.uk) for the last four years. While we spent a long time discussing and designing how editions might work on our readers’ screens, we didn’t place any focus on how to make building these experiences manageable for our editors. We quickly shoehorned the process into our generic digital CMS, making it difficult to maintain or build upon, and creating a process for our editors that took much longer than it should have done.

**Edition Builder** was our solution to this problem, and has been two years in the making. At its heart, it is a simple drag and drop interface that allows you to place and publish stories in our edition. Beneath the surface is a huge amount of user research and UX refinement and, crucially, the foundations for future newsroom tools.

Late last year _The Times_ went live, and this week _The Sunday Times_ joined them, and now all editions from _The Times_ and _The Sunday Times_ are produced on Edition Builder. I’d love to tell you more about the story of how we got here and what it means for the future.

<ImageCap
  height={300}
  width={300}
  image="https://img.tk.gg/newspaper-bundle.png"
  caption="Editions replicate the print bundle in a digital form: timely, broad, and finite."
  invert={1}
/>

## From prototype to production

I’m well known as someone who is not easily impressed, but from day one editions excited me: the idea of publishing fewer stories, of higher quality, in a bundled format was novel. However, the way we produced them irked me. Our editors were fighting with the system to make the most basic changes. The more time went on, the more we wanted to do with editions and the more painful it came to be.

<ImageCap
  image="https://img.tk.gg/eb-prototype.png"
  caption="The first Edition Builder prototype showing the menu to create article containers."
/>

In a break from our day-to-day jobs supporting editorial, Chris Hutchinson (now a Principal Engineer at _The Times_) and I took some time out with the implicit instruction to build something cool that we thought needed to exist. We only had a week, and we went slightly over schedule, but we knew from the start we wanted to solve our edition curation problem.

The value we were able to provide from our experience understanding the editorial pain-points, being able to theorise a solution, and eventually prototyping it wouldn’t have happened outside of our cross-functional editorial development team. These groups of product and engineering-focused, journalism-obsessed individuals that have sprung up at news orgs all around the world are, in my opinion, some of the most valuable people in the business.

Our team’s dream was to empower the newsroom to do much more with editions, faster and more intuitively than before, but the current monolithic CMS made this more difficult than it should have been.

Edition Builder’s prototype came out of that week, and our demo roadshow made the rounds through the technology and editorial departments. There was ambition to build more. Finally, after a presentation that included way too many Jurassic Park gifs for the seniority of the audience, we got there, and the investment case was signed off.

<ImageCap
  image="https://img.tk.gg/methode-is-gross.png"
  caption="Our first challenge was working out what features from the old system were useful, and which were not. This is our old edition plan, built into the same system that made our printed product."
/>

We formed a team of six engineers, a user-experience designer, and myself, acting as the product manager. As a new team, with a clear purpose, we dived in and (of course) began making mistakes.

It quickly became apparent that some of our assumptions were misplaced, or just wrong. We designed an introduction screen that we thought made it clear which edition was live at any one time: nobody from editorial understood it. We thought that previewing articles was something people did habitually but didn’t really need: it turns out they did. And we thought, most wrongly, that getting articles out of the system, and publishing our eventual output would be the easy bit: publishing functionality alone took six months to build.

Inevitably there are more unknowns than you realise entering a project like this. Our objective was very clear: build a slimmer, faster, more efficient, and more extensible system for editions; and sliced up, the journey ahead looked manageable. But between the cracks were services that hadn’t seen new deployments in months, orphaned systems held together with the code equivalent of a low-budget PVA glue, and teams across the business skittish and burnt from too many lookalike projects gone wrong.

This is mostly why it took just over two years to bring Edition Builder into production. When a single tool is responsible for a disproportionate amount of your output, replacing it becomes a project of either changing what you do with it, or covering off every feature it has.

<ImageCap
  image="https://img.tk.gg/open-articles-prototype.jpeg"
  caption="Making it so that users could open up articles in the original editing system was important. This was the initial paper prototype I threw together."
/>

Making it so that users could open up articles in the original editing system was important. This was the initial paper prototype I threw together.

There was no logical physical analogy that didn’t sound absurdly risky: like shifting your entire printing press all at once to another that has identical output, but also looks and works completely differently.

Getting editorial users on side was a crucial part of this project from the start. Over time we probably ended up building a few small features that weren’t needed (and we are now in the process of gradually removing), but the goodwill gained from these relatively small features greatly outweighed the development cost.

That established trust was important to move forward with some bigger work items without consulting users every step of the way, whilst working more closely with them on items we knew they cared about. It meant we slowly built up a relationship where we felt aligned in the combined goal of transforming this workflow to make a better product, rather than being forced to replicate every step for a feeling of safety and familiarity.

Ultimately this all paid off. Even though it took a little longer than expected, and the team working on it changed in its entirety when we hit a particularly sticky deployment issue towards the end. Some of our heaviest editorial users have given us rave reviews: it is “far faster", the time savings “impressive” and the rollout “exemplary”.

In our prototype, we worked out a time saving of approximately 5x on doing some basic actions. Edition Builder in production is in some cases 50–100x faster than the previous workflow to perform the same task. We really should hold everything used this regularly to the same standards.

With that I’d like to thank the people on the team that brought us to where we are now: Himesh Ladva, Tom Chambers, Elliot Davies, Cassie Best, Leah Haskoylu, Chris Hutchinson, James Fuller, Geoff Ford, Chris Jordan, Stefano Berdusco, Jack Stevens, Tom Hoad and Adele Kufour.👏

I’d also like to thank everyone in the newsroom that worked with us to help develop the tool and get it into production, including Wes Rock, Gary Mitchell, Abhi Ahluwalia, George Lindsay-Watson, David Rankin, Sadie Gray, Hannah Rock, Hannah Scott, and Michael Brear.

<ImageCap
  image="https://img.tk.gg/eb-early.png"
  caption="An early build of the live system with a simple article search loaded into the browser’s memory."
/>

Read [part two](/posts/how-not-to-launch-your-cms) here, and learn the lessons we learned putting this system together.

And, enjoy last weekend’s _The Sunday Times_; the first Sunday edition made with Edition Builder. 🎉

<ImageCap image="https://img.tk.gg/first-eb-st.png" />


LFT notifications in your browser

Apple's Mail Privacy Protection is lazy and hurts small publishers

A retrospective on five years of digital editions

How to create a voicemail inbox for your journalism

What we learned from rebuilding the editions CMS for The Times

Building Editions at The Times