"Hope for the best, plan for the worst" - this quote is from a series of books by Lee Child where the protagonist Jack Reacher gets caught up in far-fetched conspiracies and manages to save the day every. single. time. But let’s not shoot down the wisdom of this maxim because of its source. If a more cultured source suits you, Maya Angelou said something to the same effect: “Hoping for the best, prepared for the worst, and unsurprised by anything in between”.

How does this relate to software products?

Software teams constantly have to deal with risky scenarios on a daily basis, but especially so before a first-time launch of a feature or product. A lot has to come together at the same time and it can easily feel overwhelming.

The whole team starts battling with fears such as:

• Will we break the DB if we roll out this set of data migrations to enable this new feature?
• Will we run into scaling issues on our servers if we launch the product the way it’s been built?
• Will people hate using our app if we don’t address these critical UX issues?
• We’re still finding loads of bugs, how can we possibly ship soon?

These kind of risks and fears can be dealt with quite differently based on the context of the team. In a startup, the team is keenly aware of the dwindling runway and have less to lose. In a more established context, avoiding risk can easily become the status quo and the team does everything to avoid risk as much as possible. This frustrates everyone involved because it’s slows us down so much and prevents us from iterating quickly on our products.

When we are working in a team with an overly high risk tolerance, we tend to cowboy certain concerns and just hope for the best. When in this mode, we say things like:

• It doesn’t matter because it’s an MVP and we don’t know if anyone is going to use it
• That’s an edge case, so we shouldn’t worry about it
• We’re supposed to move quickly and break things, not get stuck on the details

As with all things in life, the sweet spot is not in the extremes, but somewhere in the middle. We want to identify risks and try to mitigate them while still courageously moving forward and launching our products. We can’t wait for a risk-free launch, it doesn’t exist. We also can’t ignore risk and be so naive as to believe that everything will just work out for the best.

What can we do to live out this “hope for the best, plan for the worst” maxim?

Start by holding a pre-mortem with the whole team

Although useful for frequent risk identification, a pre-mortem is especially relevant when you are getting close to launching a new product or new feature.

The wikipedia entry for pre-mortems explains it really well - here’s an extract:

A pre-mortem, or premortem, is a managerial strategy in which a project team imagines that a project or organization has failed, and then works backward to determine what potentially could lead to the failure of the project or organization.

Pre-mortems are useful for it’s primary purpose (identifying risk) but I also found it to be really useful to alleviate the stress that the team experiences while the launch date is drawing near. The team can easily feel overwhelmed with everything that still needs to be done, all while it feels like everything is falling apart as new bugs are continually discovered.

Getting our biggest fears for the launch out in the open makes us feel better because we realize that our team members shared most of our concerns. We realize we’re not alone. On top of this, seeing a fear written down makes it lose some of its power. It becomes something we can discuss and break down.

Sometimes we find that the fear has no basis and that it’s already a non-issue or easy to mitigate. Other times, we see that some fears are justified and that we need to do some heavy lifting or manoeuvring to avoid the situation where the worst comes to the worst. Even this makes people feel better because they are addressing the problem. Software people love solving problems!

Dealing with the risks identified during the pre-mortem

Once you have your list of risks, work with your team to identify ways to mitigate the risk as cheaply as possible. You should be looking for solutions that are cheap, buy you a bit of time, and that addresses the risk enough for you to launch the product.

Try to avoid pushing out the launch date in order to deal with the issues “properly”. This is a trap. By the time you’ve addressed these issues “properly”, you’ll have new risks causing you to delay your launch yet again.

Software is messy and systems and products are temporary. The time spent on perfecting some rough edges on a product before getting data from its performance in the wild is, more often than not, time wasted. You might be polishing the part of the product that nobody ever uses. Or a competitor launches before you and start iterating on real customer feedback, pulling so far ahead you that you can’t catch up.

Add a label to the work items that you capture on your work tracking system (jira, monday, trello, etc). Make these items the highest priority. You can’t ship without addressing these items, so they have to be fixed first. It’ll be good for the team’s morale to work on them first and it will get you in a launch-able position much sooner.

Common solutions for launch risks

This is by no way meant to be an exhaustive list, that would be an impossible feat. But these approaches have helped me launch several scary products and features.

Architecture review

You might be worried sick that you took too many shortcuts or that you are too close to the problem domain to spot glaring issues. Get some battle-tested technical leaders, inside or outside of your organization, and share your architecture with them.

We are our own harshest critics. I’ve been surprised many times that “external’ folks that were not part of our project are very forgiving and supportive about our solution. This approach not only makes you feel more courageous for your launch, it also might highlight any glaring issues that you should address before launching.

It’s best to get a review on your architecture proactively and not the last minute before launch. But if you are about to launch your product and you haven’t done this yet, it’s not too late. Get the feedback but be thick-skinned. Don’t let the smallest critique push you out of your trajectory. Weigh all the feedback up and make 100% sure you understand the trade-offs. After collating your action points on all the items raised during this review, feed it back to the reviewers to ensure that you didn’t misunderstand or that your actions are not misguided. You should be able to clearly communicate your actions and decisions because you will need to do it again and again.

Gradual roll-out

You have some rough edges in your UI and you are worried about unwanted side-effects on your existing systems.

Don’t commit to rolling out the new product to all your customers! Rather roll it out gradually using some sort of feature-flag management system. Even if you aren’t working with an established customer base, you can choose to enrol only a small amount of people into your real product while keeping it closed for the majority of people ending up on your site.

Once it’s confirmed that the product and systems are stable enough, gradually open up the taps to more people.

Rollback

It’s super important to have a rollback in your launch game plan if you are worried about adverse side-effects in your existing ecosystem. Your ecosystem might be so complex that it might warrant a rollback of the product if it causes harm to the ecosystem when real customers come online.

Have the self-destruct button built, it doesn’t mean it will be necessary to push it. It’s also good for team morale to know that this exists!

Scaling

Building a first iteration of a product in a way that it can be scaled horizontally (across nodes) can sometimes be expensive and adds excessive delay before you can get real customer feedback.

On the other hand, you can’t just hope for the best. Your product might grow organically or you might have large spikes in traffic that will bring your system to its knees.

My advice here is probably a bit controversial. Don’t worry about infinitely scaling horizontally (cost-effective), rather ensure that you can scale up (expensive). You want to test early and get on the feedback train as soon as possible. If it’s possible to scale up initially and defer the cost of making it horizontally scalable to the future without an inordinate amount of technical debt - do it! Your engineering hours will be much more expensive than paying for a stronger DB server or more instances to run you code to serve your spikes (if it ever gets to that point).

Backup/Restore

If you are rolling out a risky feature or a risky upgrade, make sure that your backup/restore process is actually working and that you can roll back to a safe point if you need to. Mostly we work with migrations and other mechanisms to ensure that we can roll back, but there is nothing wrong in checking that your brute force backup/restore process is in good order and functional.

Hardening up to critical feedback

Not everyone will love your product and that’s totally fine! Bring this out in the open and shine a 1000 watt light on it. It’s totally okay to get feedback. If there are some holes in your product that you are aware of but you have a good enough version to release, do it. It’s oddly empowering when the feedback starts rolling in and validating some of the things that you were concerned about. Rather than trying to address all issues in order to avoid criticism, accept that it will happen and push the launch button.

Last thoughts

Launching a new software product or feature can be super scary but it’s also one of the most rewarding feelings you can obtain working in a software team. Try and enjoy it as much as possible and think back to the previous times that you went through it and survived! 🚀