The Rewrite Trap

Rate this content
Bookmark

Let's throw away everything and start fresh. Sounds great, right? While this can feel very good it rarely speeds up anything. I'll show you why a complete rewrite is usually not what you want.

22 min
09 Mar, 2023

AI Generated Video Summary

The Talk discusses the 'rewrite trap' in software development, where the urge to start from scratch often leads to poor outcomes. It emphasizes the importance of understanding the existing project before making big technical decisions and the benefits of gradual improvement. The Talk also highlights the challenges and pitfalls of a complete rewrite, such as false sense of productivity, problems with edge cases, and the accumulation of tech debt. It stresses the need to understand the system and its influences, cater to the needs of stakeholders outside of engineering, and focus on creating value.

1. The Rewrite Trap

Short description:

Hello, everyone. My name's Phil. I'm a software engineer at Brighter. Today I'd like to talk to you about a thing I call the rewrite trap. The urge to start from scratch often comes from not wanting to learn new concepts, not aligning with the existing architecture, and the desire to work in a way that aligns with personal preferences. However, these reasons rarely pay off in the long run. It's important to take the time to understand the existing project before making any big technical decisions. The trap lies in the belief that a complete rewrite will lead to better outcomes, when in reality, gradual improvement is often a more effective approach.

Hello, everyone. My name's Phil. I'm a software engineer at Brighter. I've been a tech lead before and also a CTO, even though I got fired from that job, more on that later.

And today I'd like to talk to you about a thing I call the rewrite trap. And first, before we start, let me set the stage. So what kind of rewrite are we talking about? It's definitely not the one where you know the project for months or years. You're super familiar with all the ins and outs, the tech, the architectural decisions, all of that, and you're absolutely sure what you need to do in order to get this, you know, off the ground, right? Maybe even there is a new project, right? And you actively make the decision to not stick with the current stack or the legacy platform or whatever, but start from scratch, right? This is not what this talk is going to be about, right? Because I think the read-write or the starting from scratch might be the best option. I'm talking about the situations where you, you know, might be the new tech lead in town, right? Or the new CTO in town. You join a company that has an existing product. There is a stack, there is software there. And you kind of have that feeling that after, you know, looking at it for, I don't know, one or two days, you just think that it's better off, you know, throwing away everything and starting over, right? This is what this talk is going to be about, right?

So, let's start with where does that urge come from? And I think there are three main reasons. The first one is you don't need to learn new concepts, right? So, let's for instance say it's a client application, it's written in React, but you, you know, you like Vue or you like Svelte or whatever better, right? And getting into React, you immediately have that repulsive feeling that you really don't want to get into this, right? So, if you throw it away and rewrite it in your stack of choice, then, you know, that would make that problem go away. It's the same for, you know, programming languages, methodologies, whatever, right? So, everything that you're not used to you kind of immediately say maybe let's not do that. The second thing also is someone else built that, right? So, if you really like a certain architecture, let's say for instance you really dig functional programming, but the current product is built in a very object oriented fashion, then, ha, you know, those two worlds might not align so well, right? And, it always, you know, would feel like, you know, a hurdle to get acquainted with everything, right? And, you might already notice there's a pattern here. And, the pattern is not that necessarily, you know, tech choices, programming languages choices, architecture choices are better or worse. It's just that they're different and that, you know, intrinsically, if something is different to what we prefer, we, you know, have to fight that urge to just, you know, throw it away and, you know, turn it into something that we like. And a colleague of mine once gave me the advice to wait 100 days until you make any bigger technical decisions once you join a new job, right? Because, in 100 days, you can learn a lot and also, you can figure out whether something, you know, you know, a choice from the past is actually worse than what you wanted to, you know, wanted to use. Like, is it, you know, a problem that it's object oriented? Should it be functional? Or whether this was just, you know, your initial gut reaction to a certain thing and you just didn't like it, you know, before, but probably 100 days in, you also got used to it and then maybe it's not that bad anymore. Personally, I have to admit, I always, shortened that period to 50 days. Because especially when I was CTO, I couldn't convince my CEO to just wait, you know, almost a third of a year before I started making bigger choices. Let's just start up life and you gotta, you know, do what makes sense at the time.

Now let's get to the third part. And that's the biggest egomaniac part of it, right? If you just throw away everything, you can essentially work the way you want, right? You make up all the rules. So this definitely feel comfortable, right? The big question to ask here is just, you know, will that, you know, is that, is that something that makes sense in the long run, right? Is your way of working, for instance, compatible with how the company works? Right. And yeah, you know, all of these things that I just outlined, kind of feel initially, maybe like good reasons to start from scratch, right. And we can definitely talk ourselves into, you know, assume benefits of either one of these. But the question is, do they really pay off? And I think they rarely do. And this is also where the trap comes in because initially, if we look at these projects, so I've drawn two made up lines, essentially, you know, the green line representing a, you know, existing software project and you don't rewrite it, but you try to gradually improve it, you know, work your way through the project and you know, make it better. And the purple line represents the complete rewrite and the X's are just time on the X axis and the Y axis is outcome.

2. The Rewrite Trap: False Sense of Productivity

Short description:

If you start a rewrite, nothing's there, nothing's holding you back. You can push out a lot of stuff in the beginning really fast. The people who stick with the existing code and try to gradually make it better first have to go through the weeds, learn the architectural choices, and gradually improve the running system. That false sense of productivity in the beginning leads people to making those rewrite choices.

Like how much can you actually achieve? And obviously if you start a rewrite, nothing's there, nothing's holding you back. You can push out a lot of stuff in the beginning really fast. So you could even then fool yourself into thinking, wow, this must have been the better choice. We're so fast. Everything's moving fast. That's great, right? Well, you know, the people who would stick with the existing code and try to gradually make it better first, you have to go through the weeds, learn the architectural choices, you know, see what's there and gradually improve the running system, essentially. And this is where the trap comes in, really, you know, that false sense of productivity in the beginning, is what I think leads people to making those rewrite choices. And I give you a hint, maybe it's also good for your career to do this, right? Because like, I don't know how long, you know, the phases of one, two, and three are. But if you leave in phase one, or at the end of phase one, all you've done so far seems very productive. You've done a great job. So you might be the, you know, the really good tech lead that just gets things done and that just does the things no one else wanted to do. And it helps you progress, you know, maybe do that if it's good for you, but I am not that person.

3. Phase One: Fast Start and Initial Progress

Short description:

Let's start with phase one of the rewrite. In this phase, you quickly develop the first features and basic functionality. While the initial progress may seem slow for those working with the existing code, it's important to understand the system's intricacies and edge cases. The rewrite feels productive, but progress may not be immediately visible.

But, you know, let's look at all these things in detail and start with phase one. So I've already said the rewrite will be, you know, off the ground really fast. You crank out the first features. You do the basic stuff, I don't know, you know, have a signup flow, have the first big features going. Everything's rough because you work iteratively and, you know, do all the fluff and you can have always something to present to people. Whereas the people who stick with the existing code probably have little to present because they need to understand everything that's there. They need to figure out, okay, what belongs together? How do things really work? Where are the edge cases and all of this stuff? So in phase one, the rewrite feels good and you crank out stuff, improving what is there, doesn't feel so good because it feels like you're making no progress at all.

4. Phase Two: Problems and Tech Debt

Short description:

Phase two of the rewrite disregards the problems and edge cases of the old system. Assumptions made during the rewrite don't fit well with these cases, leading to rework and the accumulation of tech debt. Progress slows down, outcomes suffer, and the system becomes a mess.

These things start to look different if you enter phase two because what the rewrite completely disregarded was, you know, the problems and the edge cases that people have found before. That big legacy code base probably wasn't that complicated, just, you know, out of sheer coincidence, but probably for a reason. So our big rewrite at some point will hit that part where the edge cases come in, the quirks come in and you will notice that since you didn't learn about all these things upfront and you didn't even bother to understand why the old system became the way it was. You made a lot of assumptions, building out your new software that just don't really fit that well with, you know, those edge cases and the other requirements you have. So you got to start reworking stuff. You got to somehow need to crank in other things. But you know, since you've been so productive before, right. People have that assumption of, you know, like, you know, stuff's gonna move fast and they wanted to move fast and eventually you'll end up adding tech debt again, right. You got to, you know, cut corners, try to keep that momentum moving, but eventually what will happen is your progress will slow down, outcomes will go down and yeah, you just gotta navigate yourself into essentially the same mess that the old legacy system was before.

5. The Rewrite Trap: Understanding and Improving

Short description:

In phase two, the initial efforts of understanding and gradually improving the existing system start paying off. The big rewrite in phase three becomes the new legacy system, leading to a halt in future delivery due to accumulated tech debt. Sticking with the existing system and making gradual improvements allows for a better understanding of the software and the ability to make important changes. It's important to defer judgment when joining a new project and learn why things are the way they are. Embrace the pieces you don't understand and have discussions to gain insight.

Whereas the people who, you know, took the time to really understand what's there and figure out how to make it better and improve it gradually. In phase two, these initial efforts will start paying off because by reducing complexity over time by really, you know, doing the necessary refactors in a way that, you know, matches the system, they can actually start untangling stuff, maybe, you know, pulling monoliths apart into clear modules with good cohesion, you know, not like decouple the system where everything's coupled to everything. And they will actually start getting some momentum now, right? So it's going to be faster to add new things, change things because the people working on the project or product will now know better what's in there and how things work. Right. And this leads us to phase three. In phase three, essentially the big rewrite is now the new legacy system, right? Because you didn't bother to, you know, understand stuff and because you try to keep your momentum up and navigated yourself also into a situation where the outside world kind of expected features fast from you, you've built up so much tech debt that now future delivery kind of grinds to a halt. And, um, I mean, if you're lucky, you've left the project before, uh, if you're not lucky, then probably this is also the time where some other tech lead will replace you, right? And the chances are that the story will then just repeat with another rewrite. Right. But the people again, who, you know, stuck with what was there and gradually improved it might now just reap the big benefits, right? They might not be in a state where the software is just, it's pretty good. They understand like a super good understanding of what's going on and they can just, you know, make changes in the parts, um, that are really important. And that, uh, and also I built, now build out an architecture that supports those changes and supports also the software for, you know, whatever purpose it was intended to do. Um, and yeah, this is essentially the trap here, right? That the initial euphoria around rewrite kind of leads to its demise and then to the next, um, rewrite. And once again, some, um, career advice, you know, try to leave at the end of stage one, because then you're the cool person who just got things done. Um, and everything that happens after, uh, is then mostly accounted to, you know, you not being there anymore. I think, Oh, now my name still. So now Phil's gone. So, uh, the moment he left everything just went to pieces. Right. But you know that now know wasn't really the case. Right. But, um, you can use that if you're, you know, I don't say you should use that, but you could. So let's look at what should we do instead? Right. And I kinda outlined this already, uh, and, uh, before, but once again, you know, three key things. First of all, if you join a new project, try to defer judgment as much as possible, right? Always assume that the people who worked on the product before worked with the best intentions and just, you know, made the best choices they could at any given point in time. Right. Um, we all know that, you know, uh, you know, the best choice with the best knowledge in, you know, five months ago is probably not the best choice now. Um, but this kind of sets the stage for a climate where you want to learn where you want to understand why certain things are the way they are. Uh, and you need to do this to actually really improve them, right? Because if you throw out something just because you don't like it, well, you're probably prone to make the same mistakes again, right? So learning, talking to people and understanding why things are a certain way is, uh, the first key takeaway here. Second thing is that you need to understand or like there, there are going to be pieces that you don't understand, right? Uh, and that's okay, right? Some stuff might just seem super weird and you can't come up with any good explanation of why would it be this way? And this is where you need to go in and really have discussions with people and say, you know, I, you know, and be, you know, also be clear about what you don't understand, what you, you know, what you think how it should be. And that you'd like to learn, you know, why it's not a certain way, no, in some situations or in a lot of situations, probably people will tell you that we just didn't have the time to do it.

6. Understanding the System and Its Influences

Short description:

The system around developers often doesn't provide the support they need to build the right thing. As a leader, it's your responsibility to create a system where they can do their best work. Understanding the company's dynamics and talking to people outside of engineering is crucial. Sales, marketing, and other factors influence the system. It's important to cater to their needs and not just focus on technical improvements. Ignoring the existing system and solely focusing on tech improvements can lead to failure and being replaced by someone who does a complete rewrite.

Probably we know that we should have done it like this, but well, you know, um, the deadline was approaching. Um, but these discussions help you get along with people also get an understanding and probably also you learn that people really, we really have a desire to build the right thing, but you know, the system around them just didn't give them the support they needed to, to actually do it. And, you know, as a leader, tech lead, CTO, whatever, um, this is now your call, essentially, you now need to make sure that you create such a system for them, where they can do the proper work. Right.

And sometimes you also, uh, you know, should also talk to people outside the development area, right? Talk to sales, talk to marketing because, you know, chances are, they also had an influence on your current architecture and you need to understand why certain things were important, right? If you're in a startup, you know, money needs to get, come in, right? So sometimes you got to, you know, crank out the one thing that just going to sign the next deal, because otherwise the company wouldn't exist anymore. Um, these are all good reasons to make, um, some shortcuts in tech. Uh, I didn't really want to believe that early in my career. Now I think I understand some sales people much better. And this is also the third thing we get to know the system, right? You're a group of people and engineering or R&D department inside a company is not its own universe. There are factors around it that influence the system. Um, the CEO, sales, marketing, you know, all, all, all of them essentially combined. And you need to understand how the company works in order to navigate it in a way that you can actually, um, achieve, you know, success. And, uh, this is not a, an easy feat. This also requires talking a lot also to people outside of, um, uh, outside of engineering and also understanding their needs. Like, what do people need? And I'll give you the example for instance, you know, when I joined a CTO, um, you know, I couldn't believe what it was seeing, right? There were no tests, there was no definition of what the product was supposed to do. Um, we didn't have any error reporting. Um, and, and the CEO was always asking me for a plan for when the features are done. Like, you know, I can't like, I don't know, even know what's there. So I can't tell you anything about when the next thing will be done. And I tried to do a lot of explaining and essentially it didn't work out in the end. But this is also me not completely acknowledging the system there, right? Because that the system with the existing CEO, the existing customer, the chief customer officer kind of led to this, right? And, and I didn't bother understanding exactly what their needs are and trying to also always cater for that. I went in there and just with a complete tech kind of, do you want all of that, right? To say, okay, I know all of this is bad. We need to do this, right? We need to add monitoring. We need to add some tests. We need to add, you know, some automation around releases. And then we can just work our way through the weeds. And essentially I also actively did not rewrite, but improved what was there. And with that essentially failed in their view because it was really slow and essentially it, you know, I got fired when the curve just started tipping up when we, you know, didn't have more bugs in then, uh, we could actually fix or when we actually started delivering features on a predictable schedule or more predictable schedule, um, where then funny enough, someone else came in and the CEO hired someone who did a complete rewrite and they were super fast at fast. And then I essentially got the news, said, you know, look at them in, in a month they've built that. And in three quarters of a year, you didn't achieve that.

7. The Importance of Understanding and Creating Value

Short description:

Get to know the system and the needs of people to cater to their respective needs. Focus on what matters and creates value. Step out of your comfort zone and understand the needs of people who are foreign to you. Learning is a crucial part of the job. Rebuild the parts that truly matter and create value for your customers. There is no standard rule for finding these parts, but trust yourself to do so.

So you are out, you know, understandable decision now. Um, um, but still, right, this is what I mean. Get to know the system, get to know the needs of people so that you can also cater for the respective needs that people have and give them a proper story, right? You know how, or you know how I think you should work, but you still need to create a story for all the other people, um, that, that they also feel comfortable with that approach.

So in essence, the conclusion is do what matters, right? Um, don't try to be busy, um, but you know, focus on the things that actually, you know, create value, right? You got to move out of your comfort zone. And especially with that, I mean, you got to talk to a lot of people and, uh, you gotta try to also understand needs of people that are so foreign to you. Um, for me personally, it was always sales, like, how does this this is a complete different world. Um, you need to acknowledge that you just don't have all the answers. You don't know everything. And that's fine. Learning is, you know, as you probably know, the biggest part of the job, uh, and you need to, you know, you're gonna have to learn a lot more.

Okay. And in the end, once you've figured all of that out, you need to rebuild the parts that really matter. And this is a talk in itself. And you know, you're lucky because I did that talk already. It's called, uh, embrace legacy. You know, if you want to watch it, you find it on my blog, uh, philbeazit.com, um, okay. Sponsorship segment over, but really, right. So a lot of the things that you want to redo in the beginning are just not important. Right? Um, the parts that really matter all the parts that create value for your customers and also creating value for our customers could mean that essentially make you faster and make you or enable you to deliver high quality software faster. Right. And, um, there is no standard rule how to find them, but I guess you will do this. I trust you. And with that, I'll send you off. So this was the rewrite trap. My name is Phil Giese.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TechLead Conference 2023TechLead Conference 2023
35 min
A Framework for Managing Technical Debt
Let’s face it: technical debt is inevitable and rewriting your code every 6 months is not an option. Refactoring is a complex topic that doesn't have a one-size-fits-all solution. Frontend applications are particularly sensitive because of frequent requirements and user flows changes. New abstractions, updated patterns and cleaning up those old functions - it all sounds great on paper, but it often fails in practice: todos accumulate, tickets end up rotting in the backlog and legacy code crops up in every corner of your codebase. So a process of continuous refactoring is the only weapon you have against tech debt.In the past three years, I’ve been exploring different strategies and processes for refactoring code. In this talk I will describe the key components of a framework for tackling refactoring and I will share some of the learnings accumulated along the way. Hopefully, this will help you in your quest of improving the code quality of your codebases.

React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
React Day Berlin 2022React Day Berlin 2022
29 min
Fighting Technical Debt With Continuous Refactoring
Let’s face it: technical debt is inevitable and rewriting your code every 6 months is not an option. Refactoring is a complex topic that doesn't have a one-size-fits-all solution. Frontend applications are particularly sensitive because of frequent requirements and user flows changes. New abstractions, updated patterns and cleaning up those old functions - it all sounds great on paper, but it often fails in practice: todos accumulate, tickets end up rotting in the backlog and legacy code crops up in every corner of your codebase. So a process of continuous refactoring is the only weapon you have against tech debt. In the past three years, I’ve been exploring different strategies and processes for refactoring code. In this talk I will describe the key components of a framework for tackling refactoring and I will share some of the learnings accumulated along the way. Hopefully, this will help you in your quest of improving the code quality of your codebases.
React Advanced Conference 2022React Advanced Conference 2022
22 min
Monolith to Micro-Frontends
Many companies worldwide are considering adopting Micro-Frontends to improve business agility and scale, however, there are many unknowns when it comes to what the migration path looks like in practice. In this talk, I will discuss the steps required to successfully migrate a monolithic React Application into a more modular decoupled frontend architecture.
React Advanced Conference 2023React Advanced Conference 2023
22 min
Power Fixing React Performance Woes
Next.js and other wrapping React frameworks provide great power in building larger applications. But with great power comes great performance responsibility - and if you don’t pay attention, it’s easy to add multiple seconds of loading penalty on all of your pages. Eek! Let’s walk through a case study of how a few hours of performance debugging improved both load and parse times for the Centered app by several hundred percent each. We’ll learn not just why those performance problems happen, but how to diagnose and fix them. Hooray, performance! ⚡️
React Summit 2023React Summit 2023
24 min
Video Editing in the Browser
Video editing is a booming market with influencers being all the rage with Reels, TikTok, Youtube. Did you know that browsers now have all the APIs to do video editing in the browser? In this talk I'm going to give you a primer on how video encoding works and how to make it work within the browser. Spoiler, it's not trivial!

Workshops on related topic

React Summit Remote Edition 2021React Summit Remote Edition 2021
87 min
Building a Shopify App with React & Node
WorkshopFree
Shopify merchants have a diverse set of needs, and developers have a unique opportunity to meet those needs building apps. Building an app can be tough work but Shopify has created a set of tools and resources to help you build out a seamless app experience as quickly as possible. Get hands on experience building an embedded Shopify app using the Shopify App CLI, Polaris and Shopify App Bridge.We’ll show you how to create an app that accesses information from a development store and can run in your local environment.
JSNation 2022JSNation 2022
41 min
Build a chat room with Appwrite and React
WorkshopFree
API's/Backends are difficult and we need websockets. You will be using VS Code as your editor, Parcel.js, Chakra-ui, React, React Icons, and Appwrite. By the end of this workshop, you will have the knowledge to build a real-time app using Appwrite and zero API development. Follow along and you'll have an awesome chat app to show off!
GraphQL Galaxy 2021GraphQL Galaxy 2021
164 min
Hard GraphQL Problems at Shopify
WorkshopFree
At Shopify scale, we solve some pretty hard problems. In this workshop, five different speakers will outline some of the challenges we’ve faced, and how we’ve overcome them.

Table of contents:
1 - The infamous "N+1" problem: Jonathan Baker - Let's talk about what it is, why it is a problem, and how Shopify handles it at scale across several GraphQL APIs.
2 - Contextualizing GraphQL APIs: Alex Ackerman - How and why we decided to use directives. I’ll share what directives are, which directives are available out of the box, and how to create custom directives.
3 - Faster GraphQL queries for mobile clients: Theo Ben Hassen - As your mobile app grows, so will your GraphQL queries. In this talk, I will go over diverse strategies to make your queries faster and more effective.
4 - Building tomorrow’s product today: Greg MacWilliam - How Shopify adopts future features in today’s code.
5 - Managing large APIs effectively: Rebecca Friedman - We have thousands of developers at Shopify. Let’s take a look at how we’re ensuring the quality and consistency of our GraphQL APIs with so many contributors.
JSNation 2023JSNation 2023
57 min
0 To Auth In An Hour For Your JavaScript App
WorkshopFree
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.js backend + Vanilla JS frontend) to authenticate users with One Time Passwords (email) and OAuth, including:
- User authentication – Managing user interactions, returning session / refresh JWTs- Session management and validation – Storing the session securely for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
JSNation 2023JSNation 2023
87 min
Build a Collaborative Notion-Like Product in 2H
WorkshopFree
You have been tasked with creating a collaborative text editing feature within your company’s product. Something along the lines of Notion or Google Docs.
CK 5 is a feature-rich framework and ecosystem of ready-to-use features targeting a wide range of use cases. It offers a cloud infrastructure to support the real-time collaboration system needs. During this workshop, you will learn how to set up and integrate CK 5. We will go over the very basics of embedding the editor on a page, through configuration, to enabling real-time collaboration features. Key learnings: How to embed, set up, and configure CK 5 to best fit a document editing system supporting real-time collaboration.
Table of contents:- Introduction to the CK 5 ecosystem.- Introduction to a “Notion-like” project template.- Embedding CK 5 on a page.- Basic CK 5 configuration.- Tuning up CK 5 for a specific use case.- Enabling real-time editing features.