Great Migrations: Upgrading a Component Library at Scale

All right. So, thank you for joining me today for this talk about great migrations, upgrading this component library thingy. And what I'm going to tell you or the story that I'm going to be sharing is something that works for us, for our organization, and it doesn't necessarily directly apply to your organization, but hopefully you can take some inspiration or some parts of it to apply within your own company or organization. And while it says component library here at the title, it's basically about the entire environment that we have because everything that we have is built in vue and Nuxt. So, we'll be upgrading and migrating everything basically during this talk, if you bear with me. So, let's get started. So, my name is Joran. As you can see, I'm a front-end developer. I work for YUMBO. I'll explain a bit later about what that is. You can find me on Twitter. You can find me on Mastodon or on my personal webpage. If you have any questions related to this topic or any other topic, feel free to reach out to me after this presentation or just chat with me regarding these topics. Now, this whole presentation is sort of inspired by BBC's series about nature's great events and in this particular case, the Great Migration episode. And you can see here a picture of a young David Attenborough wrangling something that resembles a lizard or something. I was watching these series when I was younger, usually with my dad. So, I have very fond memories of this series and that's why I think it has a nice analogy to what we are doing in software development. And we will be seeing how the pros do it in nature migrating and what we can learn from it in our day-to-day job. So, I picked a couple of migrations that stand out for various reasons and let's take a look at them. So, I think the most famous one is of course the wildebeest migration. I've written some things down about because I don't know all of the details by heart, but this is like the biggest migration there is. It's actually a superherd made up of wildebeest, zebras and gazelles and they are like with a million or more than a million beasts migrating. And they sort of take a journey that's 1800 miles or 2900 kilometers and it's a round trip from Tanzania to Kenya and back. And while they are doing this, like with the massive scale that they have, they actually play a role in forming or adjusting the landscape. So, that's how big this is. They have to traverse crocodile infested rivers, they have to overcome droughts, they are preyed upon by lions and what they are actually doing is they rear calves while they are migrating. So, if you take a look at the analogy, they are actually pushing stuff to production while migrating. I think this is really an interesting and an impressive feat. Now we have the caribou or reindeer, whatever you want to call them, and they have the longest migration path that we have on land. So, they cover up to 3100 miles or 5000 kilometers each year and they move from their wintering grounds to their calving grounds in spring and then back to their wintering grounds during fall in sort of a circular pattern. So, the timing and distance about those migrations can vary on a certain factor such as the weather patterns, availability of food and water and risk of predators. So, this is the longest migration. And then we have, this is peak collaboration. We have the fire ants in the forests of Brazil, the rainforests, and these can form up to bodies of more than 100,000 ants clinging together and they do this in order to traverse water. And this is an interesting way of moving about, right? So, they are experts at collaborating and the ants at the bottom of this body, they need to cling on for dear life in order to avoid being submerged, while the ones at the top, they run the risk of being eaten by birds and other predators. And then there is the final bit of the puzzle. This is the elusive pack of developers and they typically move about in scrub teams of various sizes and they have to deal with other dangers such as moving requirements, Friday deployments, and bug infestations and getup outages. So, what we will do is we will investigate patterns for optimizing migration patterns and see if there's anything we can learn from nature. Let's first start with a definition. So, what do we mean with a migration? And for the sake of this talk, let's consider this. So, it is a period because it doesn't happen instantly. It takes time and effort of moving. Well, the moving could be anything from ants to people or even code. Large volumes because it's not about just one gazelle looking what's on the other side of the value, what's going on there. It's about a group. Otherwise, it doesn't really make a lot of sense. And we go from point A to point B because we have a job to do at point B, right? So, this is the definition that we will be using. And then there is the question, so why do we migrate? And if you look at the wildebeest, they have a very good reason. They follow the rain through lush feeding grounds. And if they wouldn't, the environment where they are living in wouldn't be able to sustain the herd or support the numbers. So, they will die a horrible death. And the software analogy is we want to have support of our software, of course. So, if we take a look at the life of a wildebeest, it's not really a pretty life, but it's in constant movement. So, let's see how we can apply this to us. And of course, it's not actually about following lunch and coffee. It's about the ability to quickly push new features to production and have some stability in the environment to release confidently. Now, this stability thing is sort of the opposite of migrating. And there's a risk here just as the wildebeest face. Those wildebeests would probably be better off if they would just stay in their feeding grounds. They don't have to cross those rivers. They have little risk if they would stay, but they have a reason to do this. And that's all about resources. For them, it's food. And for us, developers, it's the support of the platform, as I said. So, combine that support with an improved developer experience and the ability to create new stability. That's a good reason to migrate. So, that's why we're doing this. So, this is a silly analogy. Let's investigate. And what we will be doing is we're going to observe a migratory pattern in a relatively isolated environment, and that is the wild place of Jumbo. So, if you're not from the Netherlands or Belgium, you will probably have never heard of this or us, but we are a grocery chain and we do a lot of online groceries. We're actually in the top of the Netherlands and Belgium. And we develop a lot of our software in-house. And where we are doing this is the Jumbo Tech Campus or JTC as we call it. There are over 450 developers who are collaborating on building our digital products. And this is not the herd. These are the stewards of the herd and they need to make sure that our herd of software or code gets to point B safely. So, now that we've met the stewards of the herd, it's time to take a look at the herd and the landscape. Now, we have among things a big e-commerce monolithic engine and that's been around for over a decade, I think. And what we've been doing over time is we started to offload a bit of the responsibility of the e-commerce engine into more specific vue.js and Nuxt applications. And we started to get involved with vue at the beginning of the V2 release. So, we're doing this for quite some time now. And we gradually added Nuxt to also mitigate or migrate a bit from the monolith to its specific domain or applications and groups. And what we have here is a team setup where teams are individually responsible for their own domain. And if we zoom in a bit more, then we can take a look at what a team setup might look like. So, this could be a typical team setup. It might differ in complexity from team to team, but this is what you would normally expect. So, there is a Nuxt application at the center. That could also be a vue application. And that depends on some microservices. That could be RESTful or graphql or apollo. We don't really care because that's part of the team's responsibility. And then in terms of e-commerce, most of the Nuxt applications, they depend on a couple of standardized modules and plugins we developed. And they deal with sort of the generic stuff such as analytics and tracking. And so, they just help out the Nuxt application with generic stuff. And then in terms of facilitating to the customer needs, if you serve the page, we also include some micro applications in order to fulfill the e-commerce process. And one example of this could be the basket, which is an isolated application, which is just attached to the UI of that Nuxt application. And then lastly, we have a component library. And that just holds hundreds of components that you build or that you use in order to build UIs, which is, I think, a sensible approach. If you imagine this setup with different types of complexities spread between the company, then you sort of have a decent view of what our herd looks like. So, this is the herd that we need to move because this is all done in vue 2 with Nuxt 2. So, we need to upgrade. We need to migrate. And this is because, of course, last year, vue 3 and Nuxt 3 were announced, but also the end of life of vue 2 was announced, which is sort of that's when we run out of resources more or less. So, this is the reason that we need to migrate. And now that we have a very good reason of doing this, we need sort of a general direction or a migration path. And with all of our knowledge combined, we started to sort of plot a migration path based on the dependencies and the bottlenecks that we could identify. And in terms of the wildebeest analogy, bottlenecks could be considered the river crossings or the rivers we need to cross in order to move forward. And one of the first and more important bottlenecks or river crossings we have to do are those micro-frontends. So, what we did in the past or what we're currently doing, we're migrating away from this, is we have these micro-frontends and they are basically vue applications that we attach to the DOM. And these are using, they're using vuex in order to mutate and modify the internal workings of that micro-application, like adding product to a basket, removing or updating the quantity. And that's something we need to get rid of because we don't want to support vuex in vue 3 environments anymore. So, what in this case, the team who was responsible for the basket is doing, is they are actually removing the whole vuex api. And of course, having a vuex api for public operations can be considered somewhat of a risky idea, but it worked for us. But they are moving this to more of a tech agnostic way to interact with the basket. And then internally, they are moving to Pinya or to apollo Client, if I'm not mistaken. But they're, basically, they're removing the vuex dependency, which allows everybody else to upgrade as well, because as long as they have vuex, everybody is dependent on this. So, we are already crossing this stream at the moment. And then we have, just like this OxPacker is supporting the, this is a water buffalo, actually, not a wildebeest, but just like this OxPacker is a, plays a supporting role on our modules or on our Nuxt applications, we have modules. And those modules, as I said, they deal with stuff like gathering analytics or making sure that the seo is up to spec. And what we did here is, because not everybody is depending on those, we have gathered the teams who are using these modules and plugins and each individual team sat together and they either rewritten the module and plugin to be compatible with Nuxt 3, in this case, writing wrappers or just rewriting it entirely so that they can all move forward. So, we are, again, we're crossing another river here. And then finally, this is sort of a challenge because like this caterpillar represents Vue2 and it needs to morph in a beautiful vue 3 compliant butterfly. The issue being that we need to keep the caterpillar alive until everybody else has been able to migrate. So, there's a challenge here. And we're going to take a look at the component library more specifically. So, our component library, this is something that we built a couple of years ago and it contains hundreds of components. We built it in vue, of course. We use storybook for documentation. This is just a sensible approach. And the way that we are maintaining and collaborating on our component library is what we call a sort of a distributed approach. This means that every developer who has a stake in a component library is able to provide a contribution or to collaborate on adding new features, adding or doing maintenance, and just keeping it in a really good shape. And I'll discuss these four topics further on. So, let's first take a look at this investment. So, I think this is the most important factor or decision that we made and that is that all users can actually modify the library. And this is important because I think that this creates an investment in the library and it makes sure that everybody cares about the library. Sort of a nurturing, if you look at this picture, making sure that everything is in tip-top shape and a shared responsibility. And I think this is key for all of the other things that we do. And then we have like, it sometimes happens that stewards for some reason leave the herd. And if you do this, then if you do this with a small team, I have to say, then there is a risk of having a lot of knowledge leaving your herd or your company or organization. And again, with having this distributed approach, we also have distributed the knowledge about our component library, which means that if somebody leaves for whatever reason, that the knowledge drain isn't as big and that the risk is mitigated actually in between all of those other contributors. And it also creates sort of an encouragement of everybody to keep sharing their knowledge as well, which I think is also key in making sure that we keep everything up to date and have everybody on the same page as well. And then just as these wildebeests are crossing the stream, what they do with their body is somebody moves forward and that body then creates space for another wildebeest to move into that gap and to move forward as well. So they are blocking the stream to help others get across. And I don't want to focus on the blocking part here, but what happens here is that they create space for other teams to move forward or for other wildebeests to move forward. And it's not much different with our distributed model. So each team here contributes and moves forward, which creates space for other teams to move forward and accelerate as well. I think this is also a key benefit of having this approach. And once we've crossed that river, our whole herd can move forward. And so we wait until a large body of goat or animals has moved across before we cross the next challenge or face the next problem. And in addition to any improvement that we do is immediately shared across the entire organization, which makes everybody, even POs, happy. Everybody benefits. So once you cross that river, everybody can move forward. We couldn't just morph the component library from vue 2 into vue 3 because we needed to also support vue 2. And what we did here is we used vue Demi in order to bring our vue 2 application as close to vue 3 as possible. And then we started to place a new package that is vue 3 compatible next to the vue 2 package so that we can support both. Now if we look at where we are right now, we're not there yet, but we've already cleared a lot of crocodile infested rivers in this case. What we're doing or what we learned is that we are going to start new projects if they have no dependencies on legacy software in vue 3 and Nuxt 3 already to make sure that those are already prepared for our next step. And we're also making sure that if we have any APIs that are dependent on a certain tech stack such as the vuex case in the case of the basket, then we're now making sure to make that a more tech agnostic approach so that we have the APIs ready, but everything that is more tech related is done inside of the application. So it doesn't have any external dependencies anymore. And we're currently in the process of upgrading those micro applications as well as the component library. So we're well underway. And this means that our herd has found green pastures again, overcoming the challenges as a team. And we're not quite there yet though, but we're closing the gap. And I think it's good to share these takeaways that we have, that we have learned during our journey. And this common direction is really important to know where you want to go and also what you need to do in order to get there. So this really helped us in plotting our entire path. And those increments, they're basically making sure that we have deliverables along the way and they add value once they are released or complete. And you can think about those plugins that are already NUX3 capable, which makes sure that we can already start building NUX3 projects for new scaffold out projects. And making sure that nobody gets left behind basically means that we are supporting everything that is still dependent on vue 2 until our last team or application is ready to make the jump to vue 3. And this is why we are doing this with the component library to support both of them, because it's just in the end, everything that we deliver, we do this for the customer. And the customer actually doesn't really care if we are using vue 2 or vue 3. He just wants to buy his groceries. So we are making sure that our teams are able to deliver those features. And this distributed approach that I explained a bit about is that makes sure that we can continue to deliver features to production. This is like the rearing of the cows in the wildebeest herd while working on our migration. So we're migrating and delivering at the same time. And I think that's really important also to make sure that you're not blocking any features while you're doing this, because it's such a big endeavor. I'd like to close with an inspirational quote by the Sir David Attenborough himself. I hope that you've also taken a bit of inspiration from this talk and that you can apply some of our learnings in your own migration path. And of course, this quote is not necessarily about the natural world. It's about front-end development as well, which is also a great source of excitement. And what also was a great source of excitement for me was giving this talk to you and being able to share our story. As I said before, if you want to reach out to me, feel free to contact me via any means possible, and I will be happy to talk to you. So thank you very much and hope to see you at the next conference.