Large scale projects challenges (NextJS - Contentful)

Hi, I'm Leonidas. I have worked as a front-end engineer for about 15 years and I'm currently working as a front-end chapter lead for Vodafone Greece. Today I'm going to talk to you about challenges that you may face if you try to implement or even worse, migrate a legacy stack to a modern React framework, such as Next.js along with a headless CMS such as Contentful, but in a large-scale project.

Next.js, as you already know, is an excellent full-stack React node framework and Contentful is a well-known headless CMS. But when we talk about large-scale projects, the challenges are completely different than the one that you may face in a small to medium-scale project. In most cases, having to migrate one or more large-scale projects means that you are already part of a big organization, which by its own raises additional challenges.

With the time available, it is really hard to discuss also solutions about these challenges. Except from the time factor, each organization follows its own business model and have very specific business needs, making one solution not appropriate for all cases. So the purpose of this presentation is to raise your awareness on challenges that you may have not faced until now, and what you need to consider if you and your team decide to take this big step and begin this exciting journey. So let's begin. Let's begin with the fact that applies to all modern frameworks and not especially the ones mentioned before.

If you work on a project that uses a lot of legacy code, you and your team will sooner or later start thinking the benefits of a modern stack framework. It will be also very easy to convince business people of your company. More stable environment, better performance, faster development time, streamlined and efficient CI, CD automation, and all this stuff will lead to a faster time to market, improve the SEO ranking, help your customers.

So what are you going to do? You will have to analyze the project and design and end up with a rough estimation of time and resources needed for the migration to happen. And you're ready to go. Sounds perfect. Not exactly. In most cases of large-scale projects, analyzing and making the initial design in order to have this rough estimation is not even feasible. And if you manage to do it, you will end up with something like, hey boss, we need to two to three years to migrate to our new stack. Large organizations won't stop developing new features for their clients, so they won't allocate their full resources to migrate to a new stack. You will end up with a vague estimation and, most probably, if the migration happens, when you are done, the technologies that you chose will already be outdated.

What you should consider, your new stack should coexist throughout the whole migration period with your old stack. You will have to follow a granular approach and find workarounds for logging handling for both stacks if you have a logging mechanism, how you will share sessions between these two stacks, how you will handle cross-stack journeys, journeys that will start from your old stack and end up to your new stack, or the other way around, how you can reduce, duplicate code and maintenance while you will have two different code bases. Finally, you will need to find ways how you will share UI components to make this thing work.

Next.js is a great full-stack framework. As front-end or full-stack developers, we find it very easy to have the back-end and the front-end code always available within the same project. I know where to write my server-side code, I know where my APIs live, where my front-end code lives. Development is much faster, I find it easy to create end-to-end features, and the build process is just one click. Since I understand the logic behind the framework and its structure, everything is straightforward.

Let's suppose that you are part of a big global organization, most probably you will have to cooperate with other completely isolated teams because that's how most big organizations do source allocation. Your team may have to work with back-end teams that are from the other side of the planet and may have not ever used NextJS. Let's see this example. Your team comes with a great idea for a new feature that will be great added value for your customers. Your team roadmap is already defined and you will have to find ways to fit this new feature inside the roadmap. Your PO communicates the feature and finds that there are a number of other countries that also find this feature useful. It's a great opportunity to use shared resources from all other countries and develop the feature together in order to develop faster, increase ownership and also increase reusability.

So people may be allocated from your team for the React part of the AI feature and another team that will work on Node.js will be selected. In the best-case scenario, the back-end team will find it weird to work with the same files and code base as the front-end team works. In the worst-case scenario this team may be too opinionated and they may be need to work with a specific Node.js framework, like for example Nest.js. Is everything ruined? No, you just need to consider some things. First is how to organize two separate teams working together in the same repo with the same files. Second, how to maximize code separation between those teams. And finally, how you can integrate other frameworks with Next.js.

Let's discuss Headless CMS for a while. In Vodafone, Headless CMS was the CMS of our choice when we had to make this big decision for a future CMS that will be our source of dynamic content and data for all our applications and devices for the years to come. In specific, we decided to go with Contentful, but there is a big number of great Headless CMS out there that more or less what we're going to say here will apply. Headless CMSs have great benefits. Your content is ready to be served to any device. You have to worry less about the content and more about the presentation layer, which is something that all developers want. It's front end agnostic, React, Vue, Angular, you name it. As a result, you have a CMS that is completely decoupled from your rendering part. So it makes it very easy to switch from the technology or CMS in the future if you need to. So headless CMS is a perfect match for Next.js since Next.js provides all the functionalities missing from headless CMS. For example, routing, SEO and in general Next.js also supports static size generation which is a technology where headless CMS is your best bet.

Although headless CMS is the way to go when you have an in-house development team and you are working with modern JavaScript frameworks, some of the small disadvantages of a headless CMS may become challenges when you work in a large-scale project, especially in a large organization with a lot of content authors and more advanced SEO requirements from a dedicated SEO team. I will summarize them into categories, content authoring experience and SEO tools.

I think you can get away with both of these challenges in a small to medium-sized project, but not in a large one. There will be cases where the SEO team may need features like maintaining a list of url directs, editing a robot.txt file, previewing how Google renders your pages or metadata. Also when it comes to content authors experience, although most early CMSs have done great improvements in the last few years, things like content preview, content editing and in general the enhanced user experience are not the same level as in some traditional CMSs. With Contentful in specific there's nothing that you cannot achieve due to its custom apps features and this also I think stands true for a number of other headless CMSs. You can achieve everything but it requires some effort from your team to develop target features.

So what do you have to consider? Choose your headless CMS wisely based on your current and future needs. Focus on extensibility, flexibility and support. Most SaaS products today offer sufficient uptime SLAs like 99.99999% etc. Even if you do your own headless CMS implementation, you will most probably host it in a cloud service like AWS which also offers almost 100% uptime. Calculating the fact that you can also use a number of additional caching layers for your data in order to ensure uptime, even if your SaaS products fail, then you can be sure that you will have 100% uptime for your great features. The metric that counts most is the uptime your end user experiences and not the uptime that your servers have. Most of the time, the features in large-scale and complex projects are dependent on a number of legacy backend systems to retrieve any kind of data. These systems, most of the time, are handled by different departments in the organizations and may have low uptimes, bad patch update practices with long maintenance periods, and a number of other issues. Moreover, some of your web app features may be strongly related to business functions and services which may rise up issues unrelated to their network. For example, a click-to-call service where suddenly your call center goes down and your feature will appear unresponsive to your user. Maybe a live chat could be a great example. And the list can go on. Unavailability on all of these cases may not mean actual unavailability to your web app, but if it's not handled correctly, it may lead to crashes and bad user experience.

So, what you should consider. You need to analyze all these cases for each feature. You need to create mechanisms, like, for example, feature flags, that may help you change your user experience on the fly when such a crisis happens. Also, you need to have advanced error handling, and finally, you need to have an effective logging and monitoring mechanism to catch unhandled cases. Most of us are used to the global state concept and we have been applying this pattern in our apps extensively. There is a number of popular third party libraries, such as Redux and MobX, that can help you manage your global state in the application. React also offers context API and user-to-user hook for the same reason. Global state is a great way to avoid prop drilling and store stateful information when many of your components need access to it. I am pretty sure that this is not the first time that you are hearing someone talking about global state overuse or abuse or something like that. As we mentioned before, global state is a very useful tool.

Unfortunately, a large-scale project is the best playground to end up with a huge and hard-to-maintain global state if you design your application to be global state heavy. The extensive use of global state increases boilerplate code and may give a really hard time to an inexperienced or new developer to grasp what's happening. In addition, a number of performance issues may be related to the extensive or bad use of global state. We were surprised when we found out how many times we tend to use global state without even needing it. Global state is not evil, but overusing it may turn it to be evil.

So, what you should consider? Use global state wisely. Try a combination of a global state management library with a fetching library such as React Query or SWR. These libraries offer request duplication, client-side caching, and data updates without having you to touch the global state. Finally, I just want to mention that there is a number of modern state management libraries that are more focused on performance and ease of use like Recoil, Jetty, and Zoostack, which are very great libraries and alternatives to consider.

Code reusability is one of the most important things when it comes to projects, especially large-scale ones. React by itself is based on the philosophy of reusable components. There is a number of things that you can make reusable inside your project, for example functions, components, hooks, GraphQL queries, APIs, and the list can go on.

In a headless CMS, you need to focus on reusable content types to ensure reusability. With all these choices, it seems easy to have a scalable and easy-to-maintain project. The challenge here is not framework or CMS-specific, but project-scale specific. Think of an environment where 15 different local teams plus an unlimited number of teams around the globe are working on the same project. It's a challenge to effectively communicate the details and requirements for code and component reuse, and it's difficult to provide adequate guidance and feedback on their use of code. It's very hard to have everyone aligned with the process. And the issue with UI component reusability will soon lead to issues with content types reusability in your CMS since your content types need to be mapped with your UI, at least with your core UI components.

Also, you will find that the amount of GraphQL query that you use to retrieve your data starts to become huge and you will find yourself using the same queries again and again. Introducing GraphQL fragments is a great way to improve this, but they may lead to bad practices such as the use of nested fragments if you end up overusing this pattern. So, there is a number of things that you need to consider in order to overcome these challenges. First, try to use inheritance and composition patterns for your content types in order to achieve maximum usability. Find naming convention patterns for your content types that are not feature-specific, so you can reuse them within other content types with a meaningful way for your content authors. Use appropriate documentation tools for your UI components, such as a storybook or a style guide to have a clear representation of all your components and all their variations. And also find the right workflows and ways to communicate them to your designers and business team. Use a meaningful structure for your components library. Streamline the process of contributing reusable components to a global scale, focusing on how fast a new developer can grasp the process. Finally, investigate on ways to reuse GraphQL queries, avoiding pitfalls.

I think after six laps, I have to come to a conclusion. Don't get me wrong, Next.js, as other popular frameworks, are great to use under any circumstances and headless CMSs are the way to go, and most probably what we will use in the future. That's why we chose to use them and we will still do. Working with large-scale projects needs from the developers to do the extra mile and push framework boundaries to their limits. What you should be looking for in a framework or a CMS is accessibility and flexibility so you can overcome all the obstacles. After raising the challenges, let me thoroughly present you a list of techniques that we already use or we are in the process of implementing that may sound interesting to you. We have separated our frontend from our backend code in the filesystem level. We also decided on a microfrontend approach for team isolation and easier code maintenance, converting one large project into multiple small ones. We use feature flags to handle user experience on-the-fly when we need to address an availability. We use storybook and other documentation tools to document our components. We are using a combination of context API and SWR for our global state management. We create a number of custom apps inside Contentful to customize user experience and integrate Contentful with other third-party services and create features that are hard to find in handless CMSes out-of-the-box. We use the content-as-code approach for all our content, and finally we use the Atomic Design pattern to structure our components. As I told you in the introduction, there is no remedy for all ills or difficulties.