Large scale projects challenges (NextJS - Contentful)

Rate this content
Bookmark

NextJS is an excellent full stack framework. Contentful is a well-known flexible headless CMS. Together are a great match, but when we talk large scale projects the challenges are completely different than the ones you may face in a small to medium scale project. Leonidas will try to raise your awareness on such challenges based on Greece's experience on redesigning Vodafone's site to create beautiful self-serve and guided journeys for Vodafone customers.

20 min
21 Jun, 2022

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk discusses the challenges faced when implementing or migrating a legacy stack to Next.js and Contentful in large-scale projects. It emphasizes the need for careful analysis and estimation of time and resources. The Talk also highlights the benefits of Next.js in facilitating collaboration with isolated teams and integrating with other frameworks. It addresses the challenges of using a headless CMS in large-scale projects and suggests strategies for handling unavailability and crashes. The importance of using global state wisely and promoting code reusability is also emphasized, along with techniques for overcoming challenges in large-scale projects.

1. Introduction to Challenges in Large-Scale Projects

Short description:

Hi, I'm Leonidas. I will talk about challenges faced when implementing or migrating a legacy stack to Next.js and Contentful in large-scale projects. These challenges are different from small to medium-scale projects. Each organization has its own business model and specific needs, so there is no one-size-fits-all solution. The purpose is to raise awareness and provide considerations for this journey.

Hi, I'm Leonidas. I have worked as a front-end engineer for about 15 years and I'm currently working as a front-end chapter lead for Vodafone Greece. Today I'm going to talk to you about challenges that you may face if you try to implement or even worse, migrate a legacy stack to a modern React framework, such as Next.js along with a headless CMS such as Contentful, but in a large-scale project.

Next.js, as you already know, is an excellent full-stack React node framework and Contentful is a well-known headless CMS. But when we talk about large-scale projects, the challenges are completely different than the one that you may face in a small to medium-scale project. In most cases, having to migrate one or more large-scale projects means that you are already part of a big organization, which by its own raises additional challenges.

With the time available, it is really hard to discuss also solutions about these challenges. Except from the time factor, each organization follows its own business model and have very specific business needs, making one solution not appropriate for all cases. So the purpose of this presentation is to raise your awareness on challenges that you may have not faced until now, and what you need to consider if you and your team decide to take this big step and begin this exciting journey. So let's begin. Let's begin with the fact that applies to all modern frameworks and not especially the ones mentioned before.

2. Challenges in Large-Scale Projects

Short description:

If you work on a project with legacy code, consider the benefits of a modern stack framework. Analyze the project and estimate the time and resources needed for migration. However, in large-scale projects, this estimation may not be feasible. The new stack should coexist with the old stack during the migration period, requiring workarounds for logging, session sharing, cross-stack journeys, code duplication, and UI component sharing.

If you work on a project that uses a lot of legacy code, you and your team will sooner or later start thinking the benefits of a modern stack framework. It will be also very easy to convince business people of your company. More stable environment, better performance, faster development time, streamlined and efficient CI, CD automation, and all this stuff will lead to a faster time to market, improve the SEO ranking, help your customers.

So what are you going to do? You will have to analyze the project and design and end up with a rough estimation of time and resources needed for the migration to happen. And you're ready to go. Sounds perfect. Not exactly. In most cases of large-scale projects, analyzing and making the initial design in order to have this rough estimation is not even feasible. And if you manage to do it, you will end up with something like, hey boss, we need to two to three years to migrate to our new stack. Large organizations won't stop developing new features for their clients, so they won't allocate their full resources to migrate to a new stack. You will end up with a vague estimation and, most probably, if the migration happens, when you are done, the technologies that you chose will already be outdated.

What you should consider, your new stack should coexist throughout the whole migration period with your old stack. You will have to follow a granular approach and find workarounds for logging handling for both stacks if you have a logging mechanism, how you will share sessions between these two stacks, how you will handle cross-stack journeys, journeys that will start from your old stack and end up to your new stack, or the other way around, how you can reduce, duplicate code and maintenance while you will have two different code bases. Finally, you will need to find ways how you will share UI components to make this thing work.

3. Next.js and Collaboration with Isolated Teams

Short description:

Next.js is a great full-stack framework that allows front-end and full-stack developers to easily work with both the back-end and front-end code in the same project. It provides a straightforward development experience, enabling the creation of end-to-end features with ease. When collaborating with isolated teams, it's important to consider how to organize separate teams working together in the same repository and maximize code separation. Additionally, integrating other frameworks with Next.js is possible. Headless CMS, such as Contentful, is a suitable choice for Next.js as it provides the missing functionalities and allows for easy switching of technology or CMS in the future.

Next.js is a great full-stack framework. As front-end or full-stack developers, we find it very easy to have the back-end and the front-end code always available within the same project. I know where to write my server-side code, I know where my APIs live, where my front-end code lives. Development is much faster, I find it easy to create end-to-end features, and the build process is just one click. Since I understand the logic behind the framework and its structure, everything is straightforward.

Let's suppose that you are part of a big global organization, most probably you will have to cooperate with other completely isolated teams because that's how most big organizations do source allocation. Your team may have to work with back-end teams that are from the other side of the planet and may have not ever used NextJS. Let's see this example. Your team comes with a great idea for a new feature that will be great added value for your customers. Your team roadmap is already defined and you will have to find ways to fit this new feature inside the roadmap. Your PO communicates the feature and finds that there are a number of other countries that also find this feature useful. It's a great opportunity to use shared resources from all other countries and develop the feature together in order to develop faster, increase ownership and also increase reusability.

So people may be allocated from your team for the React part of the AI feature and another team that will work on Node.js will be selected. In the best-case scenario, the back-end team will find it weird to work with the same files and code base as the front-end team works. In the worst-case scenario this team may be too opinionated and they may be need to work with a specific Node.js framework, like for example Nest.js. Is everything ruined? No, you just need to consider some things. First is how to organize two separate teams working together in the same repo with the same files. Second, how to maximize code separation between those teams. And finally, how you can integrate other frameworks with Next.js.

Let's discuss Headless CMS for a while. In Vodafone, Headless CMS was the CMS of our choice when we had to make this big decision for a future CMS that will be our source of dynamic content and data for all our applications and devices for the years to come. In specific, we decided to go with Contentful, but there is a big number of great Headless CMS out there that more or less what we're going to say here will apply. Headless CMSs have great benefits. Your content is ready to be served to any device. You have to worry less about the content and more about the presentation layer, which is something that all developers want. It's front end agnostic, React, Vue, Angular, you name it. As a result, you have a CMS that is completely decoupled from your rendering part. So it makes it very easy to switch from the technology or CMS in the future if you need to. So headless CMS is a perfect match for Next.js since Next.js provides all the functionalities missing from headless CMS. For example, routing, SEO and in general Next.js also supports static size generation which is a technology where headless CMS is your best bet.

4. Challenges of Headless CMS in Large-Scale Projects

Short description:

Although headless CMS may present challenges in large-scale projects, such as content authoring experience and SEO tools, it is possible to overcome them with careful consideration and choosing the right CMS. The uptime of your end user's experience is crucial, even if your servers have high uptime. Large-scale projects often rely on legacy backend systems, which can have low uptimes and other issues. It is important to handle potential unavailability and crashes by analyzing each feature, implementing mechanisms like feature flags, and having advanced error handling, logging, and monitoring. Global state management is a useful tool to avoid prop drilling and store stateful information in complex applications.

Although headless CMS is the way to go when you have an in-house development team and you are working with modern JavaScript frameworks, some of the small disadvantages of a headless CMS may become challenges when you work in a large-scale project, especially in a large organization with a lot of content authors and more advanced SEO requirements from a dedicated SEO team. I will summarize them into categories, content authoring experience and SEO tools.

I think you can get away with both of these challenges in a small to medium-sized project, but not in a large one. There will be cases where the SEO team may need features like maintaining a list of url directs, editing a robot.txt file, previewing how Google renders your pages or metadata. Also when it comes to content authors experience, although most early CMSs have done great improvements in the last few years, things like content preview, content editing and in general the enhanced user experience are not the same level as in some traditional CMSs. With Contentful in specific there's nothing that you cannot achieve due to its custom apps features and this also I think stands true for a number of other headless CMSs. You can achieve everything but it requires some effort from your team to develop target features.

So what do you have to consider? Choose your headless CMS wisely based on your current and future needs. Focus on extensibility, flexibility and support. Most SaaS products today offer sufficient uptime SLAs like 99.99999% etc. Even if you do your own headless CMS implementation, you will most probably host it in a cloud service like AWS which also offers almost 100% uptime. Calculating the fact that you can also use a number of additional caching layers for your data in order to ensure uptime, even if your SaaS products fail, then you can be sure that you will have 100% uptime for your great features. The metric that counts most is the uptime your end user experiences and not the uptime that your servers have. Most of the time, the features in large-scale and complex projects are dependent on a number of legacy backend systems to retrieve any kind of data. These systems, most of the time, are handled by different departments in the organizations and may have low uptimes, bad patch update practices with long maintenance periods, and a number of other issues. Moreover, some of your web app features may be strongly related to business functions and services which may rise up issues unrelated to their network. For example, a click-to-call service where suddenly your call center goes down and your feature will appear unresponsive to your user. Maybe a live chat could be a great example. And the list can go on. Unavailability on all of these cases may not mean actual unavailability to your web app, but if it's not handled correctly, it may lead to crashes and bad user experience.

So, what you should consider. You need to analyze all these cases for each feature. You need to create mechanisms, like, for example, feature flags, that may help you change your user experience on the fly when such a crisis happens. Also, you need to have advanced error handling, and finally, you need to have an effective logging and monitoring mechanism to catch unhandled cases. Most of us are used to the global state concept and we have been applying this pattern in our apps extensively. There is a number of popular third party libraries, such as Redux and MobX, that can help you manage your global state in the application. React also offers context API and user-to-user hook for the same reason. Global state is a great way to avoid prop drilling and store stateful information when many of your components need access to it. I am pretty sure that this is not the first time that you are hearing someone talking about global state overuse or abuse or something like that. As we mentioned before, global state is a very useful tool.

5. Challenges of Global State in Large-Scale Projects

Short description:

A large-scale project can easily become burdened with a hard-to-maintain global state. Overusing global state can lead to excessive boilerplate code, performance issues, and confusion for inexperienced developers. It is important to use global state wisely and consider using state management libraries like React Query, SWR, Recoil, Jetty, and Zoostack for better performance and ease of use.

Unfortunately, a large-scale project is the best playground to end up with a huge and hard-to-maintain global state if you design your application to be global state heavy. The extensive use of global state increases boilerplate code and may give a really hard time to an inexperienced or new developer to grasp what's happening. In addition, a number of performance issues may be related to the extensive or bad use of global state. We were surprised when we found out how many times we tend to use global state without even needing it. Global state is not evil, but overusing it may turn it to be evil.

So, what you should consider? Use global state wisely. Try a combination of a global state management library with a fetching library such as React Query or SWR. These libraries offer request duplication, client-side caching, and data updates without having you to touch the global state. Finally, I just want to mention that there is a number of modern state management libraries that are more focused on performance and ease of use like Recoil, Jetty, and Zoostack, which are very great libraries and alternatives to consider.

6. Code Reusability and Overcoming Challenges

Short description:

Code reusability is crucial in large-scale projects. React promotes reusable components, but there are challenges in effectively communicating and guiding code and component reuse. UI component reusability affects content type reusability in headless CMS. Consider inheritance and composition patterns, naming conventions, documentation tools, workflows, and component library structure. Streamline the process of contributing reusable components and explore ways to reuse GraphQL queries. Next.js and headless CMSs are great choices, but developers need to push boundaries and prioritize accessibility and flexibility. Techniques like separating frontend and backend code, microfrontend approach, feature flags, documentation tools, context API and SWR, custom apps, content-as-code, and Atomic Design pattern can help overcome challenges.

Code reusability is one of the most important things when it comes to projects, especially large-scale ones. React by itself is based on the philosophy of reusable components. There is a number of things that you can make reusable inside your project, for example functions, components, hooks, GraphQL queries, APIs, and the list can go on.

In a headless CMS, you need to focus on reusable content types to ensure reusability. With all these choices, it seems easy to have a scalable and easy-to-maintain project. The challenge here is not framework or CMS-specific, but project-scale specific. Think of an environment where 15 different local teams plus an unlimited number of teams around the globe are working on the same project. It's a challenge to effectively communicate the details and requirements for code and component reuse, and it's difficult to provide adequate guidance and feedback on their use of code. It's very hard to have everyone aligned with the process. And the issue with UI component reusability will soon lead to issues with content types reusability in your CMS since your content types need to be mapped with your UI, at least with your core UI components.

Also, you will find that the amount of GraphQL query that you use to retrieve your data starts to become huge and you will find yourself using the same queries again and again. Introducing GraphQL fragments is a great way to improve this, but they may lead to bad practices such as the use of nested fragments if you end up overusing this pattern. So, there is a number of things that you need to consider in order to overcome these challenges. First, try to use inheritance and composition patterns for your content types in order to achieve maximum usability. Find naming convention patterns for your content types that are not feature-specific, so you can reuse them within other content types with a meaningful way for your content authors. Use appropriate documentation tools for your UI components, such as a storybook or a style guide to have a clear representation of all your components and all their variations. And also find the right workflows and ways to communicate them to your designers and business team. Use a meaningful structure for your components library. Streamline the process of contributing reusable components to a global scale, focusing on how fast a new developer can grasp the process. Finally, investigate on ways to reuse GraphQL queries, avoiding pitfalls.

I think after six laps, I have to come to a conclusion. Don't get me wrong, Next.js, as other popular frameworks, are great to use under any circumstances and headless CMSs are the way to go, and most probably what we will use in the future. That's why we chose to use them and we will still do. Working with large-scale projects needs from the developers to do the extra mile and push framework boundaries to their limits. What you should be looking for in a framework or a CMS is accessibility and flexibility so you can overcome all the obstacles. After raising the challenges, let me thoroughly present you a list of techniques that we already use or we are in the process of implementing that may sound interesting to you. We have separated our frontend from our backend code in the filesystem level. We also decided on a microfrontend approach for team isolation and easier code maintenance, converting one large project into multiple small ones. We use feature flags to handle user experience on-the-fly when we need to address an availability. We use storybook and other documentation tools to document our components. We are using a combination of context API and SWR for our global state management. We create a number of custom apps inside Contentful to customize user experience and integrate Contentful with other third-party services and create features that are hard to find in handless CMSes out-of-the-box. We use the content-as-code approach for all our content, and finally we use the Atomic Design pattern to structure our components. As I told you in the introduction, there is no remedy for all ills or difficulties.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Top Content
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
TechLead Conference 2023TechLead Conference 2023
35 min
A Framework for Managing Technical Debt
Let’s face it: technical debt is inevitable and rewriting your code every 6 months is not an option. Refactoring is a complex topic that doesn't have a one-size-fits-all solution. Frontend applications are particularly sensitive because of frequent requirements and user flows changes. New abstractions, updated patterns and cleaning up those old functions - it all sounds great on paper, but it often fails in practice: todos accumulate, tickets end up rotting in the backlog and legacy code crops up in every corner of your codebase. So a process of continuous refactoring is the only weapon you have against tech debt.In the past three years, I’ve been exploring different strategies and processes for refactoring code. In this talk I will describe the key components of a framework for tackling refactoring and I will share some of the learnings accumulated along the way. Hopefully, this will help you in your quest of improving the code quality of your codebases.

React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
React Advanced Conference 2022React Advanced Conference 2022
22 min
Monolith to Micro-Frontends
Top Content
Many companies worldwide are considering adopting Micro-Frontends to improve business agility and scale, however, there are many unknowns when it comes to what the migration path looks like in practice. In this talk, I will discuss the steps required to successfully migrate a monolithic React Application into a more modular decoupled frontend architecture.
React Advanced Conference 2023React Advanced Conference 2023
22 min
Power Fixing React Performance Woes
Next.js and other wrapping React frameworks provide great power in building larger applications. But with great power comes great performance responsibility - and if you don’t pay attention, it’s easy to add multiple seconds of loading penalty on all of your pages. Eek! Let’s walk through a case study of how a few hours of performance debugging improved both load and parse times for the Centered app by several hundred percent each. We’ll learn not just why those performance problems happen, but how to diagnose and fix them. Hooray, performance! ⚡️
React Summit 2023React Summit 2023
24 min
Video Editing in the Browser
Video editing is a booming market with influencers being all the rage with Reels, TikTok, Youtube. Did you know that browsers now have all the APIs to do video editing in the browser? In this talk I'm going to give you a primer on how video encoding works and how to make it work within the browser. Spoiler, it's not trivial!

Workshops on related topic

React Summit 2022React Summit 2022
173 min
Build a Headless WordPress App with Next.js and WPGraphQL
Top Content
WorkshopFree
In this workshop, you’ll learn how to build a Next.js app that uses Apollo Client to fetch data from a headless WordPress backend and use it to render the pages of your app. You’ll learn when you should consider a headless WordPress architecture, how to turn a WordPress backend into a GraphQL server, how to compose queries using the GraphiQL IDE, how to colocate GraphQL fragments with your components, and more.
React Day Berlin 2022React Day Berlin 2022
53 min
Next.js 13: Data Fetching Strategies
Top Content
WorkshopFree
- Introduction- Prerequisites for the workshop- Fetching strategies: fundamentals- Fetching strategies – hands-on: fetch API, cache (static VS dynamic), revalidate, suspense (parallel data fetching)- Test your build and serve it on Vercel- Future: Server components VS Client components- Workshop easter egg (unrelated to the topic, calling out accessibility)- Wrapping up
React Summit Remote Edition 2021React Summit Remote Edition 2021
87 min
Building a Shopify App with React & Node
Top Content
WorkshopFree
Shopify merchants have a diverse set of needs, and developers have a unique opportunity to meet those needs building apps. Building an app can be tough work but Shopify has created a set of tools and resources to help you build out a seamless app experience as quickly as possible. Get hands on experience building an embedded Shopify app using the Shopify App CLI, Polaris and Shopify App Bridge.We’ll show you how to create an app that accesses information from a development store and can run in your local environment.
React Advanced Conference 2023React Advanced Conference 2023
102 min
Fetch, useEffect, React Query, SWR, what else?
WorkshopFree
In this workshop, first, we’ll go over the different ways you can consume APIs in React. Then, we’ll test each one by fetching content from a headless CMS (with both REST and GraphQL) and checking in detail how they work.
While there is no advanced React knowledge required, this is going to be a hands-on session, so you’ll need to clone a preconfigured GitHub repository and utilize your preferred React programming editor, like VS Code.
You will learn:- What diverse data fetching options there are in React- What are advantages and disadvantages of each- What are the typical use cases and when each strategy is more beneficial than others
React Summit 2023React Summit 2023
139 min
Create a Visually Editable Next.js Website Using React Bricks, With Blog and E-commerce
WorkshopFree
- React Bricks: why we built it, what it is and how it works- Create a free account- Create a new project with Next.js and Tailwind- Explore the directory structure- Anatomy of a Brick- Create a new Brick (Text-Image)- Add a title and description with RichText visual editing- Add an Image with visual editing- Add Sidebar controls to edit props (padding and image side)- Nesting Bricks using the Repeater component- Create an Image gallery brick- Publish on Netlify or Vercel- Page Types and Custom fields- Access Page meta values- Internationalization- How to reuse content across pages: Stories and Embeds- How to create an E-commerce with Products’ data from an external database and landing pages created visually in React Bricks- Advanced enterprise features: flexible permissions, locked structure, custom visual components
JSNation 2022JSNation 2022
41 min
Build a chat room with Appwrite and React
WorkshopFree
API's/Backends are difficult and we need websockets. You will be using VS Code as your editor, Parcel.js, Chakra-ui, React, React Icons, and Appwrite. By the end of this workshop, you will have the knowledge to build a real-time app using Appwrite and zero API development. Follow along and you'll have an awesome chat app to show off!