Remixing a Symfony

Rate this content
Bookmark

In late 2020, I ran a Lighthouse test on a simple content page on Harvie, our farm management platform and Symfony app, and received a performance score of 31/100. The JavaScript bundle, the API requests, the database lookups, even with minimal UI to render, had a baseline score in the thirties! Along with customer feedback, this helped to catalyze a renewed commitment to performance at Harvie. Through numerous discussions, we walked through each step of page load, from networking to rendering, and identified where we could improve. After a year of rewrites and upgrades, our remaining detriment to overall performance was our frontend. We had been converting our Symfony twig templates into React SPA components and fell into the common problem of creating "request waterfalls", while our user had to stare at a loading screen. We needed a change, and for us, that was Remix. In this talk, I'll walk you through our team's journey with performance and how Remix has become a natural progression of that.

19 min
18 Nov, 2022

Video Summary and Transcription

This Talk discusses Harvey's performance journey and how it led to the adoption of Remix. The engineering team addressed scaling and performance issues through backend fixes and frontend improvements. The redesign focused on loading products by category and prioritizing performance. The implementation of Remix resulted in improved performance and a reduction in API requests. The focus on long-term scalability is essential for handling a growing product list and customer base.

1. Harvey's Performance Journey

Short description:

Hey everyone, welcome! My name is Emily Kaufman. Today I'll talk about Harvey's performance journey and how it led us to Remix. Harvey started as a CSA program, but during the pandemic, it grew into a full grocery store. We faced growing pains and poor performance despite the move to React. The underlying architecture couldn't handle the load.

[♪ Music playing ♪ Hey everyone, welcome! My name is Emily Kaufman. I'm a software engineer based out of Pittsburgh, Pennsylvania. I actually gave this talk at the- I was a backup speaker at the first Remix conference, so if you ended up watching that already this might sound pretty familiar. But my talk today is going to be about the performance journey that Harvey, the company where I work, has undergone over the past few years and how that ultimately led us to Remix.

Alright, so Harvey is a grocery delivery service, where all of our products come from local farms and producers. So it started out, I think about ten years ago, as a community supported agriculture program, if you're familiar with that. It's a CSA. Basically, you pay the farm some amount of money per year and then every week or every other week, you get a box of whatever they happen to have produced in that time. So it's a really great way to support local, support your local farms and producers. So what Harvey did is it provided a platform so you could actually customize what you were getting in your box. So up until about three years ago, two and a half years ago, that's all Harvey really did. We had a number of farms on the platform from all over and we had provided the way for the customers to come in and login, view the contents of their box, make additions if they wanted to swap out stuff and then they would wait for their delivery.

And then the pandemic hit. So, you might remember at the beginning the world was starting to close down. A lot of people in the Pittsburgh area, turned to Harvey as their main source of groceries, to avoid having to go into a grocery store. And on the other side of that, all these producers that were used to going to farmers markets, setting up booths somewhere, so that you could come and actually make purchases, they didn't really have a place to go anymore. And so they were coming on to Harvey as a producer in order to stay in business. So, as I'm sure you can imagine, we had this massive influx of both customers and producers and Harvey began to grow and evolve from this CSA program into a full grocery store.

So, of course, during any kind of large scale growth in a short amount of time like this you're going to experience some growing pains and we absolutely did. This is a Lighthouse test that I ran in late 2020, and it was just a simple content page in Harvey, which is a Symfony application, and we got this performance score. This was the JavaScript bundle, the API requests, database lookups, even with the minimal UI to render on this basic content page, we had a baseline score in the 30s. So this wasn't correct. Something wasn't adding up. This, along with some upset customers, some customer feedback helped to catalyze this renewed interest and commitment to performance at Harvey. The catalog page, which is where you go to view all the products, had recently, I think a year prior to that, been converted from the Symfony, jQuery, Twig combination into part of a new React single page application. And this was hit the hardest. So the dump to React had addressed many UX concerns that we had. It modernized our tech stack, but it was still falling short in terms of performance. So the underlying architecture just couldn't handle the weight of all the new products. It was taking upwards of tens of seconds just to add something to your cart or remove something or do a swap.

2. Fixes for Scaling and Performance

Short description:

Many members were dropping off the site due to poor scaling caused by the significant increase in products offered. The engineering team triaged the issues into quick wins, involved fixes, and future redesign plans. DevOps and networking were mostly handled by services and tooling. Back-end fixes included image optimization, caching, and updating endpoints and database lookups. Front-end improvements focused on reducing bundle size and removing unnecessary localization packages.

And so many members were just dropping off the site entirely. But we have to remember that we had gone from offering maybe 30 to 40 products a couple of years ago to probably over 600 at this point. And so the page just wasn't scaling correctly. So when we think about fixes for this, our first iteration was kind of this crisis mode. Our engineering team got together and we said what can we do in the short term to fix a few of these issues.

So we spent a few hours sitting around the network panel in the performance tab and just walking through every step of page load and organizing what we saw into a few groups based on who would actually be working on them. So we had DevOps and networking. We had the back end, which would include API and the database, and then we had the front end. And from there, we took what we found and we triaged this into quick and easy wins, involved fixes, and this could be part of a future redesign.

For DevOps and networking, we didn't have to do too much here because it was mostly handled by services and tooling. But it was worth it to at least sit down and walk through it and make sure there weren't any bottlenecks. For the back end API, we had quite a few issues around image loading on the site. This is a hero banner that we have on almost all of our pages, and for some reason it was taking like seven seconds to load and seemed to be blocking first paint. So we did a bunch of work around image optimization and caching, and that took actual seconds off of our page load time. So I can't speak to everything that my back end coworkers did, but more involved fixes included updating the endpoints and database lookups to make sure that we're only querying the minimum needed, trying to avoid unnecessary and computationally expensive operations. An example of this is like anything that would have to go through a third party provider. Like if we were checking a user's credit card information for one, we might only be doing this on like one or two pages. So it shouldn't be in a top level like layout component because then it's going to be happening way more than it needs to. Overall, all of these updates came around just because we sat down as a team and we stared at the dev panel for a few hours and just identified issues.

For the front end, aka my problem, because I'm the only front end developer at Harvey, this is how long it takes to actually download the content, how big the script is, testing slower connections and all that kind of thing. We identified several low hanging fruit that we knew would be a relatively small development effort but would drastically increase the user's experience. So we started there. For one, the bundle was just so big, it was too big. There was so much code that a user had to download before even getting to interact with the page. And we used Webpacks. We used Webpacks Bundle Analyzer plugin, if you're familiar with it, and this helped us identify many problem areas that we were able to tackle. One of these being our use of Moment.js localization packages. So we're mostly in the Pittsburgh area. We have farms all over, but mostly at least in the United States. So there were many localization packages that weren't really relevant to us at this time, so we removed those, setting the goal that we would eventually switch to using something else other than Moment.

3. Frontend Performance Improvements

Short description:

I went through every dependency in the package.json and removed unnecessary ones. We had a huge number of elements on the catalogue page, causing performance issues. We added lazy loading and split our code into separate projects to reduce bundle size. Text compression and minimizing third-party scripts also had a positive effect. Moving expensive API calls improved page load and interactions.

I went through every dependency in the package.json and just tried to figure out why it was there, if it was still needed, if it could be updated. And we were able to remove a good chunk from there.

So I have this fun, anxiety-inducing screenshot of the performance tab. This is our catalogue page. So one of the biggest problems that we had in the frontend, especially on the catalogue page, was just the sheer number of elements that we had. So I said before, at one point, maybe we had 40 product cards, each had an image and a few buttons, you know, normal ecommerce stuff. And now we have almost 600. So even without all the optimizations we're making now, two years ago, this page was fully functional, it was working fine. Because remember, Harvey never was meant to be bigger than a small CSA program. So one of the first things we did here, after seeing this, was to add lazy loading to the page, and it immediately stopped crashing, which makes sense. But you can see in the screenshot here, we're basically trying to load 500 images at once.

OK, so I won't dive too deeply into everything we did on the front end, because I only have 20 minutes, but I have a non-exhaustive list here. So better code splitting. This could even be more thoughtful code splitting. Just really taking the time to think about what needs to actually be loaded on every route. So with our single-page application, we didn't put a lot of time into considering which modules actually needed to be on each page. So one of the things this led to was we split our member and our admin code into two separate projects with their own builds, and then I made a components library where the two could share from it. So that was able to reduce our bundle size by a lot. We started using text compression. We added this to all of our JavaScript files. And I think our CSS using Gzip, and that had a positive effect. Minimize and defer third-party scripts. Sometimes this can feel like a constant battle with marketing. This is stuff like Zendesk, Hotjar, Analytics, anything that, you know, maybe marketing is using to track events on your website. And so we went through these and just made sure that all the ones that we were using were actually still needed, and tried to identify places that we could defer them where we could. Minimize expensive API calls. I kind of touched on this before, but a lot of this was just moving where we were making these calls, like out of a layout and into a component or into a route level. Just so that they weren't all being made on every page. So all these changes that I mentioned led to a substantial difference in terms of page load, and especially on mobile. And they helped somewhat with the page interactions after loading, like adding, removing, swapping card items.

4. Redesign and Performance Commitment

Short description:

Our redesign focused on loading products by category instead of loading every single product on the page. We moved parts of the product card to a details page and dropped functionality that didn't add enough value for the customer. This commitment to performance as an essential design tenet led to a leaner design. Although the redesign improved load time, the lighthouse score remained low. In early 2020, I purchased a remix license.

But we knew that our current design just wasn't going to support the new business model, the bigger grocery store model that we were trying to achieve. And so we knew it was time for a redesign.

One thing we did, this is our redesigned page here. Instead of loading every single product on the page, we loaded them by category. So we just weren't having as many elements on the page anymore. Fun note on this, though. Our categories are growing big enough now that we're starting to look into pagination or something like that, because of the growth.

We removed parts of the product card, the description, the producer information, and we moved these into a details page so that we weren't trying to load all of those for every product in the catalog. We generated multiple image sizes for each of these, so we had a thumbnail, which you can see at the bottom of this image here. We had a details view, which is this bigger one, the card view, and they matched how they'd actually be displayed on the page. We dropped some functionality, like the swap button. We used to have it so if you were in your shopping cart, you could swap out. Like, say you didn't want broccoli, you could swap it out for carrots. But it just wasn't adding enough value for the customer for the performance issue that it was causing, so we decided to get rid of it.

And that kind of led to this way of thinking. Like, when you include a commitment to performance as an essential tenet of design, so when you're thinking about performance every time you're building a new feature, the design's going to become leaner. So I find myself doing this a lot. So we'll be like, oh, we should add this to the site. This would do this. It would be really cool. And then we realize there's going to be a performance impact. And it's like, is it really worth it to add this much time onto page loads? Is it worth it to take that hit? And a lot of times the answer's no.

Okay, let's see if this loads. So our redesign page is going to be on the left and the existing page is on the right. The redesign page you can see is loading substantially faster than the existing page. But it's still not enough. I wanted to get rid of all the spinners. And despite the fact that our load time was so much better, our lighthouse score was still fairly low, which was surprising.

All right, into a remix. In early 2020, I had bought myself a remix license as a birthday present to myself.

5. The Power of Remix for Performance

Short description:

I've been testing Remix on hobby projects and realized its potential to solve our performance issues. We successfully implemented Remix on our new grocery pages, resulting in improved performance. Remix minimizes the request waterfall, reducing API requests and improving user experience. We plan to migrate more pages to Remix and prioritize performance and scalability in all our releases. Our growing product lists and customer base necessitate a focus on long-term scalability.

And I've been messing around with it on several hobby projects, just testing out what it could do and just in general having a good time. It was for myself. And I realized that the co-location of page load data requests with the component, with the layout, might help us break out of this spin again problem that we're having, as the remix team calls it. So I don't have anything deployed publicly yet, as much as I want to, but I was able to get our new grocery pages up and running with remix. And I think the results speak for themselves.

I ran this lighthouse report just to get an idea of if, you know, it's going to make any difference for us. And yeah, you can see. So one of remix's selling points is that it minimizes the request waterfall. And that was the case for us in some parts of the application. We were making multiple API requests on page load and you just had that spinner going in the UI. Let's look at this video. Is that playing? I'll do it again. So fast. There we go.

Okay, so this video is slightly skewed because the existing application has more third-party integrations like a support widget and a few more features overall. But you can still see that second wave of the requests that were fired in the use effects in the component. So that's on our existing page. If I pop over to Remix, this is the same waterfall but on the Remix version. So in the Remix version, we have a longer document load time but a minimal number of additional requests. And since we're only loading what's needed for that one page, this JavaScript that does load is a lot smaller. So this is all without taking full advantage of some of the features like nested routing which we're starting to work on now. But yeah, it was like an immediate improvement for us. It was really exciting.

So going forward, our plan is to continue to keep moving these existing pages that are currently in our member-facing code into our Remix application till we get to a point where we have enough that we can deploy it and have it be public. But using the framework has just become a natural progression of our performance journey at Harvey, and it's absolutely been worth the effort, and it helps that I really enjoy working in it. So if we've learned anything over the past two years, it's that performance and building for scale are non-negotiable when we're releasing new features. It has to be baked into everything that we do and constantly check for regression. So our product lists, our customer base, they're still growing at a rapid pace. And yeah, I feel confident that what we're doing now is going to support what the business looks like maybe like five to ten years down the road. So it's an exciting time. All right. Thank you for listening. Again, my name is Emily Kaufman. If you have any questions, you can find me on Twitter or you can read the blog post behind this presentation on my website.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
Top Content
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Top Content
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Advanced Conference 2021React Advanced Conference 2021
39 min
Don't Solve Problems, Eliminate Them
Top Content
Humans are natural problem solvers and we're good enough at it that we've survived over the centuries and become the dominant species of the planet. Because we're so good at it, we sometimes become problem seekers too–looking for problems we can solve. Those who most successfully accomplish their goals are the problem eliminators. Let's talk about the distinction between solving and eliminating problems with examples from inside and outside the coding world.
Remix Conf Europe 2022Remix Conf Europe 2022
23 min
Scaling Up with Remix and Micro Frontends
Top Content
Do you have a large product built by many teams? Are you struggling to release often? Did your frontend turn into a massive unmaintainable monolith? If, like me, you’ve answered yes to any of those questions, this talk is for you! I’ll show you exactly how you can build a micro frontend architecture with Remix to solve those challenges.
React Summit 2023React Summit 2023
32 min
Speeding Up Your React App With Less JavaScript
Too much JavaScript is getting you down? New frameworks promising no JavaScript look interesting, but you have an existing React application to maintain. What if Qwik React is your answer for faster applications startup and better user experience? Qwik React allows you to easily turn your React application into a collection of islands, which can be SSRed and delayed hydrated, and in some instances, hydration skipped altogether. And all of this in an incremental way without a rewrite.
React Summit 2023React Summit 2023
23 min
React Concurrency, Explained
React 18! Concurrent features! You might’ve already tried the new APIs like useTransition, or you might’ve just heard of them. But do you know how React 18 achieves the performance wins it brings with itself? In this talk, let’s peek under the hood of React 18’s performance features: - How React 18 lowers the time your page stays frozen (aka TBT) - What exactly happens in the main thread when you run useTransition() - What’s the catch with the improvements (there’s no free cake!), and why Vue.js and Preact straight refused to ship anything similar

Workshops on related topic

React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Featured WorkshopFree
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
React Summit 2022React Summit 2022
136 min
Remix Fundamentals
Top Content
Featured WorkshopFree
Building modern web applications is riddled with complexity And that's only if you bother to deal with the problems
Tired of wiring up onSubmit to backend APIs and making sure your client-side cache stays up-to-date? Wouldn't it be cool to be able to use the global nature of CSS to your benefit, rather than find tools or conventions to avoid or work around it? And how would you like nested layouts with intelligent and performance optimized data management that just works™?
Remix solves some of these problems, and completely eliminates the rest. You don't even have to think about server cache management or global CSS namespace clashes. It's not that Remix has APIs to avoid these problems, they simply don't exist when you're using Remix. Oh, and you don't need that huge complex graphql client when you're using Remix. They've got you covered. Ready to build faster apps faster?
At the end of this workshop, you'll know how to:- Create Remix Routes- Style Remix applications- Load data in Remix loaders- Mutate data with forms and actions
JSNation 2023JSNation 2023
170 min
Building WebApps That Light Up the Internet with QwikCity
Featured WorkshopFree
Building instant-on web applications at scale have been elusive. Real-world sites need tracking, analytics, and complex user interfaces and interactions. We always start with the best intentions but end up with a less-than-ideal site.
QwikCity is a new meta-framework that allows you to build large-scale applications with constant startup-up performance. We will look at how to build a QwikCity application and what makes it unique. The workshop will show you how to set up a QwikCitp project. How routing works with layout. The demo application will fetch data and present it to the user in an editable form. And finally, how one can use authentication. All of the basic parts for any large-scale applications.
Along the way, we will also look at what makes Qwik unique, and how resumability enables constant startup performance no matter the application complexity.
React Summit 2023React Summit 2023
106 min
Back to the Roots With Remix
Featured Workshop
The modern web would be different without rich client-side applications supported by powerful frameworks: React, Angular, Vue, Lit, and many others. These frameworks rely on client-side JavaScript, which is their core. However, there are other approaches to rendering. One of them (quite old, by the way) is server-side rendering entirely without JavaScript. Let's find out if this is a good idea and how Remix can help us with it?
Prerequisites- Good understanding of JavaScript or TypeScript- It would help to have experience with React, Redux, Node.js and writing FrontEnd and BackEnd applications- Preinstall Node.js, npm- We prefer to use VSCode, but also cloud IDEs such as codesandbox (other IDEs are also ok)
Remix Conf Europe 2022Remix Conf Europe 2022
195 min
How to Solve Real-World Problems with Remix
Featured Workshop
- Errors? How to render and log your server and client errorsa - When to return errors vs throwb - Setup logging service like Sentry, LogRocket, and Bugsnag- Forms? How to validate and handle multi-page formsa - Use zod to validate form data in your actionb - Step through multi-page forms without losing data- Stuck? How to patch bugs or missing features in Remix so you can move ona - Use patch-package to quickly fix your Remix installb - Show tool for managing multiple patches and cherry-pick open PRs- Users? How to handle multi-tenant apps with Prismaa - Determine tenant by host or by userb - Multiple database or single database/multiple schemasc - Ensures tenant data always separate from others
Remix Conf Europe 2022Remix Conf Europe 2022
156 min
Build and Launch a personal blog using Remix and Vercel
Featured Workshop
In this workshop we will learn how to build a personal blog from scratch using Remix, TailwindCSS. The blog will be hosted on Vercel and all the content will be dynamically served from a separate GitHub repository. We will be using HTTP Caching for the blog posts.
What we want to achieve at the end of the workshop is to have a list of our blog posts displayed on the deployed version of the website, the ability to filter them and to read them individually.
Table of contents: - Setup a Remix Project with a predefined stack- Install additional dependencies- Read content from GiHub- Display Content from GitHub- Parse the content and load it within our app using mdx-bundler- Create separate blog post page to have them displayed standalone- Add filters on the initial list of blog posts