Remixing a Symfony
AI Generated Video Summary
This Talk discusses Harvey's performance journey and how it led to the adoption of Remix. The engineering team addressed scaling and performance issues through backend fixes and frontend improvements. The redesign focused on loading products by category and prioritizing performance. The implementation of Remix resulted in improved performance and a reduction in API requests. The focus on long-term scalability is essential for handling a growing product list and customer base.
1. Harvey's Performance Journey
Hey everyone, welcome! My name is Emily Kaufman. Today I'll talk about Harvey's performance journey and how it led us to Remix. Harvey started as a CSA program, but during the pandemic, it grew into a full grocery store. We faced growing pains and poor performance despite the move to React. The underlying architecture couldn't handle the load.
[♪ Music playing ♪ Hey everyone, welcome! My name is Emily Kaufman. I'm a software engineer based out of Pittsburgh, Pennsylvania. I actually gave this talk at the- I was a backup speaker at the first Remix conference, so if you ended up watching that already this might sound pretty familiar. But my talk today is going to be about the performance journey that Harvey, the company where I work, has undergone over the past few years and how that ultimately led us to Remix.
Alright, so Harvey is a grocery delivery service, where all of our products come from local farms and producers. So it started out, I think about ten years ago, as a community supported agriculture program, if you're familiar with that. It's a CSA. Basically, you pay the farm some amount of money per year and then every week or every other week, you get a box of whatever they happen to have produced in that time. So it's a really great way to support local, support your local farms and producers. So what Harvey did is it provided a platform so you could actually customize what you were getting in your box. So up until about three years ago, two and a half years ago, that's all Harvey really did. We had a number of farms on the platform from all over and we had provided the way for the customers to come in and login, view the contents of their box, make additions if they wanted to swap out stuff and then they would wait for their delivery.
And then the pandemic hit. So, you might remember at the beginning the world was starting to close down. A lot of people in the Pittsburgh area, turned to Harvey as their main source of groceries, to avoid having to go into a grocery store. And on the other side of that, all these producers that were used to going to farmers markets, setting up booths somewhere, so that you could come and actually make purchases, they didn't really have a place to go anymore. And so they were coming on to Harvey as a producer in order to stay in business. So, as I'm sure you can imagine, we had this massive influx of both customers and producers and Harvey began to grow and evolve from this CSA program into a full grocery store.
2. Fixes for Scaling and Performance
Many members were dropping off the site due to poor scaling caused by the significant increase in products offered. The engineering team triaged the issues into quick wins, involved fixes, and future redesign plans. DevOps and networking were mostly handled by services and tooling. Back-end fixes included image optimization, caching, and updating endpoints and database lookups. Front-end improvements focused on reducing bundle size and removing unnecessary localization packages.
And so many members were just dropping off the site entirely. But we have to remember that we had gone from offering maybe 30 to 40 products a couple of years ago to probably over 600 at this point. And so the page just wasn't scaling correctly. So when we think about fixes for this, our first iteration was kind of this crisis mode. Our engineering team got together and we said what can we do in the short term to fix a few of these issues.
So we spent a few hours sitting around the network panel in the performance tab and just walking through every step of page load and organizing what we saw into a few groups based on who would actually be working on them. So we had DevOps and networking. We had the back end, which would include API and the database, and then we had the front end. And from there, we took what we found and we triaged this into quick and easy wins, involved fixes, and this could be part of a future redesign.
For DevOps and networking, we didn't have to do too much here because it was mostly handled by services and tooling. But it was worth it to at least sit down and walk through it and make sure there weren't any bottlenecks. For the back end API, we had quite a few issues around image loading on the site. This is a hero banner that we have on almost all of our pages, and for some reason it was taking like seven seconds to load and seemed to be blocking first paint. So we did a bunch of work around image optimization and caching, and that took actual seconds off of our page load time. So I can't speak to everything that my back end coworkers did, but more involved fixes included updating the endpoints and database lookups to make sure that we're only querying the minimum needed, trying to avoid unnecessary and computationally expensive operations. An example of this is like anything that would have to go through a third party provider. Like if we were checking a user's credit card information for one, we might only be doing this on like one or two pages. So it shouldn't be in a top level like layout component because then it's going to be happening way more than it needs to. Overall, all of these updates came around just because we sat down as a team and we stared at the dev panel for a few hours and just identified issues.
For the front end, aka my problem, because I'm the only front end developer at Harvey, this is how long it takes to actually download the content, how big the script is, testing slower connections and all that kind of thing. We identified several low hanging fruit that we knew would be a relatively small development effort but would drastically increase the user's experience. So we started there. For one, the bundle was just so big, it was too big. There was so much code that a user had to download before even getting to interact with the page. And we used Webpacks. We used Webpacks Bundle Analyzer plugin, if you're familiar with it, and this helped us identify many problem areas that we were able to tackle. One of these being our use of Moment.js localization packages. So we're mostly in the Pittsburgh area. We have farms all over, but mostly at least in the United States. So there were many localization packages that weren't really relevant to us at this time, so we removed those, setting the goal that we would eventually switch to using something else other than Moment.
3. Frontend Performance Improvements
I went through every dependency in the package.json and removed unnecessary ones. We had a huge number of elements on the catalogue page, causing performance issues. We added lazy loading and split our code into separate projects to reduce bundle size. Text compression and minimizing third-party scripts also had a positive effect. Moving expensive API calls improved page load and interactions.
I went through every dependency in the package.json and just tried to figure out why it was there, if it was still needed, if it could be updated. And we were able to remove a good chunk from there.
So I have this fun, anxiety-inducing screenshot of the performance tab. This is our catalogue page. So one of the biggest problems that we had in the frontend, especially on the catalogue page, was just the sheer number of elements that we had. So I said before, at one point, maybe we had 40 product cards, each had an image and a few buttons, you know, normal ecommerce stuff. And now we have almost 600. So even without all the optimizations we're making now, two years ago, this page was fully functional, it was working fine. Because remember, Harvey never was meant to be bigger than a small CSA program. So one of the first things we did here, after seeing this, was to add lazy loading to the page, and it immediately stopped crashing, which makes sense. But you can see in the screenshot here, we're basically trying to load 500 images at once.
4. Redesign and Performance Commitment
Our redesign focused on loading products by category instead of loading every single product on the page. We moved parts of the product card to a details page and dropped functionality that didn't add enough value for the customer. This commitment to performance as an essential design tenet led to a leaner design. Although the redesign improved load time, the lighthouse score remained low. In early 2020, I purchased a remix license.
But we knew that our current design just wasn't going to support the new business model, the bigger grocery store model that we were trying to achieve. And so we knew it was time for a redesign.
One thing we did, this is our redesigned page here. Instead of loading every single product on the page, we loaded them by category. So we just weren't having as many elements on the page anymore. Fun note on this, though. Our categories are growing big enough now that we're starting to look into pagination or something like that, because of the growth.
We removed parts of the product card, the description, the producer information, and we moved these into a details page so that we weren't trying to load all of those for every product in the catalog. We generated multiple image sizes for each of these, so we had a thumbnail, which you can see at the bottom of this image here. We had a details view, which is this bigger one, the card view, and they matched how they'd actually be displayed on the page. We dropped some functionality, like the swap button. We used to have it so if you were in your shopping cart, you could swap out. Like, say you didn't want broccoli, you could swap it out for carrots. But it just wasn't adding enough value for the customer for the performance issue that it was causing, so we decided to get rid of it.
And that kind of led to this way of thinking. Like, when you include a commitment to performance as an essential tenet of design, so when you're thinking about performance every time you're building a new feature, the design's going to become leaner. So I find myself doing this a lot. So we'll be like, oh, we should add this to the site. This would do this. It would be really cool. And then we realize there's going to be a performance impact. And it's like, is it really worth it to add this much time onto page loads? Is it worth it to take that hit? And a lot of times the answer's no.
Okay, let's see if this loads. So our redesign page is going to be on the left and the existing page is on the right. The redesign page you can see is loading substantially faster than the existing page. But it's still not enough. I wanted to get rid of all the spinners. And despite the fact that our load time was so much better, our lighthouse score was still fairly low, which was surprising.
5. The Power of Remix for Performance
I've been testing Remix on hobby projects and realized its potential to solve our performance issues. We successfully implemented Remix on our new grocery pages, resulting in improved performance. Remix minimizes the request waterfall, reducing API requests and improving user experience. We plan to migrate more pages to Remix and prioritize performance and scalability in all our releases. Our growing product lists and customer base necessitate a focus on long-term scalability.
And I've been messing around with it on several hobby projects, just testing out what it could do and just in general having a good time. It was for myself. And I realized that the co-location of page load data requests with the component, with the layout, might help us break out of this spin again problem that we're having, as the remix team calls it. So I don't have anything deployed publicly yet, as much as I want to, but I was able to get our new grocery pages up and running with remix. And I think the results speak for themselves.
I ran this lighthouse report just to get an idea of if, you know, it's going to make any difference for us. And yeah, you can see. So one of remix's selling points is that it minimizes the request waterfall. And that was the case for us in some parts of the application. We were making multiple API requests on page load and you just had that spinner going in the UI. Let's look at this video. Is that playing? I'll do it again. So fast. There we go.
So going forward, our plan is to continue to keep moving these existing pages that are currently in our member-facing code into our Remix application till we get to a point where we have enough that we can deploy it and have it be public. But using the framework has just become a natural progression of our performance journey at Harvey, and it's absolutely been worth the effort, and it helps that I really enjoy working in it. So if we've learned anything over the past two years, it's that performance and building for scale are non-negotiable when we're releasing new features. It has to be baked into everything that we do and constantly check for regression. So our product lists, our customer base, they're still growing at a rapid pace. And yeah, I feel confident that what we're doing now is going to support what the business looks like maybe like five to ten years down the road. So it's an exciting time. All right. Thank you for listening. Again, my name is Emily Kaufman. If you have any questions, you can find me on Twitter or you can read the blog post behind this presentation on my website.