Optimising Images in Web and Native

Rate this content

It's safe to say that a lot of websites and apps these days are heavily reliant on images: product images, avatars, social sharing. With continued technological advances, the variety of device sizes you need to support keeps growing. How exactly are we to make the same page look great and feel performant on both a 16'' MacBook Pro with a retina display as well as a small 5 year old Android phone? We will look at some of the modern approaches to tackle just this problem on web and native - as well as what happens if you don't.

21 min
02 Jun, 2023

Video Summary and Transcription

This Talk focuses on optimizing images for mobile and web. It emphasizes the importance of image dimensions in optimizing images and shares a story about crashes encountered in a mobile app due to heavy image content. The Talk discusses image rendering and sizing, using dimensions and pixel density, optimizing images for different devices and screen sizes, and the importance of image size and format optimization. It recommends tools like Cloudinary for image optimization.

Available in Español

1. Introduction to Optimizing Images

Short description:

Hi, my name is Kati Cremmann. I'm the lead developer of ReactJS. This is a speech about optimizing images for mobile and web. Optimizing images involves optimizing the content and the file size. For content optimization, aspect ratio is crucial. For file size optimization, we need to minimize the assets.

Hi, my name is Kati Cremmann. I'm the lead developer of ReactJS. And this is a speech from our new guest speaker, Kati. Hello! Thank you. You are my true friends. Thank you, everyone, who stayed just before lunch. I will try to keep it brief. Thank you for that intro, Janne. It was amazing, as always.

So, hi, my name's Kati Cremmann. I'm the director of engineering for mobile at the moment at Promotable. We're an amazing JavaScript consultancy. We build things in React and React Native. And a lot of these things include images, which brings us to the topic of my talk, which is about optimizing images for mobile and web.

So, what exactly do I mean by optimizing images? You can kind of think of it in two ways. One way is around optimizing the content of the image. So, optimizing the cropping of the image. Are you going to use portrait or landscape? Where do you want to fit the subject of the image so it fits best the position you have on the screen? So, to give you an example, so, on the left, we have an original image, which is in portrait mode. And say that our goal was to use it as a hero image on a website or maybe like a banner image on Twitter. This is not an optimal image for this use case, because, as you can see, there isn't really a good way to crop it, so we end up with half a child. But this talk is not about cropping. If it was about cropping, the TLDR at the end would be to use an aspect ratio. So, as developers, we are not usually in charge of actually taking the images and positioning the images. That's for a job for the experts. But the way that we communicate what we need, the positions that we have on the screens, are using an aspect ratio. So, an aspect ratio is a ratio between the width and the height of an image. So, rather than saying, I need an image that's 750 by 500, you can tell your designer or a photographer that you need an image with an aspect ratio of three to two, and they can then give you the highest quality image they're able.

The other way to look at images is to look at the file size. So, this is something that we are more used to optimizing, because we are well aware that when we are rendering a mobile app or rendering a website, then the user has to download all the assets from whatever server before we can display them. So, we want to have these assets, images, JavaScript bundles, CSS, as small as possible.

2. Importance of Image Dimensions

Short description:

The width and height of an image matter a lot in optimizing images. As technology advances, the problem of image optimization keeps getting harder. For example, a high-resolution image can have a large file size, making it unsuitable for a good user experience. Reducing image dimensions can help in optimizing images. I will share a story that prompted this talk about the importance of image dimensions. When building a mobile app with heavy image content, we encountered crashes after a few pages.

And the other thing I want to talk about is the image dimensions. So, specifically, the width and the height of the image, which also matters. And I've added a question mark here that's because you might not realize just how much it matters.

When I was talking about optimizing images yesterday with a couple of people, a lot of people wondered how come we're still solving an image optimizing problem? People were trying to optimize images ten years ago. When I was talking about the topic of my talk, we kind of came to the conclusion that people have been doing this similar kind of thing for years and years and years. But the problem is that as technology advances, the problem we are solving keeps getting harder.

So to give you an example, this is an image that my friend Taz took of me a couple of weeks ago. Taz is a very good camera, top of the range. And it's great. I can zoom in, go all CSI, and figure out exactly what time it was when this image was taken. However, this kind of resolution comes with the cost of file size. So if I dig into the metadata of the file, I see that this image alone is 21 megabytes. And it's 8,000 by 5,000 pixels. So this is the source image that your photographer these days with a good camera is going to give you to use on your websites and mobile apps. And obviously you can't plop that in straight away because that is not going to give a good user experience. So if I wanted to use this on my website, I might reduce the dimensions, run it through the trustworthy Panda at TinyPNG, and I might end up with something much smaller. So this is 100 megabytes. Still not ideal, but it's usable. And I've reduced the image dimensions. And you notice that I've mentioned image dimensions several times. And you might be wondering why it's even important. And I'm going to tell you a story, and this is actually the story that prompted this talk.

And this is from a couple of years ago now, quite a few years ago, when I was building a mobile app. The mobile app was ready. It wasn't live yet, it wasn't live to users. But it was ready for content managers to add content and test things out to make sure everything was ready before releasing it. And the actual app doesn't really matter for the purposes of this story. But all you need to know is that it was very image heavy. The whole app, like every single page, had lots of really professionally shot custom images in various configurations. And what we found is that the content managers came back to us, and they kept saying that after a couple of pages, the app keeps crashing.

3. Image Rendering and Sizing

Short description:

Not just after hours of use. It would literally be after five or six pages. We found that if you comment out all the images, all the performance problems magically go away. Rendering images in native apps is more expensive. The larger the difference between the image and the size that we're rendering it in, the more expensive the calculation. So to fix this, the straightforward solution is to size the image based on what you actually need.

Not just after hours of use. It would literally be after five or six pages. And immediately, we were asking, oh, what device are you using? Is it a low end Android device? Because as React native developers, we tend to blame everything on low end Android devices. Unfortunately, in this case, they were all pretty much using the latest iPhones. So, we couldn't blame Android. But we were obviously thinking, if the iPhones can't do it, Android phones have no chance. So, we have to solve this.

So, what we did is, we, well, basically commented out code until we figured out what keeps breaking the app. And we found that if you comment out all the images, all the performance problems magically go away. This is obviously great, but not a good go to market strategy for an application that's all about images. So, then my friend console logged to the rescue. And what I did is, I just console logged the sizes of all the images that were being sent from the API. And I actually managed to find the actual screen shots I took at the time. And this is basically what we arrived at. This is just opening the app and navigating to a couple of pages. And it's kind of a mess, but I've highlighted some of the biggest culprits. And as you can see, based on the image that I showed before, these are clearly the source images that their photographers gave them as a source, as the beginning image, rather than cropped versions. And what you might not know, especially if you come from the web, is that rendering images in native apps is more expensive. So the way that rendering happens in native is that we calculate the size of the image, the area that you have available, and then we convert the image to fit that size. And this is done in memory. And the larger the difference between the image and the size that we're rendering it in, the more expensive the calculation. And obviously, if you're doing tons and tons of these expensive in-memory calculations all at the same time, you're going to run out of memory and your app's going to crash. So to fix this, the straightforward solution is to size the image based on what you actually need. So I'm going to talk you through how I would go about it. We're going to use this beautiful green image throughout. I'm going to give myself two constraints. We're going to use the screen width as the width of the image. And we're going to use an aspect ratio of 3.2 for the height. This is React Native code. We have a handy hook called use Window Dimensions.

4. Using Dimensions and Pixel Density

Short description:

You can use dimensions directly as an export. The image on the left is blurry because of pixel density. Pixel density determines how many pixels make up a display point. Most iPhones have a pixel density of 3, meaning one point is nine pixels.

You can also use dimensions directly as an export, but using a hook obviously means that you're running it on the web. Someone resizes the browser, you get the new value. I'm using a screen width from using window dimensions, and I'm calculating the relative height based on the width. And in the third arrow, I'm passing it straight into the image.

Now, this is a screen shot from my phone. And I've also done a crop so you can see it a bit, the quality of the image a bit better. And I printed out the width and the height that I get from React Native. So it's telling me that my width is 390 and my height is 260. So I go away, I take this image, and I crop it to 390 by 260. And I end up with this.

I'm not sure how well you can see it, but the image on the left is really blurry. But why? I've just taken the dimensions, exactly what I need from my phone, and I'll crop my image. So why is the image blurry? Is there anyone here that can tell me? Yes. Yes. Correct. Pixel density. I have a whole slide for it. Yeah, that is spot on. And if you're coming from the web, especially, that's not something that you usually have to think about. But on mobile, it is very important. So pixel density basically tells you how many... Sorry. Let me reverse it a little bit. So when you come from the web, especially, and you're starting writing styles, you might be thinking, oh, I no longer need to write pixels, or Ms, or Rems, after the style numbers. And the reason you don't have to write those is because you're not talking in pixels and not talking in Ms. You're talking in display points. And a display point may equal a pixel, but it may not. And pixel density basically tells you how many pixels does it take to make up a display point. So one-to-one, so all the iPhones would have used this, would be one pixel is one display point. However, most iPhones these days will use a pixel density of 3, which means that one point is actually nine pixels.

5. Optimizing Images for Different Devices

Short description:

To optimize images for different devices, you need to consider pixel densities. The size of the image in pixels is determined by the area in display points multiplied by the pixel ratio. The pixel ratio varies between one and 3.5. By multiplying the display points with the pixel ratio, you can obtain the actual image size needed for optimal display.

So, and if you ever had, for example, added an app icon, you would have been very familiar with this page, where you have to upload each image in 1x1x2x3. The reason you do that is to account for the pixel densities of the different iPhones. You'll have a similar thing on Android, but I think you have about six different versions to account for. And this is basically the formula. It's pretty straightforward. The size of the image in pixels is the size of the area in display points times the pixel ratio. There is an export in React Native, so you can import pixel ratio and then get the pixel ratio of the device or website you're currently on. And there's some examples. I just took the screenshot from the React Native website. We can see that the pixel ratio tends to vary between one and 3.5. It doesn't usually go higher, but knowing technology, it probably will eventually. And now, if I multiply the display points with the pixel ratio, I'll get the actual image that I needed, which was 1170 by 780. And now this will look crisp for my screen.

6. Optimizing Images for Different Screen Sizes

Short description:

Not everyone uses the same phone, so optimizing images for different screen sizes is crucial. Uploading one image and scaling it for different devices can burden lower-end devices. CDNs can help by caching images in specific sizes. However, managing multiple images can be costly. To optimize performance, use an array of supported sizes and calculate the width needed for the image.

Now, the problem is not everyone uses the same phone as me. I wish they did. That would make my life much simpler. However, even if you just look at iPhones, there are tons and tons of different sizes. So here, if I compare an iPhone 6 and an iPhone 12 Pro Max, you can see that the same image using the width of the screen and a 3 to ratio will be wildly different.

So if you only have one source of truth image that you're going to upload, you might be tempted to just upload the image in 1290 by 860 and everyone else will have to scale. The problem with that is that now your lower end devices that are already struggling for the ones bearing the burden of having to convert this image to the size you have available. Thankfully, this is a problem that has been solved. If you're using a CDN, pretty much all modern CDNs will have this functionality or you can build your own where basically the device calculates the area that they need, and they ask the image in specific size. So, your server would go, oh, you want 750 by 500, I've got this in the cache, there you go. Or if they don't have it, they would calculate it on the fly, save it in the cache for the next time, and return it.

The problem with this solution is that there is still too many different images. So, the first time, firstly, the first time we generate the image is going to have a couple seconds of cost where the image is being generated, so the user has to wait. And secondly, if you're using a managed service, it gets really expensive because most of these services will charge you based on transformations, so you're much cheaper, much better off using the cache if you can. So, the way to get around that is usually you'll end up having an array of supported sizes. So, I've gone for quite a few here. So, I've gone for like 15, but you will have supported stops where you say that I'm going to support an image that's 600 width or 1,200 width. And so, you calculate the width that you need the image in and you'll take the next best image, which means that you're still not having to rent anything too difficult, but you're still getting the optimization of not having to do so many on the fly transformations.

All right. I've talked a lot about this. I did do a demo as well. So, you can try it out. So, this is something that's the most fun, I think, if you can try it on your phone. I wasn't expecting this to be quite so big. Oh wow. Okay. I'm not sure I can do this. I guess you can see the bits on the right, which is the important part. So, the code is on the left. If you can't see it, just trust me.

7. Image Rendering and Optimization

Short description:

I'm rendering three different images, including the full-size image and images with different pixel ratios and sizes. By generating a range of fixed sizes, you can optimize image rendering for different devices. Cloudinary is a recommended tool for image optimization, but there are other options depending on the available tools.

It's there. But, I've got three examples here. So, on the right, I'm rendering it in React Native Web at the moment. But if you go here, and you have the Expo Snack app. I'm going to put this slide with the QR code up later. You can just run it on your phone, and you can see how these images run on your own phone. But on the web, so I'm rendering three different images.

So, I've got the full size image in all its glory. Then, I've got... So, at the top here, you can see what the pixel ratio is. And here, we've got the pixel ratio of 1, thank God. But if I'm running it just normally on my display, it's a pixel ratio of 2. You can see the screen width is 333, and you can see this is the full-size image. This is the exact size image. This one, on your phone, if you try it, it's going to look blurry. This is the exact size image with the pixel ratio. And this is the exact size image with the particular sizes. So, for example, here, you can see that the width in pixels that we're looking for the image for is 333, but we go for the next best width, which is 350. Let me just go back to my slides. There you go. This is the slides to take a picture of, if you want to try it out. and you can just try it out on your phone, and you can see how the same image renders to you versus your friends. Hopefully, you found it useful, and thank you very much.

So, firstly, in this solution, you have this array of fixed sizes, so you actually generate all of those images during the build time? Yes. So, you can choose. Depending on your service as well, you can create the images in advance. If, for example, you don't want to use a service at all, you can just decide you're going to use these three images, you're going to generate them in advance, and you know that your API is always going to support them. But the more optimal solution would be to create them on the fly. But in your case, can you recommend any tools, for example, bundlers, or things that do this image optimization for you, or is that exercise left for the reader? Personally, I like using Cloudinary, but they can be quite expensive. So, I think it's also the tools that you have available.

8. Optimizing Image Size and Format

Short description:

If you already have an image CDN, they will probably do it for you. Is there a proper way to size images without having to choose the next best version? The original method is to size the image exactly for the screen. Another way to optimize image loading is to choose an appropriate image format. Use tools like Cloudinary's format auto parameter to serve the best format for the platform. Splurging on tools is worth it when it comes to image optimization. Time is more important than money in this scenario. Thank you, Cady, for joining us.

If you already have an image CDN, then they will probably do it for you. Nice.

Here's the next question. Is there a proper way to size images without having to choose the next best version, requiring you to have an array of sizes at the ready? So, basically, if you don't do this fixed size kind of thing, what is the best way to go about it? So, it would be the original method that I showed, which is exactly the size for the screen. So, you will get the best size always for the area that's available, but it could be slower if you don't have that many users with that device. So, if you have one user on an iPhone X and 10,000 users on an iPhone 12, the iPhone X user will always, whenever they go on the page, the server will have to generate those images, whereas the iPhone 12 users will be all using the same cache. Nice.

And the other way, of course, to optimize image loading is to choose an appropriate image format for the type of image. Have you played around with these modern image formats and how those sort of effects solve the same problem? I think, to be honest, image is one of those things where I tend to pay to use the good tool for the job. For example, not that I'm advertising them in particular, but if you were to use Cloudinary, they have a parameter called format auto, which will serve the best format for the platform that you're requesting from. If you go on Chrome versus native iPhone versus native Android, it actually sends the same image in a different file format. Yeah, but I like the point about splurging on tools a little bit. This is not something that you want to try to solve for yourself if you do have the budget for it. Yeah, absolutely. I mean, it's like time versus money, right? In this scenario, I feel like my time is more important than some people's money. Yes. Image is nothing. Thirst is everything. Exactly.

Okay, you're thirsty. All right. Thank you, Cady. Speaking of thirst and hunger, we'll let people go for lunch, but you can find Cady at the speaker Q&A room after this. Thank you so much, Cady, for joining us. Awesome. Thank you, Yanni. I appreciate it.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
Top Content
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
React Summit 2023React Summit 2023
32 min
Speeding Up Your React App With Less JavaScript
Too much JavaScript is getting you down? New frameworks promising no JavaScript look interesting, but you have an existing React application to maintain. What if Qwik React is your answer for faster applications startup and better user experience? Qwik React allows you to easily turn your React application into a collection of islands, which can be SSRed and delayed hydrated, and in some instances, hydration skipped altogether. And all of this in an incremental way without a rewrite.
React Summit 2023React Summit 2023
23 min
React Concurrency, Explained
React 18! Concurrent features! You might’ve already tried the new APIs like useTransition, or you might’ve just heard of them. But do you know how React 18 achieves the performance wins it brings with itself? In this talk, let’s peek under the hood of React 18’s performance features: - How React 18 lowers the time your page stays frozen (aka TBT) - What exactly happens in the main thread when you run useTransition() - What’s the catch with the improvements (there’s no free cake!), and why Vue.js and Preact straight refused to ship anything similar
JSNation 2022JSNation 2022
21 min
The Future of Performance Tooling
Top Content
Our understanding of performance & user-experience has heavily evolved over the years. Web Developer Tooling needs to similarly evolve to make sure it is user-centric, actionable and contextual where modern experiences are concerned. In this talk, Addy will walk you through Chrome and others have been thinking about this problem and what updates they've been making to performance tools to lower the friction for building great experiences on the web.

Workshops on related topic

React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Featured WorkshopFree
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
JSNation 2023JSNation 2023
170 min
Building WebApps That Light Up the Internet with QwikCity
Featured WorkshopFree
Building instant-on web applications at scale have been elusive. Real-world sites need tracking, analytics, and complex user interfaces and interactions. We always start with the best intentions but end up with a less-than-ideal site.
QwikCity is a new meta-framework that allows you to build large-scale applications with constant startup-up performance. We will look at how to build a QwikCity application and what makes it unique. The workshop will show you how to set up a QwikCitp project. How routing works with layout. The demo application will fetch data and present it to the user in an editable form. And finally, how one can use authentication. All of the basic parts for any large-scale applications.
Along the way, we will also look at what makes Qwik unique, and how resumability enables constant startup performance no matter the application complexity.
React Day Berlin 2022React Day Berlin 2022
53 min
Next.js 13: Data Fetching Strategies
Top Content
- Introduction- Prerequisites for the workshop- Fetching strategies: fundamentals- Fetching strategies – hands-on: fetch API, cache (static VS dynamic), revalidate, suspense (parallel data fetching)- Test your build and serve it on Vercel- Future: Server components VS Client components- Workshop easter egg (unrelated to the topic, calling out accessibility)- Wrapping up
React Advanced Conference 2022React Advanced Conference 2022
81 min
Introducing FlashList: Let's build a performant React Native list all together
Top Content
In this workshop you’ll learn why we created FlashList at Shopify and how you can use it in your code today. We will show you how to take a list that is not performant in FlatList and make it performant using FlashList with minimum effort. We will use tools like Flipper, our own benchmarking code, and teach you how the FlashList API can cover more complex use cases and still keep a top-notch performance.You will know:- Quick presentation about what FlashList, why we built, etc.- Migrating from FlatList to FlashList- Teaching how to write a performant list- Utilizing the tools provided by FlashList library (mainly the useBenchmark hook)- Using the Flipper plugins (flame graph, our lists profiler, UI & JS FPS profiler, etc.)- Optimizing performance of FlashList by using more advanced props like `getType`- 5-6 sample tasks where we’ll uncover and fix issues together- Q&A with Shopify team
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
React Summit Remote Edition 2021React Summit Remote Edition 2021
60 min
How to Build an Interactive “Wheel of Fortune” Animation with React Native
Top Content
- Intro - Cleo & our mission- What we want to build, how it fits into our product & purpose, run through designs- Getting started with environment set up & “hello world”- Intro to React Native Animation- Step 1: Spinning the wheel on a button press- Step 2: Dragging the wheel to give it velocity- Step 3: Adding friction to the wheel to slow it down- Step 4 (stretch): Adding haptics for an immersive feel