Transforming Images Using AI Without Leaving Your React App

Rate this content
Bookmark

Image optimization libraries abstract away the hard parts of working with images so that you can focus on shipping. But these tools don’t let you do complex transformations to your existing images. Adding text, removing distracting backgrounds, or upscaling images to higher resolution all require you to download, transform, and re-upload your images. This talk will explore using AI image APIs and unpic-image to handle complex image transformations inside our React applications. We’ll remove and replace image backgrounds and use context-aware facial cropping to create responsive layouts.

5 min
13 Nov, 2023

Comments

Sign in or register to post your comment.

AI Generated Video Summary

Today's Talk discusses the importance of transforming images with AI in React apps and the benefits of using image CDNs. The speaker emphasizes the significance of images as a crucial component of websites and the time-consuming nature of adopting image best practices. They propose image transformation on the edge as a faster and easier alternative. The Talk also highlights the use of AI in image transformation, including removing backgrounds, cropping images, and upscaling pixelated images in a Contentful Next JS app. The combination of image CDNs and AI parameters ensures that images always look their best without leaving the React app.

1. Introduction to Image Transformation

Short description:

Today we're talking about transforming images with AI in your React app. It's important not to forget about images as they are the connective tissue to your sites. Adopting image best practices takes time, so we can explore image transformation on the edge as a faster and easier alternative.

Alright, so my name is Luis Paul. I'm a senior software engineer with ImageX. Today we're going to be talking about transforming images with AI without leaving your React app. But I'm going to go fast because there's a lot of content so excuse me, I'll try to slow down.

So first off, a little story time. As developers, we love to ship product. We're laser focused on features, customer experience and satisfaction as we should be. That's our job. But that makes it easy to forget about the parts that we don't code, about things like images. So you might find yourself in this situation where you shipped a viral product, people love it, but they're also DMing you saying, hey, it's fantastic but you're shipping 100 megabytes of images. Let's not do that. And you got to run and fix it. So this is why it's important not to forget about images. They are the connective tissue to your sites. Images are the way the customers experience your product for the first time and they matter more than we tend to realize.

So what should we do about it? We should adopt image best practices. There's a bunch of them, right? You got source sizes, aspect ratio, layout shifts, lazyloading, etc. And ensuring you hit all of them takes time. So it requires you to do things like generate variants for all of your images. How do we make this easier? So we gravitated to doing things like image transformation as a build step. But it's not great. Transforming at build time can take forever. The feature set is more limited in terms of transformations you can achieve and you have to do some careful maintenance of your tooling to do it right long-term. The DX in short just isn't great.

So what else can we do? We can do things like image transformation on the edge. This is fast, easy to integrate into any application. It's always up-to-date. You can spin up a new replicate endpoint pretty easily and it stays up-to-date for you. But it comes with some overhead. You gotta spin up new functions and every time you need a new transformation, and you have to do things like maintaining and invalidating caches, syncing the generated asset back to your bucket where your assets live.

2. Image Transformation with CDN and AI

Short description:

With an image CDN, you can take care of all the heavy lifting and progressively adopt best practices. In our Contentful Next JS app, we remove backgrounds, crop images using AI, and upscale pixelated images. We leverage the image CDN and AI parameters to ensure our images always look their best. That's transforming images using AI without leaving your React app.

And so it can get a little complicated which is why people gravitate to image CDNs. With an image CDN, you can take care of all that heavy lifting. And then you can progressively adopt it using something like unpick image which will take care of hitting all these best practices for you. It's fantastic because it's zero config with most of the big CDNs out there. You just drop your image URL that lives in the CDN. It knows what to do. You just give it a width and a height if you need it. And that's pretty much it.

Long story short, you don't need to reinvent the wheel to achieve best practices. So let's take a look at how you could actually use this in your applications. We're gonna have a Contentful Next JS app, it's gonna use an image optimization library. It'll use an image CDN to remove all that overhead and will leverage the CDN's AI parameters to do some fun transformations.

So, first off, we're going to remove backgrounds using AI. We wanna do some consistency enforcement on our site, make sure that things match our designs. So here we have our Contentful Next JS application. It's got a hero image. We wanna remove the background. All we need to do, using in CDN, is use its background removal parameter, save, and by appending that to the URL, we'll have a removed background. That was quick, easy, painless, thanks to using Unpick Image in the CDN. Let's go ahead and keep going. We got some AI facial cropping we wanna do. We wanna make sure the images are always focused on what's actually highly relevant and centered. So let's take a look at how we can do that. We've got an image here, and we have somebody walking down the hallway, but we can't really see this person's face, and we'd like to, so, why don't we go ahead and crop it around this person's face? The way we do that is we add some parameters to the URL. We'll set Frick crop, we'll add some width and height dimensions. And once we've done that, now we also wanna make sure that we use some facial recognition to automatically center the subject's face in the image. Easy. And now, last thing we wanna do, do some upscaling. This is my personal favorite, it's like a magic trick. You can add some definition to the pixelated images, you're going to increase the pixel density, and you're gonna do some things like, you know, enhance lightning and contrast. So, if we're thinking about our Contentful NxJS application, we can have an image component, that all our images use, but maybe we have a lot of user-generated content, it gets a little hairy, right? Not everybody has the latest and greatest stuff, they have some old content, we wanna upscale it. What we have to do in this case is add the upscale parameter, save that, and so now, with the upscale equals true in our image URL, our image gets automatically upscaled and that carousel looks great. So just recapping, what did we do? We talked a lot about a lot of stuff. We did a Contentful NxJS application, we used an image optimization library called Unpick Image, and then we used an image CDN to remove all that nasty overhead of having to invalidate caches or store images in the cloud and finally, we leveraged that CDN to use some AI parameters like facial recognition cropping, like background removal, like upscaling and make sure our images always looked their best when they were being displayed. That's transforming images using AI without leaving your React app and I am sufficiently out of breath. Thank you all so much.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

JSNation 2023JSNation 2023
24 min
AI and Web Development: Hype or Reality
In this talk, we'll take a look at the growing intersection of AI and web development. There's a lot of buzz around the potential uses of AI in writing, understanding, and debugging code, and integrating it into our applications is becoming easier and more affordable. But there are also questions about the future of AI in app development, and whether it will make us more productive or take our jobs.
There's a lot of excitement, skepticism, and concern about the rise of AI in web development. We'll explore the real potential for AI in creating new web development frameworks, and separate fact from fiction.
So if you're interested in the future of web development and the role of AI in it, this talk is for you. Oh, and this talk abstract was written by AI after I gave it several of my unstructured thoughts.
React Summit US 2023React Summit US 2023
30 min
The Rise of the AI Engineer
We are observing a once in a generation “shift right” of applied AI, fueled by the emergent capabilities and open source/API availability of Foundation Models. A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023. Emergent capabilities are creating an emerging title: to wield them, we'll have to go beyond the Prompt Engineer and write *software*. Let's explore the wide array of new opportunities in the age of Software 3.0!
JS GameDev Summit 2023JS GameDev Summit 2023
37 min
Building the AI for Athena Crisis
This talk will dive into how to build an AI for a turn based strategy game from scratch. When I started building Athena Crisis, I had no idea how to build an AI. All the available resources were too complex or confusing, so I just started building it based on how I would play the game. If you would like to learn how to build an AI, check out this talk!
TestJS Summit 2023TestJS Summit 2023
8 min
Code coverage with AI
In this lightning demo I will showcase how Codium, a cutting-edge generative AI tool, is revolutionizing code integrity. We will demonstrate Codium's ability to generate useful Mocha tests, taken from a public repository and highlight the seamless integration. You can see Codium as it transforms complex test scenarios into actionable insights, propelling code coverage forward. Join us for an insightful peek into the future of automated testing where speed meets quality!
React Summit 2023React Summit 2023
29 min
Improving Developer Happiness with AI
GitHub Copilot is an AI pair programmer that can help you write code faster and spend less time writing repetitive code.This session will cover some interesting use cases for Copilot that could shine a light on its possibilities. This ranges from prompting Copilot to suggest a function based on a comment, learning how to use a new framework, tackling a security or accessibility bug, better documenting your code, translating  code from one language to another, etc.Agenda:
Introduction to CoPilot
- What is Copilot
- How can you use it
- How it can help you write code faster
- Copilot Labs experimental features I will pick examples from the React ecosystem and show how we can fix Security Vulnerabilities and Accessibility issues in some components.

Workshops on related topic

DevOps.js Conf 2024DevOps.js Conf 2024
163 min
AI on Demand: Serverless AI
Featured WorkshopFree
In this workshop, we discuss the merits of serverless architecture and how it can be applied to the AI space. We'll explore options around building serverless RAG applications for a more lambda-esque approach to AI. Next, we'll get hands on and build a sample CRUD app that allows you to store information and query it using an LLM with Workers AI, Vectorize, D1, and Cloudflare Workers.
React Advanced Conference 2023React Advanced Conference 2023
98 min
Working With OpenAI and Prompt Engineering for React Developers
Workshop
In this workshop we'll take a tour of applied AI from the perspective of front end developers, zooming in on the emerging best practices when it comes to working with LLMs to build great products. This workshop is based on learnings from working with the OpenAI API from its debut last November to build out a working MVP which became PowerModeAI (A customer facing ideation and slide creation tool).
In the workshop they'll be a mix of presentation and hands on exercises to cover topics including:
- GPT fundamentals- Pitfalls of LLMs- Prompt engineering best practices and techniques- Using the playground effectively- Installing and configuring the OpenAI SDK- Approaches to working with the API and prompt management- Implementing the API to build an AI powered customer facing application- Fine tuning and embeddings- Emerging best practice on LLMOps
JSNation Live 2021JSNation Live 2021
81 min
Intro to AI for JavaScript Developers with Tensorflow.js
Workshop
Have you wanted to explore AI, but didn't want to learn Python to do it? Tensorflow.js lets you use AI and deep learning in javascript – no python required!
We'll take a look at the different tasks AI can help solve, and how to use Tensorflow.js to solve them. You don't need to know any AI to get started - we'll start with the basics, but we'll still be able to see some neat demos, because Tensorflow.js has a bunch of functionality and pre-built models that you can use on the server or in the browser.
After this workshop, you should be able to set up and run pre-built Tensorflow.js models, or begin to write and train your own models on your own data.