Exploring AR Try-On with React Native

Rate this content
Bookmark

React Native can be much more than a toolkit for building mobile UIs in JavaScript. We’ll explore how to use features beyond the core library, and use the built-in native module system to integrate AR capabilities into your mobile app.

Kadi Kraman
Kadi Kraman
20 min
17 Jun, 2022

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk discusses exploring AR Tryon with React Native, implementing AR try-on experiences in e-commerce apps, and considerations for AR development. It also covers the integration of AR platforms like ARKit and ARCore with React Native using the Duvero bridge. The Talk highlights the use of off-the-shelf solutions like Wanna's SDK for virtual try-on and Snap's AR technology and shopping extension. The importance of creating 3D models for AR Try-On and the challenges of writing native code for AR development are also mentioned.

Available in Español

1. Introduction to AR and VR

Short description:

I'm excited to talk about exploring AR Tryon with React Native. I'll discuss what's happening in the AR and VR space and share a case study of implementing a virtual try on feature. AR enhances the real world with computer-aided graphics, while VR offers a fully simulated experience. AR is becoming an expected part of the shopping experience, as seen with Amazon's AR-powered virtual shoe tryout. Examples include Bailey Nelson's AR try-on for glasses and Gucci's app with various AR features.

I'm really excited to be here today, and I'm going to be talking about exploring AR Tryon with React Native. So, here is the intro slide.

Hi, my name is Kadi. As Yanni said, amongst his very kind words, I'm currently the head of mobile development at Formidable. I've been an engineer for about ten years, but for the past five years since 2017, I've been building things in React Native. I've been very fortunate to be able to work on some really exciting projects in React Native, and I'm very excited to share one of them with you today.

So, in this talk, it's going to be in two parts. So, first, we are going to talk in general about what's happening in the AR VR space, and in the second part, I'm going to go through a bit of a case study of how we actually implemented a virtual try on feature in a React Native application.

So to kick us off, what's currently happening in the AR and VR space? So I'll do a quick clarification on the difference between AR and VR, because we tend to use those terms together. VR stands for virtual reality and it is a fully simulated experience, so it may look like the real world or it may look completely different. So the distinguishing feature in a VR experience is that you will have some sort of a VR headset, so it will be an Oculus or a Google Cardboard or an Index VR, but the point is, you need to wear something in order to have this immersive experience, and an example of that is, for example, Beatsaber which you'll see on the screen.

AR stands for Augmented Reality and it is a process in which the real world is enhanced by some computer-aided graphics. So usually, for an AR experience, you'll be looking through a screen, so it could be your phone screen, it could be your laptop screen that has a camera that's recording the real world, and then your device will add some kind of computer-aided enhancements. So for example, you could use it to place furniture in your space. If you've followed the news recently, you might have seen that last week there was a news story making rounds that Amazon has launched an AR-powered virtual shoe tryout experience in their iOS app. Now this in itself is not particularly groundbreaking. Amazon are nowhere near the first, and they're not going to be the last company that launches something like that. But the reason that I found it significant is that it's speaking to us of a trend that is happening more and more. Which is that in e-commerce, AR is going to stop being a gimmick. It's not stopping the cool thing that maybe 5% of the people would use. And it's more and more starting to be an expected part of our shopping experience. And just to give you a couple of examples of myself using AR in my shopping experience. So, these were a couple of things that I used before I even knew that I was going to be giving this talk. So, Bailey Nelson is a company that sells glasses. I like their glasses quite a lot. And on their website, they have an AR try-on where you can try on glasses before you purchase them. So, another thing, so, Gucci have an app that's just for experience features. They have a couple of different AR features. They have a shoe try on. They have a makeup try on and in this example, you can also try on different nail polishes.

2. Moving into an Unfurnished House

Short description:

I recently moved into a completely unfurnished house and had to furnish it from scratch. I struggled with placing a sofa in my weirdly shaped living room, but ultimately settled for a two-seater sofa.

I recently moved house. Well, I say recently. It was the beginning of the year. And it was my first time moving into a completely unfurnished house. It was my first time moving into, sorry, I can hear some echo. It was my first time moving into a completely unfurnished house, so I needed to furnish it from scratch. And here was me trying to figure out how to place a sofa into my very weirdly shaped living room. So, as you can see, that corner sofa was wishful thinking, but I did end up going with a smaller version of this, um, the two-seater sofa.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Building Fun Experiments with WebXR & Babylon.js
JS GameDev Summit 2022JS GameDev Summit 2022
33 min
Building Fun Experiments with WebXR & Babylon.js
Top Content
During this session, we’ll see a couple of demos of what you can do using WebXR, with Babylon.js. From VR audio experiments, to casual gaming in VR on an arcade machine up to more serious usage to create new ways of collaboration using either AR or VR, you should have a pretty good understanding of what you can do today.
Check the article as well to see the full content including code samples: article. 
Raising the Bar: Our Journey Making React Native a Preferred Choice
React Advanced Conference 2023React Advanced Conference 2023
29 min
Raising the Bar: Our Journey Making React Native a Preferred Choice
At Microsoft, we're committed to providing our teams with the best tools and technologies to build high-quality mobile applications. React Native has long been a preferred choice for its high performance and great user experience, but getting stakeholders on board can be a challenge. In this talk, we will share our journey of making React Native a preferred choice for stakeholders who prioritize ease of integration and developer experience. We'll discuss the specific strategies we used to achieve our goal and the results we achieved.
Opensource Documentation—Tales from React and React Native
React Finland 2021React Finland 2021
27 min
Opensource Documentation—Tales from React and React Native
Documentation is often your community's first point of contact with your project and their daily companion at work. So why is documentation the last thing that gets done, and how can we do it better? This talk shares how important documentation is for React and React Native and how you can invest in or contribute to making your favourite project's docs to build a thriving community
Bringing React Server Components to React Native
React Day Berlin 2023React Day Berlin 2023
29 min
Bringing React Server Components to React Native
Top Content
React Server Components are new topic in community, bunch of frameworks are implementing them, people are discussing around this topic. But what if we could use React Server Components in React Native? And bring all optimisation features that RSC allows to mobile apps? In this talk I would present what we are able to do with RSC in React Native!
Building Cross-Platform Component Libraries for Web and Native with React
React Advanced Conference 2021React Advanced Conference 2021
21 min
Building Cross-Platform Component Libraries for Web and Native with React
Top Content
Building products for multiple platforms such as web and mobile often requires separate code-based despite most of the components being identical in look and feel. Is there a way where we could use shared React component library on different platforms and save time? In this presentation I'll demonstrate one way to build truly cross-platform component library with a unique approach of using React & React Native in combination.
React Native Kotlin Multiplatform Toolkit
React Day Berlin 2022React Day Berlin 2022
26 min
React Native Kotlin Multiplatform Toolkit
Top Content
Combining the best of two cross-platform frameworks to build the ultimate multiplatform development experience.

Workshops on related topic

Introducing FlashList: Let's build a performant React Native list all together
React Advanced Conference 2022React Advanced Conference 2022
81 min
Introducing FlashList: Let's build a performant React Native list all together
Top Content
WorkshopFree
David Cortés Fulla
Marek Fořt
Talha Naqvi
3 authors
In this workshop you’ll learn why we created FlashList at Shopify and how you can use it in your code today. We will show you how to take a list that is not performant in FlatList and make it performant using FlashList with minimum effort. We will use tools like Flipper, our own benchmarking code, and teach you how the FlashList API can cover more complex use cases and still keep a top-notch performance.You will know:- Quick presentation about what FlashList, why we built, etc.- Migrating from FlatList to FlashList- Teaching how to write a performant list- Utilizing the tools provided by FlashList library (mainly the useBenchmark hook)- Using the Flipper plugins (flame graph, our lists profiler, UI & JS FPS profiler, etc.)- Optimizing performance of FlashList by using more advanced props like `getType`- 5-6 sample tasks where we’ll uncover and fix issues together- Q&A with Shopify team
Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
WorkshopFree
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
How to Build an Interactive “Wheel of Fortune” Animation with React Native
React Summit Remote Edition 2021React Summit Remote Edition 2021
60 min
How to Build an Interactive “Wheel of Fortune” Animation with React Native
Top Content
Workshop
Oli Bates
Oli Bates
- Intro - Cleo & our mission- What we want to build, how it fits into our product & purpose, run through designs- Getting started with environment set up & “hello world”- Intro to React Native Animation- Step 1: Spinning the wheel on a button press- Step 2: Dragging the wheel to give it velocity- Step 3: Adding friction to the wheel to slow it down- Step 4 (stretch): Adding haptics for an immersive feel
Effective Detox Testing
React Advanced Conference 2023React Advanced Conference 2023
159 min
Effective Detox Testing
Workshop
Josh Justice
Josh Justice
So you’ve gotten Detox set up to test your React Native application. Good work! But you aren’t done yet: there are still a lot of questions you need to answer. How many tests do you write? When and where do you run them? How do you ensure there is test data available? What do you do about parts of your app that use mobile APIs that are difficult to automate? You could sink a lot of effort into these things—is the payoff worth it?
In this three-hour workshop we’ll address these questions by discussing how to integrate Detox into your development workflow. You’ll walk away with the skills and information you need to make Detox testing a natural and productive part of day-to-day development.
Table of contents:
- Deciding what to test with Detox vs React Native Testing Library vs manual testing- Setting up a fake API layer for testing- Getting Detox running on CI on GitHub Actions for free- Deciding how much of your app to test with Detox: a sliding scale- Fitting Detox into you local development workflow
Prerequisites
- Familiarity with building applications with React Native- Basic experience with Detox- Machine setup: a working React Native CLI development environment including either Xcode or Android Studio
Deploying React Native Apps in the Cloud
React Summit 2023React Summit 2023
88 min
Deploying React Native Apps in the Cloud
WorkshopFree
Cecelia Martinez
Cecelia Martinez
Deploying React Native apps manually on a local machine can be complex. The differences between Android and iOS require developers to use specific tools and processes for each platform, including hardware requirements for iOS. Manual deployments also make it difficult to manage signing credentials, environment configurations, track releases, and to collaborate as a team.
Appflow is the cloud mobile DevOps platform built by Ionic. Using a service like Appflow to build React Native apps not only provides access to powerful computing resources, it can simplify the deployment process by providing a centralized environment for managing and distributing your app to multiple platforms. This can save time and resources, enable collaboration, as well as improve the overall reliability and scalability of an app.
In this workshop, you’ll deploy a React Native application for delivery to Android and iOS test devices using Appflow. You’ll also learn the steps for publishing to Google Play and Apple App Stores. No previous experience with deploying native applications is required, and you’ll come away with a deeper understanding of the mobile deployment process and best practices for how to use a cloud mobile DevOps platform to ship quickly at scale.
Introduction to React Native Testing Library
React Advanced Conference 2022React Advanced Conference 2022
131 min
Introduction to React Native Testing Library
Workshop
Josh Justice
Josh Justice
Are you satisfied with your test suites? If you said no, you’re not alone—most developers aren’t. And testing in React Native is harder than on most platforms. How can you write JavaScript tests when the JS and native code are so intertwined? And what in the world are you supposed to do about that persistent act() warning? Faced with these challenges, some teams are never able to make any progress testing their React Native app, and others end up with tests that don’t seem to help and only take extra time to maintain.
But it doesn’t have to be this way. React Native Testing Library (RNTL) is a great library for component testing, and with the right mental model you can use it to implement tests that are low-cost and high-value. In this three-hour workshop you’ll learn the tools, techniques, and principles you need to implement tests that will help you ship your React Native app with confidence. You’ll walk away with a clear vision for the goal of your component tests and with techniques that will help you address any obstacle that gets in the way of that goal.you will know:- The different kinds React Native tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting text, image, and native code elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RNTL tests and how to handle them- Options for handling native functions and components in your JavaScript tests
Prerequisites:- Familiarity with building applications with React Native- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Native Testing Library- Machine setup: Node 16.x or 18.x, Yarn, be able to successfully create and run a new Expo app following the instructions on https://docs.expo.dev/get-started/create-a-new-app/