WHOA, I Wrote This React App With My Voice!

Rate this content
Bookmark

Have you ever imagined writing code without even touching the keyboard? In this talk, I'll show you how I wrote a React app with my voice. But more importantly, I'll demonstrate how this technology can developers with disabilities to write code with ease and efficiency. Together, we'll explore the ways in which voice-activated AI assistants can revolutionize the way we think about coding. Join me as we explore the exciting possibilities of voice-activated AI programming in the React community and the ways in which it can make React coding more accessible and fun for everyone.

9 min
06 Jun, 2023

AI Generated Video Summary

Today we're going to build a React application with just our voice using GitHub Copilot, an AI peer programmer powered by OpenAI Codecs. It's important to be specific in your comments to get accurate suggestions from Copilot. Prompt engineering tips can be used to create different applications, such as a basic markdown editor and a simple to-do app. The application was tested successfully by adding and deleting to-do items using voice commands.

1. Introduction to GitHub Copilot

Short description:

Today we're going to build a React application with just our voice. GitHub Copilot is an AI peer programmer that helps you code faster. It draws context from your comments and code to suggest lines and functions. It's powered by OpenAI Codecs, a machine learning model that translates natural language into code. Prompt engineering is the practice of using prompts to get the desired output.

♪♪♪ Hey, folks, today we're going to build a React application with just our voice. Super excited, but I'm just going to give you a little bit of background before we dive into that. We're going to talk about GitHub Copilot, prompt engineering tips with GitHub Copilot, and then we're going to use our voices to build a React app.

You might be wondering, who am I? Who's this awesome person who could build apps with their voices? My name is Rizal Scarlett, I'm a developer advocate at GitHub. I have a background in software engineering and teaching underrepresented groups how to code. I'm also addicted to social media, so if you wanted to connect with me, my handle is at BlackGirlBytes on most platforms, including BlueSky and Mastodon, so find me on there.

Okay. GitHub Copilot, what is it? It's an AI peer programmer that helps you to code faster with less work. And so it feels like magic to me, just saying. It feels like it can read my mind, but being honest, it can't do either of those things, and sometimes it doesn't always spit out perfectly well-written, up-to-date code. Maybe sometimes it's referencing an older API, which is why I always encourage people to go back, review the code that is generated just as you would with a human peer programmer.

Okay, so what's happening if it's not magic? Under the hood, GitHub Copilot is drawing context from your comments and your code and suggesting individual lines and whole functions instantly. It's being powered by OpenAI Codecs. And you may be familiar with OpenAI because of ChatGPT or DALI, which is the AI image generation tool, but Codecs is another thing they created. And it's a machine learning model that translates natural language into code. Some background on Codecs is that it's a later version of GPT-3, which stands for Generative Pre-trained Transformer 3, which uses deep learning to produce human-like text. A lot of our favorite apps, like Duolingo, use GPT-3 for grammar correction. And the biggest difference between GPT-3 and Codecs is that Codecs has been fine-tuned for programming. So this is what's power in GitHub Copilot. Here's an example of GitHub Copilot in action. We have Melmykdev from Twitter, who is using GitHub Copilot to translate the strings, answer question and date. And all that they do is they provide context through the language code, so FR for French, JP for Japanese, ES for Spanish, and GitHub Copilot takes that context and translates it into the right words. So, maybe you're like, I've used GitHub Copilot and it's not working for me, it's not doing it. Let me introduce you to the concept of prompt engineering. And that's the practice of using prompts to get the output that you want. Here are my top three tips for prompt engineering. Give it a high-level task description. At the top of the file, describe the app's purpose so that it knows what it's doing. This is especially helpful if you're starting from a clean slate, there's no code in it. GitHub Copilot has no context.".

2. Providing Examples and Iterating

Short description:

In machine learning models, there's a concept called few shot learning where you provide examples to refine the model's output. It's important to be specific in your comments to get accurate suggestions from GitHub Copilot.

So, you have to give it somewhat a comment. Also providing examples, right? In machine learning models, they learn from examples. So, there's this concept of few shot learning where you feed your model examples and it gets a more refined idea of what it should be outputting based on certain inputs. And then also, iterating more for specificity. Maybe you wrote a really vague comment that says get data, but GitHub Copilot's like get what data? Tell it that you want to get the user data or the user IDs that belong to this specific individual or whatever. Get more specific. Go ahead and delete that comment that you wrote and delete the suggestion and reiterate on it.

3. Using Prompt Engineering and Copilot Voice

Short description:

Here's an example of using prompt engineering tips to create a basic markdown editor. GitHub Copilot can help with typing, making coding more accessible. Now, let's use Copilot Voice to create a simple to-do app. It can generate functions and use state variables. We'll also render input forms and lists of to-do items.

Okay, so here's an example of me using these prompt engineering tips that I just shared with you. At the top here, you can see that I wrote like a high level task description. I want to create a basic markdown editor. I want to use React hooks, I want to have a live preview of the markdown text and I get all that, but it's not exactly what I wanted. So I delete some parts of it and I write a comment that says I want to have a default value for my use date variable and I want that value to say type markdown here and then I have that and I try it out and it works pretty well. Like I got headings, I got bullets, I got bold, I got italics, I got links, it's working great.

But let's move into the voice part. Like why does that matter, right? GitHub Copilot can help you brainstorm, jog your memory, retain focus, but it could also help you type and you might be like, why do I need help typing? Maybe you have physical dexterity issues, maybe you have visual impairment. As we all get older, our vision will decline so this is something that's helpful in terms of coding accessibility. So let's dive into it. I'm gonna use Copilot Voice to create a simple to-do app. Let's get myself set up here. Okay. Go to line 16. Create a state variable for new to-do items. Create a function to handle adding new to-do items. Create a function for deleting to-do items. Save file.

So we see here that it was able to create a couple functions for us and use state variables, a few state variables, I mean, to say. So I just want to move over here for time's sake and get it to render a few elements for us. Create an input form to add to-do items. Create an unordered list of to-do items with a button to delete each item. Save file. So as you can see, this is not doctored because there's a little bit of error here. We don't really want the comments here. But that's all right. We're doing this live. All right. Next thing I want to do is I wanted to tell it to run the program. Run program.

4. Testing the Application and Conclusion

Short description:

The application is now live on Localhost 3000. I tested it by adding and deleting to-do items using my voice. To learn more about GitHub Copilot, visit github.com/features/Copilot. Thank you to React Summit for the opportunity to speak.

Awesome. So now this is live on Localhost 3000. So I can move over to my browser. I'm going to refresh. And we do have each to-do item here. I want to just maybe try it out to see if it works. Speak at React Summit. And let's try to delete. So it does work. We were able to build a simple application with our voices.

If you wanted to learn more about GitHub Copilot, you could go on over to github.com slash features slash Copilot. I want to thank React Summit for allowing me to speak. And thank you for listening. Bye. Thank you.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TechLead Conference 2023TechLead Conference 2023
35 min
A Framework for Managing Technical Debt
Let’s face it: technical debt is inevitable and rewriting your code every 6 months is not an option. Refactoring is a complex topic that doesn't have a one-size-fits-all solution. Frontend applications are particularly sensitive because of frequent requirements and user flows changes. New abstractions, updated patterns and cleaning up those old functions - it all sounds great on paper, but it often fails in practice: todos accumulate, tickets end up rotting in the backlog and legacy code crops up in every corner of your codebase. So a process of continuous refactoring is the only weapon you have against tech debt.In the past three years, I’ve been exploring different strategies and processes for refactoring code. In this talk I will describe the key components of a framework for tackling refactoring and I will share some of the learnings accumulated along the way. Hopefully, this will help you in your quest of improving the code quality of your codebases.

React Summit 2023React Summit 2023
24 min
Debugging JS
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
React Advanced Conference 2022React Advanced Conference 2022
22 min
Monolith to Micro-Frontends
Many companies worldwide are considering adopting Micro-Frontends to improve business agility and scale, however, there are many unknowns when it comes to what the migration path looks like in practice. In this talk, I will discuss the steps required to successfully migrate a monolithic React Application into a more modular decoupled frontend architecture.
React Advanced Conference 2023React Advanced Conference 2023
22 min
Power Fixing React Performance Woes
Next.js and other wrapping React frameworks provide great power in building larger applications. But with great power comes great performance responsibility - and if you don’t pay attention, it’s easy to add multiple seconds of loading penalty on all of your pages. Eek! Let’s walk through a case study of how a few hours of performance debugging improved both load and parse times for the Centered app by several hundred percent each. We’ll learn not just why those performance problems happen, but how to diagnose and fix them. Hooray, performance! ⚡️
React Summit 2023React Summit 2023
24 min
Video Editing in the Browser
Video editing is a booming market with influencers being all the rage with Reels, TikTok, Youtube. Did you know that browsers now have all the APIs to do video editing in the browser? In this talk I'm going to give you a primer on how video encoding works and how to make it work within the browser. Spoiler, it's not trivial!
JSNation 2023JSNation 2023
24 min
AI and Web Development: Hype or Reality
In this talk, we'll take a look at the growing intersection of AI and web development. There's a lot of buzz around the potential uses of AI in writing, understanding, and debugging code, and integrating it into our applications is becoming easier and more affordable. But there are also questions about the future of AI in app development, and whether it will make us more productive or take our jobs.
There's a lot of excitement, skepticism, and concern about the rise of AI in web development. We'll explore the real potential for AI in creating new web development frameworks, and separate fact from fiction.
So if you're interested in the future of web development and the role of AI in it, this talk is for you. Oh, and this talk abstract was written by AI after I gave it several of my unstructured thoughts.

Workshops on related topic

DevOps.js Conf 2024DevOps.js Conf 2024
163 min
AI on Demand: Serverless AI
Featured WorkshopFree
In this workshop, we discuss the merits of serverless architecture and how it can be applied to the AI space. We'll explore options around building serverless RAG applications for a more lambda-esque approach to AI. Next, we'll get hands on and build a sample CRUD app that allows you to store information and query it using an LLM with Workers AI, Vectorize, D1, and Cloudflare Workers.
React Advanced Conference 2023React Advanced Conference 2023
98 min
Working With OpenAI and Prompt Engineering for React Developers
Workshop
In this workshop we'll take a tour of applied AI from the perspective of front end developers, zooming in on the emerging best practices when it comes to working with LLMs to build great products. This workshop is based on learnings from working with the OpenAI API from its debut last November to build out a working MVP which became PowerModeAI (A customer facing ideation and slide creation tool).
In the workshop they'll be a mix of presentation and hands on exercises to cover topics including:
- GPT fundamentals- Pitfalls of LLMs- Prompt engineering best practices and techniques- Using the playground effectively- Installing and configuring the OpenAI SDK- Approaches to working with the API and prompt management- Implementing the API to build an AI powered customer facing application- Fine tuning and embeddings- Emerging best practice on LLMOps
React Summit Remote Edition 2021React Summit Remote Edition 2021
87 min
Building a Shopify App with React & Node
WorkshopFree
Shopify merchants have a diverse set of needs, and developers have a unique opportunity to meet those needs building apps. Building an app can be tough work but Shopify has created a set of tools and resources to help you build out a seamless app experience as quickly as possible. Get hands on experience building an embedded Shopify app using the Shopify App CLI, Polaris and Shopify App Bridge.We’ll show you how to create an app that accesses information from a development store and can run in your local environment.
JSNation 2022JSNation 2022
41 min
Build a chat room with Appwrite and React
WorkshopFree
API's/Backends are difficult and we need websockets. You will be using VS Code as your editor, Parcel.js, Chakra-ui, React, React Icons, and Appwrite. By the end of this workshop, you will have the knowledge to build a real-time app using Appwrite and zero API development. Follow along and you'll have an awesome chat app to show off!
GraphQL Galaxy 2021GraphQL Galaxy 2021
164 min
Hard GraphQL Problems at Shopify
WorkshopFree
At Shopify scale, we solve some pretty hard problems. In this workshop, five different speakers will outline some of the challenges we’ve faced, and how we’ve overcome them.

Table of contents:
1 - The infamous "N+1" problem: Jonathan Baker - Let's talk about what it is, why it is a problem, and how Shopify handles it at scale across several GraphQL APIs.
2 - Contextualizing GraphQL APIs: Alex Ackerman - How and why we decided to use directives. I’ll share what directives are, which directives are available out of the box, and how to create custom directives.
3 - Faster GraphQL queries for mobile clients: Theo Ben Hassen - As your mobile app grows, so will your GraphQL queries. In this talk, I will go over diverse strategies to make your queries faster and more effective.
4 - Building tomorrow’s product today: Greg MacWilliam - How Shopify adopts future features in today’s code.
5 - Managing large APIs effectively: Rebecca Friedman - We have thousands of developers at Shopify. Let’s take a look at how we’re ensuring the quality and consistency of our GraphQL APIs with so many contributors.
React Summit 2023React Summit 2023
109 min
Web Accessibility for Ninjas: A Practical Approach for Creating Accessible Web Applications
Workshop
In this hands-on workshop, we’ll equip you with the tools and techniques you need to create accessible web applications. We’ll explore the principles of inclusive design and learn how to test our websites using assistive technology to ensure that they work for everyone.
We’ll cover topics such as semantic markup, ARIA roles, accessible forms, and navigation, and then dive into coding exercises where you’ll get to apply what you’ve learned. We’ll use automated testing tools to validate our work and ensure that we meet accessibility standards.
By the end of this workshop, you’ll be equipped with the knowledge and skills to create accessible websites that work for everyone, and you’ll have hands-on experience using the latest techniques and tools for inclusive design and testing. Join us for this awesome coding workshop and become a ninja in web accessibility and inclusive design!