WHOA, I Wrote This React App With My Voice!

Rate this content
Bookmark

Have you ever imagined writing code without even touching the keyboard? In this talk, I'll show you how I wrote a React app with my voice. But more importantly, I'll demonstrate how this technology can developers with disabilities to write code with ease and efficiency. Together, we'll explore the ways in which voice-activated AI assistants can revolutionize the way we think about coding. Join me as we explore the exciting possibilities of voice-activated AI programming in the React community and the ways in which it can make React coding more accessible and fun for everyone.

Rizel Scarlett
Rizel Scarlett
9 min
06 Jun, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Today we're going to build a React application with just our voice using GitHub Copilot, an AI peer programmer powered by OpenAI Codecs. It's important to be specific in your comments to get accurate suggestions from Copilot. Prompt engineering tips can be used to create different applications, such as a basic markdown editor and a simple to-do app. The application was tested successfully by adding and deleting to-do items using voice commands.

Available in Español

1. Introduction to GitHub Copilot

Short description:

Today we're going to build a React application with just our voice. GitHub Copilot is an AI peer programmer that helps you code faster. It draws context from your comments and code to suggest lines and functions. It's powered by OpenAI Codecs, a machine learning model that translates natural language into code. Prompt engineering is the practice of using prompts to get the desired output.

♪♪♪ Hey, folks, today we're going to build a React application with just our voice. Super excited, but I'm just going to give you a little bit of background before we dive into that. We're going to talk about GitHub Copilot, prompt engineering tips with GitHub Copilot, and then we're going to use our voices to build a React app.

You might be wondering, who am I? Who's this awesome person who could build apps with their voices? My name is Rizal Scarlett, I'm a developer advocate at GitHub. I have a background in software engineering and teaching underrepresented groups how to code. I'm also addicted to social media, so if you wanted to connect with me, my handle is at BlackGirlBytes on most platforms, including BlueSky and Mastodon, so find me on there.

Okay. GitHub Copilot, what is it? It's an AI peer programmer that helps you to code faster with less work. And so it feels like magic to me, just saying. It feels like it can read my mind, but being honest, it can't do either of those things, and sometimes it doesn't always spit out perfectly well-written, up-to-date code. Maybe sometimes it's referencing an older API, which is why I always encourage people to go back, review the code that is generated just as you would with a human peer programmer.

Okay, so what's happening if it's not magic? Under the hood, GitHub Copilot is drawing context from your comments and your code and suggesting individual lines and whole functions instantly. It's being powered by OpenAI Codecs. And you may be familiar with OpenAI because of ChatGPT or DALI, which is the AI image generation tool, but Codecs is another thing they created. And it's a machine learning model that translates natural language into code. Some background on Codecs is that it's a later version of GPT-3, which stands for Generative Pre-trained Transformer 3, which uses deep learning to produce human-like text. A lot of our favorite apps, like Duolingo, use GPT-3 for grammar correction. And the biggest difference between GPT-3 and Codecs is that Codecs has been fine-tuned for programming. So this is what's power in GitHub Copilot. Here's an example of GitHub Copilot in action. We have Melmykdev from Twitter, who is using GitHub Copilot to translate the strings, answer question and date. And all that they do is they provide context through the language code, so FR for French, JP for Japanese, ES for Spanish, and GitHub Copilot takes that context and translates it into the right words. So, maybe you're like, I've used GitHub Copilot and it's not working for me, it's not doing it. Let me introduce you to the concept of prompt engineering. And that's the practice of using prompts to get the output that you want. Here are my top three tips for prompt engineering. Give it a high-level task description. At the top of the file, describe the app's purpose so that it knows what it's doing. This is especially helpful if you're starting from a clean slate, there's no code in it. GitHub Copilot has no context.".

2. Providing Examples and Iterating

Short description:

In machine learning models, there's a concept called few shot learning where you provide examples to refine the model's output. It's important to be specific in your comments to get accurate suggestions from GitHub Copilot.

So, you have to give it somewhat a comment. Also providing examples, right? In machine learning models, they learn from examples. So, there's this concept of few shot learning where you feed your model examples and it gets a more refined idea of what it should be outputting based on certain inputs. And then also, iterating more for specificity. Maybe you wrote a really vague comment that says get data, but GitHub Copilot's like get what data? Tell it that you want to get the user data or the user IDs that belong to this specific individual or whatever. Get more specific. Go ahead and delete that comment that you wrote and delete the suggestion and reiterate on it.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

A Framework for Managing Technical Debt
TechLead Conference 2023TechLead Conference 2023
35 min
A Framework for Managing Technical Debt
Top Content
Let’s face it: technical debt is inevitable and rewriting your code every 6 months is not an option. Refactoring is a complex topic that doesn't have a one-size-fits-all solution. Frontend applications are particularly sensitive because of frequent requirements and user flows changes. New abstractions, updated patterns and cleaning up those old functions - it all sounds great on paper, but it often fails in practice: todos accumulate, tickets end up rotting in the backlog and legacy code crops up in every corner of your codebase. So a process of continuous refactoring is the only weapon you have against tech debt.In the past three years, I’ve been exploring different strategies and processes for refactoring code. In this talk I will describe the key components of a framework for tackling refactoring and I will share some of the learnings accumulated along the way. Hopefully, this will help you in your quest of improving the code quality of your codebases.

Debugging JS
React Summit 2023React Summit 2023
24 min
Debugging JS
Top Content
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
Power Fixing React Performance Woes
React Advanced Conference 2023React Advanced Conference 2023
22 min
Power Fixing React Performance Woes
Top Content
Next.js and other wrapping React frameworks provide great power in building larger applications. But with great power comes great performance responsibility - and if you don’t pay attention, it’s easy to add multiple seconds of loading penalty on all of your pages. Eek! Let’s walk through a case study of how a few hours of performance debugging improved both load and parse times for the Centered app by several hundred percent each. We’ll learn not just why those performance problems happen, but how to diagnose and fix them. Hooray, performance! ⚡️
Monolith to Micro-Frontends
React Advanced Conference 2022React Advanced Conference 2022
22 min
Monolith to Micro-Frontends
Top Content
Many companies worldwide are considering adopting Micro-Frontends to improve business agility and scale, however, there are many unknowns when it comes to what the migration path looks like in practice. In this talk, I will discuss the steps required to successfully migrate a monolithic React Application into a more modular decoupled frontend architecture.
Building a Voice-Enabled AI Assistant With Javascript
JSNation 2023JSNation 2023
21 min
Building a Voice-Enabled AI Assistant With Javascript
Top Content
In this talk, we'll build our own Jarvis using Web APIs and langchain. There will be live coding.
Video Editing in the Browser
React Summit 2023React Summit 2023
24 min
Video Editing in the Browser
Video editing is a booming market with influencers being all the rage with Reels, TikTok, Youtube. Did you know that browsers now have all the APIs to do video editing in the browser? In this talk I'm going to give you a primer on how video encoding works and how to make it work within the browser. Spoiler, it's not trivial!

Workshops on related topic

AI on Demand: Serverless AI
DevOps.js Conf 2024DevOps.js Conf 2024
163 min
AI on Demand: Serverless AI
Top Content
Featured WorkshopFree
Nathan Disidore
Nathan Disidore
In this workshop, we discuss the merits of serverless architecture and how it can be applied to the AI space. We'll explore options around building serverless RAG applications for a more lambda-esque approach to AI. Next, we'll get hands on and build a sample CRUD app that allows you to store information and query it using an LLM with Workers AI, Vectorize, D1, and Cloudflare Workers.
Working With OpenAI and Prompt Engineering for React Developers
React Advanced Conference 2023React Advanced Conference 2023
98 min
Working With OpenAI and Prompt Engineering for React Developers
Top Content
Workshop
Richard Moss
Richard Moss
In this workshop we'll take a tour of applied AI from the perspective of front end developers, zooming in on the emerging best practices when it comes to working with LLMs to build great products. This workshop is based on learnings from working with the OpenAI API from its debut last November to build out a working MVP which became PowerModeAI (A customer facing ideation and slide creation tool).
In the workshop they'll be a mix of presentation and hands on exercises to cover topics including:
- GPT fundamentals- Pitfalls of LLMs- Prompt engineering best practices and techniques- Using the playground effectively- Installing and configuring the OpenAI SDK- Approaches to working with the API and prompt management- Implementing the API to build an AI powered customer facing application- Fine tuning and embeddings- Emerging best practice on LLMOps
Building a Shopify App with React & Node
React Summit Remote Edition 2021React Summit Remote Edition 2021
87 min
Building a Shopify App with React & Node
Top Content
WorkshopFree
Jennifer Gray
Hanna Chen
2 authors
Shopify merchants have a diverse set of needs, and developers have a unique opportunity to meet those needs building apps. Building an app can be tough work but Shopify has created a set of tools and resources to help you build out a seamless app experience as quickly as possible. Get hands on experience building an embedded Shopify app using the Shopify App CLI, Polaris and Shopify App Bridge.We’ll show you how to create an app that accesses information from a development store and can run in your local environment.
Build a chat room with Appwrite and React
JSNation 2022JSNation 2022
41 min
Build a chat room with Appwrite and React
WorkshopFree
Wess Cope
Wess Cope
API's/Backends are difficult and we need websockets. You will be using VS Code as your editor, Parcel.js, Chakra-ui, React, React Icons, and Appwrite. By the end of this workshop, you will have the knowledge to build a real-time app using Appwrite and zero API development. Follow along and you'll have an awesome chat app to show off!
Hard GraphQL Problems at Shopify
GraphQL Galaxy 2021GraphQL Galaxy 2021
164 min
Hard GraphQL Problems at Shopify
WorkshopFree
Rebecca Friedman
Jonathan Baker
Alex Ackerman
Théo Ben Hassen
 Greg MacWilliam
5 authors
At Shopify scale, we solve some pretty hard problems. In this workshop, five different speakers will outline some of the challenges we’ve faced, and how we’ve overcome them.

Table of contents:
1 - The infamous "N+1" problem: Jonathan Baker - Let's talk about what it is, why it is a problem, and how Shopify handles it at scale across several GraphQL APIs.
2 - Contextualizing GraphQL APIs: Alex Ackerman - How and why we decided to use directives. I’ll share what directives are, which directives are available out of the box, and how to create custom directives.
3 - Faster GraphQL queries for mobile clients: Theo Ben Hassen - As your mobile app grows, so will your GraphQL queries. In this talk, I will go over diverse strategies to make your queries faster and more effective.
4 - Building tomorrow’s product today: Greg MacWilliam - How Shopify adopts future features in today’s code.
5 - Managing large APIs effectively: Rebecca Friedman - We have thousands of developers at Shopify. Let’s take a look at how we’re ensuring the quality and consistency of our GraphQL APIs with so many contributors.
Web Accessibility for Ninjas: A Practical Approach for Creating Accessible Web Applications
React Summit 2023React Summit 2023
109 min
Web Accessibility for Ninjas: A Practical Approach for Creating Accessible Web Applications
Workshop
Asaf Shochet Avida
Eitan Noy
2 authors
In this hands-on workshop, we’ll equip you with the tools and techniques you need to create accessible web applications. We’ll explore the principles of inclusive design and learn how to test our websites using assistive technology to ensure that they work for everyone.
We’ll cover topics such as semantic markup, ARIA roles, accessible forms, and navigation, and then dive into coding exercises where you’ll get to apply what you’ve learned. We’ll use automated testing tools to validate our work and ensure that we meet accessibility standards.
By the end of this workshop, you’ll be equipped with the knowledge and skills to create accessible websites that work for everyone, and you’ll have hands-on experience using the latest techniques and tools for inclusive design and testing. Join us for this awesome coding workshop and become a ninja in web accessibility and inclusive design!