Figma to React With AI, Are We There Yet?

Rate this content
Bookmark

What if we could generate production-ready React components (including props, responsiveness, CSS styling, TypeScript typings, and more) from Figma designs, automatically? This seemingly impossible task is becoming a reality thanks to the latest advancements in AI (such as ChatGPT).


In this talk, we’ll discuss some of the techniques we are using at Anima to automate the boring parts of frontend development, generating React components from Figma designs. We’ll cover some algorithmic techniques, processes and deep-learning based approaches that could help you speed up your day-to-day frontend work.

21 min
23 Oct, 2023

Video Summary and Transcription

Today's Talk explores how AI can empower developers to write better React code and automate the process of converting Figma designs into code. It discusses the use of heuristics to convert Figma designs into working HTML code and the potential of AI in generating better CSS class names. The Talk also highlights the importance of generating code that follows team conventions and automating design and code updates. Finally, it emphasizes the benefits of using large language models to automate tasks and improve developer productivity.

Available in Español

1. AI Empowering React Code and Figma Translations

Short description:

Today we're going to talk about AI and how it will empower us to write better React code. We'll explore automating the tedious task of recreating designs into code and discuss translating Figma representations into React representations. Figma provides a hierarchy of nodes, supports components, properties, subcomponents, actions, and events. Accessing Figma information is easy with its API.

Hey, everyone. Today we're going to talk about AI and in particular, how the recent advancements in AI technologies are going to empower us in the near future to write better React code, faster while also automating away some of the boring parts of our job.

So before we start a short presentation, I'm Federico, I'm a software architect at Anima and our mission is to automate as much as possible the boring parts of front-end development. So if you're interested, visit animaapp.com. Enough said, let's get started.

As a front-end developer, I used to focus a lot on how to write good React code. But that's not what we are going to discuss today. Today we won't be developers trying to write React code, but rather developers trying to create a machine that writes React code for us automatically. And we're going to start this journey with a story.

I worked for about a year as a front-end developer. And my day-to-day job involved many of the problems I'm sure you're all very familiar with, like interfacing with APIs, state management, performance optimizations. But there was one particular task I found extremely tedious. Every time we had to implement a feature, a designer would come to me with a design, usually in Figma, and then asked me to make it happen as an actual feature. But before implementing the actual functionality, the business logic, what I really liked, my first job was to take the design and then painfully recreate it into code, with all the styling, structure, elements, and so on. And because recreating a pixel-perfect design with code is not easy, the process often requires some back and forth between developers and designers. Because I'm sure you know, designers are really good at spotting slight differences between their designs and the production code. Anyway, very tedious. To the point where many developers started to ask themselves a question, can we automate it? And if we go from a high-level perspective, the question is, can we translate the information in a Figma design into a code representation? And at first, it might seem like an impossible task, as the two representations look quite different.

So, let's dive a bit into what's possible in Figma. First of all, in a Figma design, nodes are structured in a hierarchy, which is very similar to the DOM structure in an HTML page. Figma also supports components, and those components support properties, and those properties can define things like text content, visibility, instances, and many other things. Figma also supports subcomponents. So we can have a button component instead of a car component. And finally, Figma even supports actions and events. So for example, I can say, when I click on this button, open this web page. And by now you might have realized that this information is not that different from our React code. It's kind of the same information, but in a totally different representation. And so our mission today is to translate the Figma representation into a React representation. Before we dive into this translation process, we briefly need to discuss how to actually access the information contained in a Figma design. And it turns out that Figma makes it really easy, because it not only provides an API to access the information about a design, but it provides two.

2. Converting Figma Design to Code with Heuristics

Short description:

We can convert a Figma design into code using heuristics. By using a small JavaScript function, we can convert any Figma design into working HTML code. The function takes a Figma node as input and decides the corresponding HTML tag. It then uses the getCSSAsync method to get an approximate CSS representation of the node styles. The function is called recursively for all the children, and the information is put together in an HTML string. Although the resulting code may not be production-ready, it works and can be extended to handle more cases and produce higher quality code.

One that is REST-based, and another that is JavaScript-based. If you're interested, I highly recommend checking out the documentation, but for the sake of this talk, you can basically think of a Figma design as a JSON document. This JSON structure contains all the design nodes, along with their style properties and other useful information. It's like HTML, but in JSON and using Figma-specific properties. The contained information is roughly the same.

So, we have this JSON document representing a design, right? How do we convert it into code? And there are multiple possible approaches, so let's see some of them. We start from the most basic one, heuristics. And you might have heard of the term heuristic before. For the sake of this talk, we'll refer to heuristics as a collection of rules that define the behavior of our system. In simple words, imagine some code that says, if this happens, do this, otherwise do that. Pretty straightforward.

And we can actually convert a Figma design into code by just using heuristics. In fact, here we've prepared a small JavaScript function which is less than 20 lines of code that can convert any Figma design into somewhat working HTML code. So, let's see what it does step by step. First of all, the function takes as input a Figma node. Then we need to decide which HTML tag the current node corresponds to and we use a very simple rule. If it's a text node, make it a span, otherwise make it a div. Then we use a Figma method called getCSSAsync which basically gives us an approximate CSS representation of the node styles. And finally, we basically call the function recursively for all the children. We can then put together all this information in an HTML string and if I open up Figma, I have a video here to show you. So basically if you open Figma and go to Plugin, Development, Open Console, this opens up a console you can paste any JavaScript you want to execute on. If I paste the function we just created, this produces some HTML code. Now if we get that HTML code and we paste it into CodePen, give it a sec, this is the card. That's pretty impressive.

And the thing is, let me go next, the thing is this is the resulting code and for sure it's not production ready. It's ugly, it's verbose, it uses inline styles, but the thing is, it works and it took us less than 20 lines of code. So you can imagine that even though it's not perfect, you can extend these heuristics to handle more and more cases and produce higher quality code. But even though heuristic can get us pretty far in creating high quality code, it's not a bulletproof solution and have its problems. For example, let's say that you want to generate code that uses CSS classes rather than inline styles. One of the problems you need to face is that you need to come up with good CSS class names for each node.

3. AI and Generating Better CSS Class Names

Short description:

The body node in Figma can be converted into a div with a class name 'body', but real-world designs often have poorly named layers. Heuristics lack creativity in generating good CSS class names. However, large language models like Chadjupt, Claude, and Lama have the ability to generate better names. These models are trained on vast datasets and can predict what comes next in code. By leveraging AI, we can now solve the problem of generating better CSS class names in seconds. This is just one example of the possibilities offered by AI in software development.

For example, the body node on Figma was, we need to come up with a class for that node inside the code. The first approach could be to use the Figma node name as a CSS class name. So, for example, you can see that the body node in Figma becomes a div with class body, the content becomes a div with class content and so on.

Now, in theory this works, but I'm not sure how familiar you are with real world Figma designs, but usually designers don't really care about proper layer naming. Instead of good names like body, content, title, you will see things like frame 23, frame 32, button 3 and so on. And if we feed this to our heuristics, the resulting code will have terrible class names.

And to expand on this, I prepared one of my favorite quotes, there are two hard problems in computer science, caching validation, naming things and off by one errors. And I love this quote because it's funny, but it's also really true. Naming things is hard also for humans. And we might not always realize it, but our job requires a high degree of creativity to solve problems. And it turns out that heuristics lack creativity. It's really hard to encode as a heuristic, a task that requires creativity. Like coming up with good CSS class names. Five years ago, this would have been a very hard problem to solve. You would have needed an entire team of machine learning engineers and weeks to tackle it. But things have changed.

You might have heard of this transformative recent technology that is, among other things, really good at creative tasks. And of course, I'm talking about large language models like Chadjupt, Claude, Lama, or many others. We won't be able to go in depth into large language models during this talk, but in a large language models are trained on enormous datasets, including open source projects on GitHub, Wikipedia, Stack Overflow, in general, a good chunk of the Internet. And their job, in a nutshell, is to always predict what comes next. So, for example, if I feed the model with, the CAD is on, it might decide that the table is what comes next. And because it has been trained on open source code, as well, it also knows the structure and conventions of popular frameworks like React. And so, if we go back to our previous example with bad CSS class names, it turns out we can solve the problem in 20 seconds by just asking, charge a bit. And here, I ask it to rewrite this React component to use better CSS names, and 20 seconds later, I have code that is using much better CSS class names. For example, user car container, user profile, user name heading, and so on.

And so, thanks to the recent advancements in AI, what used to take an entire team weeks now takes seconds. But this is just the tip of the iceberg of what's possible with these technologies. So, let's see another use case. At Anima, our goal is to go from any Figma design into code you would actually put in production. And that's a very difficult problem to solve.

4. Generating Code that Follows Team Conventions

Short description:

We generated high-quality code, but it didn't follow the team's conventions. Heuristic-based code generation is limited in handling various conventions. We can use a system that combines heuristic-generated code and team conventions to refactor the code. By using large language models, we can customize the code to follow the team's convention. These techniques are useful for migrations or rewrites. In the future, integrating similar technology into workflows will be common.

One of the common complaints we got from our users is that even though we generated high-quality code, they couldn't put it in production because it didn't follow their team's conventions. And that's one of the problems of heuristic-based code generation. It would be extremely hard to make a heuristic that is general enough to handle all the possible variations in conventions, code style, styling frameworks, state management, and so on.

So how do we actually generate code that follows every team's conventions? A few years ago, you might have seen an image like this. This is called neural style transfer, and it's a deep learning-based technique. The idea is you start from an image, in this case a Lamborghini, and a reference, in this case a Van Gogh painting. You feed them into a deep neural network, and you get as output a Lamborghini painted in Van Gogh style. In a nutshell, this process transfers the style of the reference, the Van Gogh painting, to the original image.

You might be wondering why are we even discussing this at a React conference. The reason is that, from a high-level perspective, we could frame our problem in a similar way. Let's say we have a heuristic that generates high-quality code, but unfortunately it's not personalized, it does not follow the user's conventions. What if there could be a system that takes as input both the high-quality heuristic-generated code and your team conventions, combining them, basically taking the original code and refactoring it to follow your team's convention?

Now, let's see an example. This is some basic React code that one of our heuristics generated. The code works fine, but our hypothetical user is not very happy about it because it does not follow their team's convention. Let's say they want to use arrow functions, they want to use named exports and many other things. Now, how can we solve that? We could extend our heuristics to be more and more and more complex, to handle more and more cases, but that's not scalable because you know how variable the conventions and styling and state management can be among teams.

What we can do today is, again, ask charging PT. I won't go too much into the details of the prompt we use, but let's see the key points. I start by framing the program. You are an expert frontend developer. You should refactor a React project to follow the given guidelines. Then there is some boiler plates, and then finally the guidelines. You should use Arrow functions to define components, and then components should use named exports. And then we provide the component code, and after about 30 seconds, we have React code that uses Arrow functions, that use named exports, and that's exactly what the user wanted.

So again, to reiterate on this, we started from a high quality heuristic-generated code that was generic, and then we used large language models to customize it to actually follow your team's convention. And by the way, these techniques are also extremely useful if your team is going through a migration or a rewrite. Those things can save weeks of time. And this is not the most exciting thing I want to show you today. I want to close the talk with what I think is a sneak peek into the future, because even though the technology is not there yet, I'm confident that in two or three years most of us will integrate something similar in their workflow. So, so far we discussed how you can turn a Figma design into React code.

5. Automating Design and Code Updates

Short description:

Updating existing components and screens is crucial, especially if your company uses a design system. With the help of large language models, tedious back-and-forth between designers and developers can be automated. Changes made in Figma can be automatically reflected in the code base, and vice versa. However, there may be challenges in handling more complex scenarios, as the AI may not always update the code as expected.

But that's a one-time thing. And what we need to do, what we really need to do, is updating existing components and screens rather than creating new ones. And that's especially relevant if your company uses a design system.

For example, a designer might add an icon on a bottom component in Figma and then some developer has to actually update the code to support that icon. And so I prepared a small proof of concept in one of our internal hackathons at Anima And I want to share it with you. So we start from a button in Figma, you can see here, and here's the corresponding code in our code base. We have the JSX, we have the CSS and let's pretend we are designers and we update this design. We changed the border radius to five. We changed the background color. We removed the icon.

Now, imagine if the designer could click, send changes, and then after a few seconds, our code base would be automatically updated. As you can see, the icon has been removed and the styles have been updated. The border radius, the background color, now all of this was done automatically using large language models. And the thing is, those large language models are so powerful that the process can also be done in reverse. And so, for example, I can go inside the style, update the radius from 20 to 1, and then a designer on the other side can hit pull changes. And after a few seconds, here we go, the border radius reflects the code.

Now the first time I saw this, I was mind blown. Imagine how much you can automate with a solution like this, like all the tedious back and forth between designers and developers. Even if you could automate 30, 40 percent of those small updates, that would be such a time saver. I'm going to show you the next slide. So here we go. Some might argue that the example we discussed so far was too trivial. How does it work with more complex scenarios? And to answer that question, let me discuss another key feature of large language models. So let's say you have this card in Figma will be the one on the top left and on the bottom right, you have the corresponding React code. Let's say that our designer changes the default description of the card from this is a test description to, I have changed the description. If I feed this design to the previous code that I showed the demo before, this is the result. As you can see, the AI, instead of updating the default description constant as I expected, basically updated the text in line. And that's not what I expected. I wanted it to update a constant. Now, let's pretend for a second that this code was not AI generated, but rather it was made by a new intern at the company.

6. Automating Tasks with Large Language Models

Short description:

Large language models can handle suggestions and iterate on their answers based on natural language suggestions. They can adapt to different scenarios and code bases, automating boring tasks and allowing us to focus on building features and delivering value. If you want to know more or have any questions, reach out to me on Twitter, YouTube, LinkedIn, or my website.

What would you do? Well, if there are problems in a pull request, you usually write review comments, and in this case, I suggest you should not remove the constant. And the thing is, large language models are trained to be conversational, which means they can handle suggestion and iterate on their answer based on those suggestions.

And if we rerun the system with my comment, this is the result. Bingo. The AI updated the constant exactly as we expected. And what makes, again, large language models so transformative is that they can adapt to very different scenarios and code bases while also being able to iterate on their answer based on suggestions in natural language, which is something that we are very used to do in pull requests.

And what makes me so excited today is that those technologies have the potential to automate away many of the boring tasks of our job so that we can actually focus on the really important things like building features and delivering value to the users. With that being said, thank you so much for listening. If you want to know more, you want to chat, you can reach out to me on Twitter, YouTube, LinkedIn and also my website. And if you want to know more about Anima and what we do, you can visit the website animaapp.com. Keep in mind that many of the things I've shown today are actually usable products that you can use today. So if you're interested and you want to know more, just reach out to me. I'll be happy to answer all of your questions.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Advanced Conference 2021React Advanced Conference 2021
47 min
Design Systems: Walking the Line Between Flexibility and Consistency
Top Content
Design systems aim to bring consistency to a brand's design and make the UI development productive. Component libraries with well-thought API can make this a breeze. But, sometimes an API choice can accidentally overstep and slow the team down! There's a balance there... somewhere. Let's explore some of the problems and possible creative solutions.
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
Atomic Deployment for JS Hipsters
Deploying an app is all but an easy process. You will encounter a lot of glitches and pain points to solve to have it working properly. The worst is: that now that you can deploy your app in production, how can't you also deploy all branches in the project to get access to live previews? And be able to do a fast-revert on-demand?Fortunately, the classic DevOps toolkit has all you need to achieve it without compromising your mental health. By expertly mixing Git, Unix tools, and API calls, and orchestrating all of them with JavaScript, you'll master the secret of safe atomic deployments.No more need to rely on commercial services: become the perfect tool master and netlifize your app right at home!
TestJS Summit 2021TestJS Summit 2021
36 min
Effective Performance Testing to your Server with Autocannon
Top Content
Performance testing expertise that is developed for a long time. In order to measure your server performance you need a tool that can efficiently simulate a lot of abilities and give you good measurements according your analysing criteria.Autocannon NPM library gave me exactly that - that library is super easy to install and has a very simple API to work with. Within a really short amount of time you can start do performance testing to your application and get good measurements in development environment and in your performance labs, and generate complicated testing scenarios.In this talk I will introduce Autocannon, explain how to efficiently analyse your server performance with it, and show how it helped me to understand complicated performance issues in my Node.js servers. At the end of this lecture, developers will be able to have the ability to integrate a fast and easy tool in order to measure your server performance.
Vue.js London 2023Vue.js London 2023
14 min
Domain Driven Design with Vue Applications
Introduction to Domain Driven Design- What is DDD?- Key principles of DDD- Benefits of using DDD in web application developmentDomain Modeling in Vue 3 Applications- How to design and implement domain models in Vue 3- Strategies for integrating domain logic with Vue's reactive data model and component-based architectureBest Practices for Implementing DDD in Vue 3- Strategies for organizing code in a way that follows DDD principles- Techniques for reducing coupling between domain and application logic- Tips for testing and debugging domain logic in Vue 3 applications

Workshops on related topic

TestJS Summit 2021TestJS Summit 2021
85 min
Automated accessibility testing with jest-axe and Lighthouse CI
Workshop
Do your automated tests include a11y checks? This workshop will cover how to get started with jest-axe to detect code-based accessibility violations, and Lighthouse CI to validate the accessibility of fully rendered pages. No amount of automated tests can replace manual accessibility testing, but these checks will make sure that your manual testers aren't doing more work than they need to.
TestJS Summit 2022TestJS Summit 2022
163 min
Automated Testing Using WebdriverIO
Workshop
In this workshop, I cover not only what WebdriverIO can do, but also how you'll be using it day-to-day. I've built the exercises around real-world scenarios that demonstrate how you would actually set things up. It's not just "what to do," but specifically "how to get there." We'll cover the fundamentals of Automated UI testing so you can write maintainable, useful tests for your website and/or web app.
TestJS Summit 2021TestJS Summit 2021
111 min
JS Security Testing Automation for Developers on Every Build
WorkshopFree
As a developer, you need to deliver fast, and you simply don't have the time to constantly think about security. Still, if something goes wrong it's your job to fix it, but security testing blocks your automation, creates bottlenecks and just delays releases...but it doesn't have to...

NeuraLegion's developer-first Dynamic Application Security Testing (DAST) scanner enables developers to detect, prioritise and remediate security issues EARLY, on every commit, with NO false positives/alerts, without slowing you down.

Join this workshop to learn different ways developers can access Nexploit & start scanning without leaving the terminal!

We will be going through the set up end-to-end, whilst setting up a pipeline, running security tests and looking at the results.

Table of contents:
- What developer-first DAST (Dynamic Application Security Testing) actually is and how it works
- See where and how a modern, accurate dev-first DAST fits in the CI/CD
- Integrate NeuraLegion's Nexploit scanner with GitHub Actions
- Understand how modern applications, APIs and authentication mechanisms can be tested
- Fork a repo, set up a pipeline, run security tests and look at the results
GraphQL Galaxy 2021GraphQL Galaxy 2021
82 min
Security Testing Automation for Developers on Every Build
WorkshopFree
As a developer, you need to deliver fast, and you simply don't have the time to constantly think about security. Still, if something goes wrong it's your job to fix it, but security testing blocks your automation, creates bottlenecks and just delays releases, especially with graphQL...but it doesn't have to...

NeuraLegion's developer-first Dynamic Application Security Testing (DAST) scanner enables developers to detect, prioritise and remediate security issues EARLY, on every commit, with NO false positives / alerts, without slowing you down.

Join this workshop to learn different ways developers can access NeuraLegion's DAST scanner & start scanning without leaving the terminal!

We will be going through the set up end-to-end, whilst setting up a pipeline for a vulnerable GraphQL target, running security tests and looking at the results.

Table of contents:
- What developer-first DAST (Dynamic Application Security Testing) actually is and how it works
- See where and how a modern, accurate dev-first DAST fits in the CI/CD
- Integrate NeuraLegion's scanner with GitHub Actions
- Understand how modern applications, GraphQL and other APIs and authentication mechanisms can be tested
- Fork a repo, set up a pipeline, run security tests and look at the results
React Advanced Conference 2022React Advanced Conference 2022
118 min
Rapid UI Development in React: Harnessing Custom Component Libraries & Design Systems
Workshop
In this workshop, we'll take a tour through the most effective approaches to building out scalable UI components that enhance developer productivity and happiness :-) This will involve a mix of hands-on exercises and presentations, covering the more advanced aspects of the popular styled-components library, including theming and implementing styled-system utilities via style props for rapid UI development, and culminating in how you can build up your own scalable custom component library.
We will focus on both the ideal scenario---where you work on a greenfield project---along with tactics to incrementally adopt a design system and modern approaches to styling in an existing legacy codebase with some tech debt (often the case!). By the end of the workshop, you should feel that you have an understanding of the tradeoffs between different approaches and feel confident to start implementing the options available to move towards using a design system based component library in the codebase you work on.
Prerequisites: - Familiarity with and experience working on large react codebases- A good understanding of common approaches to styling in React