Let's build a TV Spatial Navigation

Rate this content
Bookmark
Slides

In this talk, I'll take you through my journey as I joined the team supporting our Smart TVs application and share my experience learning one of the most overlooked but essential pieces of functionality we have.

Sergio Avalos
Sergio Avalos
34 min
12 Dec, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Today's Talk is about building a spatial navigation library for Smart TVs. The speaker shares their experience and challenges in building applications for Smart TVs. They demonstrate the functionality of spatial navigation using React and React Router. The navigation engine class is developed to handle TV control events and navigate through elements. Circular navigation is implemented to make navigation easier for users in TV applications.

Available in Español

1. Introduction to Spatial Navigation

Short description:

Today we're going to talk about spatial navigation and how to build our own. I'll share what I learned and how we can create our own library. Let's discuss why we need to build our library and the challenges of building applications for Smart TVs. Although the web browsers are sophisticated, spatial navigation is still a work in progress.

Welcome everyone, thank you very much for joining this session. Today we're going to talk about the spatial navigation. But rather than talking, we're going to be building our own.

My name is Sergio Abalos. I'm a husband, father, and a parent of a small little sausage dog. And when I'm not running after any of them, I'm working also as a front-end developer at Spotify, where I am part of the team that is building the Spotify client that can run on your smart TV. That's the reason why even when this is a React conference, we won't be talking about mobile, no, we won't be talking about desktop or laptop and especially we won't be talking about your mouse. Instead we're going to be talking about the TV control, that little boring device that I'm sure all of you have in your living rooms.

So in case you were wondering, special navigation refers, as the definition says it here, a user input modality that allows mouseless navigation around the page. Basically what this means is like the action that you do with the directional keys, up, down, left and right keys of your TV control is what you do, like selecting an app and selecting any item or like just executing any action. So the reason why I choose this topic is because as most of you in the audience, I bet that you've been working on mobile or desktop. It was the same for me until I came to develop an application for smart TVs and I wanted to bring those topics that I felt that they were unique to, well obviously to me, but most of you that you've been working on other platforms and I didn't need to look very far away. I saw that Spotify develop internally a library for handling this specific case, so I decided to take a look at it, just out of curiosity and I was fascinated, not because the code was wow, no, the code was fine, but I felt that it was a very interesting problem scope to solve, so this is what this talk is about, I want to share with you what I learned and how I started digging into the code and, like I said, instead of just be talking about it, we're going to build our own. Excuse me because I don't have a lot of experience, I'm not going to be doing any live coding, but I'm going to show you step by step how you can build your own library, but first of all, I think it's very important to like ask ourselves like why do we need to build our library, like are we using the time efficiently because if we're talking about building an application on a Smart TV, shouldn't the special navigation be provided by the TV platform? And the answer is like, yes, it is if we are building a full native application. So let me try to explain in the Smart TV market, there are so many brands is really quite segmented and each of these players have their own operating system. So for them, one has to be old and native applications so that you can install using their own application store. So you cannot get around these, but to make our life a lot easier, we decided to use a hybrid application where like only the user interface is done using a web application. That gave us incredibly interoperability because we can use the same source code, the same project for running on all these platforms. But of course, it came out at cost. And in this case, we lost the support for some of the features that the platform provides like the spatial navigation. After this, I was thinking like, okay, okay, is 2023. Today, the web browsers are a very sophisticated piece of software. Isn't it that the spatial navigation already supported by the web browsers? Not yet. It's a work in progress. There is a draft for proposing this new API from 2019. But again, this market is very new. I think Smart TV's application started coming out in 2017. So, it feels like yesterday. So, it's understandable that the browsers are catching up. Because I bet not all the players out there are using this hybrid most as we're doing it.

2. Building Applications and Limitations

Short description:

There is an open source project for Smart TV applications, but we couldn't use it because we didn't have their library. I recommend using it. We can start building our own application by wrapping navigational notes, which are elements the user can interact with, and assigning identifiers to them. However, this approach has limitations with dynamic views, is error-prone, and makes debugging harder.

Then I was thinking, okay, okay, but we're not the only company building a Smart TV application. There must be an open source project. So, I look up and yes, there's actually one. And Norwegian media, thank you very much for contributing to that. Unfortunately we didn't have their library by the time we started building our own applications. So, that's the reason we couldn't use theirs. But I totally recommend to use that one.

So, having answered that question, we are not wasting our time, let's get started. If I ask you from top of your head, your intuition, what would you build? I ask this question because this is the first thing I had in mind when I knew there was a library for solving spatial navigation. And I even read it that the same approach was used by other companies in their blog. I mean, they put it themself. It's something similar to this. Imagine that you have, for example, in the Spotify client, the side menu where you have the different elements of it. If you remove all the images and styles, then you get the skeleton with the links to home, search your library and so on. And well, if you're using a web application, it will be a link to each of these views. So our intention was to like just wrap all those elements that we call navigational notes. Meaning like those elements that the user can interact with the TV control and just put an identifier to just identify from each other and tell them where they should go if they go up and down and so on and so forth. So this is like very naive approach. This actually gets the job done. There's no problem with that. But of course, there are a few small caveats. And the first one is that it is very difficult to work with dynamic views. And nowadays, almost every application offers some sort of recommendation personalization. So if you're building the user interface, most likely you don't know what you're going to get. And the next one is that it's error prone because, as a developer, you need to put the unique identifiers. We're human. We make mistakes. That's common. And finally, it's just that there is a lot of information that has to do with the navigational, the algorithm for the spatial navigation, but has nothing to do with the layout or the view. So it just makes debugging a lot harder.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

A Guide to React Rendering Behavior
React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
Top Content
React is a library for "rendering" UI from components, but many users find themselves confused about how React rendering actually works. What do terms like "rendering", "reconciliation", "Fibers", and "committing" actually mean? When do renders happen? How does Context affect rendering, and how do libraries like Redux cause updates? In this talk, we'll clear up the confusion and provide a solid foundation for understanding when, why, and how React renders. We'll look at: - What "rendering" actually is - How React queues renders and the standard rendering behavior - How keys and component types are used in rendering - Techniques for optimizing render performance - How context usage affects rendering behavior| - How external libraries tie into React rendering
Building Better Websites with Remix
React Summit Remote Edition 2021React Summit Remote Edition 2021
33 min
Building Better Websites with Remix
Top Content
Remix is a new web framework from the creators of React Router that helps you build better, faster websites through a solid understanding of web fundamentals. Remix takes care of the heavy lifting like server rendering, code splitting, prefetching, and navigation and leaves you with the fun part: building something awesome!
React Compiler - Understanding Idiomatic React (React Forget)
React Advanced Conference 2023React Advanced Conference 2023
33 min
React Compiler - Understanding Idiomatic React (React Forget)
Top Content
React provides a contract to developers- uphold certain rules, and React can efficiently and correctly update the UI. In this talk we'll explore these rules in depth, understanding the reasoning behind them and how they unlock new directions such as automatic memoization. 
Using useEffect Effectively
React Advanced Conference 2022React Advanced Conference 2022
30 min
Using useEffect Effectively
Top Content
Can useEffect affect your codebase negatively? From fetching data to fighting with imperative APIs, side effects are one of the biggest sources of frustration in web app development. And let’s be honest, putting everything in useEffect hooks doesn’t help much. In this talk, we'll demystify the useEffect hook and get a better understanding of when (and when not) to use it, as well as discover how declarative effects can make effect management more maintainable in even the most complex React apps.
Routing in React 18 and Beyond
React Summit 2022React Summit 2022
20 min
Routing in React 18 and Beyond
Top Content
Concurrent React and Server Components are changing the way we think about routing, rendering, and fetching in web applications. Next.js recently shared part of its vision to help developers adopt these new React features and take advantage of the benefits they unlock.In this talk, we’ll explore the past, present and future of routing in front-end applications and discuss how new features in React and Next.js can help us architect more performant and feature-rich applications.
(Easier) Interactive Data Visualization in React
React Advanced Conference 2021React Advanced Conference 2021
27 min
(Easier) Interactive Data Visualization in React
Top Content
If you’re building a dashboard, analytics platform, or any web app where you need to give your users insight into their data, you need beautiful, custom, interactive data visualizations in your React app. But building visualizations hand with a low-level library like D3 can be a huge headache, involving lots of wheel-reinventing. In this talk, we’ll see how data viz development can get so much easier thanks to tools like Plot, a high-level dataviz library for quick & easy charting, and Observable, a reactive dataviz prototyping environment, both from the creator of D3. Through live coding examples we’ll explore how React refs let us delegate DOM manipulation for our data visualizations, and how Observable’s embedding functionality lets us easily repurpose community-built visualizations for our own data & use cases. By the end of this talk we’ll know how to get a beautiful, customized, interactive data visualization into our apps with a fraction of the time & effort!

Workshops on related topic

React Performance Debugging Masterclass
React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Top Content
Featured WorkshopFree
Ivan Akulov
Ivan Akulov
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
Concurrent Rendering Adventures in React 18
React Advanced Conference 2021React Advanced Conference 2021
132 min
Concurrent Rendering Adventures in React 18
Top Content
Featured WorkshopFree
Maurice de Beijer
Maurice de Beijer
With the release of React 18 we finally get the long awaited concurrent rendering. But how is that going to affect your application? What are the benefits of concurrent rendering in React? What do you need to do to switch to concurrent rendering when you upgrade to React 18? And what if you don’t want or can’t use concurrent rendering yet?

There are some behavior changes you need to be aware of! In this workshop we will cover all of those subjects and more.

Join me with your laptop in this interactive workshop. You will see how easy it is to switch to concurrent rendering in your React application. You will learn all about concurrent rendering, SuspenseList, the startTransition API and more.
React Hooks Tips Only the Pros Know
React Summit Remote Edition 2021React Summit Remote Edition 2021
177 min
React Hooks Tips Only the Pros Know
Top Content
Featured Workshop
Maurice de Beijer
Maurice de Beijer
The addition of the hooks API to React was quite a major change. Before hooks most components had to be class based. Now, with hooks, these are often much simpler functional components. Hooks can be really simple to use. Almost deceptively simple. Because there are still plenty of ways you can mess up with hooks. And it often turns out there are many ways where you can improve your components a better understanding of how each React hook can be used.You will learn all about the pros and cons of the various hooks. You will learn when to use useState() versus useReducer(). We will look at using useContext() efficiently. You will see when to use useLayoutEffect() and when useEffect() is better.
React, TypeScript, and TDD
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Web3 Workshop - Building Your First Dapp
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Top Content
Featured WorkshopFree
Nader Dabit
Nader Dabit
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn