Going Live from your Browser without WebRTC

Rate this content
Bookmark

When developers think about broadcasting live from the browser, the immediate assumption is to use WebRTC. While WebRTC is amazing technology, server-side implementations are...lacking right now. We'll talk about a (totally hacky) way to get video from the browser via technology you're using today.

Matt McClure
Matt McClure
13 min
02 Aug, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Mux provides an API for live streaming and aims to keep users in their own applications. Live broadcast and live chat are different, with live chat using WebRTC and live broadcast using RTMP and HLS. WebRTC can be implemented using headless Chrome or the getUserMedia process. Mux targets developers building platforms and suggests using semantic HTML. Ionic supports native apps and custom native views.

Available in Español

1. Introduction to Live Streaming and User Experience

Short description:

I'm Matthew McClure, co-founder of Mux. We provide an API for live streaming. Users often ask how to let their users go live from a browser. Current recommendations involve using native software, but people want to keep users in their own applications.

So, let's get started. My name is Matthew McClure, I'm a co-founder of a company called Mux. We provide an API to online video infrastructure and one of the things that we provide is this live streaming solution. So, you can create live streams, we give you back stream key, and then you can just push RTMP feeds to it. So, it's great for live broadcasts, but a really common question we get is how can I let my users go live directly from a browser? So, right now the current recommendations are usually use native software like Open Broadcast Studio or Wirecast or something along those lines. But people want to be able to just keep people in their applications themselves and not ship them off to another solution and make them download and learn a new technology. So, it's understandable why they want to, but unfortunately it's not quite that easy.

2. Difference between Live Broadcast and Live Chat

Short description:

Live broadcast and live chat are often confused, but they are different. Live chat is for direct communication between two users, while live broadcast is for one person broadcasting to many. Live chat uses WebRTC, while live broadcast uses RTMP and HLS. WebRTC cannot directly communicate with RTMP, so a server is needed for the conversion.

So, let's talk about a hack I've been working on that's probably a really terrible idea. So, first of all let's talk about what we mean by live broadcast. Some quick background here. Live broadcast is not live chat. So, it's a really common misconception, but these two things are actually quite different.

So, in live chat you have just two people, two users talking directly back and forth. They can just share video potentially even peer-to-peer. So, it doesn't need to get routed through a centralized server. It can just go directly from one to the other. This latency needs to be like 300 milliseconds or less. You start getting up to like 500 milliseconds, it gets really hard to actually have that one-to-one conversation. You can maybe even have a few peers here. So, that can get 3, 5, 10, really depends on how much bandwidth each user can have, because you're kind of constrained by whoever has the least bandwidth to be able to share video back and forth between every person in the chat.

Live broadcast, on the other hand, is from one camera feed to hundreds, thousands, tens of thousands, hundreds of thousands of people at once. So now, you're not talking any more about a communication, one-to-one communication anymore, it's one person kind of broadcasting out to a bunch. So you need to be able to scale it, you need to be able to have affordable costs, but then those viewers don't necessarily need to be talked back in real time. So, latency in, you know, 10 seconds, 15 seconds, is pretty fine. By the time a person's responded in chat, it should be pretty responsive. The same technology doesn't really work well for both, for a few reasons.

So, live chat is powered by browser technologies like WebRTC, so that's a suite of APIs that can be used for browsers to communicate with each other, peer-to-peer, get the browsers' media, all those kind of things. Live broadcasting, on the other hand, is powered by technologies like RTMP. So RTMP is a server protocol, a communication protocol for delivering video. So it used to be used a lot more on delivery, but now it's kind of standard for getting a broadcast feed into a server. Then that server will transcode that to something for delivery to the end users that's a little bit more cheap and scalable, like HLS. And HLS is a format that basically takes video, chunks it up into small chunks, lists those chunks in a manifest, and then players can download the manifest and then keep pulling it for updates. But it can just be hosted on normal CDNs, it's just delivered like normal files, it's just HTTP, so it's really really easy and understandable to scale, and it's cheap, relatively speaking. Ok, so you're probably thinking, if I need to get to RTMP first, let's just go WebRTC to RTMP in the browser. The browsers are mostly like, nah. You unfortunately can't get low level enough in the networking level to be able to communicate over RTMP. Ok, so what about the technology we can't access? What about the things that we do have in our toolkit? So spoiler alert, a server is going to be involved either way.

3. WebRTC to RTMP Streaming

Short description:

You can implement WebRTC to server side WebRTC using headless Chrome, which allows you to incorporate browser technologies and overlays into the live stream. However, running Chrome at scale for each user can be complex. Alternatively, you can use the get user media process to capture the browser's microphone and camera, and send the stream over WebSockets. On the server side, you can use FFmpeg to process the WebSocket data and deliver it via RTMP. This method works well and there is a demo available on Glitch.

You're just not going to get from a browser to RTMP without in some way involving a server. So WebRTC to server side implementations of WebRTC, it can be done, it's a little bit complicated, the WebRTC spec is large and daunting. This has gotten a lot better recently, there's projects like Pyon that really make this much easier, but it's still pretty daunting.

So you're thinking WebRTC to a server side WebRTC implementation, but now that implementation is just headless Chrome, and this can be done. It's actually done really well, once you basically can have a chat, the headless Chrome instance can just join that chat and record it. And the nice thing here is you're just using browser technologies, you can do overlays, you can do whatever you would do on the browser there, and then it's just in the stream. It's really, really cool. The problem is now you're having to run Chrome at scale for every single person that wants to do a live stream, which can be complicated.

Okay, so what if we just used a piece of that WebRTC spec, get user media, which is the process of getting the browser microphone and camera, and then we'll just send that over WebSockets. WebSockets are understandable. The server set of limitations are common, and things that we've all worked with, or a lot of us have worked with in the past, so let's try that. You might be thinking, how would that work? So first, we would request the browser's media, so get user media is what I was talking about earlier. You can set different constraints. We'll just set audio and video to true, but you could adjust that if you wanted to. We'll set that stream, we'll add that stream to a video element so we can see it, then we'll capture that stream, and then we'll pass that stream to the media recorder, to a media recorder instance, or we'll create a new instance of the media recorder API, which just allows you to basically record content from a browser. And then, that recorder will expose this data available event, so every time that event fires, we have a chunk of video, so we'll just fire that chunk of video down a WebSocket connection. Now, that's going to be complying with all the process of creating that WebSocket connection, but assuming we have a WebSocket connection, now we can just send that video down that WebSocket connection, which is great. And then, the server side of things is also pretty simple and straightforward. We have this WebSocket, and every time we get a new connection, we'll spin up a FFmpeg process. Here, I'm using a MUX RTMP endpoint, but that could be anything. We'll do some cleanup if the FFmpeg process dies, or if the WebSocket closes, but otherwise, every time we get a new message and it's a buffer, we'll just write it to FFmpeg and then FFmpeg will deliver it via RTMP. So this actually works pretty well. If you want to see a demo of this, you can check out a glitch. It's got everything running. You can see it working in the browser. It's actually pretty cool. It works pretty well. Here, you just put a stream key. If you wanted to remix the glitch and use a different RTMP endpoint, totally cool. And I also wrote a blog post on this whole thing.

QnA

Q&A: MUX Target Market and Using a Div as a Button

Short description:

I go a little bit more into it. We're a developer-facing product, purely just APIs for developers to build into their platform. If you're a streamer just looking to go live without writing any code whatsoever, Twitch and YouTube are great platforms to use. If you're trying to build a platform, we're probably a better fit there. Putting HTML inside of a button is not actually semantic HTML. So they may want to wrap that content into a div and make that an accessible button instead of putting a button around it.

So if you want to check it out and get more details, I go a little bit more into it. Thanks everybody! Wow! That's a lot of knowledge in just 20 or 28 minutes it was. Four great topics. I would like to invite all the Lightning Talk speakers with me on the stage to do the last round of Q&A of the day.

Hey everyone! Hello! Hey there. Hello! Good evening, night, whatever it is for you. I'm going to go straight into the questions. I'm going to start with the first question from Matt McClure. What market are you targeting and why would someone use MUX instead of Twitch or YouTube? Yeah, that's a valid question. We're a developer-facing product so we're purely just APIs for developers to build into their platform as opposed to Twitch and YouTube which are much more consumer-facing products. So if you're a streamer just looking to go live without writing any code whatsoever, those are great platforms you should probably use them. If you're trying to build a platform, we're probably a better fit there. Okay, so it's more about the target audience, I guess, and that you have more control over what you're doing? Yeah, I would think about it a little bit like a bad analogy that I mentioned in Slack is they're more like the PayPal or Venmo. We're more of the Stripe, if you're thinking about in terms of like payment APIs. Okay, thank you.

Next question is for Jen. What are the reasons why someone like the React Native web team would want to use a div as a button? The reason is that putting HTML inside of a button is not actually semantic HTML. So they may want to wrap that content, for instance, a card or a block of like an image and text into a div and make that an accessible button instead of putting a button around it. Yeah, so if you have a completely clickable card with different elements inside it, you can't do that semantically within a button. Correct. So in that case, you'll want to make an accessible div. Well, you should want to, at least. Perhaps. And can I say, if you don't, Jen is going to come and get you? I will very kindly tap you on the shoulder and make suggestions. How about that? Yes. Yeah, but tap doesn't work. Then I might hack it. Yeah. Yeah. Not really a question, but just for you, a nice tap on the shoulder from Martin van Houten.

Appreciation for Mesh and Ionic Native Support

Short description:

Not really a question, just wanted to express appreciation for Mesh. At Albert Heijn, we've been using it and it's been a pleasure. Does Ionic support native apps like React Native, or is it more like a Cordova standard app with a web UI? It's a mix of both, allowing integration with custom native views. Thanks everyone for the talk, and goodbye for now.

Not really a question. Just want to say, mesh looks awesome. Well, that's always nice to hear. Thank you very much. I hope that you feel the same and not hate me. Well, actually, at the company I work for, Albert Heijn, we are using it. And I have to say, it's been a pleasure. So, thanks a lot.

Oh, you are my neighbor. I can come visit you. Would be gezellig. Mike, does Ionic support native apps, similar to React Native? Or it's like a Cordova standard app where it's a web UI instead of a native app? So, it's kind of a mix of both where the majority of the UI is displayed in a web view. You can integrate with custom native views or activities on Android and kind of mix which one gets displayed, the web view or the native view, or even just overlay the native view on top of the web view. So, you get kind of the best of both worlds. That feels powerful.

Okay. Thanks guys and my lady for this great talk. For the people watching, they're also going to be in the Zoom rooms for questions. But the formal part is now over. I'm going to say goodbye to you for a little bit. So, thanks for joining. Thank you. Thank you.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Don't Solve Problems, Eliminate Them
React Advanced Conference 2021React Advanced Conference 2021
39 min
Don't Solve Problems, Eliminate Them
Top Content
Humans are natural problem solvers and we're good enough at it that we've survived over the centuries and become the dominant species of the planet. Because we're so good at it, we sometimes become problem seekers too–looking for problems we can solve. Those who most successfully accomplish their goals are the problem eliminators. Let's talk about the distinction between solving and eliminating problems with examples from inside and outside the coding world.
Jotai Atoms Are Just Functions
React Day Berlin 2022React Day Berlin 2022
22 min
Jotai Atoms Are Just Functions
Top Content
Jotai is a state management library. We have been developing it primarily for React, but it's conceptually not tied to React. It this talk, we will see how Jotai atoms work and learn about the mental model we should have. Atoms are framework-agnostic abstraction to represent states, and they are basically just functions. Understanding the atom abstraction will help designing and implementing states in your applications with Jotai
Debugging JS
React Summit 2023React Summit 2023
24 min
Debugging JS
Top Content
As developers, we spend much of our time debugging apps - often code we didn't even write. Sadly, few developers have ever been taught how to approach debugging - it's something most of us learn through painful experience.  The good news is you _can_ learn how to debug effectively, and there's several key techniques and tools you can use for debugging JS and React apps.
Fighting Technical Debt With Continuous Refactoring
React Day Berlin 2022React Day Berlin 2022
29 min
Fighting Technical Debt With Continuous Refactoring
Top Content
Let’s face it: technical debt is inevitable and rewriting your code every 6 months is not an option. Refactoring is a complex topic that doesn't have a one-size-fits-all solution. Frontend applications are particularly sensitive because of frequent requirements and user flows changes. New abstractions, updated patterns and cleaning up those old functions - it all sounds great on paper, but it often fails in practice: todos accumulate, tickets end up rotting in the backlog and legacy code crops up in every corner of your codebase. So a process of continuous refactoring is the only weapon you have against tech debt. In the past three years, I’ve been exploring different strategies and processes for refactoring code. In this talk I will describe the key components of a framework for tackling refactoring and I will share some of the learnings accumulated along the way. Hopefully, this will help you in your quest of improving the code quality of your codebases.
AHA Programming
React Summit Remote Edition 2020React Summit Remote Edition 2020
32 min
AHA Programming
Top Content
Are you the kind of programmer who prefers to never see the same code in two places, or do you make liberal use of copy/paste? Many developers swear the Don't Repeat Yourself (DRY) philosophy while others prefer to Write Everything Twice (WET). But which of these produces more maintainable codebases? I've seen both of these approaches lay waste to codebases and I have a new ideology I would like to propose to you: Avoid Hasty Abstractions (AHA). In this keynote, we'll talk about abstraction and how you can improve a codebase applying and creating abstractions more thoughtfully as well as how to get yourself out of a mess of over or under-abstraction.
The Epic Stack
React Summit US 2023React Summit US 2023
21 min
The Epic Stack
Top Content
Modern web development is fantastic. There are so many great tools available! Modern web development is exhausting. There are so many great tools available! Each of these sentiments is true. What's great is that most of the time, it's hard to make a choice that is wrong. Seriously. The trade-offs of most of the frameworks and tools you could use to build your application fit within the constraints of the vast majority of apps. Despite this, engineers consistently struggle with analysis paralysis.Let's talk about this, and a solution I am working on for it.

Workshops on related topic

React, TypeScript, and TDD
React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
Paul Everitt
Paul Everitt
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
Web3 Workshop - Building Your First Dapp
React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Top Content
Featured WorkshopFree
Nader Dabit
Nader Dabit
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
Remix Fundamentals
React Summit 2022React Summit 2022
136 min
Remix Fundamentals
Top Content
Featured WorkshopFree
Kent C. Dodds
Kent C. Dodds
Building modern web applications is riddled with complexity And that's only if you bother to deal with the problems
Tired of wiring up onSubmit to backend APIs and making sure your client-side cache stays up-to-date? Wouldn't it be cool to be able to use the global nature of CSS to your benefit, rather than find tools or conventions to avoid or work around it? And how would you like nested layouts with intelligent and performance optimized data management that just works™?
Remix solves some of these problems, and completely eliminates the rest. You don't even have to think about server cache management or global CSS namespace clashes. It's not that Remix has APIs to avoid these problems, they simply don't exist when you're using Remix. Oh, and you don't need that huge complex graphql client when you're using Remix. They've got you covered. Ready to build faster apps faster?
At the end of this workshop, you'll know how to:- Create Remix Routes- Style Remix applications- Load data in Remix loaders- Mutate data with forms and actions
Vue3: Modern Frontend App Development
Vue.js London Live 2021Vue.js London Live 2021
169 min
Vue3: Modern Frontend App Development
Top Content
Featured WorkshopFree
Mikhail Kuznetcov
Mikhail Kuznetcov
The Vue3 has been released in mid-2020. Besides many improvements and optimizations, the main feature of Vue3 brings is the Composition API – a new way to write and reuse reactive code. Let's learn more about how to use Composition API efficiently.

Besides core Vue3 features we'll explain examples of how to use popular libraries with Vue3.

Table of contents:
- Introduction to Vue3
- Composition API
- Core libraries
- Vue3 ecosystem

Prerequisites:
IDE of choice (Inellij or VSC) installed
Nodejs + NPM
Developing Dynamic Blogs with SvelteKit & Storyblok: A Hands-on Workshop
JSNation 2023JSNation 2023
174 min
Developing Dynamic Blogs with SvelteKit & Storyblok: A Hands-on Workshop
Top Content
Featured WorkshopFree
Alba Silvente Fuentes
Roberto Butti
2 authors
This SvelteKit workshop explores the integration of 3rd party services, such as Storyblok, in a SvelteKit project. Participants will learn how to create a SvelteKit project, leverage Svelte components, and connect to external APIs. The workshop covers important concepts including SSR, CSR, static site generation, and deploying the application using adapters. By the end of the workshop, attendees will have a solid understanding of building SvelteKit applications with API integrations and be prepared for deployment.
Back to the Roots With Remix
React Summit 2023React Summit 2023
106 min
Back to the Roots With Remix
Featured Workshop
Alex Korzhikov
Pavlik Kiselev
2 authors
The modern web would be different without rich client-side applications supported by powerful frameworks: React, Angular, Vue, Lit, and many others. These frameworks rely on client-side JavaScript, which is their core. However, there are other approaches to rendering. One of them (quite old, by the way) is server-side rendering entirely without JavaScript. Let's find out if this is a good idea and how Remix can help us with it?
Prerequisites- Good understanding of JavaScript or TypeScript- It would help to have experience with React, Redux, Node.js and writing FrontEnd and BackEnd applications- Preinstall Node.js, npm- We prefer to use VSCode, but also cloud IDEs such as codesandbox (other IDEs are also ok)