WebBluetooth – the Missing Link

Hello, everyone. My name is Nico. As Maxim said, I'm a freelance front-end developer from Switzerland. I'm also part of the Google Developer Experts programme which basically means that I just spend way too much of my free time playing around with all the new web technologies and browser APIs. Today, I'm here to take you through my journey with the web Bluetooth API. So browsers have been built around this idea that they would request data from a server and they would render that data. So browsers have always been great at communicating with servers that might be hundreds of kilometres away but it was always quite difficult to interact with devices that were right next to the browser. Now with the whole movement of progressive web apps, with the project Fugoo, we have seen a couple of new APIs. We have seen the web USB, web serial API, web HID API, and so on and so forth, and they all try to fill this gap between the browser and the peripheral devices. Let's start with some basics. So Bluetooth is a wireless technology standard to exchange data that uses the same frequency as wireless LAN, but it sends really small amounts of data over a rather short distance and if you think about this old and buggy and frustrating experience you had with your first smartphones, or first cell phones, I have very good news for you because that was the old Bluetooth, Bluetooth Classic. Right now, with Bluetooth 4.0, we have BLE which stands for Bluetooth Low Energy. That's a completely new standard that was built specifically for IoT devices, so it's very energy-efficient and it also runs over quite a long distance. With web Bluetooth, we always talk about BLE because that's the standard that was implemented. So Bluetooth normally works like this. You would have a central device which is the more capable device in terms of CPU and battery usage, and then you would have peripheral devices. Now we also refer to them as the client and the server, and that's very important that you can connect one central device to multiple peripheral devices, but one peripheral device can only be connected to one central device at the same time. That's also the reason why, for example, you can connect your headphones and your smartwatch to the same smartphone, but you can't connect your headphones to two smartphones at the same time. That's just not possible. Now as part of BLE, the Bluetooth special interest group introduced the GATT, the generic attribute profile, and that's basically the way how the peripheral device exposes its data to the client. So you would have the GATT server that contains a list of GATT services, each service is then just a group of characteristics, and now one characteristic identifies a value, and it also defines what operations can be performed on that value. For example, if you have a battery level, there it makes sense that you can read the battery level and you also want to be notified when the battery level changes on the device, but it doesn't make sense to make that writeable because you can't just overwrite the battery level and then expect it to be magically at 100 per cent. If that would be the case, all our energy problems, climate change problems would be solved, but that's not. But if you have the flight direction of a drone, for example, there it does make sense or it's crucial that you can control the direction that you can control your drone, it needs to be writeable. Now a GATT value is then a very low-level data structure, so it's just an array of bytes. If you now look at the right side, here I have the implementation of a GATT server from the Play Bulb Sphere which is a BLE light bulb, and we have the services, we have the characteristics, and now it's very important to mention that services and characteristics are not really identified by their names, but by UUIDs. So there's the list of standard UUIDs maintained by the Bluetooth Special Interest Group that cover common use cases like information about the device, or battery levels, or, we do know that on each device, the characteristic, or let's say the service 180F would be the battery service, but then we also have non-standard services, like in that example the light service and the light characteristic, and there, we can reach for either 128 bits or long unique UUIDs, or we can choose 16-bit UUIDs that are not yet specified, in that case, we have FF0F for the light service, and FFFC for the characteristic. The tricky part is now how are those values structured? The first two are pretty basic, that's basically just a UTF-8 encoded string. The battery level is just the decimal representation of our byte, so 5A would be 90 per cent. Now for the custom characteristics, there, we need to reach out for the documentation, and in most cases, there is no documentation. But in our case, we have documentation, so we do know that the Playbop sphere has four bytes for LEDs inside the light bulb, and with the value of each byte, you can control the intensity of one of those LEDs. The cool thing is now, with red, green, and blue, we can basically create any colour that we have in the RGB colour space. Now that we have the basics, let's have a look at the web API. The web API is pretty straightforward. First we have to request the device. We can pass a set of filters and we need to specify the services that we will use later on. When that code is executed, the user will see this prompt. There we can select one of all the devices that match the filters we defined. In our case, we want to filter by the name Playbop sphere, and therefore, we can then select the Playbop sphere. Once the user connects to that, we can call device.gatt.connect to connect to the server. To read the value, we can first get the service using the UUID and also the characteristic using the UUID. Now, on the characteristic, we can call characteristic.readValue to read the value, and, in that case, we will receive a data view, and we are now interested in the first byte of that data view in a decimal format, so, right now, we just want to console log the battery level of the characteristic or of that device. Since the battery characteristic also allows the notify operation, we can add an event listener characteristic value changed, and that will be fired whenever the battery level changes on the device. It's very important here as well that we need to call.star notifications because, for the sake of battery efficiency, those events are not sent by default. To write values, again, we need to request the service, the characteristic, this time for the light service. Now we can read the value as before, but now we're interested in the RGB value, so the second, third, and fourth value. To write a value, we can call the write value without a response, or with a response that depends on the device you're using, but both function, they accept a new data view, so, in that case, we will just overwrite the current data view, and that will change the characteristic on the device. Now, let's start with a little demo. First of all, my slides are browser, or they run in the browser, so what I can do is I can just open up the console. And those are exactly the commands you have seen before. First of all, let's copy this. I'm not a trackpad person, but I think that will work. Now we can connect to the Playbop sphere. On the device, we can then select server. On the server, we can then select the slide service, and on the light service, we can now write a new value, so, in our case, we want to overwrite that value, and we expect the light bulb to turn. Yes! Okay! I forgot to get the characteristic. Let's try again. Here we go. No, sorry. Wait. Okay, let's try that again. It's always nice to do live coding or live demos on a conference. So let's go again. Connect. Service now. Characteristic. And now the moment of truth. Will it be blue? Yes, it is! Yes, it will! Okay! So, in the end, with web Bluetooth, we are basically now able to control any BLE device that we have around us, but there's one problem. Reverse engineering Bluetooth devices is a huge pain. Before I found the Playbop sphere, I tried a couple of different RGB light bulbs, and the problem is, since there is no standard for light, each manufacturer has their own set of characteristics, their own UUIDs, and I was just there trying to make sense of this, and that was a huge waste of time and energy, and, on the other hand, there is this whole new world right outside the window, the world of Arduinos, Raspberry Pi, Picos, hardware, and I'm a front-end developer. I never came in touch with hardware at any point in my career, but I thought, well, let's just go on an adventure. The adventure turned out to be a pretty deep rabbit hole, and I'm still somewhere down there, but I managed to create this BLE buzzer which is an Arduino Nano RP2040 which is the latest Arduino chip or module, and there's also a Bluetooth chip on the board. So what I have here basically is just one server, one service, one characteristic that has one byte, and that byte can either be zero when it's in default state, or it can be 255 when it's pressed, and now my web application uses the Bluetooth to listen to the characteristic value change event, and whenever it changes from zero to 255, I will just show the next slide. The next thing... I'm using this the whole time, but the problem is maybe you don't see it. The next thing I built is this little car. It's again an Arduino Nano. It's called Speedwheels. It's not that fast, but my son came up with the name, and I'm fine with that. So it's an Arduino Nano connected to those motors and to an LED RGB ring. The problem I had here is my C++ knowledge is very, very limited, and I'm still, it's still very limited, and my ideas were huge. So what I had to do is I wanted to keep the device part very, very small, very easy, and move all the heavy logic to the client, to JavaScript, where I feel comfortable. What I did is I have basically one function that accepts two values, one for the left wheel, one for the right wheel, between zero, which would be full speed backward, 100, which would be stop, and 200, full speed forward. Since those values are in the range of one byte, we can now use one characteristic with two bytes, one for the left wheel, one for the right wheel. I also have a demo, and let's hope that this time it will work as expected. I'm also, since I have my stage here, so everyone can see it, so, since a web application is cross-platform by default, I can use my smartphone with the same web application I have on my desktop. I can now connect two speed wheels, and I can now... Okay, that works. But the problem is, I mean, it's cool, arrow keys work, but they're very, very boring. So what I came up is, or I tried to get the full or to work with the full potential of the web platform, so, for example, we do have a gyroscope on the smartphone, so what I can do now is I can use the device orientation API to convert the movement of my... No! Okay. It's still running, so it does work. But I mean, the device orientation is great, but we also do have some fancy stuff in the web. We have machine learning, and, as you might know, we can use TensorFlow.js inside the browser, and what I did is I have a hand, or I took a hand gesture model. There we go. And now I really hope that the light is right, but I can... What I do is basically I analyse the given video input, and I try to convert the hand gestures into characteristic right events. And it has some delay, but it does work. So to sum it all up, we have this very powerful new APIs. We can connect them to the full potential of the web platform. I asked myself how far can we go? We do have, or I said in the beginning that we can connect multiple peripheral devices to one central device, but can we also connect multiple devices to one web application? If so, where are the limits? I didn't find any limits. So me and my little friends here, we created a little demo where I'm using the web audio API to analyse a given audio input, and there I can now interact with the audio, and, for example, I can get the volume of a given frequency, or the beat, and everything, and let's hope the audio is on as well. That should be it. I didn't want to do that in a live demo because there are so many things that need to work out. We can connect the lightbulb. I have two little LED matrices where I can set the value of each of those LEDs. I need to speed up the process a little because it just takes too much time. But once all the devices are connected, we can take you a little bit back in time, and we can see what's going on. So, in the end, all you see in the video is the web application that analyses the audio input and sends a lot of small commands to my devices. So that was it from my side. It was great being here. I have one thing to say. If you are a web developer, if you're a front-end developer, and I think most of you are, and you ever wanted to dip your toes into hardware development, now it is the perfect time. Thank you very much. My slides are all online, so you will have all the links to the threads and to the Twitter threads and the GitHub profiles as well. And if there are any questions, I think we do have a couple of minutes. Yes, yes, yes, there are some questions. Let's just take our places here. MC, speaker, and I kindly ask our AV team to put questions on the big screen. Yay! Yes, so, actually, how do you control two devices if that's ever possible at all? Well, it is possible. Maybe you have seen the two LED matrixes. They do have the same name. And you can just connect to those, and you have then two instances of that device. So in the end, that's perfectly fine. No problem. Great. Well, that's a very interesting question. Can you use Bluetooth to control your device if you're far away from it? Well, of course it needs to be in the range, in the Bluetooth range, because you can't connect otherwise. I think this is not a very simple question. I mean, yeah, of course, it's not possible by Bluetooth, right? So the whole point of Bluetooth is being in proximity. But why not create some kind of library to provide fallback? So if you're out of range with Bluetooth, maybe you could try your luck with some cloud connection and bypass commands from your control centre via other channels. And also, Bluetooth is not that, the range is not that small, because with Bluetooth 5.0, I have read that it should be possible to communicate over 100 metres or more. So it's pretty OK, I would say. I'm not sure how long it will take, but it should be possible. Yeah, just think about this kind of fallback library could be at least nominee for the next year in one of these nominations. Maybe I don't know, the most fun project or most exciting project. OK, nevertheless, the last question, or not actually last, so let's take two more minutes. First simple one, is this API available in all browsers and what about cross-platform support? Well, no, it's not. The problem is it is available in all Chromium-based browsers already since 2017. It's not available, for example, in WebKit. That also means that we can't use it on iOS because all the browsers on iOS need to use WebKit under the hood. But there is some hope because I think right now there are some regulatory changes where maybe Apple will be forced to allow other browser engines on iOS, and also I've seen lately some movement in the whole WebKit ecosystem, so now that they will allow push notifications, maybe they will also change their mind about all the hardware API because that's also the case for WebUSB, WebSerial, all those cool things. But right now, it's only available in Chromium-based browsers. And I say progressive enhancement and feature detection are your best friends. Thank you very much, Nico, give me five. Thank you very much. Thank you.