WebBluetooth – the Missing Link

Please welcome Nicole. Hello, everyone. My name is Nicole. I was running a little bit late today because I was so busy at work that I spent way too much of my free time playing around with all the new web technologies and browser API APIs and today, I'm here to take you through my journey with the web Bluetooth API.

So, browsers have been built about this idea that they would request data from a server, and they would render that data. So, browsers have always been great at rendering data. But it was always quite difficult to interact with devices that were right next to the browser. Now, with the whole movement of progressive web apps, with the project Fugu, we have seen a couple of new APIs, we have seen the web USB, web serial API, web HID API, and so on and so forth, and they all try to fill this gap between the browser and the peripheral devices.

Now, let's have a look at some basics. So, Bluetooth is a wireless technology standard to exchange data that uses the same frequency as wireless LAN, but it sends really small amounts of data over rather short distance. If you now think about this old and buggy and frustrating experience you had with your first smartphones or your first cell phones, I have very good news for you, because that was the old Bluetooth, Bluetooth 4.0, we also have BLE, which stands for Bluetooth Low Energy, and that's a completely new standard that was built specifically for IoT devices, so it's very energy efficient, and it also runs over quite a long distance. And with Web Bluetooth, we always talk about BLE, because that's a standard that was implemented.

Bluetooth normally works like this. You would have a central device which is the more capable device in terms of CPU and battery usage, and then you would have peripheral devices. Now, we also refer to them as the client and the server, and that's very important that you can connect one central device to multiple peripheral devices, but one peripheral device can only be connected to one central device at the same time. That's also the reason why, for example, you can't connect your headphones and your smart watch to the same smart phone, but you can't connect your headphones to two smart phones at the same time. That's just not possible. Now, as part of BLE, the Bluetooth special interest group introduced the GATT, the generic attribute profile, and that's basically the way how the peripheral device exposes its data to the client. You would have the GATT server that contains a list of GATT services, each service is then just a group of characteristics, and now, one characteristic identifies a value and it also defines what operations can be performed on that value.

For example, if you have a battery level, there it makes sense that you can read the battery level and you also want to be notified when the battery level changes on the device, but it doesn't make sense to make that writeable because you can't just override the battery level and then expect it to be magically 100 per cent if that would be the case, or our energy problems, climate change problems would be solved. But that's not. But if you have the flight direction of a drone, for example, there it does make sense or it's crucial that you can control the direction that you can't control your drone. It needs to be writeable. Now, a GATT value is a low-level data structure, so it's just an array of bytes. If you now look at the right side, here I have the implementation of a GTT server from the play bulb sphere which is a BLE light bulb, and we have the services, we have the characteristics, and that's very important to mention that services and characteristics are not really identified by their names, but by UUIDs. So there's the list of standard UUIDs maintained by the Bluetooth special interest group that cover common use cases like information about the device, or battery levels, so we do know that on each device, the characteristic, or let's say the service 180F, would be the battery service, but then we also have non-standard services like in that example the light service and the light characteristic, and there we can reach for either 128-bits, so long, unique UUIDs, or we can choose 16-bit UUIDs that are not yet specified, in that case we have FF0F for the light service, and FFFC for the characteristic. The tricky part is now, how are those values structured? The first two are pretty basic, that's basically just a UTF-8 encoded string. The battery level is the decimal representation of our byte, so 5a would be 90 per cent, and now for the custom characteristics, there we need to reach out for the documentation, and in most cases, there is no documentation, but in our case, we have documentation, so we do know that the play box here has four bytes, four LEDs inside the light bulb, and with the value of each byte, you can control the intensity of one of those LEDs, and the cool thing is now with red, green, and blue, we can basically create any colour that we have in the RGB colour space. So, now that we have the basics, let's have a look at the web API. So, the web API is pretty straightforward.

First you have to request a device, pass a set of filters, we need to specify the services we will use later on, and, when that code is executed, the user will see this prompt, there we can select one of all the devices that match the filters we defined.

In our case, we want to filter by the name Playbop sphere, and therefore we can select the playbop sphere, once the user connects to that, we can call device.connect to connect to the server.

To read the value, we can get the service using the UUID and also the characteristic using the UUID. Now, on the characteristic, we can call characteristic.read value to read the value, and, in that case, we will receive a data view, and we are now interested in the first byte of the data view in the decimal format, so, right now, we just want the console log the battery level of the characteristic of the device.

Since the battery characteristic also allows the notify operation, we can add an event listener characteristic value changed, and that will be fired whenever the battery level changes on the device. It is very important here as well that we need to call notifications because in the sake of, or for the sake of battery efficiency, those events are not sent by default.

To write values, again, we need to request the service, the characteristic, this time for the light service. Now we can read the value as before, but now we're interested in the RGB value. So the second, the third, and the fourth value. And now to write a value, we can call the write value without a response, or with response. That depends on the device you're using. But both functions, they accept a new data view. So in that case, we will just override the current data view and that will change the characteristic on the device.

Now let's start with a little demo. First of all, my slides are browser or they're run in the browser. What I can do is I can just open up the console. And those are exactly the commands you have seen before. First of all, let's copy this. I'm not a trackpad person but I think that will work. Now we can connect to the Playbop Sphere. On the device, we can then select server. On the server, we can then select the slide service. And on the light service, we can now write a new value. So in our case, we want to overwrite that value and we expect the light bulb to turn. Yeah, OK. I forgot to sorry, I forgot to get the characteristic. Let's try again. Here we go. No, sorry. Wait.

OK, let's try that again. It's always nice to do live coding or live demos on a conference. conference. So let's go again. Let's connect. Service. Now. Characteristic. And now the moment of truth. Will it be blue? Yes, it will.

OK. So in the end, with Web Bluetooth, we are basically now able to control any BLE device that we have around us. But there's one problem. Reverse engineering Bluetooth device is a huge pain. Before I found the Play Bulbsphere, I tried a couple of different RGB light bulbs. And the problem is, since there is no standard for light, each manufacturer has their own set of characteristics, their own UUIDs. And I was just there trying to make sense of this. And that was just a huge waste of time and energy.

And on the other hand, there is this whole new world right outside the window, the world of Arduinos, Raspberry Pis, PICOs, hardware. I mean, I'm a front end developer. I never came in touch with hardware at any point in my career. But I thought, well, let's just go on an adventure. And the adventure turned out to be a pretty deep rabbit hole. And I'm still somewhere down there. But I managed to create this BLE Buzzer, which is an Arduino Nano RP2040, which is the latest Arduino chip or module, I need say. And there's also a Bluetooth chip on the board.

So what I have here, basically, is just one server, one service, one characteristic that has one byte. And that byte can either be zero when it's in default state, or it can be 255 when it's pressed. And now, my web application uses the Bluetooth to listen to that characteristic value change event. And whenever it changes from zero to 255, I'll just show the next slide.

The next thing, I'm using this all the whole time, but the problem is maybe you don't see it. OK, the next thing I built is this little car. It's again an Arduino Nano. It's called Speed Wheels. It's not that fast, but my son came up with the name, and I'm fine with that. So it's an Arduino Nano connected to those motors and to an LED RGB ring.

And the problem I had here is my C++ knowledge is very, very limited. And it's still very limited. And my ideas were huge. So what I had to do is I wanted to keep the device part very, very small, very easy. And move all the heavy logic to the client, to JavaScript, where I feel comfortable. So what I did is I have basically one function that accepts two values, one for the left wheel, one for the right wheel, between 0, which would be full speed backward, 100, which would be stop, and 200, full speed forward. And since those values are in the range of one byte, we can now use one characteristic with two bytes, one for the left wheel, one for the right wheel.

I also have a demo. And let's hope that this time it will work as expected. I'm also, since I have my stage here so everyone can see it, since a web application is cross-platform by default, I can just use my smartphone with the same web application I have on my desktop. I can now connect two speed wheels. And I can now. Woo! Woo! Woo! Woo! OK, that worked. But the problem is it's cool arrow keys work, but they're very, very boring. So what I came up is, or I try to get the full or to work with the full potential of the web platform. So for example, we do have a gyroscope on the smartphone. So what I can do now is I can use the device orientation API to convert the movement of my, ah, no. OK. It's still running, so it does work. But I mean, the device orientation is great, but we do have some fancy stuff in the web. For example, we have machine learning. And as you might know, we can use TensorFlow.js inside the browser. And what I did is I took a hand gesture model. Here we go.

And now I really hope that the light is right. What I do is basically I analyze the given video input, and I try to convert the hand gestures into characteristic right events. And it has some delay, but it does work. So to sum it all up, we have these very powerful new APIs. We can connect them to the full potential of the web platform. And I ask myself, how far can we go? I mean, we do have, or I said in the beginning that we can connect multiple peripheral devices to one central device. But can we also connect multiple devices to one web application? If so, where are the limits? And I didn't find any limits.

So me and my little friends here, we created a little demo where I am using the Web Audio API to analyze a given audio input. And there I can now interact with the audio. And for example, I can get the volume of a given frequency or the beat and everything. And let's hope the audio is on as well. That should be it. I didn't want to do that in a live demo because there are so many things that need to work out.

We can connect the light bulb. I have two little LED matrixes where I can set the value of each of those LEDs. I need to speed up the process a little because it just takes too much time. But once all the devices are connected, we can take you a little bit back in time. And... So in the end, all you have seen in the video is the web application that analyzes the audio input and then sends a lot of small commands to my devices.

So that was it from my side. It was great being here. I have one thing to say. If you are a web developer, if you are a front-end developer, and I think most of you are, and you ever wanted to dip your toes into hardware development, now it is the perfect time. Thank you very much. My slides are online, so you will have all the links to the Twitter threads and the GitHub profiles as well. And if there are any questions, I think we do have a couple of minutes. There are some questions! Let's just take our places here, MC, speaker, and I kindly ask our AV team to put questions on the big screen.

Yes, so, actually, how do you control two devices if that is ever possible at all? Well, it is possible. Maybe you have seen the two LED matrixes, they do have the same name, and you can just connect to those and you have then two instances of that device. So in the end, that is perfectly fine, no problem. Great. Well, that is a very interesting question. Can you use Bluetooth to control your device if you are far away from it? Well, of course it needs to be in the range, in the Bluetooth range, because you can't connect otherwise. I think this is not a very simple question. I mean, yeah, of course, it's not possible by Bluetooth, right? So the whole point of Bluetooth is being in proximity. But why not create some kind of library to provide fallback? So if you're out of range with Bluetooth, maybe you could try your luck with some cloud connection and bypass commands from your control center via other channels. Also, Bluetooth, the range is not that small, because with Bluetooth 5.0, I have read that it should be possible to communicate over 100 meters or more.

So it's pretty okay, I would say. I'm not sure how long it will take, but it should be possible. Yeah, just think about this kind of fallback library could be at least a nominee for the next year in one of these nominations. Maybe, I don't know. The most fun project or most exciting project. Okay.

Nevertheless, the last question, or not actually last, so let's dig in two more minutes. First, simple one. Is this API available in all browsers, and what's about cross-platform support? Well, no, it's not. The problem is it is available in all Chromium-based browser already since 2017. It's not available, for example, in WebKit. That also means that we can't use it on iOS, because all the browsers need to use WebKit under the hood, but there is some hope because I think right now there are some regulatory changes where maybe Apple will be forced to allow other browser engines on iOS, and also I've seen lately some movement in the whole WebKit ecosystem, so now that they will allow push notifications, maybe they will also change their mind about all the hardware APIs, because that's also the case for WebUSB, WebSerial, all those cool things, but right now, it's available in Chromium-based browsers. Yeah, and I'd say, progressive enhancement and feature detection are your best friends.

Thank you very much, Niko. Give me five.