Build an IoT App With InfluxDB

Hello everybody, I would like to start with a workshop called iot Center, which is a very simple iot application that we're going to improve today. We are not going to build the full application from scratch because we have limited time. So my name is Miroslav Malecha, I'm a product manager at Bonintu and today this is the agenda that we're going to go through. In the beginning, I would like to briefly introduce InflexDB, which is a database that we are using behind the iot Center and the iot devices. Then I will introduce briefly the iot Center application itself. Then we all together will install this application from Git and then we're going to play directly with InflexDB because we need to know how to get the data from this database that are stored by the iot devices. Then we will look also to javascript client library that is used in the iot Center as a client that is accessing the InflexDB. Then the most challenging workshop item is to add a new widget into the iot Center, into a dashboard there that will show also the location of the device, the track of the device. And the last chapter and last part is focused to the monitoring of any node.js application because we implemented into iot Center also a basic monitoring that is also storing data into InflexDB and you can build your own dashboard that are monitoring the application itself. So this is the agenda for today. During the whole workshop, if you have any questions, please use Discord. There is a channel called June iot App with InflexDB, so feel free to ask there for help. You can ask for the questions that we will go through at the end of this workshop as well as feel free to add there your comments. All the decks I have uploaded into this channel as well, so it's very helpful to open these files as well to copy paste the commands from the deck into your command line or the source code of the iot Center because the workshop part, the practical part will require some typing and it is much faster to copy paste it in from the deck itself. Before I start, let me introduce briefly also Bonik2, which is an R&D company delivering services, almost all kinds of services around the development beginning with architecture, development ending with also the operation of the systems. We are also focusing to monitoring solutions and we are tightly collaborating with the InflexData company. We develop various types of software assets for this company, including the client libraries that we're going to use during this workshop. Let's start with the introduction of InflexDB. InflexDB is a time series platform. It can handle a huge amount of data with timestamps. This is the primary purpose of this system or this solution and you can easily share the data across multiple teams. The InflexDB is able to get the data from many different data sources. Later on, I will show you some examples and it's available as open source or there is an open source version that you can freely download and use it as you need. I would say the biggest benefit of InflexDB or one of the big benefits is also that you can use it very quickly in a few minutes. You can install it and use it without any extensive training and later on you will see that it's really simple. There is a very strong focus to developers, except the client libraries that I mentioned in the beginning. There are also a lot of templates of dashboards and data sources that you can download from the repository. There is a very strong open source community so you can get a lot of information about InflexDB as well as to contribute to this community. It's not just about the database where you can store the time series data but part of this ecosystem is also Telegraph server which is able to process data and get data from various type of systems more than 200 and push it into InflexDB as well as to other data sources. So what are the key capabilities of Inflex data platform? The first part is focused to the data accumulation or ingestion. How to get the data from various systems into InflexDB. In most cases you can use Telegraph but there are some other approaches as well like you can use internal internal language flux to get and join data from other databases and then process it inside InflexDB. You can store the data, you can visualize the data, you can analyze the data inside InflexDB as well as you can configure alerts and notifications in case that some specific value reaches some threshold that is defined. You can run it on your machine locally as well as there is a cloud version of InflexDB that you just need to register. From the InflexDB you can again push the data into various systems. You can use other visualizations technologies like Grafana as well as in case of the alerts you can use various technologies to notify the right users like Slack or page duty and so on. So this is the whole ecosystem. We're going to use mainly the InflexDB itself in this example but it can be easily extended. There is another option if you don't want to install it and this is also the approach that we're going to use during this workshop. We can also use a cloud version of InflexDB. So you can log in into this address and you can register your email or account there, create the account, select the subscription. There is a free subscription which is time unlimited. There are some limitations from the data point of view but you can freely use it as long as you wish and in case you need more you can also upgrade to some better subscription. As I already mentioned you can also run it locally so the open source version is available. You can download it and use it. It is supporting many different platforms. So this was the brief introduction of InflexDB and now let me jump to the next section and that next part which is the introduction of the iot center itself. So what is iot center? iot center is a very simple application which is able to capture data from unlimited amount of iot devices. This application has been developed for education purposes so it's not a production application that you can use as it is but it's very helpful to understand or helpful to quickly understand how you can develop similar applications for your solutions that is able to configure the iot devices, process data from the iot devices as well as visualize data in your application. It is showing how to integrate with InflexDB so we can write measurements into InflexDB and there are many examples inside this application. In opposite in order to show data in dashboards you will see also how to query InflexDB and because of the management of iot devices and these devices have I separated dedicated access rights per device it is also automatically creating tokens that these devices are using to push the data directly into InflexDB. iot center is using node.js or it's based on node.js and is using react framework to visualize the data. A part of this solution we develop also something called virtual device which is emulating a real iot device. The main purpose of this is to be able to use and play with the iot center even when you don't have any iot devices connected into the iot center. Even in this case you can use it and we're going to use it during this workshop as well. Everything is available on Git. Later on you will get the address and because we're going to use InflexDB cloud we don't need to install any database behind this. The application is going to use your InflexDB cloud account. As a part of the source code we also developed some examples of the iot devices so there is a source code for two different technologies that you can connect and use as an iot device. Later on I will show you also some details. So let me introduce the architecture which is here. In the middle there is the iot center, the green part and as I already mentioned is a node.js application. It's using react and part of this application also InflexDB client which is responsible for the communication between InflexDB which is the blue box and there is also a virtual device which is able to emulate the connected iot devices so the iot center is able to generate data into InflexDB to be able to visualize them and use them for education purposes. iot center is accessible through the web browser so anybody can access it from the web and the last component in the architecture is this unlimited amount of iot devices which are typically connected to or which are communicating with iot center because of the configuration so the iot, the only information that iot device has built in is the location of the iot center. It is connecting once it is started or power on. The iot center provides the configuration for the iot device and based on the configuration which means the location of the InflexDB credentials and other information like the refresh rate and so on. So once it gets the configuration according to this configuration it is pushing data into InflexDB. The iot device is for iot center can measure various types of measurements. It can measure temperature which is mandatory and some other optional ones like humidity and GPS coordinates in case that the device is mobile as well as some other values like pressure or CO2 level and so on. It depends on the connected sensors to the iot device. So this is the structure of the iot center. Here is the list of the three main functions that iot center is supporting. The first one is the registration of the devices. Then there is a virtual device option to be able to generate the data and we have a dashboard there. So how does it look? The registration of the devices looks like the screenshot on the right side. You can see there the list of devices including some additional options like you can delete the device from the configuration. You can jump to the dashboard which is showing the latest measured values and so on. You can also register the device by pressing the register button. It's not necessary typically because the device is automatically registered once it is connected to the iot center. The second functionality is the virtual device. Virtual device is an option to be able to generate the data into InfluxDB virtually so you don't need any real iot device and the way how to do it is on the right side. The top screen is a button with pencil that if you click it it will always generate the data for the last seven days or a shorter period when you run it periodically. So this always reads the latest sample and generates this up to now. And the last item in iot center is an example how you can visualize the data. So this page is showing what values have been measured. The current ones as well as the historical values. You can have their filters that can select the specific device you want to see as well as you can select the time period that you want to see. So now let me show quickly the real demo. This is the iot center running on my laptop. There is a home page that is just explaining the purpose of this solution. In all cases you can see on the right side in context help which is explaining what is done inside the specific page as well as there is a link into the source code so you can immediately click it and see in git the source code of the specific page as well as there is a developer guide that you can which is linked with this application but you can see some details related to the specific page. So there are three options. You have the iot center itself, then you have the link to the source code as well as there is a developer guide which is explaining in detail how it is implemented the specific component. So this is the device registration. The second one is the virtual device. I'm going to generate the data so first of all we have to wait because currently it is communicating with the iot with the InfluxDB in order to get minimum maximums and some other information and now if I click to the pencil it will generate the data. As you see it wrote 53 measurements so you can always generate the data as you need and the last option is a dashboard where you can see some current values on the top of the screen as well as some historical values. As you see these data are generated, these data are not real but I can switch to different time periods so I can see more data like last three days for example as well as I can select a different device so I can select another real device and here I can see some real measured values. Hopefully. Okay so here you can see the real sensor in the real data including some real values, current values. So this is the brief introduction of the iot center. Some information regarding the source code. This is the address where you can download the source code. We're going to do it in the next phase or in the next chapter we're going to install it so you will get this address again and here let me a little bit introduce and explain also the structure explain also the structure of the repository. There is an app directory inside the repository that contains the iot center itself. There is a server side and UI side which is running in browser so it's separated so there are two sub directories there and then there is also a client Arduino which is a source code for the Arduino devices based on ESP chips as well as there is a very simple client Python or Python client for Raspberry Pi that you can use as a iot device as well. So these are the two directories where you can download the code which is not directly related to iot center but you can download it into your device and then measure the values and push it into iot center. Let me briefly summarize what are the goals of this workshop. So we're gonna install the iot center and the first I would say big task is to add a Geo widget. So we have the current dashboard and what we're gonna do is to add there a map with the track of the iot device. So this is the one task so we're gonna update and improve the existing iot center code and the second goal is to install or define and install a dashboard for node.js application monitoring. So here you will see how easily you can measure CPU usage, memory consumption, you can get also the information about the HTTP request, response time and so on in few clicks. So these are the two key goals during this workshop. So now let's start with the next part, number three, which is the installation of the iot center. We're gonna do the following steps. The first one is to register InfoSDb account. Of course, for those who already have the cloud account, you can skip this part. Then we're gonna prepare the you can skip this part. Then we're gonna prepare the environment. We need free tools to be installed in order to run InfoSDb which is a gate command line. I suppose that most of you already have it. Then we need node.js. Again, I expect that most of you have it and yarn package manager to be able to run iot center. Once we have these tools, we will get the source code from git repository. Then we will build the iot center from the source code and then we set the configuration in order to be able to run the iot center. We're gonna use the credentials from your cloud account. So you need your cloud account created otherwise you can't connect with your iot center instance and then we will run it. The last step is that we will generate data by the virtual device because we will need it for during the next or in the next chapter because we cannot play with the data from the iot center. So to summarize it, we will start with the cloud installation or the creation of the account for InfoSDb cloud and then we will install the iot center. So how to create the InfoSDb cloud? It's very simple. You need to enter the following address cloud2influxdata.com and then you have three options. You can either use your existing Google account or Microsoft account which is preferred because in this case it's very fast or you can fill your name and email address and then create the account. If you do this, there is an extra step to verify your email. So once you register it by pressing create account you need to look to your inbox and click to the link inside the email that you will receive from InfoSDb cloud. Once you have this, you should be able to continue with the registration. The next step is to select the provider. You have three options. You can either select Amazon Web Services or Google account or Microsoft azure as well as you can select the location where the specific instance should be available. Then you need to enter your company name. It can be whatever you want and then agree the service agreement. Once you do all these three steps you can click to the continue and select the specific plan. In our case for iot center the free account, free free plan, a free tier is good enough so feel free to select it. There is no payment connected with this account. It's completely free. So once you click it you should open the main page of InfoSDb. It looks like this and my instance looks like this. So it is running on InfoX cloud and this is the initial screen of InfoSDb. Once you open this page the next step is to create a new token. This token is necessary for iot center because this is the way how the iot center will be authenticated with your instance of InfoSDb. So how to get there? I will do it together with you. So let me close the presentation and let me do it together with you. So the first step is on the left side there is an icon called the data. So let's click it. Once we select the data there is an option called tokens. So let's click to tokens and on this page on the right side there is a generate token option. If you click it there is a menu and we need all access token. The reason why we need all access token is that iot center is automatically creating tokens per device and in order to do be able to do this you need all access token or just read the right token which is able to just read and write data into or from InfoXDB. So once we click all access token we need to name it. You can name it as you want. I typically call it iot center in order to understand what's the purpose of this token. And click to save it. Now a new token has been created. I have to select the right one which is the last one. In your case it will be empty of course the list and once I open the token you will see here and once I open the token you will see here a long string that we need later on for the configuration of your iot center. So in your case the string will be definitely different because it's unique per each token. Okay, so this is the preparation of the token. Let me return back to the presentation and now we will continue with the iot center installation. The first one is the node installation. I hope that all of you already have it. If not if not there is an address where you can install it for Mac or iOS. In case of Linux node.js is typically a part of the installation or you can use a package called node.js on your Linux machine. How to test that your node.js is properly installed? It's very simple just by typing node minus minus version and you should get the version. In case of Linux or Mac I've got the option here as well so here I have a Linux machine so again if I type node minus minus version I should get some version. The next application or the tool that we need is yarn. So again I'm gonna test it whether yarn is installed. In case you don't have it there is a package on Linux called yarn in case of Windows or iOS you can download it from the page that you can see in the video or on the in the deck. So again how to test it so let's type yarn minus minus version and the same for for Linux. And you should get some version. The last tool you need is git. I am almost sure that all of you have git already installed. If not for Linux there is a git-all package that you can install. In case of Windows there is this address where you can download the git package. Once you install it again to verify whether you have git, git minus minus version the same as in case of previous tools. These three tools are necessary to be able to install iot center. Once we verify that we have all these tools and once we install them the next step is to clone the repository from the github. The command is to clone the repository from the github. Repository from the github. The command that you can use is in the presentation so I'm going to copy paste it as well. So this is the command and now I will paste it into the windows and I am going to do the same for Linux as well. So now we downloaded the source code of the iot center into your machine. Now in order to be able to run it first of all we need to install all the packages. It's done through yarn so we need to call yarn install but before we will run it we have to move to the right directory which is iot center and add so we have to move to the server side and now we can call yarn install. It will take some time it depends on your speed of your internet connection. I am going to do the same for windows and typically takes one minute to install it. For those who have a problem with the installation of the package feel free to use discord. They are my colleagues that should be able to help you with the installation. There can be some issues sometimes with the version of node.js. The node.js is older than version 10. It can generate some error during the installation and in this case you need to upgrade it into the version which is higher than version 10. So once we install the libraries into the iot center by the yarn install the next step it will be the configuration of the iot center because we need to connect the influxdb with your iot center instance. So the linux part is already done i'm still waiting for windows and the next step is we need to configure it so let's let's jump to the next part. In order to do this we need to edit a file called def.bat in case of windows or def.sh in case of linux. So let me start with the linux part so let me call the i sh this is our windows so linux is here so now i have a file where i need to enter three values the first one is url feel free to use any editor that you are typically using doesn't need to be vi so we need to enter their influx url so let's open the page where influxdb is running and we will copy paste only the server address of the of the url which is accessible in the address bar of your web browser so let me copy the address and paste it into the file script file the next one is the token the token is the exact token we just generated so let me copy the token and let me paste it into the file the last thing that we need to add is the organization organization is typically the email that you use during the registration it can be changed and if you click to the user on the left side and about you will see here the name of the organization so let's copy paste the organization i'm gonna do it again for windows so for those that were not able to follow me i will do it again for windows as well once i entered all these three values i can save the file and the file can be used for the start of the influx iot center so let's do the same for windows so let's use notepad for this and in case of windows the name will be def but instead of def sh for linux okay so let's use notepad only so now this is the same file but for windows and again i need to copy there all three values the first one will be the organizations let's copy the organization the next one is the url so let's copy the url and the last one is the token which means that i need to click on data tokens and select the right token in your case you will have only one record there there and i've got a token so once i enter these three values please do not copy mine values here you should use the values from your cloud account you will save the file and now we can start the iot center in case of windows we will run def but in case of linux we we need to run def sh in my case because i cannot use both systems simultaneously on one machine i will select only the windows part but in case of the linux the only difference is that you will type a little bit different command uh so let me start it in case of linux or megos you're gonna use dot slash def dot sh but i am not good i am not going to run it because in this case i would run two instances on one machine and there will be a chord collision to be able to run both now we have to wait yeah so the windows part is still starting it will automatically open webpage but we are still waiting for the start yeah the first start that typically takes longer time we are using this in a developer mode because later on we're gonna change the files inside iot center and we want to see immediately the changes in the UI so that's why we're gonna run it this way of course we can also build it and run it in a production mode in this case the start would be much faster but before we do this we need to build the iot center which is also time consuming so let's wait once it is started all of you should see the page in your web browser it typically takes again around one minute or a little bit more depends on the computer that you are using so it's absolutely necessary to enter correctly all the values that we entered into the the script files if you do there some typo then you will still get some error later on yeah so the same values that you type it into the script files either the batch file or sh file should be visible here except token because the token is sensitive so there are just stars instead of token and now you can see what you should see in your case as well so this is the iot center up and running on your laptop and it is open on the home page page and the main step that we need to do is to jump into virtual device so on the left side in the menu there is a virtual device so and then in your case you will probably get this table empty because you have haven't written any data there yet and what we need to do is to click the small pencil button on the top right or right top screen so it's called write missing data for last seven days so let's click it it will take some time in your case it would it should uh write more than few thousand or around few thousand points in my case only 27 because i did it recently in case you will get an error these are the most typical errors that you can get there yeah on the top screen if you get cannot load the device data and there will be some error in case there is a knot in half memory which is caused by a lot of data in my instance you need to just refresh it but in most cases you will get one of these errors the first one if you get that the organization name is not found you probably incorrectly write the organization so you need to fix it in the uh script file dev bot or dev sh and correct the organization from the instance of your influxdb in case the url is incorrect yeah you will get either unsupported protocol it typically means that you wrote there only the address without https dot slash slash so this is a mandatory part of this or you will get the issue that the existing address doesn't doesn't exist which typically means a typo in the name of the server the last issue could be the message called out authorities are authorized which is typically caused by incorrecting a token so you copy paste it incorrectly the token into the script file that we recently updated so please fix these these values and then please run the influxdb iot center again the last step once we generate the data is to validate whether we wrote all the data so let me close this and let me open the dashboard by default you should see the virtual device on the top of the screen and you should see some values the actual values as well as historical values all these values were written by the virtual device directly from the iot center so we don't need any iot device connected but if we have a connected iot device we should see also some other devices in the list yeah so in my case there are multiple devices so you can see some other ones as well okay so if you have any issue if you don't know what don't know how to fix it feel free to use discord chat there are colleagues that should help you with your issue in order to save some time i will briefly also introduce the iot devices that we currently support by our center we develop a iot center we develop a education code for arduino devices we selected development boards based on esp chips esp 8266 and 832 so you can open it in arduino ide adjust few values like the uh name of the wi-fi network password and the location of your iot center and then you will be able to run this this arduino board of course you need to also connect some sensor and define the pin where this sensor is connected the second options that we prepared for you is a raspberry pi again with some connected sensor that's where you can run the python code that will get the data from the sensor and push it into influx db based on the configuration of the iot center so there are two devices and there is a scope also as a part of the repository we also developed a example of the application mobile application implemented in flutter in this case the the repository address is a little bit different but in this case you can run and access the iot center from your mobile phone from both android as well as ios devices or apple devices iphones directly see the values there from the iot center so now let me start five minutes break for those who have issues with the iot center you can use this time for the fixing of the issues and it will start in five minutes so let me let me continue with the next part hopefully all of you already have up and running influx db including the generated data from the virtual device there was one question that in the list of the devices you don't see any device of course because you are using your own cloud account you don't have any data there yet once you run the virtual device the virtual device should automatically be visible in the list of the devices as well so in your case the device registrations you will see there only one item called virtual device which is here so this is what you should see there in the registered devices and of course in the dashboard you will see there only one device which is the virtual device the default one that is automatically selected if you open the dashboard so this is the iot center installation the next part is focused to how to play with the data uh in influx db so we're going to use the influx db ui so what we're going to do we will log in into influx db all of you probably have it already opened if not we will log in there and we're going to use a data explorer to to see some measured data in our case the data from the virtual device and then we're gonna also try to build a query that will be able to provide also the coordinates the gps coordinates that we will use later on for the adjustment of the dashboard page in the iot center to be able to show the location or track of the device uh historic historical locations of the of the iot device in the dashboard so this this is what we're gonna do now so the first step is we need to log in into influx db in this case uh you need to enter again select google microsoft or your email to log in and all of you we need to move to this page so let's me let me do it as well so and if you click the top icon on the left side you will see the same screen here as well and now we're gonna use the exploration or data explorer feature it's accessible through the left menu or the left icon called explorer so let's click to the explorer and now let's look to the data so we would like to see data from the iot center so the first item that we need to select is the iot center the next one is the environment the iot center is generating different types of measurements these these ones are from the monitoring of the application but we would like to see the environment because this is the the measurement where we see the temperature and some other values from the virtual device then the next step is to select and filter only the data from virtual device so let's click to client id so there is a drop down and let's select the client id in your case there will be just one virtual device in my case there are two because i have also the real iot device which is running on my table which is generating some data so in your case we will select the virtual device then from the next drop down list we're gonna select field option and select temperature and that's it so iot center environment virtual device and the temperature once you have this on the right side of the screen there is a blue button called submit so let's run it and you should see something similar like i have on my screen you can play with the data so you can for example select different window instead of one hour i can select six hours and i can see and i can see longer period of the time this has been generated by the virtual device so this is the wizard based the way how you can browse the data but this wizard is generating a code to the query in a flux language that is accessible if you click to the button next to the submit called script editor so if you click it you will see the flux command that is used to get the data from influxdb now let me a little bit deeper explain the command itself so i will jump directly back to the presentation so data explorer slide number seven and this is the explanation of the of the command so let's start with from from the top from bucket iot center is selecting the specific database it's something like database it's called bucket but in a real database world it can be considered as a database with a retention policy so we can define how long you want to keep data there and it is automatically truncated according to the configuration the next branch start and so on is the standard filter inside the query which defines from time to specific time we want to see the data there is a v time arrange start and stop which are the variables which are coming from the ui of the influxdb so these values are automatically generated based on the time window that you will select in the drop down box that we used when we were looking to the data so this is the filter the next one defines that the measurements is called environment this is something like in the real database world something like select the specific database table so in our case we're going to select the environment then we would like to see only field called temperature so again it's something like selection of the specific column inside the table and then we would like to see only the rows where the client id is virtual device so we will filter only the temperature from the virtual device and then the last item is the aggregation we are using mean function because if we are showing in a visualization just viewpoint because there is not enough room it is going to use mean as a function that is aggregating the data into into the specific amount of points which are needed by the visualization so this is the structure of the query uh what is also quite important is that if i return back to the ui session so let me log in back so we see the the measured values as a line chart if we want to see how what's the format of the data that we can get from the infoxdb which is extremely useful if you want to develop and against the database there is a option to view raw data so i can click to view raw data and then i will see a table this is the format how you will get it into the application so if you run the this query from your application from node.js application for example and then you will get this this table you will get csv that will have these following values this is very useful because then if you want to process the data here you can see what data you will get from from the specific query so these are the fields all the values that you will get and now there is a task how we can get also not just temperature but also the gps coordinates there are two fields called let and long latitude and longitude that we need to add into the the into the query in order to get the in order to get to the these these following items let me return back so how to do it yeah still we're gonna use the same measurement called environment we're gonna use the same device so this this row remains the same as well but in case of field which is the selection of the specific columns we need to add there some additional fields so how to do this there are two options one option is to use the data explorer so i can click to query builder back to to the button next to the submit and see the selected the selected options and instead of temperature i will select let and long latitude and longitude so this is one of the options i can use the bizarre and i can run the submit and i can see the values of course i can also switch to from raw data into the visualization but in this case because this location is not changed as as quickly you will see almost the same value all the time and now let me jump back to the script editor and let's look what has been changed and you will see is that the only item that is changed is this one so instead of temperature we can see here latitude and longitude and this is something that we're gonna do inside our iot center applications once we want to add the visualization of the location of the device so we need to add this into our application as well so please remember it because later on we will use it for the extending of the iot center application okay so let me return back to the presentation just to summarize this the flux language is used to get the data from influxdb it contains multiple rows each row is defining typically some filtering or aggregation or selection of the data and the result is typically a csv response that you will process inside the application and there is a data explorer that is able to play with the data so you can see the visualization of the data directly inside influxdb the flux language is not used only for querying yeah it's a full language that can be used for almost anything you want and there is a documentation that is describing the flux language on the influxdb data pages this is the address so feel free to look at it and so feel free to look at it and and play with flux language okay so this was the data explorer and now there is a theoretical part again which is the introduction of the client libraries before i start with the client libraries let me mention one thing which is interesting and important you don't need any library in case you want to push data into influxdb and you can use a pure http or https approach this is the command yeah this is one of the options how you can push the data so it's a standard post command into http where you need to enter some basic information like the organization bucket as well as the token inside the header and then there is a sentence that is defining the data the format is called line protocol and it always contains the contain the measurement name of the measurement in this case it's cpu load short but in our case it was for example environment then there is a set of tags which are defining for example the device id and so on in this case it's a host and region and then there is a value which is the real value that you want to write into the database and then there is an optionally stamp time stamp so you want to define the exact time you can either push it if the timestamp is not there influxdb automatically writes the record with the time when this record has been written into influxdb so you can use the line protocol in case of some devices if you don't have the library it's one of the options on the other hand the client libraries that you i will i am going to introduce are doing much more than just the translation into the line protocol they are doing also buffering if some if you have many records there are automatically aggregated as well as it is handling if the connection fails and it needs to be reconnected and and so on so the client libraries are much more powerful than this basic approach to http protocol so we developed or in fact the data have developed many different libraries for different languages here you can see the list of supportive languages for influxdb too so almost for all key technologies you typically can find the client library that you can use to access the influxdb and there is a documentation where you can open and see the details around all these libraries but in order to show what these libraries are able to do please let me describe in deeper detail the javascript line library other libraries are typically used in the same way but there are some specific things that are related to javascript so first of all javascript libraries is separated into two packages the first one is called influxdb client influxdb client is used for query and writing data into influxdb and it can be used on both server side as well as in the web browser side on the web browser side and there is an influxdb client api so as a second package where you can do some more advanced things like management of the data sources configuration of the authority authentication like definition of tokens buckets and so on you can do also health check of the of the database there are many examples attached to this library accessible from from this webpage of course there is also a documentation that describes in detail the capabilities of this client library how you can use it yeah i purposefully selected few examples that are showing how simple it is yeah so how to write the data for example you need just few lines of this code yeah the first two rows here of the first row is responsible for the connection to influxdb connection to influxdb and you can also define something we call default tags so all the rights will always contain will always contain the host tag with the host name so everything what you write into the database influxdb it will always contains the whole host name and then there is a structure called point in this case is a temperature yeah where you can add additional tags related to this including the value yeah in this case the value is generated using a random function but there can be of course some real values from the sensor and then you need to write it yeah so the right point is the way how you will write it into influxdb then once everything is written you can post the api so in few lines you can write very complex data uh into influxdb the right is non-blocking which is also important to mention and it is automatically using the buffering capability you can define how how big should be the buffer and what's the time out in order to write the data safely to into influxdb another example is in opposite how to query influxdb in this case again you need to be connected into influxdb which is this this row then you need to define the query this is the flux language query from my bucket that starts with the specific time i'm filtering the specific measurement and then you can run the query rows function where you can process by the next part the specific data which we get from influxdb and we can also handle of course the errors as well as process once the list is completed the last example is a little bit more complicated and it is using the second package which is able to manage the configuration in our case we would like to create a bucket called example bucket so in this case we need to be connected which is this command the definition of the organization structure so we will get the organization from influxdb we will select the first one then we will look whether there are some buckets if yeah and the bucket name is the and if there are some buckets we will delete the buckets with the same name with the same name as the bucket we want to create and then we will create a new bucket so there are the options like deletion creation of the bucket or the list of the specific items so this is the example of the management as i already mentioned there is a documentation on this address where you can see the details what are the capabilities of the javascript client and what you can do with the javascript client from the node.js application okay so this is the introduction of the javascript client and now let me continue with the hopefully the most interesting part of this workshop which is how we can add some additional visualizations into the node.js applications which is getting data from influxdb the goal is to extend the existing dashboard because we have the latest values using this visualization we have the history with historical data using the line chart and we would like to see also the location history so we would like to add into the dashboard something like you can see here so the visualization with the track of the specific device how to do this first of all we need to open the specific file that contains the dashboard so let me open it let's open the windows part it will be the same directory for both windows as well as linux so let me start another window so okay so here i have the window so let me move to the directory which is defined here so it should be iod center slash app slash ui slash src slash pages slash so let's move to this this directory and there is a file called the dashboard page dot psx that we need to open so let's open knit by any editor i am using a basic notepad in order to simplify it as much as possible so we need to adjust the configuration of this file so what we're gonna do oh well i will open it in some better editor because this editor is really is this not using simpax highlighting so let me open it using something else so this is the file this file contains the definition of the whole dashboard and now we're gonna use the presentation to look what changes we have to do in order to add a visualization geo visualization there are several steps the first one is we need to adjust the query that are that we use to get the data from inflexdb the second step that we're gonna do is to add the that we're gonna do is to add the visualization which means that we need to add additional imports import additional libraries then we need to define the visualization component that needs to have data in the right format as well as we need to add this visualization into the dashboard so these are the steps that we're gonna do the first one is to adjust the query if you look to the function called fetch device measurements you will see that a part of this is also a definition of the query so there is a flux command that is getting temperature tvoc pressure humidity and so on and in order to get also the coordinates we need to adjust the query to add there the latitude and longitude so let's copy paste the green part the highlighted part from the deck the file is called 06 geo visualization it's a on the page five so let me open the file let me scroll down into the part where this flux query is defined there is only one and it's in the function called fetch device measurements and i will paste there the additional fields so currently we will get also latitude and longitude from influx tv not only the fields which are listed here in the current source code so this is the first step i have already mentioned that this application is for education purposes so we have only one select command or one one flux command to visualize the current values as well as historical values as well as as coordinates which is not typical for production use because we will get a huge amount of fields and only some of them we're going to use for the visualizations it's also can take a lot of memory so it's quite typical is that each visualization have each own query and for geo visualization we also have inside influx db a geo package that is able to filter the data in a more smart way and this is the set of functions which are available so i can filter specific data according to specific region for example so if i zoom into the specific map and see only the subset of the uh specific locations of the device we can use the filtering based on the current view and see only the viewpoints generated for the viewpoint of the current visualization so we can use the functions like the ones that you can see on this screen to filter it on the server side and get only some subset of the data in our case it's not as necessary we are playing with just few and a few points or a few thousand of points for larger data sets it's definitely necessary the next one is we need to add there the library in our case we're going to use leaflet to visualize the data so we need to add these imports into the leaflet so again the green part the highlighted green part please add into the the tsx file so let's add it here so additional imports so we added their additional imports to be able to use leaflet library to visualize the data from influx tv another step is to add the visualization itself it's the definition that you can copy paste from the deck and we're gonna do it and edit before the render plot function yeah because there are some some definitions that we need to use in in the source code and we need to add it into the right place in order to have all the definition visible page controls render plot so let's edit here just copy paste the geo okay so geo is here what does it contain it contains the transformation of flat and latitude latitude into the right format that you can process by leaflet which should be an array and then there is a definition of the leaflet visualization so what is the default zoom what should be the tile server which will be used for the visualization the tile server which will be used for the visualization and so on so this is the definition of the visualization and the last step is we need to also add the geo visualization into the dashboard so if you scroll down to the end you see the go-goges there and the plots and in between we would like to add there also geo so let's add there the geo and save the file so there will be there there are three changes we need to adjust the query add there the imports add there the definition of the visualization and the last thing which means four changes also add the geo into the dashboard and now let's open the iot center and as you see we have the visualization yeah so it's automatically reloaded because i did some changes in the code and you can see the track of the iot device in the last during the last day i can switch to the for example seven days because we generated data for seven days and i should see the longer track the longer track of the device so now i can see the history of the data of the location of the virtual device we of course generate this data in the virtual device but in case of the real device that is installed in some track or car you will see the historical location and the track of the specific device within the specific time okay so this is this is all as you see just by adding few lines of the code you are able to build very nice component that is able to show the visualization of the location of the specific device iot device if you don't if you don't see it please refresh the page let's have additional or another five minutes break for for you to be able to fix the potential issues in case you still have the the issues with the update of the file you can also move to this directory where we were editing the file and run this command if you run this command all these changes will be automatically populated into the dashboard page file and you should see this visualization even when there were some issues with the entering of the code into the file so this is the second option if you don't want to edit all the things that i have just done edit all the things that i have just done based on these steps in this this presentation so let's have a five minutes break and we will continue with the last part which is the monitoring of node.js application so let's let's continue so what we're gonna do is to look how we can monitor any node.js application we will start with the process monitoring so how to measure the cpu and memory consumption of the node.js processes then we will look how we can measure the response time of the HTTP requests and then we will look how we can quickly create the dashboards using the templates in InfluxDB and the last step is how you can do and add this monitoring capability to any application node.js application because the code from inside iot center is very generic so how you can monitor the node.js process the first step is we need to connect the InfluxDB yeah so this is the command in our case this is the command from iot center but very similar you can use in your case as well and then the next step is to write the measured values from the process from the node.js into InfluxDB in this case we're going to use functions of node.js memory usage cpu usage and resource usage usage but the format generally or provided by these functions is not fully compatible so there is a create point function which is translating the measurements from these functions into a point that you can write into InfluxDB so using these following lines you can write the data into InfluxDB and later on you will see the result by default we measure we write these measurements once at 10 seconds in case of the monitoring of the the HTTP requests that are processed by the node.js application you can use this approach so you will capture or catch the response time and again create the point with some specific name you can add there some additional tags in this case the path also the use method as well as the status code and we measure the response time as a field so this is the way how you can measure the response time by the way the same approach you can use for any measurement so if you want to for example store the information about some custom events inside node.js application very similar approach the creation root point and the write point you can use for any information InfluxDB doesn't need to define the schema so you can write there any value you want using this this approach without any configuration previous configuration of InfluxDB later on I will show you some example in case of the iot center what we did we created a file called monitor.js which is a part of iot center that you can copy to any project this monitor.js is also contains the link to env.js that contains the information about the location of InfluxDB or if the location is not defined in this file it can use environment variables like the ones that we configured when we were installing iot center so in the script and then the only thing you need to do is to run during the application start this function to start the monitoring of your InfluxDB node.js application how we can visualize the data of course we can use the data explorer to play with the data or we can use also some predefined template in our case let me open the the UI of InfluxDB how to install the template the first step is we will click to settings which is the last icon on the left side of your screen in InfluxDB UI then there is an option called templates in order to find the right template the first step is that you can click to browse community templates in this case it will open a page it's a it's a git repository where you can see a long list of various templates that the community has created if you have some interesting template you can edit here as well and we will select the node.js monitoring which is here yeah so i will click to the node.js and now i can see here the list of files the file i need is this node.js yaml so i will copy the link address yeah or i will open the yaml and copy paste the the url from from the web browser yeah from the address bar once you have it you need to paste it into the location of the template once you have it you can click to look up the template button that will show what is included into this specific template you can see that there are some buckets there is a one dashboard there are two variables and labels which are some additional items that are used either by the dashboard or by the queries used by the dashboard or some there can be also some configurations and so on and the only thing you need to do is the install template i'm not going to do this because you probably noticed that the install template that i have in this ui is already the node.js template so i cannot do it but once you click the installation you will see this link the list of or the install template in the list so there will be no js template i have additional one which is out of center that we cannot do later on so now let's look to the dashboard so i will click to the boards on the left side there is a icon called boards and now after the installation of the template i will see there a new dashboard called node.js application monitoring if i click to this on its ring i will see the dashboard and this is the dashboard which is capturing the data from iot center application so i can see the response time i can see the cpu usage the memory usage i can see the response time i can sort it according to the time so i can see the worst or the best responses like as you see the longest ones took 1.5 seconds almost 1.6 i can see the io i can see some additional information like response time in the histogram i can see some additional values that are provided by the resources and so on of course you can edit and adjust the the dashboard according to your needs on the top of the screen i can see also the name of the service or the name of the application as well as i can see also the host so if i have multiple iot centers running on the multiple hosts i can filter and see the exact ones that i want to analyze so this is the example of the filters i can also see on the right side the filter so if i want to see some longer longer period of the time i can open it and see it as well so now you can also see that there was one query that took almost 20 seconds and it was the query that was getting the data from inflexDB so if you remember the spinning wheel during this workshop this was the this was the case that we see here so this is the monitoring of node.js application as i already mentioned you can also add there so your own user events yeah like this one which is here the user registered that you can define for example his name email and result and the response time or how long it took and and so on so you can create there your own custom events that you can use for the monitoring of the application even internally at the monitor we are using this point massively for for the monitoring of our applications it's very powerful and a very quick way how you can capture the information that you need in a few rows of the code inside your application this is of course some basic monitoring there are also some much more advanced solutions that are able to monitor node.js application like the company called node source which is using a very similar approach it is using internally also inflexDB to capture and store all the information about the running applications and you can get much more precise and more detailed information about your applications but for basic monitoring you can use the approach that i was showing let me briefly describe also the templates structure or templates we downloaded the template that edit there the dashboard but the template itself can contain also the definition of data sources it's not mandatory but it's optional the specific dashboards as well as the the queries there are many dashboards you can select the interesting ones including the configuration the always the approach is first of all you will browse the templates like the specific template add there the url of the yaml file which defines the specific template and import it there is a very similar template in this this repository for the iot center so if i open and import the template for iot center and you will see something like this this is a very basic template but here you can see similar values well i have to define different filter that are showing similar values like in iot center application so you can see similar visualizations involve in iot center for all applications as well as in the ui if in case you build your own visualizations okay so that's all for my side for today are there any questions ivan were there any questions that are interesting to answer to the whole audience okay first of all mirek thanks a lot for your presentations with and the demos with regards to the questions we have quite a lot of fun in the discord chat and well the questions are whether the influx db is the conceptual one is a relational database or the database that can be used to store other data than the time series data that was answered by blaster influx db is a time series database only but of course it is possible to store also the configuration data into a specific bucket technical bucket i think what what is worth to mention is that you can also combine the relational database with influx db so in flux command you can get data from relational database and combine them with with the time series from influx db that's true the flux language has connectors to relational databases and of course it is possible to query or write to the relational databases and use the integrated environment with a time series as well as the relational database in one in one pocket and there were a couple of the questions related to deployments to the exercises thanks for them um well but i do not see any other questions so maybe we can recap what was already what was already mentioned or what was already presented in this in this in this webinar okay so let me let me continue so what we did we used the iot center application that we developed primary for education purposes like this workshop you can use it you can play with this you can add there your own code you can base on your your solution based on this this this application feel free to play with this as the next steps it's also definitely worth to look to the gallery of the templates and see what else you can do of course the telegraph is able to do much much more than what you can see in the gallery so it's worth to see also what you can capture from from from from the sources based on the capabilities of telegraph application and yeah there is if you are interested you can join the community yeah so there are multiple addresses you can use slack you can use github projects many github projects where influx data is involved as well as the community influxdata.com webpage if you want to join more tightly with influxdata so thank you very much for your time thank you very much for the questions and and that you were playing with with iot center and hopefully it will help you to build similar applications in the future so thank you very much