Hello everybody, I would like to start with a workshop called
iot Center, which is a very simple
iot application that we're going to improve today. We are not going to build the full application from scratch because we have limited time. So my name is Miroslav Malecha, I'm a product manager at Bonintu and today this is the agenda that we're going to go through. In the beginning, I would like to briefly introduce InflexDB, which is a
database that we are using behind the
iot Center and the
iot devices. Then I will introduce briefly the
iot Center application itself. Then we all together will install this application from Git and then we're going to play directly with InflexDB because we need to know how to get the
data from this
database that are stored by the
iot devices. Then we will look also to
javascript client library that is used in the
iot Center as a client that is accessing the InflexDB. Then the most challenging workshop item is to add a new widget into the
iot Center, into a dashboard there that will show also the location of the device, the track of the device. And the last chapter and last part is focused to the monitoring of any
node.js application because we implemented into
iot Center also a basic monitoring that is also storing
data into InflexDB and you can build your own dashboard that are monitoring the application itself. So this is the agenda for today. During the whole workshop, if you have any questions, please use Discord. There is a channel called June
iot App with InflexDB, so feel free to ask there for help. You can ask for the questions that we will go through at the end of this workshop as well as feel free to add there your comments. All the decks I have uploaded into this channel as well, so it's very helpful to open these files as well to copy paste the commands from the deck into your command line or the source code of the
iot Center because the workshop part, the practical part will require some typing and it is much faster to copy paste it in from the deck itself. Before I start, let me introduce briefly also Bonik2, which is an R&D company delivering services, almost all kinds of services around the development beginning with
architecture, development ending with also the operation of the systems. We are also focusing to monitoring solutions and we are tightly collaborating with the InflexData company. We develop various types of software assets for this company, including the client libraries that we're going to use during this workshop. Let's start with the introduction of InflexDB. InflexDB is a time series platform. It can handle a huge amount of
data with timestamps. This is the primary purpose of this system or this solution and you can easily share the
data across multiple teams. The InflexDB is able to get the
data from many different
data sources. Later on, I will show you some examples and it's available as open source or there is an open source version that you can freely download and use it as you need. I would say the biggest benefit of InflexDB or one of the big benefits is also that you can use it very quickly in a few minutes. You can install it and use it without any extensive training and later on you will see that it's really simple. There is a very strong focus to developers, except the client libraries that I mentioned in the beginning. There are also a lot of templates of dashboards and
data sources that you can download from the repository. There is a very strong open source
community so you can get a lot of information about InflexDB as well as to contribute to this
community. It's not just about the
database where you can store the time series
data but part of this
ecosystem is also Telegraph server which is able to process
data and get
data from various type of systems more than 200 and push it into InflexDB as well as to other
data sources. So what are the key capabilities of Inflex
data platform? The first part is focused to the
data accumulation or ingestion. How to get the
data from various systems into InflexDB. In most cases you can use Telegraph but there are some other approaches as well like you can use internal internal language flux to get and join
data from other databases and then process it inside InflexDB. You can store the
data, you can visualize the
data, you can analyze the
data inside InflexDB as well as you can configure alerts and notifications in case that some specific value reaches some threshold that is defined. You can run it on your machine locally as well as there is a
cloud version of InflexDB that you just need to register. From the InflexDB you can again push the
data into various systems. You can use other visualizations technologies like Grafana as well as in case of the alerts you can use various technologies to notify the right users like Slack or page duty and so on. So this is the whole
ecosystem. We're going to use mainly the InflexDB itself in this example but it can be easily extended. There is another option if you don't want to install it and this is also the approach that we're going to use during this workshop. We can also use a
cloud version of InflexDB. So you can log in into this address and you can register your email or account there, create the account, select the subscription. There is a free subscription which is time unlimited. There are some limitations from the
data point of view but you can freely use it as long as you wish and in case you need more you can also upgrade to some better subscription. As I already mentioned you can also run it locally so the open source version is available. You can download it and use it. It is supporting many different platforms. So this was the brief introduction of InflexDB and now let me jump to the next section and that next part which is the introduction of the
iot center itself. So what is
iot center?
iot center is a very simple application which is able to capture
data from unlimited amount of
iot devices. This application has been developed for education purposes so it's not a production application that you can use as it is but it's very helpful to understand or helpful to quickly understand how you can develop similar applications for your solutions that is able to configure the
iot devices, process
data from the
iot devices as well as visualize
data in your application. It is showing how to integrate with InflexDB so we can write measurements into InflexDB and there are many examples inside this application. In opposite in order to show
data in dashboards you will see also how to query InflexDB and because of the management of
iot devices and these devices have I separated dedicated access rights per device it is also automatically creating tokens that these devices are using to push the
data directly into InflexDB.
iot center is using
node.js or it's based on
node.js and is using
react framework to visualize the
data. A part of this solution we develop also something called virtual device which is emulating a real
iot device. The main purpose of this is to be able to use and play with the
iot center even when you don't have any
iot devices connected into the
iot center. Even in this case you can use it and we're going to use it during this workshop as well. Everything is available on Git. Later on you will get the address and because we're going to use InflexDB
cloud we don't need to install any
database behind this. The application is going to use your InflexDB
cloud account. As a part of the source code we also developed some examples of the
iot devices so there is a source code for two different technologies that you can connect and use as an
iot device. Later on I will show you also some details. So let me introduce the
architecture which is here. In the middle there is the
iot center, the green part and as I already mentioned is a
node.js application. It's using
react and part of this application also InflexDB client which is responsible for the communication between InflexDB which is the blue box and there is also a virtual device which is able to emulate the connected
iot devices so the
iot center is able to generate
data into InflexDB to be able to visualize them and use them for education purposes.
iot center is accessible through the web browser so anybody can access it from the web and the last component in the
architecture is this unlimited amount of
iot devices which are typically connected to or which are communicating with
iot center because of the configuration so the
iot, the only information that
iot device has built in is the location of the
iot center. It is connecting once it is started or power on. The
iot center provides the configuration for the
iot device and based on the configuration which means the location of the InflexDB credentials and other information like the refresh rate and so on. So once it gets the configuration according to this configuration it is pushing
data into InflexDB. The
iot device is for
iot center can measure various types of measurements. It can measure temperature which is mandatory and some other optional ones like humidity and GPS coordinates in case that the device is mobile as well as some other values like pressure or CO2 level and so on. It depends on the connected sensors to the
iot device. So this is the structure of the
iot center. Here is the list of the three main functions that
iot center is supporting. The first one is the registration of the devices. Then there is a virtual device option to be able to generate the
data and we have a dashboard there. So how does it look? The registration of the devices looks like the screenshot on the right side. You can see there the list of devices including some additional options like you can delete the device from the configuration. You can jump to the dashboard which is showing the latest measured values and so on. You can also register the device by pressing the register button. It's not necessary typically because the device is automatically registered once it is connected to the
iot center. The second functionality is the virtual device. Virtual device is an option to be able to generate the
data into InfluxDB virtually so you don't need any real
iot device and the way how to do it is on the right side. The top screen is a button with pencil that if you click it it will always generate the
data for the last seven days or a shorter period when you run it periodically. So this always reads the latest sample and generates this up to now. And the last item in
iot center is an example how you can visualize the
data. So this page is showing what values have been measured. The current ones as well as the historical values. You can have their filters that can select the specific device you want to see as well as you can select the time period that you want to see. So now let me show quickly the real demo. This is the
iot center running on my laptop. There is a home page that is just explaining the purpose of this solution. In all cases you can see on the right side in context help which is explaining what is done inside the specific page as well as there is a link into the source code so you can immediately click it and see in git the source code of the specific page as well as there is a developer guide that you can which is linked with this application but you can see some details related to the specific page. So there are three options. You have the
iot center itself, then you have the link to the source code as well as there is a developer guide which is explaining in detail how it is implemented the specific component. So this is the device registration. The second one is the virtual device. I'm going to generate the
data so first of all we have to wait because currently it is communicating with the
iot with the InfluxDB in order to get minimum maximums and some other information and now if I click to the pencil it will generate the
data. As you see it wrote 53 measurements so you can always generate the
data as you need and the last option is a dashboard where you can see some current values on the top of the screen as well as some historical values. As you see these
data are generated, these
data are not real but I can switch to different time periods so I can see more
data like last three days for example as well as I can select a different device so I can select another real device and here I can see some real measured values. Hopefully. Okay so here you can see the real sensor in the real
data including some real values, current values. So this is the brief introduction of the
iot center. Some information regarding the source code. This is the address where you can download the source code. We're going to do it in the next phase or in the next chapter we're going to install it so you will get this address again and here let me a little bit introduce and explain also the structure explain also the structure of the repository. There is an app directory inside the repository that contains the
iot center itself. There is a server side and UI side which is running in browser so it's separated so there are two sub directories there and then there is also a client Arduino which is a source code for the Arduino devices based on ESP chips as well as there is a very simple client Python or Python client for Raspberry Pi that you can use as a
iot device as well. So these are the two directories where you can download the code which is not directly related to
iot center but you can download it into your device and then measure the values and push it into
iot center. Let me briefly summarize what are the goals of this workshop. So we're gonna install the
iot center and the first I would say big task is to add a Geo widget. So we have the current dashboard and what we're gonna do is to add there a map with the track of the
iot device. So this is the one task so we're gonna update and improve the existing
iot center code and the second goal is to install or define and install a dashboard for
node.js application monitoring. So here you will see how easily you can measure CPU usage, memory consumption, you can get also the information about the HTTP request, response time and so on in few clicks. So these are the two key goals during this workshop. So now let's start with the next part, number three, which is the installation of the
iot center. We're gonna do the following steps. The first one is to register InfoSDb account. Of course, for those who already have the
cloud account, you can skip this part. Then we're gonna prepare the you can skip this part. Then we're gonna prepare the environment. We need free tools to be installed in order to run InfoSDb which is a gate command line. I suppose that most of you already have it. Then we need
node.js. Again, I expect that most of you have it and
yarn package manager to be able to run
iot center. Once we have these tools, we will get the source code from git repository. Then we will build the
iot center from the source code and then we set the configuration in order to be able to run the
iot center. We're gonna use the credentials from your
cloud account. So you need your
cloud account created otherwise you can't connect with your
iot center instance and then we will run it. The last step is that we will generate
data by the virtual device because we will need it for during the next or in the next chapter because we cannot play with the
data from the
iot center. So to summarize it, we will start with the
cloud installation or the creation of the account for InfoSDb
cloud and then we will install the
iot center. So how to create the InfoSDb
cloud? It's very simple. You need to enter the following address cloud2influxdata.com and then you have three options. You can either use your existing Google account or Microsoft account which is preferred because in this case it's very fast or you can fill your name and email address and then create the account. If you do this, there is an extra step to verify your email. So once you register it by pressing create account you need to look to your inbox and click to the link inside the email that you will receive from InfoSDb
cloud. Once you have this, you should be able to continue with the registration. The next step is to select the provider. You have three options. You can either select Amazon Web Services or Google account or Microsoft
azure as well as you can select the location where the specific instance should be available. Then you need to enter your company name. It can be whatever you want and then agree the service agreement. Once you do all these three steps you can click to the continue and select the specific plan. In our case for
iot center the free account, free free plan, a free tier is good enough so feel free to select it. There is no payment connected with this account. It's completely free. So once you click it you should open the main page of InfoSDb. It looks like this and my instance looks like this. So it is running on InfoX
cloud and this is the initial screen of InfoSDb. Once you open this page the next step is to create a new token. This token is necessary for
iot center because this is the way how the
iot center will be authenticated with your instance of InfoSDb. So how to get there? I will do it together with you. So let me close the presentation and let me do it together with you. So the first step is on the left side there is an icon called the
data. So let's click it. Once we select the
data there is an option called tokens. So let's click to tokens and on this page on the right side there is a generate token option. If you click it there is a menu and we need all access token. The reason why we need all access token is that
iot center is automatically creating tokens per device and in order to do be able to do this you need all access token or just read the right token which is able to just read and write
data into or from InfoXDB. So once we click all access token we need to name it. You can name it as you want. I typically call it
iot center in order to understand what's the purpose of this token. And click to save it. Now a new token has been created. I have to select the right one which is the last one. In your case it will be empty of course the list and once I open the token you will see here and once I open the token you will see here a long string that we need later on for the configuration of your
iot center. So in your case the string will be definitely different because it's unique per each token. Okay, so this is the preparation of the token. Let me return back to the presentation and now we will continue with the
iot center installation. The first one is the node installation. I hope that all of you already have it. If not if not there is an address where you can install it for Mac or iOS. In case of Linux
node.js is typically a part of the installation or you can use a package called
node.js on your Linux machine. How to test that your
node.js is properly installed? It's very simple just by typing node minus minus version and you should get the version. In case of Linux or Mac I've got the option here as well so here I have a Linux machine so again if I type node minus minus version I should get some version. The next application or the tool that we need is
yarn. So again I'm gonna test it whether
yarn is installed. In case you don't have it there is a package on Linux called
yarn in case of Windows or iOS you can download it from the page that you can see in the
video or on the in the deck. So again how to test it so let's type
yarn minus minus version and the same for for Linux. And you should get some version. The last tool you need is git. I am almost sure that all of you have git already installed. If not for Linux there is a git-all package that you can install. In case of Windows there is this address where you can download the git package. Once you install it again to verify whether you have git, git minus minus version the same as in case of previous tools. These three tools are necessary to be able to install
iot center. Once we verify that we have all these tools and once we install them the next step is to clone the repository from the github. The command is to clone the repository from the github. Repository from the github. The command that you can use is in the presentation so I'm going to copy paste it as well. So this is the command and now I will paste it into the windows and I am going to do the same for Linux as well. So now we downloaded the source code of the
iot center into your machine. Now in order to be able to run it first of all we need to install all the packages. It's done through
yarn so we need to call
yarn install but before we will run it we have to move to the right directory which is
iot center and add so we have to move to the server side and now we can call
yarn install. It will take some time it depends on your speed of your internet connection. I am going to do the same for windows and typically takes one minute to install it. For those who have a problem with the installation of the package feel free to use discord. They are my colleagues that should be able to help you with the installation. There can be some issues sometimes with the version of
node.js. The
node.js is older than version 10. It can generate some error during the installation and in this case you need to upgrade it into the version which is higher than version 10. So once we install the libraries into the
iot center by the
yarn install the next step it will be the configuration of the
iot center because we need to connect the influxdb with your
iot center instance. So the linux part is already done i'm still waiting for windows and the next step is we need to configure it so let's let's jump to the next part. In order to do this we need to edit a file called def.bat in case of windows or def.sh in case of linux. So let me start with the linux part so let me call the i sh this is our windows so linux is here so now i have a file where i need to enter three values the first one is url feel free to use any editor that you are typically using doesn't need to be vi so we need to enter their influx url so let's open the page where influxdb is running and we will copy paste only the server address of the of the url which is accessible in the address bar of your web browser so let me copy the address and paste it into the file script file the next one is the token the token is the exact token we just generated so let me copy the token and let me paste it into the file the last thing that we need to add is the organization organization is typically the email that you use during the registration it can be changed and if you click to the user on the left side and about you will see here the name of the organization so let's copy paste the organization i'm gonna do it again for windows so for those that were not able to follow me i will do it again for windows as well once i entered all these three values i can save the file and the file can be used for the start of the influx
iot center so let's do the same for windows so let's use notepad for this and in case of windows the name will be def but instead of def sh for linux okay so let's use notepad only so now this is the same file but for windows and again i need to copy there all three values the first one will be the organizations let's copy the organization the next one is the url so let's copy the url and the last one is the token which means that i need to click on
data tokens and select the right token in your case you will have only one record there there and i've got a token so once i enter these three values please do not copy mine values here you should use the values from your
cloud account you will save the file and now we can start the
iot center in case of windows we will run def but in case of linux we we need to run def sh in my case because i cannot use both systems simultaneously on one machine i will select only the windows part but in case of the linux the only difference is that you will type a little bit different command uh so let me start it in case of linux or megos you're gonna use dot slash def dot sh but i am not good i am not going to run it because in this case i would run two instances on one machine and there will be a chord collision to be able to run both now we have to wait yeah so the windows part is still starting it will automatically open webpage but we are still waiting for the start yeah the first start that typically takes longer time we are using this in a developer mode because later on we're gonna change the files inside
iot center and we want to see immediately the changes in the UI so that's why we're gonna run it this way of course we can also build it and run it in a production mode in this case the start would be much faster but before we do this we need to build the
iot center which is also time consuming so let's wait once it is started all of you should see the page in your web browser it typically takes again around one minute or a little bit more depends on the computer that you are using so it's absolutely necessary to enter correctly all the values that we entered into the the script files if you do there some typo then you will still get some error later on yeah so the same values that you type it into the script files either the batch file or sh file should be visible here except token because the token is sensitive so there are just stars instead of token and now you can see what you should see in your case as well so this is the
iot center up and running on your laptop and it is open on the home page page and the main step that we need to do is to jump into virtual device so on the left side in the menu there is a virtual device so and then in your case you will probably get this table empty because you have haven't written any
data there yet and what we need to do is to click the small pencil button on the top right or right top screen so it's called write missing
data for last seven days so let's click it it will take some time in your case it would it should uh write more than few thousand or around few thousand points in my case only 27 because i did it recently in case you will get an error these are the most typical errors that you can get there yeah on the top screen if you get cannot load the device
data and there will be some error in case there is a knot in half memory which is caused by a lot of
data in my instance you need to just refresh it but in most cases you will get one of these errors the first one if you get that the organization name is not found you probably incorrectly write the organization so you need to fix it in the uh script file dev bot or dev sh and correct the organization from the instance of your influxdb in case the url is incorrect yeah you will get either unsupported protocol it typically means that you wrote there only the address without https dot slash slash so this is a mandatory part of this or you will get the issue that the existing address doesn't doesn't exist which typically means a typo in the name of the server the last issue could be the message called out authorities are authorized which is typically caused by incorrecting a token so you copy paste it incorrectly the token into the script file that we recently updated so please fix these these values and then please run the influxdb
iot center again the last step once we generate the
data is to validate whether we wrote all the
data so let me close this and let me open the dashboard by default you should see the virtual device on the top of the screen and you should see some values the actual values as well as historical values all these values were written by the virtual device directly from the
iot center so we don't need any
iot device connected but if we have a connected
iot device we should see also some other devices in the list yeah so in my case there are multiple devices so you can see some other ones as well okay so if you have any issue if you don't know what don't know how to fix it feel free to use discord chat there are colleagues that should help you with your issue in order to save some time i will briefly also introduce the
iot devices that we currently support by our center we develop a
iot center we develop a education code for arduino devices we selected development boards based on esp chips esp 8266 and 832 so you can open it in arduino ide adjust few values like the uh name of the wi-fi network password and the location of your
iot center and then you will be able to run this this arduino board of course you need to also connect some sensor and define the pin where this sensor is connected the second options that we prepared for you is a raspberry pi again with some connected sensor that's where you can run the python code that will get the
data from the sensor and push it into influx db based on the configuration of the
iot center so there are two devices and there is a scope also as a part of the repository we also developed a example of the application mobile application implemented in flutter in this case the the repository address is a little bit different but in this case you can run and access the
iot center from your mobile phone from both android as well as ios devices or apple devices iphones directly see the values there from the
iot center so now let me start five minutes break for those who have issues with the
iot center you can use this time for the fixing of the issues and it will start in five minutes so let me let me continue with the next part hopefully all of you already have up and running influx db including the generated
data from the virtual device there was one question that in the list of the devices you don't see any device of course because you are using your own
cloud account you don't have any
data there yet once you run the virtual device the virtual device should automatically be visible in the list of the devices as well so in your case the device registrations you will see there only one item called virtual device which is here so this is what you should see there in the registered devices and of course in the dashboard you will see there only one device which is the virtual device the default one that is automatically selected if you open the dashboard so this is the
iot center installation the next part is focused to how to play with the
data uh in influx db so we're going to use the influx db ui so what we're going to do we will log in into influx db all of you probably have it already opened if not we will log in there and we're going to use a
data explorer to to see some measured
data in our case the
data from the virtual device and then we're gonna also try to build a query that will be able to provide also the coordinates the gps coordinates that we will use later on for the adjustment of the dashboard page in the
iot center to be able to show the location or track of the device uh historic historical locations of the of the
iot device in the dashboard so this this is what we're gonna do now so the first step is we need to log in into influx db in this case uh you need to enter again select google microsoft or your email to log in and all of you we need to move to this page so let's me let me do it as well so and if you click the top icon on the left side you will see the same screen here as well and now we're gonna use the exploration or
data explorer feature it's accessible through the left menu or the left icon called explorer so let's click to the explorer and now let's look to the
data so we would like to see
data from the
iot center so the first item that we need to select is the
iot center the next one is the environment the
iot center is generating different types of measurements these these ones are from the monitoring of the application but we would like to see the environment because this is the the measurement where we see the temperature and some other values from the virtual device then the next step is to select and filter only the
data from virtual device so let's click to client id so there is a drop down and let's select the client id in your case there will be just one virtual device in my case there are two because i have also the real
iot device which is running on my table which is generating some
data so in your case we will select the virtual device then from the next drop down list we're gonna select field option and select temperature and that's it so
iot center environment virtual device and the temperature once you have this on the right side of the screen there is a blue button called submit so let's run it and you should see something similar like i have on my screen you can play with the
data so you can for example select different window instead of one hour i can select six hours and i can see and i can see longer period of the time this has been generated by the virtual device so this is the wizard based the way how you can browse the
data but this wizard is generating a code to the query in a flux language that is accessible if you click to the button next to the submit called script editor so if you click it you will see the flux command that is used to get the
data from influxdb now let me a little bit deeper explain the command itself so i will jump directly back to the presentation so
data explorer slide number seven and this is the explanation of the of the command so let's start with from from the top from bucket
iot center is selecting the specific
database it's something like
database it's called bucket but in a real
database world it can be considered as a
database with a retention policy so we can define how long you want to keep
data there and it is automatically truncated according to the configuration the next branch start and so on is the standard filter inside the query which defines from time to specific time we want to see the
data there is a v time arrange start and stop which are the variables which are coming from the ui of the influxdb so these values are automatically generated based on the time window that you will select in the drop down box that we used when we were looking to the
data so this is the filter the next one defines that the measurements is called environment this is something like in the real
database world something like select the specific
database table so in our case we're going to select the environment then we would like to see only field called temperature so again it's something like selection of the specific column inside the table and then we would like to see only the rows where the client id is virtual device so we will filter only the temperature from the virtual device and then the last item is the aggregation we are using mean function because if we are showing in a visualization just viewpoint because there is not enough room it is going to use mean as a function that is aggregating the
data into into the specific amount of points which are needed by the visualization so this is the structure of the query uh what is also quite important is that if i return back to the ui session so let me log in back so we see the the measured values as a line chart if we want to see how what's the format of the
data that we can get from the infoxdb which is extremely useful if you want to develop and against the
database there is a option to view raw
data so i can click to view raw
data and then i will see a table this is the format how you will get it into the application so if you run the this query from your application from
node.js application for example and then you will get this this table you will get csv that will have these following values this is very useful because then if you want to process the
data here you can see what
data you will get from from the specific query so these are the fields all the values that you will get and now there is a task how we can get also not just temperature but also the gps coordinates there are two fields called let and long latitude and longitude that we need to add into the the into the query in order to get the in order to get to the these these following items let me return back so how to do it yeah still we're gonna use the same measurement called environment we're gonna use the same device so this this row remains the same as well but in case of field which is the selection of the specific columns we need to add there some additional fields so how to do this there are two options one option is to use the
data explorer so i can click to query builder back to to the button next to the submit and see the selected the selected options and instead of temperature i will select let and long latitude and longitude so this is one of the options i can use the bizarre and i can run the submit and i can see the values of course i can also switch to from raw
data into the visualization but in this case because this location is not changed as as quickly you will see almost the same value all the time and now let me jump back to the script editor and let's look what has been changed and you will see is that the only item that is changed is this one so instead of temperature we can see here latitude and longitude and this is something that we're gonna do inside our
iot center applications once we want to add the visualization of the location of the device so we need to add this into our application as well so please remember it because later on we will use it for the extending of the
iot center application okay so let me return back to the presentation just to summarize this the flux language is used to get the
data from influxdb it contains multiple rows each row is defining typically some filtering or aggregation or selection of the
data and the result is typically a csv response that you will process inside the application and there is a
data explorer that is able to play with the
data so you can see the visualization of the
data directly inside influxdb the flux language is not used only for querying yeah it's a full language that can be used for almost anything you want and there is a
documentation that is describing the flux language on the influxdb
data pages this is the address so feel free to look at it and so feel free to look at it and and play with flux language okay so this was the
data explorer and now there is a theoretical part again which is the introduction of the client libraries before i start with the client libraries let me mention one thing which is interesting and important you don't need any library in case you want to push
data into influxdb and you can use a pure http or https approach this is the command yeah this is one of the options how you can push the
data so it's a standard post command into http where you need to enter some basic information like the organization bucket as well as the token inside the header and then there is a sentence that is defining the
data the format is called line protocol and it always contains the contain the measurement name of the measurement in this case it's cpu load short but in our case it was for example environment then there is a set of tags which are defining for example the device id and so on in this case it's a host and region and then there is a value which is the real value that you want to write into the
database and then there is an optionally stamp time stamp so you want to define the exact time you can either push it if the timestamp is not there influxdb automatically writes the record with the time when this record has been written into influxdb so you can use the line protocol in case of some devices if you don't have the library it's one of the options on the other hand the client libraries that you i will i am going to introduce are doing much more than just the translation into the line protocol they are doing also buffering if some if you have many records there are automatically aggregated as well as it is handling if the connection fails and it needs to be reconnected and and so on so the client libraries are much more powerful than this basic approach to http protocol so we developed or in fact the
data have developed many different libraries for different languages here you can see the list of supportive languages for influxdb too so almost for all key technologies you typically can find the client library that you can use to access the influxdb and there is a
documentation where you can open and see the details around all these libraries but in order to show what these libraries are able to do please let me describe in deeper detail the
javascript line library other libraries are typically used in the same way but there are some specific things that are related to
javascript so first of all
javascript libraries is separated into two packages the first one is called influxdb client influxdb client is used for query and writing
data into influxdb and it can be used on both server side as well as in the web browser side on the web browser side and there is an influxdb client
api so as a second package where you can do some more
advanced things like management of the
data sources configuration of the authority
authentication like definition of tokens buckets and so on you can do also health check of the of the
database there are many examples attached to this library accessible from from this webpage of course there is also a
documentation that describes in detail the capabilities of this client library how you can use it yeah i purposefully selected few examples that are showing how simple it is yeah so how to write the
data for example you need just few lines of this code yeah the first two rows here of the first row is responsible for the connection to influxdb connection to influxdb and you can also define something we call default tags so all the rights will always contain will always contain the host tag with the host name so everything what you write into the
database influxdb it will always contains the whole host name and then there is a structure called point in this case is a temperature yeah where you can add additional tags related to this including the value yeah in this case the value is generated using a random function but there can be of course some real values from the sensor and then you need to write it yeah so the right point is the way how you will write it into influxdb then once everything is written you can post the
api so in few lines you can write very complex
data uh into influxdb the right is non-blocking which is also important to mention and it is automatically using the buffering capability you can define how how big should be the buffer and what's the time out in order to write the
data safely to into influxdb another example is in opposite how to query influxdb in this case again you need to be connected into influxdb which is this this row then you need to define the query this is the flux language query from my bucket that starts with the specific time i'm filtering the specific measurement and then you can run the query rows function where you can process by the next part the specific
data which we get from influxdb and we can also handle of course the errors as well as process once the list is completed the last example is a little bit more complicated and it is using the second package which is able to manage the configuration in our case we would like to create a bucket called example bucket so in this case we need to be connected which is this command the definition of the organization structure so we will get the organization from influxdb we will select the first one then we will look whether there are some buckets if yeah and the bucket name is the and if there are some buckets we will delete the buckets with the same name with the same name as the bucket we want to create and then we will create a new bucket so there are the options like deletion creation of the bucket or the list of the specific items so this is the example of the management as i already mentioned there is a
documentation on this address where you can see the details what are the capabilities of the
javascript client and what you can do with the
javascript client from the
node.js application okay so this is the introduction of the
javascript client and now let me continue with the hopefully the most interesting part of this workshop which is how we can add some additional visualizations into the
node.js applications which is getting
data from influxdb the goal is to extend the existing dashboard because we have the latest values using this visualization we have the history with historical
data using the line chart and we would like to see also the location history so we would like to add into the dashboard something like you can see here so the visualization with the track of the specific device how to do this first of all we need to open the specific file that contains the dashboard so let me open it let's open the windows part it will be the same directory for both windows as well as linux so let me start another window so okay so here i have the window so let me move to the directory which is defined here so it should be iod center slash app slash ui slash src slash pages slash so let's move to this this directory and there is a file called the dashboard page dot psx that we need to open so let's open knit by any editor i am using a basic notepad in order to simplify it as much as possible so we need to adjust the configuration of this file so what we're gonna do oh well i will open it in some better editor because this editor is really is this not using simpax highlighting so let me open it using something else so this is the file this file contains the definition of the whole dashboard and now we're gonna use the presentation to look what changes we have to do in order to add a visualization geo visualization there are several steps the first one is we need to adjust the query that are that we use to get the
data from inflexdb the second step that we're gonna do is to add the that we're gonna do is to add the visualization which means that we need to add additional imports import additional libraries then we need to define the visualization component that needs to have
data in the right format as well as we need to add this visualization into the dashboard so these are the steps that we're gonna do the first one is to adjust the query if you look to the function called fetch device measurements you will see that a part of this is also a definition of the query so there is a flux command that is getting temperature tvoc pressure humidity and so on and in order to get also the coordinates we need to adjust the query to add there the latitude and longitude so let's copy paste the green part the highlighted part from the deck the file is called 06 geo visualization it's a on the page five so let me open the file let me scroll down into the part where this flux query is defined there is only one and it's in the function called fetch device measurements and i will paste there the additional fields so currently we will get also latitude and longitude from influx tv not only the fields which are listed here in the current source code so this is the first step i have already mentioned that this application is for education purposes so we have only one select command or one one flux command to visualize the current values as well as historical values as well as as coordinates which is not typical for production use because we will get a huge amount of fields and only some of them we're going to use for the visualizations it's also can take a lot of memory so it's quite typical is that each visualization have each own query and for geo visualization we also have inside influx db a geo package that is able to filter the
data in a more smart way and this is the set of functions which are available so i can filter specific
data according to specific region for example so if i zoom into the specific map and see only the subset of the uh specific locations of the device we can use the filtering based on the current view and see only the viewpoints generated for the viewpoint of the current visualization so we can use the functions like the ones that you can see on this screen to filter it on the server side and get only some subset of the
data in our case it's not as necessary we are playing with just few and a few points or a few thousand of points for larger
data sets it's definitely necessary the next one is we need to add there the library in our case we're going to use leaflet to visualize the
data so we need to add these imports into the leaflet so again the green part the highlighted green part please add into the the tsx file so let's add it here so additional imports so we added their additional imports to be able to use leaflet library to visualize the
data from influx tv another step is to add the visualization itself it's the definition that you can copy paste from the deck and we're gonna do it and edit before the render plot function yeah because there are some some definitions that we need to use in in the source code and we need to add it into the right place in order to have all the definition visible page controls render plot so let's edit here just copy paste the geo okay so geo is here what does it contain it contains the transformation of flat and latitude latitude into the right format that you can process by leaflet which should be an array and then there is a definition of the leaflet visualization so what is the default zoom what should be the tile server which will be used for the visualization the tile server which will be used for the visualization and so on so this is the definition of the visualization and the last step is we need to also add the geo visualization into the dashboard so if you scroll down to the end you see the go-goges there and the plots and in between we would like to add there also geo so let's add there the geo and save the file so there will be there there are three changes we need to adjust the query add there the imports add there the definition of the visualization and the last thing which means four changes also add the geo into the dashboard and now let's open the
iot center and as you see we have the visualization yeah so it's automatically reloaded because i did some changes in the code and you can see the track of the
iot device in the last during the last day i can switch to the for example seven days because we generated
data for seven days and i should see the longer track the longer track of the device so now i can see the history of the
data of the location of the virtual device we of course generate this
data in the virtual device but in case of the real device that is installed in some track or car you will see the historical location and the track of the specific device within the specific time okay so this is this is all as you see just by adding few lines of the code you are able to build very nice component that is able to show the visualization of the location of the specific device
iot device if you don't if you don't see it please refresh the page let's have additional or another five minutes break for for you to be able to fix the potential issues in case you still have the the issues with the update of the file you can also move to this directory where we were editing the file and run this command if you run this command all these changes will be automatically populated into the dashboard page file and you should see this visualization even when there were some issues with the entering of the code into the file so this is the second option if you don't want to edit all the things that i have just done edit all the things that i have just done based on these steps in this this presentation so let's have a five minutes break and we will continue with the last part which is the monitoring of
node.js application so let's let's continue so what we're gonna do is to look how we can monitor any
node.js application we will start with the process monitoring so how to measure the cpu and memory consumption of the
node.js processes then we will look how we can measure the response time of the HTTP requests and then we will look how we can quickly create the dashboards using the templates in InfluxDB and the last step is how you can do and add this monitoring capability to any application
node.js application because the code from inside
iot center is very generic so how you can monitor the
node.js process the first step is we need to connect the InfluxDB yeah so this is the command in our case this is the command from
iot center but very similar you can use in your case as well and then the next step is to write the measured values from the process from the
node.js into InfluxDB in this case we're going to use functions of
node.js memory usage cpu usage and resource usage usage but the format generally or provided by these functions is not fully compatible so there is a create point function which is translating the measurements from these functions into a point that you can write into InfluxDB so using these following lines you can write the
data into InfluxDB and later on you will see the result by default we measure we write these measurements once at 10 seconds in case of the monitoring of the the HTTP requests that are processed by the
node.js application you can use this approach so you will capture or catch the response time and again create the point with some specific name you can add there some additional tags in this case the path also the use method as well as the status code and we measure the response time as a field so this is the way how you can measure the response time by the way the same approach you can use for any measurement so if you want to for example store the information about some custom events inside
node.js application very similar approach the creation root point and the write point you can use for any information InfluxDB doesn't need to define the schema so you can write there any value you want using this this approach without any configuration previous configuration of InfluxDB later on I will show you some example in case of the
iot center what we did we created a file called monitor.js which is a part of
iot center that you can copy to any project this monitor.js is also contains the link to env.js that contains the information about the location of InfluxDB or if the location is not defined in this file it can use environment variables like the ones that we configured when we were installing
iot center so in the script and then the only thing you need to do is to run during the application start this function to start the monitoring of your InfluxDB
node.js application how we can visualize the
data of course we can use the
data explorer to play with the
data or we can use also some predefined template in our case let me open the the UI of InfluxDB how to install the template the first step is we will click to settings which is the last icon on the left side of your screen in InfluxDB UI then there is an option called templates in order to find the right template the first step is that you can click to browse
community templates in this case it will open a page it's a it's a git repository where you can see a long list of various templates that the
community has created if you have some interesting template you can edit here as well and we will select the
node.js monitoring which is here yeah so i will click to the
node.js and now i can see here the list of files the file i need is this
node.js yaml so i will copy the link address yeah or i will open the yaml and copy paste the the url from from the web browser yeah from the address bar once you have it you need to paste it into the location of the template once you have it you can click to look up the template button that will show what is included into this specific template you can see that there are some buckets there is a one dashboard there are two variables and labels which are some additional items that are used either by the dashboard or by the queries used by the dashboard or some there can be also some configurations and so on and the only thing you need to do is the install template i'm not going to do this because you probably noticed that the install template that i have in this ui is already the
node.js template so i cannot do it but once you click the installation you will see this link the list of or the install template in the list so there will be no js template i have additional one which is out of center that we cannot do later on so now let's look to the dashboard so i will click to the boards on the left side there is a icon called boards and now after the installation of the template i will see there a new dashboard called
node.js application monitoring if i click to this on its ring i will see the dashboard and this is the dashboard which is capturing the
data from
iot center application so i can see the response time i can see the cpu usage the memory usage i can see the response time i can sort it according to the time so i can see the worst or the best responses like as you see the longest ones took 1.5 seconds almost 1.6 i can see the io i can see some additional information like response time in the histogram i can see some additional values that are provided by the resources and so on of course you can edit and adjust the the dashboard according to your needs on the top of the screen i can see also the name of the service or the name of the application as well as i can see also the host so if i have multiple
iot centers running on the multiple hosts i can filter and see the exact ones that i want to analyze so this is the example of the filters i can also see on the right side the filter so if i want to see some longer longer period of the time i can open it and see it as well so now you can also see that there was one query that took almost 20 seconds and it was the query that was getting the
data from inflexDB so if you remember the spinning wheel during this workshop this was the this was the case that we see here so this is the monitoring of
node.js application as i already mentioned you can also add there so your own user events yeah like this one which is here the user registered that you can define for example his name email and result and the response time or how long it took and and so on so you can create there your own custom events that you can use for the monitoring of the application even internally at the monitor we are using this point massively for for the monitoring of our applications it's very powerful and a very quick way how you can capture the information that you need in a few rows of the code inside your application this is of course some basic monitoring there are also some much more
advanced solutions that are able to monitor
node.js application like the company called node source which is using a very similar approach it is using internally also inflexDB to capture and store all the information about the running applications and you can get much more precise and more detailed information about your applications but for basic monitoring you can use the approach that i was showing let me briefly describe also the templates structure or templates we downloaded the template that edit there the dashboard but the template itself can contain also the definition of
data sources it's not mandatory but it's optional the specific dashboards as well as the the queries there are many dashboards you can select the interesting ones including the configuration the always the approach is first of all you will browse the templates like the specific template add there the url of the yaml file which defines the specific template and import it there is a very similar template in this this repository for the
iot center so if i open and import the template for
iot center and you will see something like this this is a very basic template but here you can see similar values well i have to define different filter that are showing similar values like in
iot center application so you can see similar visualizations involve in
iot center for all applications as well as in the ui if in case you build your own visualizations okay so that's all for my side for today are there any questions ivan were there any questions that are interesting to answer to the whole audience okay first of all mirek thanks a lot for your presentations with and the demos with regards to the questions we have quite a lot of fun in the discord chat and well the questions are whether the influx db is the conceptual one is a relational
database or the
database that can be used to store other
data than the time series
data that was answered by blaster influx db is a time series
database only but of course it is possible to store also the configuration
data into a specific bucket technical bucket i think what what is worth to mention is that you can also combine the relational
database with influx db so in flux command you can get
data from relational
database and combine them with with the time series from influx db that's true the flux language has connectors to relational databases and of course it is possible to query or write to the relational databases and use the integrated environment with a time series as well as the relational
database in one in one pocket and there were a couple of the questions related to deployments to the exercises thanks for them um well but i do not see any other questions so maybe we can recap what was already what was already mentioned or what was already presented in this in this in this webinar okay so let me let me continue so what we did we used the
iot center application that we developed primary for education purposes like this workshop you can use it you can play with this you can add there your own code you can base on your your solution based on this this this application feel free to play with this as the next steps it's also definitely worth to look to the gallery of the templates and see what else you can do of course the telegraph is able to do much much more than what you can see in the gallery so it's worth to see also what you can capture from from from from the sources based on the capabilities of telegraph application and yeah there is if you are interested you can join the
community yeah so there are multiple addresses you can use slack you can use github projects many github projects where influx
data is involved as well as the
community influxdata.com webpage if you want to join more tightly with influxdata so thank you very much for your time thank you very much for the questions and and that you were playing with with
iot center and hopefully it will help you to build similar applications in the future so thank you very much