Mastering Error Handling Node.js

Hi, everyone. I am Liz Farradi, and I'm Head Developer Advocate at a really cool company called Fusebit, focused on integrations for developers. Thanks for attending to my talk. Today, I will be talking about mastering error handling in node.js. This is my Twitter handle list for D23. And if you want to create awesome integrations, check out Fusebit.io Twitter account. Error handling should be always taken into considerations when coding, because we all want to write code that works. But errors happen in every single application, from small startups to big tech companies. And developers need to decide what to do about those errors. Simply suppress it, ignore it, maybe notify the users, report to the team, retry, but how many times? Because one does not simply write a code without bugs. And that's a fact. So this is the agenda for this talk. First, we're going to see the types of errors, operational errors, and programmer errors, because there are different strategies for the different types of errors. Then we will see how to handle those errors, and finally, patterns to improve error handling. So let's begin. A big part of programming is just finding and fixing bugs. So it's important to do it like a pro. Sometimes we're trying to fix a bug, we create three more, or even we break the whole application. So if we know the best practices of error handling, it's possible to reduce debugging time, improve the code, detect errors faster, and avoid creating more errors when fixing a bug. Now, let's first identify the type of errors. We can divide all errors into two broad categories, operational errors and programmer errors. The operational errors are runtime problems experienced by code that is well written. So the code doesn't have any problems, no bugs. These errors are caused by something else, some external circumstances that can disrupt the flow of the program execution. Some examples are the system itself, for example, when the system runs out of memory. The system configuration, there is no route to a remote host, maybe because the host is offline, or if you're trying to connect to the incorrect port. The network, where you experience socket hangups. Remote service, a 500 error, which is the famous internal server error. There is something wrong with the server that is not very specific and is not related to your code, the internet, or your computer. It's just a number with the server. Fail to connect to the server, maybe because you don't have internet. Fail to resolve hostname when you have a typo in the hostname that you're trying to connect with. Invalid user input. For example, the user needs to enter an email address, but the email is invalid, or any other invalid input provided by the user. Request a map. For example, the server took way too much time to respond and it fails. A database connection is lost, perhaps due to faulty network connection. This is an example of a system itself running out of memory, socket hangup, database connection lost, invalid input, 500 error, or remote service error, and no Wi-Fi, which can cause fail to connect to the server error. When you experience these kind of errors, there is not much code we can do to fix it. File not found is an operational error, but it doesn't necessarily mean anything is wrong. These situations do not arise due to mistakes in the application code, but they must be handled correctly. Otherwise, they could just cause more serious problems. There are five main techniques to handle operational errors. First, retry the operation. If you're experiencing a 500 error, or fail to connect to server, or the server is overloaded, you can just wait and retry to connect in a few seconds. Network requests to external service might sometimes fail, even if the request is completely valid. Such issues are normally sporadic. So, instead of reporting it or notifying it, you can retry the request in a few times until it succeeds, or it reaches the maximum amount of retries. If the HTTP status code is 500, 503, or 429, it's a good idea to retry after a few seconds, or retry using the exponential back-off strategy, which is delaying the follow-up retry and progressively increasing the delay for each consecutive retry. The second is report the error up to the stack. In many cases, it's a good practice to stop the flow of the program execution, clean up any unfinished process, and report the error to the stack so that it can be handled appropriately. This is useful when a function doesn't have enough information to handle the error directly. The third one, notify the client. For example, if the error is caused by an invalid user input, like an incorrect email address, you should let the client know and make sure to include the information that you need to construct the error message that makes sense to the user, like invalid email address. When dealing with external input from users, always assume that the input is bad by default. Notify the input before it's sent to the user so they can address it before starting any error handling processes. The next one is abort the program. Another solution is just abort the operation in case it's unrecoverable system errors. The only thing we can do is to lock the error and terminate the program immediately. Clean up what you started and start again. That's it. And finally, do nothing. Yes, just do nothing. Sometimes there's nothing you can do about it to operational error. There's nothing to retry or abort. There's no reason to crash the program. An example might be if you're keeping track of a group of remote services using DNS and one of those services falls out of the DNS. There's nothing you can do about it except to log a message and proceed with remaining services. Error handling is about what failed and why. The distinction between operational error and programmer errors is important to figure out how to deliver those errors and how to handle them. Now let's check the programmer errors, which are the errors where developers have control and can be fixed by code. Programmer errors are just bugs in the program. Those can be always avoided by fixing and changing the code or the logic. There are six types of errors that can be found in javascript. First one is type error. It's an error when an operation can't be performed because the value is not an expected type. For example, when you see kind of a property of undefined or pass a string when an object was expected or required is not a function. In this case, the variable of a parameter is not a valid type. You can also get this error when attempting to modify a value that cannot be changed or use a value in an appropriate way. The second one is syntax error, when you miss in parentheses a semicolon or quotation marks inside a string. All of those are syntax errors and are super annoying. Sometimes they're so difficult to find because the syntax of the code is incorrect. The next one is reference error, when you reference or call a variable or object that doesn't exist or is outside the scope of the executing code. For example, referencing the variable number where this hasn't been defined. Those three are the most common, but there are other three that are not that common but is useful to know. Range error is an error you receive when a numeric value exceeds the allowed range. For example, if I have a numeric range from 10 to 20, numbers outside this range will get a range error. And when the maximum call stack size is exceeded, likely because of a problem with a recursive function within your javascript code. The function keeps calling itself indefinitely. Uri error. It happens when the encodeUri or decodeUri functions are used in an incorrect manner. Uri is uniform resource identifier. For example, if you want to decode percentage signs, it will throw an Uri error. Eval error. When you use eval function in an incorrect manner. This exception is not thrown by javascript anymore. Now it's used syntax error instead. That's why I don't have an example here because you will probably never encounter this error. The eval error object remains for compatibility. Now that we know the six types of javascript errors you can find, the question is how to handle them. node.js supports several mechanisms for handling errors that occur while an application is running. How these errors are reported and handled depends entirely on the type of error and the style of the api that is called. There are four main ways to deliver an error in node.js. First one, exceptions. Draw the error for asynchronous operations. Callbacks. Pass the error to a callback. A function provides specifically for handling errors and the result of asynchronous operations. Promise rejections. Pass the error to a reject promise function using async await. Event emitters. Emit an error on an event emitter. Now let's see the first one, exceptions or throwing the error, which is a very common way for functions to deliver errors. All javascript errors are handled as exceptions that immediately generate and throw an error using the standard javascript throw mechanism. When you throw an error, it becomes an exception and needs to be caught somewhere in the stack using a try catch block. If the error is allowed to populate the stack without being caught, it becomes an uncaught exception, which causes an application to exit prematurely. As we can see in this example, if there is no handler, the node.js process will exit immediately. I will show a reference error because Z is not defined. Throw delivers an error synchronously. That is, the same context where the function was called. If the caller's callers use try catch, they can catch the error. In none of the callers, the program usually crashes. Callbacks. The second way of handling errors is callbacks. Callbacks are the most basic way of delivering an error asynchronously. Because of the synchronous nature of node.js, callbacks are heavily used. A callback function is processed as an argument to another function and executed when the function has completed. If you have been working with javascript for a while, you have probably seen the callback pattern a lot. node.js uses the error first callback pattern in most of its asynchronous methods to ensure that errors are checked correctly before using the result to open operations. This callback function is usually the last argument to a function that initiates an asynchronous operation. For example, the real fight method here expects a callback function and its last argument and is called once when an error occurs as result is available from the operation. For example, here at the end in the console log. The first argument is the error. If there is an error, it will be available in the error argument. If not, result will contain the expected result of the operation. It is important to control the flow of the content of the callback function by always checking for an error before attempting to access the result of the operation. Ignoring errors isn't safe and you should not trust the content of result before checking for errors. This is an example of how to use callbacks to throw a type error. Every caller of the square function here will need to pass a callback function to access its results or error. Note that the runtime exception will occur if the callback argument is not a function. Also, we want to avoid at all costs callback hell. A callback instead of callback instead of callback with multiple catch statements can get very messy. Now callbacks are still used for handling errors asynchronously, but they are out of fashion. Because since node.js version 8, the most popular way of handling errors asynchronously is async-await, the third way of handling errors. Promise rejection using async-await. Promises are the modern way to perform asynchronous operations in node.js, and they are generally preferred to callbacks because this approach has better flow that matches the way we analyze programs, especially with async-await pattern. If you use callbacks for async error handling, they can be converted to promises using the built-in utPromiseify method. For example, here's how the fsReadFile method can be made to utilize promises. The readFile variable is a promisify version of fsReadFile in which promise projections are used to report errors. These errors can be caught by chaining the catch method, as shown for example here. You can also use promisify's api in an async function like in this example. This is a predominant way to use promises in modern javascript because the code reads like asynchronous code, and the familiarity of try-catch mechanism can be used to handle errors. It is important to use await before the asynchronous method so that the promise is settled, fulfilled or rejected, before the function resumes its execution. If the promise rejects, the await expression throws the rejected value, which is subsequently caught by the surrounding catch block. You can utilize promises in asynchronous functions by returning a promise from the function and placing the function code in the promise callbacks. If there's an error, reject with an error object. Otherwise, resolve here the promise with the result so it can be accessible in the chain. Then method directly adds the value of the async function when using async await. Finally, event emitters. For more complicated cases, the function itself can return an event emitter object instead of using a callback, and the caller would expect to listen for error events on the emitter. In other words, return an event emitter from the function and emit the event for both success and failure cases. An example of this code is shown here. The emitCount function returns a new event emitter that reports both success and failure events in the asynchronous operation. The function increments the count variable and emits a success every second and an error if the count is divisible by 4. When count reaches 10, an end event is emitted. This pattern allows the streaming of results as they arrive instead of waiting until the entire operation is completed. Here's how you can listen and react to each event emitted from the emitCount function. As you can see here, the callback function from each event listener is executed indefinitely as soon as the event is emitted, one second at a time. The error event is a special case in node.js because if there's no listener for it, the node.js will crash. This is what happens when you don't put the error event listener and you run the program. Just crashes. For the most part, we can join callbacks and event emitters in the same bucket of asynchronous error delivery. If you want to deliver an error asynchronously, you generally want to use one of the other callback or event emitter, but not both. Finally, the last session of this talk, which is patterns to improve error handling. The first one is use error classes, creating a global error class. In this case, call application error that extends other different error classes that are more specific to what you're doing. In order to have different layers when you're handling your errors. One layer is, for example, a database layer. Another one is user facing error and so on. All of these have very specific error cases. You can move all of those error classes into a separate module. That way you can easily import the module and test again the application error. Anything else will be unexpected errors. The second pattern is make message errors and expressive as possible. Return proper errors status and codes. If it's 500 or 104, and just make the errors very descriptive. The next one is YouTube promise if I some projects and APIs only support callbacks instead of a sink weight and is not possible to refactor. So you can use promise if I because it's very useful for this case. The next one is adopt typescript. typescript is a strongly typed superset of javascript. Its primary design goal is to identify constructs likely to be errors without any runtime penalties. By adopting typescript in your projects, you can eliminate a whole class of programmer errors at compile time. For example, it was estimated that 38% of bugs in Airbnb code base were preventable by typescript. When you migrate your entire project to typescript, errors like undefined is not a function or syntax error or reference errors should no longer exist in your code base. Migrating your entire node.js application to typescript can be done incrementally. So you can start getting the rewards immediately in crucial parts of the code base. The next one is stack traces. A stack trace is a list of methods called that the application was in the middle of processing when the exception was drawn. You have all the information for where you awaited your promises. Highly recommended to use as much as a sink weight as you can. If you don't, you will lose valuable debugging information. Number six, automated testing. The javascript language doesn't do much to help you find the mistakes in the logic of your program. So you have to run the program to determine whether it works as suspected. The presence of automated tests makes it far more likely that you will find and fix various programmer errors, especially logic errors. They are also helpful in asserting how a function builds with atypical values. Using a testing framework such as Jest or Mocha is a good way to get started with testing/talks">unit testing for node.js applications. Finally, you should know that in javascript and node.js, there's a difference between an error and an exception. An error is an instance of an error class, errors that might be constructed and then passed directly to another function or term. When you throw an error, it becomes an exception. Here is an example of using an error as an exception. Throw new error, something that happened. But you can just as well create an error without throwing it. Call back new error, something that happened. And this is a much more common in node.js because more errors are asynchronous. It's very uncommon to need to catch an error from a synchronous function. Conclusion, proper error handling is a non-negotiable requirement if you're aiming to write good code and reliable software. If I employed the technique described in this talk, you will be able to handle it correctly. Thanks for watching. Happy coding. Hey, Liz. Hello again. Yeah, and that was awesome. Awesome talk. I loved your code. One simply doesn't write code without errors. Yeah, that's totally true. So, yeah. So you asked the question, HTTP error 404 is operational or programming error. And 57% of people say it could be both. 28% say it's an operational error with 9% neither. And it changes, but yeah, 4% it's a programmer error. Yeah, that's actually right. If the majority of the people that responded the poll is correct, because 404 can be both. Usually it can be an operational error because maybe there is a typo in the route. For example, a route is the slash contact and the person typed it slash contacto. And that would show 404 because it's not found, right? But sometimes, and less frequent, it can also be a programmer error because the requirement could say that the route should be contact, but the programmer creates like a different route, let me say. So the people is right. You will learn a lot in this talk. So good job. Yeah, you did a great job explaining all the errors. Yay. Okay. Let's take a look at some questions. So, Arzintile1990 asks, does infrastructure as code and the devops approach blur the line or complicate deciphering between operational and programming errors? So what I've read doing this research and yeah, everywhere is like, there's always going to be two types of errors. Like sometimes errors can be both, like we just saw, like 404 can be both, but there is like this distinction always. So yeah. So yeah, there will always be distinction depending upon the use case. Yeah. Yeah. Depending on the use case. And sometimes one error can be both. Can be. Yeah. Okay. So yeah, totally depends upon the context. I hope they got the question answered. Okay. So we have another question. Yeah. Again, I think that is what we just discussed is can error be both operational and programmer error or is it one of the other? So yeah. Yeah. Well, yeah. Sometimes you have both operational and programmer errors as part of the same root problem. For example, if a HTTP server tries to use an undefined variable and it crashes, that's a programmer error. And any client we request in flat at the time of the crash, we'll see an error, but typically reported as a socket hangup. Yeah. So it can be basically can be both. Okay. Nice. And errors are so fascinating. So, okay. So, and we have a question. Is it possible to keep another statement in between try cache and finally blocks? No, it is not recommended to include any statements between the section of try and cache and finally blocks since they form one whole unit of the exception handling mechanism. So it's not recommended. Yeah. We'll throw an error if you do that. Okay. So talking about these errors, I was wondering what are some kinds of, let's say, annoying errors that you have come across while coding? Yeah. For example, blocking the event loop can be super annoying because it could be easy to block the event loop because of the single threaded nature of node.js. But also in blocking, I call that more than once. I have done that. I'm sure a lot of people have done that too. Or expecting callbacks to run synchronously since no, no, no. Or join errors from inside callbacks. Like those are errors that are very common and they're very annoying. And it's like, oh, I did this again. Or the typical error to just forget like a semicolon or like a parentheses or whatever, and everything crashes. Yeah. Those are annoying. Or I guess errors in general can be super annoying because sometimes it's like the simplest thing ever. And it's just so hard to find. And sometimes the errors are super hard and you just like know what to do. So yeah, errors are a big part of programming. And I was very excited to give this talk because this is a big part of it. And knowing how to handle those errors depending on the use case is very important. So there are these different kinds of errors and they are super annoying. So what's your approach? Like how do you tackle these? How do you debug? And what's your approach? There are multiple ways in which we can debug the errors and all. The first thing I always do is just console log everything. I'm a big fan of console log. Some people just, there's other techniques like async trace. There is, yeah, there is like different techniques. You can also use like an APM or external tools for debugging. Yeah. But there's like a whole new different talk, how to debug applications. Yeah. I mean, I always keep reading all these different ways, but console log, nothing beats console log. Yeah. It's super handy. Okay. So another question from Metin. And I read something this week about a global error catcher, something like report error. Did you see this new api? Did you take a look at it, this new api? No, I didn't see it actually. I should check it out. Thank you, Metin. Like now we all know about this new api. Thank you. Yeah. Sounds interesting. Global error catcher. Yeah. Okay. So tell us something. Is there something that you would like to elaborate more in your talk? Like you gave a super nice talk. So is there some point which you would like to elaborate on? I would like the audience, if they can just in the Q&A, this core channel, if they can tell me like the errors and what errors they have, what are the least favorite errors or what are the easiest errors for them or the most difficult, like I would like to read their approaches on errors so they can like overscore if you can answer those questions or yeah. So yeah, do let us know what were some of your errors which you found and yeah, we will take a look. And there's another question, which is what is the most time you have spent on debugging an error and do you recall it being worse or better than expected? Sorry, can you repeat again? What was the question? Yeah, sure. So what is the most time that you have spent on debugging an error? Okay. For me, I think it was like a whole day. Like I couldn't figure it out. It was a while ago and I was just spending like a whole day thinking about this error and then as usual, like sometimes it's really good like to just completely disconnect and go to bed or wait or go for a walk or anything and then just come back and you can actually solve the error like magically because you were maybe you come with a different approach or you have a different mindset. So because sometimes when you're stuck in an error and you just try and try and try, it's just like in a loop, it gets harder. So sometimes disconnected can be, oh, maybe I shouldn't include that in my next talk. Like disconnected is also a form of error handling. Like just, yeah. Yeah. Sometimes when you see it with fresh eyes again and then you can suddenly have that moment, yes, this was the error. Sometimes like, I don't know if that happened to you, but sometimes I even before like when I was like coding a lot right now, it's like I'm developer relation, so I do less coding. I could even dream about coding and it's like I would solve the errors when I was sleeping and it's like, oh no. But yeah. I think so, J.S. Scars asks, what is your opinion about typescript versus tdd? Well, I really like typescript. I think, yeah, I think it can solve a lot of problems and also test driven development. I guess that's why you may buy tdd. Yeah. I mean, there is no like big difference. I mean, there is like, there are two different things. Like one is typescript, which is, yeah, what I say before. And the test driven development is just, it covers the test case before the software is fully deployed. Like that's what we have, CI, CD and all that. So I think both of them like tackles different things. So I would use them both really. Yeah. Yeah. Yeah. They have some different use cases and we should be using them both maybe to have a less error. Yeah. And you Anuela, I would like to know what type of errors, which one is your least favorite or your most memorable error? I hate the kind of errors in which it was very easy. Like just at my site and I couldn't see it. I mean, I'm debugging the whole thing, just going deep into the code, like analyzing lots of stuff. Yeah. This, that, all big stuff. And then it turns out to be that single, maybe a single loop issue or sort of that. A very small issue, just online chains or one word chains and you are done. So once I remember I had this very peculiar issue with concat, array concatenation and all. So like these are always super confusing. Some of the functions are returning errors. Some of them are mutating and you are using one or the other and then your result is totally going haywire. Yeah. Yeah. So, so we, that's all the questions which we have. And thank you Liz once again for sharing all this information with us and audience, you can still continue the conversation with Liz. Go over to the special chat and join the Liz speaker room and you can still talk to her. Thank you Liz for joining us again. Bye. Thank you. See you on the other side.