Web3.js - Past, Present & Future

Rate this content

Will discuss the challenges faced in Web3.js project.

Some opensource contribution statistics.

Current refactoring objectives we are working on to make web3.js better.

Release timeline.

Overview of future plans.

21 min
20 Jun, 2022

AI Generated Video Summary

Today we will discuss the Web3JS library, its history, maintenance, and community involvement. The upcoming version 4 aims to address challenges faced in version 1 by introducing native TypeScript support, reduced size, improved code readability, and increased test coverage. Version 4 also introduces a new validator for easier Ethereum data validation and allows developers to customize how they handle numbers and bytes. It brings a dynamic data format for custom formatting and introduces TypeScript for contracts without transpiling. The API in version 4 is easy to extend and has future improvements and refactoring.

1. Introduction to Web3JS Library

Short description:

Today we will discuss the Web3JS library, its history, maintenance, and community involvement. Started in 2014 by Jeffrey Bickle, the library has been maintained by various contributors and organizations. With over 100 contributors and 17,000 pull requests, the library has undergone extensive testing. Fun facts include the library's two-year beta period and failed rewrite attempts. Despite this, version one remains the most widely used.

Hi everyone, it's Nazar here. And today we are going to discuss the Web3JS, one of the most famous and widely used libraries in the Ethereum community. So we will be discussing some past aspects of the library and some future aspects as well as some present milestones that we have achieved currently. So let's get started.

So first of all, we're going to explore the past about the library. Most of you had been using this library a lot, but may be not aware of how this library was actually started. So this Web3js library was started in 2014 by a solo contributor, Jeffrey Bickle, and in 2016 it was moved to another solo maintainer, Samuel Furter. And from 2018, Ethereum JS from the Ethereum Foundation was maintaining it. And since two years, our company ChainSev has been responsible to maintain this library. And if you look on the facts that this library has been maintained for over eight years, it has over 100 contributors overall and have 17,000 more than 1700 pull requests.

And so far 172 versions were released and out of which 90 were the pre-release version. That means this library had very thorough investigation and debugging and testing from the community during the pre-release versions. 90 pre-release versions is a large number. And this library has 600,000 weekly downloads as well. And it's all because of the community who is using and trusting this library.

And there are some fun facts about the library. I have asked so version one of the library web3.js has been living in beta for two years and people have been using v1 beta in the production environment for many years. Because this v1 was in extensive testing and thorough debugging process. And we started to have a v2 rewrite in past and it was a fail and abandoned rewrite attempt because of some aspects I will discuss in further slides. We started v3 rewrite with some extensive features, which never lasted a lot. And since then, v1 is the only version which the whole community has been using so far.

2. Web3 JS Version 4 and Achieving Goals

Short description:

The Web3 JS library has faced challenges in version 1, including an obscure code base, tightly coupled modules, TypeScript issues, and build size problems. Despite failed attempts at rewriting, the upcoming version 4 aims to address these challenges. It will feature native TypeScript support, reduced size, improved code readability, and increased test coverage. The goals for version 4 have been achieved while maintaining the v1 Parity API, ensuring minimal project breakage.

This is quite an interesting aspect as well, and considering that like two rewrite attempts or two version attempts has been failed so far, but it doesn't mean that the library v1 version has been unstable. So v1 has been maintained and supported so far and had been doing it. And if you look at the aspect that v1 lived for the two years in the beta, I think this is the most longest time after another project expressjs version 5, it will last for more than two years.

And there were some challenges we were facing in the web3js version 1. Most of you can understand it because you may have already reported some of the issues or have been also encountered or facing those issues. It was very obscure code base. The code base was being maintained by over 100 contributors I explained earlier. The code base was not using some standard process or standard patterns. The modules inside the code were very tightly coupled. There were a lot of technical challenges to maintain this code and there was a lot of difficulties to introduce new features particularly in the library v1.

And most of the you guys have already been using the typescript. So there were a lot of decoupled types. Decoupled types were in the web3.js v1. And because of that, there were a lot of typescript issues being reported by the community. And if I look on the logs, I can see around 60 over 60 issues been reported of the mismatch or inaccurate types, which was one of the biggest challenge for us. And from the community perspective, the build size became around the bottleneck. And for the dev development, because this library was bundled in a huge single package file and that package file was being bundled in every dev. And this was causing a lot of problem during the dev development by a lot of community as well. And on a larger spectrum, JS community and the toolchain has been advanced a lot. A lot of new frameworks, new patterns and new tools have been introduced in the community in the ecosystem, but Web3 JS could not get way forward because of those challenges I discussed earlier.

So are we going to stick with the version one, or are we going to do something about it? Despite the two attempts or failed attempts, we are still hopeful that we are going to present the current improvements in the library and with that said, this is the new era for the Web3 JS, the version four, and this is not some speculation. It's been under development for over a year now. And we are almost ready to release, hopefully it's been released this month, but you can already explore directly from the GitHub. So version four is going to be a huge, huge, huge milestone for the Web3 JS lifespan and it's going to address a lot of stuff that we've been discussing in the community and all those people have been addressing those things. So version four will come with a native TypeScript support and it will have a reduce size. We have decoupled a lot of code and the complexity has been reduced since we are doing this rewrite completely from scratch, so the code readability has been improved a lot, and we try to make it extensible so that in the future we can add more features very easily and even the community can extend the library for their own use, and we focus on the test coverage for the version four and hopefully once, right now we are working on writing the system test for the library, and within a month when we will release the first alpha for the v4, it will be having a lot higher test coverage compared to the version one, and compared any other competitive library in the ecosystem. So the question is, does all these goals that we defined, rewrite for the version four, are these good enough, or have we achieved it or not? Yes we have achieved all these goals, and the most important thing is that we achieved all these goals with a v1 Parity API. So you don't need to be worried about that your projects are going to break. There could be slight changes that we will be documenting and sharing with the community in the change log or the migration guide, but those changes will not be a lot. Overall, the API of version one remains the same.

3. New Validator and Custom Data Handling

Short description:

We have introduced a new validator in the version 4 rewrite of the Web3.js library. This validator provides declarative validation using JSON schema, making it easier to validate Ethereum data. It is compatible with Ethereum ABI schema and eliminates the need for complex imperative validation. Additionally, version 4 introduces a feature that allows developers to customize how they handle numbers and bytes in their dApps.

We explicitly decided this architecture change, or this pattern, so that a lot of community does not break their projects, and they can easily migrate to the version four, which is more recent.

And the question is like, does it just rewrite of the v1 with the same API or does it include some new features as well or not? Yes, it is. It contains a lot of new features. It's not just the rewrite. Rewrite was the first objective, but during the rewrite, we focused on creating more features within the library so that we can be future safe and we can extend the library very easily in the future.

And some of the features which I'm going to present here, which is very unique for the version one, version four for the Web3.js library is unique in the Ethereum ecosystem. No other package or the utility as of now, which I remember may have introduced such features and hopefully you guys will like them.

So the first thing, the most important feature within our rewrite is this new introduction of the new validator. Because data is so important for any dev particularly and data validation has been an extensive job for the maintainers of the different devs and we decided to create a brand new layer of the declarative validation for the community. And we use this validator within our library as well and it is also available for you to use in your devs.

And so what is a declarative validator? So most of you know that JSON schema or JSON data is kind of a defacto data format for the JS community. JSON schema is also a defacto data standard for validation and communities already adopted it. So we use both of these schemes as a JSON data and the JSON schema and merge them together for the Ethereum community. And you don't need to have no if-else anymore and all of the validation that you're going to do using the JSON schema will be tightly coupled with the Ethereum types. So you don't need to remember the Ethereum native types or compare to the JavaScript native types. It will work seamlessly. And this validator is also compatible with Ethereum ABI schema. So if you have such an ABI, it will also validate the data based on the ABI as well.

Let's jump to some examples. So here you can see some examples. So we have some data and which contains some addresses in an array and some balance, and, if I go with the imperative validation, which we had been doing in previous library versions, you can see that, we have, some loops going on and then we are validating by using function calls and then throwing some errors and same for the balance field that we have, we are particularly validating by using some function call. But if we go with a declarative approach, you can see that we have schema defined for the data and this schema is going to present the whole anatomy of the data and if you notice that the data that we are going to present is very particular for the Ethereum. So we have an address array, we have UN256. These are Ethereum types and we can validate this data based on this schema and there are different formats of the schema and hope you can explore further.

So there is another feature that we introduced in version 4 rewrite and this is going to change the way you look into your data. So the question is that every dApp that you develop is not the same. So the need for every dApp is also different. For example, someone developing an app or using a library where he needs numbers in the beginning. Someone wants numbers in a number or someone wants number to be a number string. Similarly for the bytes.

4. Dynamic Data Format and Custom Formatting

Short description:

The previous version of the library was restrictive to one data format, which was counterintuitive for dApps using different formats. To address this, we are introducing a dynamic data format that allows custom formatting. You can define your own format for numbers and bytes and use it in function calls. Our library has been extensively using this feature.

So there can be different formats of the same data that you can use but the previous version was very strict and very restrictive to the one data format. Which was not counter which was counterintuitive for the different dApps which were using different data formats.

So we tried to cover this as well and we are creating a dynamic data format, which is quite easy. All of the Ethereum web 3js library interfaces are going to return are going to support this custom data formatting. And let's say we have this get balance and by default, this get balance will return you a balance in the hex string format.

But if you want that the balance function should return the balance into different formats, you can define your own data format, specifying the format for the numbers and specifying the format for the bytes and passing these formats to the every function call is the last parameter and you will be get the data in that particular format that you define. And you can use this formatting logic in your own project as well. Our library has been extensively using it.

5. Typescript for Contracts

Short description:

Web 3.js version 4 introduces Typescript for contracts without transpiling. Simply declare your API in a Typescript project as a constant, and you'll have a typesafe contract instance. No transpiling is required, and changes to the API will automatically update the types. Share your feedback on this unique feature.

So one more important feature. Maybe everyone should be relying on it, and I think this is one of the unique features which Web 3.js version 4 is going to introduce, is Typescript for the contracts without any transpiling. So you don't need to transpile anything. All you need is your API declaration in the Typescript project. So there is no need for the transpiling of the APIs or creating or doing some command line execution of any other third-party tools. You just need to copy your API, declare them into the Typescript project, and you will be ready to have a complete typesafe contract instance. Let's say we have this API and all you need is to have this API declared, and the trick is here that you have to declare them as a const, so the Typescript does not infer them as generic types rather have strong types for each declaration. And once you have declared this API as a constant, you can create your contract instance and this contract instance will be completely typesafe and there is no transpiling required. And if you ever change the API or update your API, your types will automatically reflect those change and Typescript can detect the validations on the compile time. This is a unique feature and I would love that we'll be sharing some feedback on it.

6. Easy Extensibility and Future Improvements

Short description:

We are introducing easy extensibility for our API, providing a consistent API for all Web3 modules. You can also create your own modules by extending the Web3 context. Version 4 has many future improvements and refactoring. It is currently in alpha and can be installed from the GitHub repository.

And there is another feature that we're going to introduce is easy extensibility for our API. So the Web3 modules have introduced a different or consistent API to support all modules. So when I'm referring to the modules, you may be aware that Web3 has different modules like contracts, Ethereum, SSH, ABI, ENS. So these are different modules. So we introduced a consistent API for all of the modules so that you don't have a different compatibility issues using them.

And you can also create your own modules if you want. And the feature is very simple and like straightforward to introduce, you just need to, we have consistent object or Web3 context and all you need is to declare your component or the module which is extended from the Web3 context and then you have a library instance Web3. You just call them a use and you will get an instance of the component which will have all of the main component of the library link. For example, a provider will be linked a request manager, a subscription manager and a lot more will be linked to your component. And that will easily, so you can use your component easily with the same consistent API which we had been using in the Ethereum or contract or ABI or ENS packages. So this is all from the version four, so the question is like, this is all the features that we're gonna introduce. I think not. There are a lot of future improvements, a lot of refactoring, a lot of things going on in the version four and I will left all those to you that you can explore in the version four and the version four is in the alpha hopefully this month but you don't need to wait it. You can go to the GitHub and check out 4.x branch and it will be all up for you to install directly from the GitHub and use it.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

TestJS Summit 2022TestJS Summit 2022
21 min
E2E Tests for Web3 Applications
We will go through a brief explanation of what is Web3 and the architecture of a web3 application. Then we will talk about how to end-to-end test, its challenges, some test tools that are available, and a demo using cypress and metamask.Agenda: What is Web3; The Architecture of a Web3 Application; Web3 E2E Tests Introduction; Web3 E2E Tests Challenges; E2E Test Tools; Demo.
React Advanced Conference 2021React Advanced Conference 2021
30 min
Building Dapps with React
Decentralized apps (dApps) are continuing to gain momentum in the industry. These developers are also now some of the highest paid in the entire trade. Building decentralized apps is a paradigm shift that requires a different way of thinking than apps built with traditional centralized infrastructure, tooling, and services – taking into consideration things like game theory, decentralized serverless infrastructure, and cryptoeconomics. As a React developer, I initially had a hard time understanding this entirely new (to me) ecosystem, how everything fit together, and the mental model needed to understand and be a productive full stack developer in this space (and why I would consider it in the first place). In this talk, I'll give a comprehensive overview of the space, how you can get started building these types of applications, and the entire tech stack broken apart then put back together to show how everything works.
React Summit 2022React Summit 2022
22 min
A Frontend Developer’s Guide to Web3
There is a lot of hype and interest around diving into the Web3 space, and it can be hard to manage the information overload and the emphasis on learning new languages to work on smart contracts. Many developers in the frontend space ask, “How much of this new information do I need to transition into web3?” The truth is as a React developer, the skills I know are essential in web3. Let’s add some context around how to leverage those existing skills to succeed in this new space.
In this talk, we’re going to explore how to build the correct context and the technologies to understand to help you dive into web3 as a front-end developer. We’ll go over popular packages like Ethers, which can be used in React applications to interact with wallets and existing smart contracts. We will also explore how you can leverage your Javascript skills to build full-stack dApps using managed services like Moralis and Thirdweb.
Let’s demystify some of the concepts in web3 and leverage our existing skills to start building.
React Summit 2022React Summit 2022
6 min
Reacting to Web3
What are the opportunities available to web devs in web3? What problems are they likely to face transitioning to web3? How the Decentology's Hyperverse solves these problems and can help them avail of these opportunities.
JSNation 2022JSNation 2022
21 min
Build Blockchain dApps using JavaScript
Have we found nirvana for Blockchain developers? This session will focus on building blockchain dApps (distributed Apps) with one set of source code and deploy to multiple blockchains! The session will cover getting started with Reach and being able to deploy the same solution to multiple blockchains including Algorand, Ethereum, and Conflux. New to Reach? We will cover how to set up your development environment as well as walk through a simple app frontend and backend. Finally, we will look one the huge benefits of Reach in the built-in verification process. Reach provides automatic verifications to ensure that your program does not lose, lock away, or overspend funds and guarantees that your applications are free from this entire category of errors.

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
145 min
Web3 Workshop - Building Your First Dapp
Featured WorkshopFree
In this workshop, you'll learn how to build your first full stack dapp on the Ethereum blockchain, reading and writing data to the network, and connecting a front end application to the contract you've deployed. By the end of the workshop, you'll understand how to set up a full stack development environment, run a local node, and interact with any smart contract using React, HardHat, and Ethers.js.
JSNation 2023JSNation 2023
64 min
Querying Blockchain Data with GraphQL
Curious about how data works in the world of blockchain? Join Simon in an engaging session about The Graph, the decentralized indexing protocol that makes it easy for blockchain developers to search and query blockchain data. 
Table of the contents:- Understanding blockchain data- Anatomy of a smart contract- Indexing blockchain data with The Graph- Accessing data on The Graph- Recommended subgraphs- Writing subgraphs overview
JSNation 2023JSNation 2023
117 min
How to Convert Crypto Currencies With GRPC Microservices in Node.js
The workshop overviews key architecture principles, design patterns, and technologies used to build microservices in the Node.js stack. It covers the theory of the GRPC framework and protocol buffers mechanism, as well as techniques and specifics of building isolated services using the monorepo approach with lerna and yarn workspaces, TypeScript. The workshop includes a live practical assignment to create a currency converter application that follows microservices paradigms. It fits the best developers who want to learn and practice GRPC microservices pattern with the Node.js platform.
Prerequistes:- Good understanding of JavaScript or TypeScript- Experience with Node.js and writing Backend applications- Preinstall Node.js, npm- Preinstall Protocol Buffer Compiler- We prefer to use VSCode for a better experience with JavaScript and TypeScript (other IDEs are also ok)
React Summit 2022React Summit 2022
134 min
How to Build UI's for Smart Contracts
There are many journeys we can take into becoming a web3 developer. Not all journeys require learning a completely new language and becoming a smart contract developer. In this workshop we will learn how to build our own React UI on top of any existing smart contracts on the Polygon Proof of Stake Blockchain. We will explore a basic overview of blockchains and learn about the advantages of building on Polygon. We will also gain an understanding of what Smart Contracts are, learn how to find existing ones, and learn tools that help us interact with them from a React application.
JSNation 2022JSNation 2022
49 min
Build Web3 apps with Javascript
The workshop is designed to help Web2 developers start building for Web3 using the Hyperverse. The Hyperverse is an open marketplace of community-built, audited, easy to discover smart modules. Our goal - to make it easy for JavaScript developers to build Web3 apps without writing a single line of smart contract code. Think “npm for smart contracts.”Learn more about the Hyperverse here.We will go over all the blockchain/crypto basics you need to know to start building on the Hyperverse, so you do not need to have any previous knowledge about the Web3 space. You just need to have JavaScript experience.
React Summit 2022React Summit 2022
51 min
Build Web3 apps with React
The workshop is designed to help Web2 developers start building for Web3 using the Hyperverse. The Hyperverse is an open marketplace of community-built, audited, easy to discover smart modules. Our goal - to make it easy for React developers to build Web3 apps without writing a single line of smart contract code. Think “npm for smart contracts.”
Learn more about the Hyperverse here.
We will go over all the blockchain/crypto basics you need to know to start building on the Hyperverse, so you do not need to have any previous knowledge about the Web3 space. You just need to have React experience.