Supercharging Developer Productivity With Advanced Code Search

Rate this content
Bookmark

Google & Facebook Engineers are able to search over their gargantuan codebase using an internal code search engine. The search engine accelerates the ability of their developers (both old & new) to understand any part of their codebase & start contributing immediately! What about the rest of us?

In this talk, I'll walk through the different types of code search, tools & software, and advanced tips & tricks to navigate any type of codebase easily. With the advent of large code repositories and sophisticated search capabilities, code search is increasingly becoming a key software development activity. Every developer in the world that spends an insane amount of time reading rather than writing code should have access to the best code search tools that amplifies their productivity. The audience will walk away with everything they need to confidently onboard, navigate & understand any small, medium-sized and daunting codebase.

10 min
22 Oct, 2021

Video Summary and Transcription

Developers can use Sourcegraph to easily search for code using literal, regular expression, and structural patterns. Sourcegraph allows searching through open source and private code indexed across GitHub and GitLab, and supports searching for unstable batch API in React.js. It also enables searching with regular expressions, finding dependencies and usage patterns. With structural search, developers can search for specific blocks of code and narrow down their search using keywords. Sourcegraph is essential for developer velocity, helping teams quickly search through repositories, find references to shared systems, and providing powerful search parameters.

1. Introduction to Sourcegraph and Code Search

Short description:

Developers spend a lot of time reading and writing code, so they need tools to easily search for code. Sourcegraph is a search engine for open source and private code. It offers features like local search, code intelligence, batch changes, code monitoring, and code insights. With Sourcegraph, developers can search using literal, regular expression, and structural patterns.

All right, so if you're on Sourcegraph.com, just follow me. I'm Prosper Otemiyuwa. I work as a Developer Advocate at Sourcegraph. We have an agenda.

So every developer in the world spends a huge amount of time reading and writing code. In fact, if you use GitHub a lot, you discover that even when you're not with your laptop, you're on GitHub, you're trying to get notifications for pull requests, comments, and every other thing, you're doing code, right? So if you spend more time reading and writing, then you should have the tools that allow you to search for code very easily.

So we have a tool called Sourcegraph. Sourcegraph.com. That's why I said if you open your phones. If you open your phones and go to Sourcegraph.com, I like to call Sourcegraph.com the Google of code search. So this is how it looks like. If your interface doesn't look like that, you're not on Sourcegraph, please. So just cross check.

So with Sourcegraph, this is the value we offer to developers. Sourcegraph has currently indexed over 2.1 million open source repositories across GitHub and GitLab. So right now, you can search code on Sourcegraph and it gives you data from GitHub and GitLab, right? And you can also set private code across several repositories. With Sourcegraph, you can do a local search on your local IDs. You can have precise code intelligence on Sourcegraph.com. And there's a feature we have called batch changes. Instead of automating, instead of, you know, opening several pull requests to different with batch changes, you can have a file and then it can make several pull requests for you. Instead of you having to do that yourself. So it's called batch changes. And then we have two features called code monitoring and code insights. But for this talk, I'm going to be talking more about the search.

All right. So, like I said, Sourcegraph can be called Google of code search. It's literally a search engine that allows you to search all of open source code and all of your private code, right? So what are the code search patterns we have with Sourcegraph? There are three types of search you can do right now. Literal, regular expression, and structural. So let's go to literal. I know many of us are familiar with the source space component, right? So if you're onboarding a new developer or you get onboarded to a codebase and you're trying to see occurrences of a particular class, symbol, or definition, you can literally copy the codebase and paste it into Sourcegraph and then it goes ahead to search for you.

2. Sourcegraph Search Capabilities

Short description:

Sourcegraph allows you to search all the open source code indexed across GitHub and GitLab. You can search through divs, time difference, committers, and APIs. It supports searching for the unstable batch API in React.js and provides results from within the React code and organization. Sourcegraph also enables searching with regular expressions, allowing you to find dependencies and usage patterns. You can easily see how many people are using your open-source packages or specific versions. Additionally, you can search through branches and connect private repositories. Structural search is another powerful feature of Sourcegraph.

So right now, this is me placing the source space components in Sourcegraph, and it searches all of the open source code it has indexed across GitHub and GitLab. Examples are results of people that are using source space across this GitHub and GitLab. So now you can see the responses. You can scroll down. On the left-hand side, you can see all the filters and operators that you can add.

So you can search through divs, you can search through the time difference, you can also search through the committers of the code, and you can have access to how people are using certain APIs. Another API that you can talk about in React JS2 is the unstable batch of this. So you want to know how many people are using unstable batch of this in the world? How many people are really using it, and how are they using this particular API method? With source graph, all you need to do is paste it in. So if you look at the arrow, it searches within the React.js organisation and it brings you out results of every usage of unstable batch of this. From within the React code itself and every other example within the React organisation, you can get results.

Now let's go to regular expression. How many of us are familiar with regular expression? I know we probably don't use it every day, but with source graph, it allows you to flex your muscles with regular expressions. So if you understand what you're trying to look for, you're trying to look for something within your file, for example. Here I was trying to look for how people are using some of the latest versions of React by using regular expression to search via the packages of several repositories. So in the source graph search engine, I toggled on the regular expression mode, and these are the results it gives me. So with these results, you can see and you can narrow it down to get the results of how many people or how many projects are dependent on a particular version of React. So some of us here are open source maintainers, others are open source authors. You want to get an idea of how many people are using your open-source package or how many people are using certain versions because you need to deprecate certain methods or because there is a new architecture you need to work on. So with this, you can easily scroll and see how people are using the webpack, Remix Run, Visualizer. You can see how many people are dependent on a particular version of your project.

This is an example of me searching through branches. I was searching through the source graph repository, and specifying with regular expression to search within every branch that has mchap within the name of the branch. So this goes into all of the branches that have mchap within it and then searches for Ubuntu latest. With that it returns all the codebase for me. This is just open-source code. Then connect all your private repositories as well. The same results it gets you. It just searches within the scope of all the repositories that you've connected to source graph. And then we have the structural search. I think this is one of my best types of search.

3. Structural Search and Code Blocks

Short description:

With structural search, you can search for specific blocks of code by pasting them into the Sourcegraph search engine. It will return all the projects, code bases, and files that contain that code. You can further narrow your search using various keywords.

With structural search you can literally take blocks of code. So you can take if conditions, try catch, a full block of code, and paste it into the Google search engine, into the source graph search engine, and it's going to give you the results you're looking for. So this is me, you know, searching through Google organization on GitHub and looking for this block of code. If not path.length. And you have the three dots, right? So it goes ahead and looks for all the types of code that fulfills this particular condition and returns them to me. So now I can see all of the projects and all of the code bases, all of the files that have this within them. And then I can keep narrowing my search, right? There's so many keywords that you can use. In fact, we have a boot out there with a cheat sheet, so I encourage you after this talk or when you're free to come down to our boot and get the cheat sheets. There's a lot of things to learn there.

4. Benefits of Sourcegraph for Developer Velocity

Short description:

Sourcegraph is essential for developer velocity, allowing teams to quickly search through repositories and enabling new hires to start committing code within a week. It helps with refactoring by finding references to shared systems across repositories. Sourcegraph offers powerful search parameters like 'repo', 'branch', and 'type: symbol'. Learn more at learn.sourcegraph.com and explore additional resources on Devtool, the Sourcegraph blog, and the DevTool Time YouTube series. Follow me on Twitter for more information and discussions.

So why do you actually need Sourcegraph? Developer velocity. As your team grows and you build more features, you want your developers to move quickly. Sourcegraph allows them to easily search through all your repositories, enabling new hires to start committing code within the first week. With Sourcegraph, everything is in one place, making it easy for developers to get up to speed on the codebase. It also helps with refactoring by finding all the references to shared systems across different repositories, allowing for faster progress.

Now, let's recap the search parameters. Sourcegraph offers various keywords for searching, such as 'repo' to search within a particular repository or organization, 'branch' to narrow the search to a specific branch, and 'type: symbol' to only return the definition of a class or function. You can also search within specific file types or commits. Sourcegraph is a powerful tool for discovering how certain code patterns are used across your codebase.

If you want to learn more about code search, visit learn.sourcegraph.com for tutorials on regular expression search, structural search, and more. Sourcegraph provides additional resources like an active presence on Devtool, a blog, and the biweekly DevTool Time on YouTube, where you can see different developer workspaces. You can also search for Sourcegraph podcasts. Follow me on Twitter for any further questions or discussions.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

JSNation 2023JSNation 2023
29 min
Modern Web Debugging
Few developers enjoy debugging, and debugging can be complex for modern web apps because of the multiple frameworks, languages, and libraries used. But, developer tools have come a long way in making the process easier. In this talk, Jecelyn will dig into the modern state of debugging, improvements in DevTools, and how you can use them to reliably debug your apps.
JSNation 2022JSNation 2022
21 min
The Future of Performance Tooling
Top Content
Our understanding of performance & user-experience has heavily evolved over the years. Web Developer Tooling needs to similarly evolve to make sure it is user-centric, actionable and contextual where modern experiences are concerned. In this talk, Addy will walk you through Chrome and others have been thinking about this problem and what updates they've been making to performance tools to lower the friction for building great experiences on the web.
DevOps.js Conf 2022DevOps.js Conf 2022
31 min
pnpm – a Fast, Disk Space Efficient Package Manager for JavaScript
You will learn about one of the most popular package managers for JavaScript and its advantages over npm and Yarn.A brief history of JavaScript package managersThe isolated node_modules structure created pnpmWhat makes pnpm so fastWhat makes pnpm disk space efficientMonorepo supportManaging Node.js versions with pnpm
GraphQL Galaxy 2022GraphQL Galaxy 2022
31 min
Your GraphQL Groove
Building with GraphQL for the first time can be anywhere between daunting and easy-peasy. Understanding which features to look for in your client-side and server-side tooling and getting into the right habits (and ridding yourself of old habits) is the key to succeed with a team of any size in GraphQL.

This talk gives an overview of common struggles I've seen numerous teams have when building with GraphQL, how they got around common sources of frustration, and the mindset they eventually adopted, and lessons learned, so you can confidently stick with and adopt GraphQL!

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
174 min
React, TypeScript, and TDD
Top Content
Featured WorkshopFree
ReactJS is wildly popular and thus wildly supported. TypeScript is increasingly popular, and thus increasingly supported.

The two together? Not as much. Given that they both change quickly, it's hard to find accurate learning materials.

React+TypeScript, with JetBrains IDEs? That three-part combination is the topic of this series. We'll show a little about a lot. Meaning, the key steps to getting productive, in the IDE, for React projects using TypeScript. Along the way we'll show test-driven development and emphasize tips-and-tricks in the IDE.
React Advanced Conference 2021React Advanced Conference 2021
168 min
How to create editor experiences your team will love
Workshop
Content is a crucial part of what you build on the web. Modern web technologies brings a lot to the developer experience in terms of building content-driven sites, but how can we improve things for editors and content creators? In this workshop you’ll learn how use Sanity.io to approach structured content modeling, and how to build, iterate, and configure your own CMS to unify data models with efficient and delightful editor experiences. It’s intended for web developers who want to deliver better content experiences for their content teams and clients.