Code on Demand: The Future of Code Collaboration

Rate this content
Bookmark

What if you could use code the same way you use Netflix or Spotify? 


Imagine your entire code base organized as small components on the cloud, where you can import only what you need into any workspace. In fact, you can create an ad hoc workspace where you can use and modify only the components you need and export them back.


In order to use code on demand, components need to be independent entities, which means that they have to include their source code, configuration and dependencies. This allows you to version them independently and import them into small dynamic workspaces which are then made much easier to navigate, setup and build. Independent component versioning also makes for independent teams, because this separates publishing updates from consuming them. 


When all your organization’s components are in the cloud, teams can discover them, discuss changes and visualize relationships, making collaboration between teams much simpler. 


Alongside its many advantages, this paradigm shift brings about considerable challenges, which may explain why this is not the standard way to manage code today. This talk is about the tools and methodologies required to overcome these challenges. 

27 min
23 Oct, 2023

Video Summary and Transcription

During the Talk, the speaker discusses the power of streaming and its impact on music consumption and production. They also explore the challenges and benefits of incorporating streaming into software development, using examples from Spotify. The future of code development is envisioned as a streaming world, where code is always live and changes are instantly available to everyone. The speaker emphasizes the importance of treating components as complete products, prioritizing component reviews, and enhancing the workflow for forking and contributing to components.

Available in Español

1. Introduction to Streaming and My Background

Short description:

During the next 20 minutes, I will demonstrate how streaming can be incorporated into our development workflow. Streaming services have replaced buying CDs and DVDs. Our code is still stuck in the 90s. Let me introduce myself as the engineering leader at Bit with 6 years of experience. Bit is an open source tool chain for crafting composable software. I have stories about code collaboration and dev methodologies. Follow me on ix for more!

So, let's take a look. Greetings to everyone and thanks for joining me today. During the next 20 minutes, I will demonstrate how streaming can be incorporated into our development workflow. Stay tuned, for a fascinating journey.

It wasn't so long ago when we all bought CDs for listening to music, or DVDs for watching movies. However, no one does this anymore. Streaming services, like Spotify and Netflix, have replaced it all. But what about our code? Here is a little secret. Our code is still jamming in the 90s on those CDs. Don't you think it's time for an upgrade?

Before we open onto this new bit, let me tell you a bit about myself and why I'm chatting about this today. So a bit about myself. I am the engineering leader over at Bit. I've been working at Bit for 6 years, especially on the open source project. To put it simply, Bit is an open source tool chain made for crafting composable software. It's kinda like building with Lego. And trust me, after all these years, I've got a whole lot of stories about code collaboration and dev methodologies. Aside from that, I chat on blogs, speak on stages, about javascript, typescript, frontend, and turning houses into smart homes. If any of that sound up your ally, give me a follow on ix. But at the end of the day, I'm mostly coding, and doing what every dedicated developer loves to do. Transforming coffee into code.

2. The Power of Streaming and Components

Short description:

Why is everyone so hyped about streaming? It changed the game of music consumption, allowing users to create their own playlists and access detailed information. Streaming enables social interaction, sharing, and easy access to additional content. It also democratizes music production, making it accessible to anyone. Although we have made progress in adopting modular development, there is still more to explore. Let's dive deeper into the concept of components and hear from developers Claudia and McDonald about their experiences with streaming.

Okay, enough talking about me. Back to our main story, streaming. Why is everyone so hyped about it? Let's break down what makes streaming tick. In a bit, we will see how we can sprinkle some of that magic into our dev game.

Remember buying a CD, and paying for the whole thing just for that one track? Streaming changed the game. Now you compose your own playlist, picking only the tracks you love. Back in the CD era, hunting for music was like a mini quest. Now with streaming, it's a breeze. Type in a mood, artist, or beat, get all the deets, like who is singing, how long is it playing, and other cool info right in front of you.

In the CD days, music was a solo trip. Streaming flips that. Now you can vibe to tunes with friends, drop comments, and share your feelings on particular tracks, all in one spot. Remember DVD's. Once you got them, they were set in stone. The erector's cut or some cool deleted scene came out. With streaming, it's right there, waiting for you. No fuss.

One cool thing about streaming, who gets to make the beats. Back in the day, dreaming of dropping a track meant cozying up with record labels and burning cash on studio time, design and distribution. Now, grab the mic, hit record, and boom, you're on Spotify. I bet some of you are thinking, wait up, aren't we already riding this wave? Yes, we are all crafty, nifty little components living that modular dream. That's a solid start for sure, but trust me, there is more juice to squeeze out of this approach. Before we dive deeper, let's clarify what we mean by component. It's essentially any piece of usable code, including UI components, utility functions, or microservices. But it's not restricted to UI components only. Let's dive deeper.

All right, let me introduce you to two developers who will paint the picture for us. Claudia is living the futuristic cloud-streaming life, and McDonald is jamming on in today's world, still spinning CDs. We will hop between their experiences, spotlighting the contrast and bringing those key streaming traits into focus. Let's roll.

3. Building the Player Menu Component

Short description:

Both front-end engineers at Spotify tasked with building a new player menu for the mobile app. McDonald's workflow involves navigating through folders, finding the right place for the component, and dealing with dependencies. He consults his teammate and discovers Spotify's design system and storybook for reference. McDonald faces challenges with button variations and suggests improving the button API.

Both were hired as front-end engineer at Spotify last week. Lucky them. As they code away for Spotify app, they are tuning into a hand-picked playlist I curated especially for this talk. Take a moment, maybe even pause the video, and really check out those track names. And the pro tip, band number six, they are the real deal. Make sure to give them a listen. Both got the same task, to build a new player menu for the Spotify mobile app.

Alright, let's kick things off with McDonalds. Its workflow, probably pretty familiar to most of you here. I will breeze through it, but pay attention to the nuances. First up, McDonalds hits git clone, or the mobile app repo, and now the waiting game begins. Right from the get go, he is pulling in a bunch of code he might not even use for his task. Next, McDonald feels like he is wandering a massive store, trying to find the perfect spot for his new component. But man, there is folder after folder. What's all this code about? Feels like amazing here. After a bit of a trek, McDonald spots the right place, and set up new folder for the Player component.

McDonald gets going, typing his first line of code. Import, button, progress bar, from… from where though? There has got to be a button somewhere he can use! But where is it hiding? With a bit of a sigh, he heads over to his senior teammate. Hey, where can I pull this button from? His teammate points him to a confluent docks. Check that out, they say, it's all about Spotify's design system. With that hurdle cleared, McDonald finishes that initial line of code. But here is the twist, he is pulling in extras, thanks to the old package! Tree-shaking has its own limit, right? Now on to the next challenge, how does he work with that button? Good news, there is a link to Spotify's storybook. But oh snap, it's not up to date. Turns out, a different team maintains it, and they have yet to roll out the latest version. Close enough, though. McDonald's next puzzle, a sea of button variations. Which one is the right fit? Pushing forward, McDonald checks out the button APIs and prompts. There is an enum for border style, options like square or round. But wait, it needs a circle button, that's not on the list. Now, where does it go from here? Let's improve the button API.

4. McDonald's Player Code and Deployment

Short description:

To update the player code, McDonald clones the design system repo, makes the necessary modifications, and submits a PR. After approval, a new version of the design system is published. McDonald copies the button code and modifies it for his player. He rolls out the new code and awaits review by the product and design teams. To deploy the changes, a new version of the app needs to be deployed, which is handled by the dev ops team.

Easy, right? All we need is clone the design system repo, read contribution docs, update the code, submit the PR, now wait for approve. Okay, it's approved and merge, okay, wait again. We need a new version. Okay, a new version of the design system has been published. But wait, but what will happen to my progress bar now? Will it break? Again, IPEI Extra.

In reality, McDonald decides, let's just copy that button code and modify it here. I mean, why let someone else's decision slow me down? What could be possibly go wrong? All right, McDonald has rolled out his new player code. Let's fast forward to the next part of his journey. Hold on, before racing ahead. You need to build this project. That means diving into more docs and then the waiting game. Building takes its sweet time, kind of like burning an entire DVD. Good thing McDonald thinks, hey, I'm clocking in hours for this. All set and looking sharp. Time for a review by the product and design teams. Now where should this update version go? To roll out the changes, we've got to deploy an entire new version of the app. It's like burning a fresh CD and getting it to the store. But hey, that's for someone else to handle, right? McDonald thinks, I will just chill while the dev ops team tackles that.

5. Cloudia's Experience and Component Integration

Short description:

Cloudia crafts a new empty folder named Player Workspace for her player project. She rolls out two new components, Player and Player Control, generated from Spotify's React Component template. The streaming platform offers powerful contextual search, comprehensive documentation, and a live demo of the component. Claudia seamlessly installs just the button she needs, forking the component to modify its API and ensuring consistent quality standards.

Now let's switch gears and peek into Cloudia world. How does their experience look like? Cloudia kick things off by simply crafting a new empty folder named Player Workspace for her player project. Without being tied to a specific app context, Cloudia's approach shifts. She's crafting for usability, ensuring clear boundaries, comprehensive docs, and user-friendly API.

She rolls out two new components, Player and Player Control. Leveraging a scaffolding tool, she generates them from Spotify's specialized React Component template. This template already bundles the essential code, comprehensive documentation, examples like stories, and all configurations, TypeScript, Linterpreter, Webpack. Best of all, these configurations are neatly tucked away. No distraction or mess, Cloudia can focus purely on what she loves, writing code.

Next, on her agenda, finding the right button, Cloudia dives into the streaming platform. It's tailored for developers like her, offering powerful contextual search with filters, such as tags like input or TypeScript, technology, in her case React, and more. What's more, she's greeted with a rich discovery experience that showcases the current version of the component, have full labels, the components bundle size, comprehensive docs, a live demo of the component, preset examples, a detailed API reference, everything she needs right on her fingerprints, fingertips.

But wait, there is more, a clear view of dependencies and which components depend on this one, details on test cases and their coverage. A list of similar components offering alternatives. Claudia can quickly assess if this component fits the bill or maybe explore the similar ones. Best part, everything is fresh and up-to-date since it's an integral part of the component. No outdated docs here. Claudia seamlessly installed just the button, mirroring the Spotify experience, downloading precisely what she requires. No bulky unnecessary part of a huge design system. Not only does this streamline her bundle size, but future updates for this particular component will be more straightforward and secure.

Claudia integrates the specific button component into her code. However, she soon notices that the button lacks the circuit style she needs. In this modern approach, Claudia has the ability to fork a component. By doing so, she pulls the source code of the button directly into her workspace. This process seamlessly includes its dependencies and configuration such as TypeScript, Jest and the linter. Even better, everything gets automatically connected with the local fork, enabling her to integrate and use the button in a player code just like before without any additional adjustments.

Claudia modified the button API to accommodate the specific style she needs. Following this change, she executes a dedicated command. This command not only compiles the button, but also lints and tests it, all while leveraging the button's native tools and configurations. This ensures the updated button remain consistent with its original quality standards.

6. Wrapping Up Development and Code Review

Short description:

Claudia wraps up development of her new player control and player components. She imports the app into her workspace, altering the original component. Importing the app brings in the necessary code and configurations. Claudia effortlessly incorporates her new player, streamlining the integration process. She gathers feedback through a change request, involving various roles and teams. Transparency is key, with a dedicated feed for updates. Claudia can gauge her impact and track engagement, driving motivation. Now, let's compare the CD world and the streaming world.

With everything in place, Claudia can now wrap up the development of her sleek new player control and player components. Claudia proceeds to import the app into her workspace as she requires updates to accommodate the new player. This action, much like forking, integrates the original component, but with a significant distinction. It alters the original component instead of creating a separate fork. Importing the app also brings in the source code, configuration, and dependencies needed for seamless integration.

With everything set up, Claudia can effortlessly incorporate her new player as if it were already installed from the NPM registry. All the necessary connections and configurations are handled automatically, streamlining the integration process. The BIT process now races ahead at lightning speed thanks to Claudia's workspace contains only four components. Claudia launches the app locally without needing to tinker with configurations. All the necessary development server settings are seamlessly integrated into the app component itself. This allows her to easily validate that everything functions as expected. To gather feedback, Claudia submit a change request on the cloud. This request encompasses two new components, player and player control, one fault component, the button, and one modified component, the app. With this, she sets the stage for reviews and potential improvements.

This isn't just your regular code review. It's a holistic 360 degree evaluation. It encompasses the entire business initiative and involves various roles, including designers, product managers, developers, technical writers, and it doesn't stop there. It also bring together different teams and different team members, such as Claudia's team, the design system team, the app team. Everyone's input and perspective contribute to the comprehensive review process. Transparency is key in this business initiative. There is a dedicated feed where everyone can stay in the loop regarding the entire process. This includes update on code changes, new examples, review comments, and much more. And the best part, all of this unfolds before any code gets merged. Now Claudia can truly gauge her impact, much like artists who have shared their songs on podcasts on Spotify. Tracking the audience is crucial. It's a source of empowerment, seeing people engage with the content, and it's a driving force behind motivation to keep creating. This holds true, even if there is a compensation involved that is, and depend on the number of listeners. Similarly, in the world of code, there is a desire to socialize and share the impact of our work. Now, let's swiftly compare the CD world and the streaming world based on the key factors we highlighted at the start. In the CD world, there is always an extra cost.

7. Challenges in the CD and Streaming Worlds

Short description:

In the CD world, you pay for unnecessary code and face the risk of updates breaking other components. This leads to distractions and reliance on luck. In contrast, streaming offers a streamlined experience with contextual search, live examples, and accessible documentation. However, finding code in a larger organization can still be challenging.

You end up paying for songs, aka code, that you don't actually need. Since it's all bundled together on one massive CD, you are left with a hefty bundle size, plus there is the ever-present risk of updates when you want to update one component. Who knows which of the other 100 components might break in the process. This approach is a clear anti-pattern for collaboration incentives.

In the CD world, you are constantly bringing in loads of code that you don't immediately need. This continuous arrival creates a significant distraction during the development process. It's a bit like searching in a physical store, looking for that how to find CD. Without proper search and discovery mechanisms, you are left relying on luck.

In stock contrast, the streaming approach offers a streamlined experience with contextual search by labels, tech stack, dependencies, and size, access to live examples, readable and searchable docs, API reference. While you might have a partial discovery experience for smaller sections of your organization's code, like the design system, most of your organization's code isn't the design system. So, sometimes the code you need is hidden away somewhere, making it challenging to locate in the city world.

8. The Future of Code Development and Streaming

Short description:

In the future world, your code is always live, allowing you to track your impact, assess code quality, and receive constant feedback. In the streaming world, changes are instantly available to everyone, streamlining the code-to-production process. Unlike the city world, anyone can participate, craft reusable components, and broaden the reach of their work. Achieving this transformation requires substantial undertaking and dedication to developing processes and technologies.

In the city world, it's a one-time deal. You bury your code onto the city and that's the end of it. But in the future world, your code is always live and it opens up a world of social possibilities. You can track your impact as an individual or as a team within the organization. Discover how many people who you never imagine are using your code. Assess the quality of your code and your personal growth over time. Evaluate how much better you are today than you were yesterday. Plus, you continually receive feedback on your code, leaving room for constant improvements. It's a dynamic and social development environment.

In the CD world, you are dealing with hard copies and hard copies can be quite challenging to modify. It involves a time-consuming process of creating a new version of the CD, gathering all the changes, burning a fresh CD, and then shipping it to stores. This results in long waits for updates to become available to you, your team, or your users. You are essentially at the mercy of unrelated changes getting merged, enduring lengthy build times and navigating the complex deployment process. In the streaming world, it's a whole different story. You can effortlessly replace files on the server, making changes instantly available to everyone. In our case, we modify, review, and deploy individual components, which streamlines the process from code to production. It's a significantly faster journey.

In the city world, only a select few can become artists and reach a broad audience. Similarly, in our case, only a limited number of developers can create and distribute reusable code. It involves numerous tools and processes, and only they can make a significant impact within the organization. But in the streaming world, it's an entirely different story. Anyone can participate. Anyone can craft a reusable component and effortlessly share it within the organization or with the wider world. You will be amazed to discover just how vast the world is. And there are people, both within and outside your organization, who have been searching for your specific piece of code. It's about democratizing development and broadening the reach of your work. This vision might sound like an utopia. But it's not just a fairy tale. So how do we turn this dream into a reality? Achieving this transformation is indeed a substantial undertaking, touching on various aspects, from philosophical and methodological shifts to organizational changes and advancement in technology and products. I've been dedicating to developing these processes and technologies full time for the past six years.

9. Key Factors for Component Development

Short description:

Shift your perspective from apps to components. Treat components as complete products and develop them with reusability in mind. Prioritize component reviews, version and ship them individually, and comprehensively document them. Simplify the process of creating and sharing components to motivate everyone in your organization. Enhance your discovery tools for seamless searching and effective categorization. Infuse a social dimension into your discovery tools and establish a direct connection between code and discovery. Enhance your workflow for forking and contributing back to other components.

Now let's delve into some key factors to consider on the journey toward this future. The first step is to shift our perspective regarding the relationship between apps and components. Instead of centering our development process around apps, which include organizational structure, repositories, CICD pipeline, and testing specific to apps, we should place component at the core. Here is how.

Treat components as complete products deserving respect. Develop components with reusability in mind, assuming they will be used more than once. Organize your structure around components. Prioritize components reviews. Version and ship components individually. Comprehensively document components. Focus your testing efforts on components and establish CI pipeline specifically for components. This approach shifts the focus to releasing individual singles to your development environment, akin to how artists release songs on Spotify instead of assembling a full CD albums.

In the end, it's the presence of well-crafted reusable components that truly builds your R&D compounding value. These components serve as the foundation for developing tomorrow's features without the need to reinvent the wheel every single time. Plus, they allow you to leave behind the burden of a giant heavy legacy monolith that holds you back.

Simplifying the process of creating and sharing individual component is key. It should be so straightforward that anyone in your organization is not only able, but also motivated to do so. Consider this critical aspect, providing easy-to-use templates for component creation, streamlining the setup of necessary development tools, simplifying the management of component dependencies, and ensuring appropriate permission for component sharing and collaboration. Allocate time and effort to enhance your discovery tools.

Consider the following aspect, make searching for components seamless and effective. Implement a tagging system to categorize components. Provide clear and useful example to showcase component functionality. Establish a ranking system to highlight popular and high quality components. Furthermore, infuse a social dimension into your discovery tools. Think about ways to measure and present impact, usage and other social metrics. Also ensure a direct connection between code and discovery tools. The developer responsible for creating a component should be accountable for its discovery experience. It's not someone else's problem. Remember, even a great song goes unheard if no one can find it. Enhance your workflow for forking and contributing back to other components.

10. Wrapping Up and Looking Ahead

Short description:

Make forking and modifying components straightforward. Encourage contribution and collaboration. Transform gradually, one step at a time. Build features with components and usability in mind. Share your thoughts and preferences. Provide feedback through the QR code or social media. Thank you for joining me on this journey. Goodbye.

This process should be straightforward. Make it easy to fork an existing component. Make changes and build it. Facilitate the process of suggesting new changes. Ensure there is a convenient way to use the fork component when waiting for the original component to be updated with the new suggestions.

Ensure encouraging, contribution and collaboration should be at the heart of your development culture. Remember that this transformation should happen gradually, one step at a time. You cannot completely change the entire organization in a single day. When you start working on your next feature, try to build it with components and usability in mind. Every small step contributes to a more streamlined and efficient development process.

As a wrap up, one final question remains. Which world do you like to live in? Feel free to share your thoughts and preferences with me in the comments or through the feedback form. Thank you for joining me on this journey. I hope you found it enjoyable and informative. I've included the QR code for a brief anonymous feedback form. You can also find my social media information in case you would like to connect. Your feedback is highly appreciated. Thank again, and I look forward to having you with me next time. Goodbye.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

React Summit US 2023React Summit US 2023
21 min
The Epic Stack
Modern web development is fantastic. There are so many great tools available! Modern web development is exhausting. There are so many great tools available! Each of these sentiments is true. What's great is that most of the time, it's hard to make a choice that is wrong. Seriously. The trade-offs of most of the frameworks and tools you could use to build your application fit within the constraints of the vast majority of apps. Despite this, engineers consistently struggle with analysis paralysis.Let's talk about this, and a solution I am working on for it.
React Summit US 2023React Summit US 2023
30 min
The Rise of the AI Engineer
We are observing a once in a generation “shift right” of applied AI, fueled by the emergent capabilities and open source/API availability of Foundation Models. A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023. Emergent capabilities are creating an emerging title: to wield them, we'll have to go beyond the Prompt Engineer and write *software*. Let's explore the wide array of new opportunities in the age of Software 3.0!
GraphQL Galaxy 2022GraphQL Galaxy 2022
31 min
Your GraphQL Groove
Building with GraphQL for the first time can be anywhere between daunting and easy-peasy. Understanding which features to look for in your client-side and server-side tooling and getting into the right habits (and ridding yourself of old habits) is the key to succeed with a team of any size in GraphQL.

This talk gives an overview of common struggles I've seen numerous teams have when building with GraphQL, how they got around common sources of frustration, and the mindset they eventually adopted, and lessons learned, so you can confidently stick with and adopt GraphQL!
DevOps.js Conf 2024DevOps.js Conf 2024
25 min
Atomic Deployment for JS Hipsters
Deploying an app is all but an easy process. You will encounter a lot of glitches and pain points to solve to have it working properly. The worst is: that now that you can deploy your app in production, how can't you also deploy all branches in the project to get access to live previews? And be able to do a fast-revert on-demand?Fortunately, the classic DevOps toolkit has all you need to achieve it without compromising your mental health. By expertly mixing Git, Unix tools, and API calls, and orchestrating all of them with JavaScript, you'll master the secret of safe atomic deployments.No more need to rely on commercial services: become the perfect tool master and netlifize your app right at home!

Workshops on related topic

React Advanced Conference 2021React Advanced Conference 2021
168 min
How to create editor experiences your team will love
Workshop
Content is a crucial part of what you build on the web. Modern web technologies brings a lot to the developer experience in terms of building content-driven sites, but how can we improve things for editors and content creators? In this workshop you’ll learn how use Sanity.io to approach structured content modeling, and how to build, iterate, and configure your own CMS to unify data models with efficient and delightful editor experiences. It’s intended for web developers who want to deliver better content experiences for their content teams and clients.