Why Scaling Bottom-up? How Teams’ Interactions Should Impact the Organizational Structure

Hi all, I'm happy to be here talking about Teams interactions and the important role they play on proper scaling and reorging processes. In the next 20 minutes, I'll talk a bit about Teams as means of delivery, Teams interactions and the importance of DDD for fast flow. Let's start. So, the first question is, why? Why is the organizational layout and Teams interactions important? Well, Mr. Mel Conway observed a behavioral pattern that would end up leveling up to a law. Conway's law is the observation that the design of any system is significantly affected by the communication structure of the organization that developed it. It's kind of a long sentence here. And is a consequence of the fact that two software modules A and B cannot interface correctly with each other unless the designer and implementer of A communicates with the designer and implementer of B. Thus, the interface structure of a software system necessarily will show a congruence with the social structure of the organization that produced it. This has significant impact on how software is built, especially if microservices and or domain-driven design are adopted. And this is directly related to the organizational shards that companies usually have. The problem with org shards is they provide a team single view with the relation between teams and lines of reporting, but usually the communication patterns are not up and down the reporting line. They are parallel and throughout the shard, which makes communication efficiency the key to effective teams. We'll go to whomever we need to solve our problems. And this creativity and problem solving method needs to be turned in the benefit of the organization, not restricted to up and down lines of reporting. So how can we solve the org shard problems? System thinking is a holistic approach process that is always repeating itself, always trying to optimize for the whole, considering the overall flow of work, identifying bottlenecks and removing them. And now back to our initial point of communication. Common organizational bottlenecks are busy team structures, poor communication, and slow delivery pace. And I think that we all have been through, if not all, some of them. Problems to which James Lewis identified the solution that he called reverse Conway maneuver that we will talk right now. So the overall goal is for the architecture to support the ability of teams to get their work done from design to the deployment without requiring high bandwidth communication between teams. And I guess that this is the nirvana of the organization, right? This means each organization needs to understand which software architecture is needed before and big capital letters here before organizing the teams. Otherwise the communication facts and incentives in the organization will end up dictating the software architecture, which is something that we don't want. And here we are going back to the communication topic. The more we fit teams for the desired architecture and not the other way around, the more contained and improved communication will be improving the efficiency of team designs by taking out the communication chaos overhead. Rearging or scaling a company based on this assumption, however obvious, is challenging to the top down as it's intrinsically related to the team's needs of collaboration and interactions, their ability to be flexible and adapt to change, and teams domains and cognitive loads boundaries. And in case you are asking, cognitive load is the limit of how much information a person or in this case a team by joining all the members can hold in their brains at a given moment. So what is the goal here with this maneuver? How can we improve the team structure and interactions? To start, we need to think in teams as living organisms within the ecosystem with individualities that respond to increasing motivators. These motivators are usually a three element reference and they are autonomy, mastery, and purpose, which when enabled within strong bounded responsibilities allows quality and increased speed of delivery. Why? I do this question a lot. Well when teams are loosely coupled and highly cohesive, it means that they are autonomous in the context they work. Context which is restricted in responsibilities and adapted to their cognitive load, limiting the switching efforts, thus increasing focus and quality. You may have heard these terms before and they apply the same. So the loose coupling is when components do not hold strong dependencies on another company and high cohesion is when components have clearly bounded responsibilities and their internal elements are strongly related. So these concepts apply on all scenarios the same. Also the clear set of domains of responsibility pushes back the jack of all trades, master of none mindset, avoid forcing teams to juggle requests and priorities from multiple sources. In summary, proper team landscapes that put the teams first positively influence responsibility boundaries, speed of delivery, quality of work, and autonomy, which consequently directly impacts the software the teams produce. Holt on top of Conway's Law team topology book mentions two core concepts that help to use the team landscapes to rethink the company organizational shards and the concepts are fundamental topologies and interaction modes. In summary, what are the best topologies teams can work in? The authors suggest four fundamental topologies that go from business to platform oriented teams. We have the stream aligned team, which is aligned to a flow of work from usually a segment of business domain. We have the enabling teams that helps streamline teams to overcome obstacles and they also detect missing capabilities. We also have complicated subsystem teams where significant mathematics calculation or technical expertise is needed. And last but not least, we have the platform team, which is a grouping of other team types that provide a compelling internal product to accelerate delivery by stream aligned teams. Has communication and interactions between the teams are the most critical aspect of organizational and strategic thinking. To allow these teams topologies to interact properly, some interaction modes are suggested as presented in the next slide. And again, going back to the communication problems where we started, how can effective teams collaborate and communicate? Dependencies will exist, the loser the better and all of that, but they exist and we will not be able to make them go away. How can we simplify our lives? How can we communicate to build upon a good team structure? In the book, again, the three essential team interaction modes goes as follows. Communication, meaning working closely together with another team, which is the driver of innovation and rapid discovery, but tends to blur the boundaries. It's a really good communication mode for rapid discovery of new things that avoids costly handing offs between teams. We also have the mode of X as a service, meaning consuming or providing something with minimal collaboration, which keeps clear responsibilities with predictable delivery, but needs good product management. It's a great choice when there is a need for one or more teams to use a shared component that can be provided as a service. And last but not least, facilitation, meaning helping or being helped by another team to clear impediments, which senses and reduces gaps in capabilities. It's good when one or more teams benefit from the active help of another team facilitating or even coaching some aspect of their work. Bottom line is all these concepts, both fundamental topologies and interaction modes, end up being a rule of thumb when it comes to think about loosely coupled and high-cohesive teams working together in a proper flow of change. At this point, we have been through the importance of having a clear vision for the design software architecture to then fit the teams as organisms that are as effective as the boundaries they are given and communication strategies they apply. We have also discussed interaction modes that improve communication, but what about the cognitive load? What about the team's boundaries and responsibilities? Well, Eric Evans, which is a widely known software architect and author of the Domain-Driven Design book, aligns with the observation that flow is difficult to achieve when each team depends on a complicated web of interactions with many other teams. When the responsibility boundaries assigned to the teams is not viable or well-defined, it results in lack of ownership, disengagement, and slow delivery rate. Is there a solution? Yes. Exploring and validating boundaries of responsibility, taking into consideration the team first, is what we will discuss next. Research by Accelerate found that the tightly coupled architectures negatively influence the capacity of having autonomous teams with clear responsibilities. Many problems in delivering software come from unclear boundaries between teams and their responsibilities, which is often shadowed by a software architecture with high coupling between its different parts, something that monolithic architectures indulge. The ultimate goal for every architecture is to support the ability of teams to get their work done with high throughput from design until development without requiring high bandwidth communication between teams, which by nature is harder to achieve in a monolithic architecture. But in the process of moving away from monolithic software, how do we consider the team needs? How do we account for their angle? Well, we think about domains by applying a well-known strategy that enables teams full ownership. We explore and discuss domain-driven design, and then assess how the teams will fit on those. This means, again, it feels like I'm repeating myself, but this is really important, that the organization needs to understand what software architecture is required before organizing the teams. Otherwise, as already mentioned, the communication paths and incentives in the organization will end up dictating the software architecture. But first, taking a step back, what is a domain? We know that good boundaries minimize cognitive load, right? But what are these boundaries? Boundaries are conceptual lines that wrap around domains, and domains are larger applicable concepts than software size, helping us to think across the board and use common heuristics. The fact they provide a vision on the problem being solved makes it also easier to assign complexity to a domain, which in turn makes it easier to assign to the teams according to their available cognitive load. This enables domain-driven design, providing some software development techniques that address both strategic planning and strategic and technical design, keeping the alignment with the business, fostering ubiquitous language, and a proper complexity evaluation, which in the end provides clear benefits, the team's ownerships, engagement, and fast flow of delivery. Having said all of the theory, let's move to a real case scenario. And I will be talking about my personal example on OLX. So every company needs an organizational chart, especially big corporations acting in several different types of markets all over the world. The online classified business is no different, and in OLX group, there is a specific customer unit to represent the car selling and buying market, which is Motors customer unit. This unit represents individual buyers and sellers, aiming to match them to conduct business. As you can imagine, what started small is now a huge business where people are empowered to make faster and better decisions and deliver better features quicker with low rates of problems for the users. With more teams, markets, and competition, Motors software architecture reached the point where the monolith that served and brought the unit until this point no longer works. At some point, the monolith hindered our efficiency from being even faster and better. And for example, high coupling and diffused boundaries constantly put our teams in each other's ways, forcing a strong alignment commitment at all times. The progress of business features was also affected, as every change triggers impact in other teams, slowing down the delivery, and even the quality as the impact of the monolith then led to small steps of delivery or even a compromise between the parts. By all this, the pace of delivery was also being affected by the monolith, as every deployment had a high probability of stopping the pipeline due to the high number of changes impacting everyone in need to deliver. Recognizing all this, and in an effort to enhance independence and excellence, the group of teams working to enable professional sellers' business growth through vehicle acquisition, management, and sales decided to put in motion a plan to move away from the monolith by putting together a group of experts in domain-driven design to help the teams get a better insight of their responsibility boundaries and identify the needs to move to a decoupled architecture allowing autonomy, independent work, and defined responsibilities. But having that decision, where to start? Strategic planning helps to understand what are the most important software investments to make, what existing software assets to leverage in order to get there faster and safer, and who must be involved. This strategic planning then involves reviewing current domains using event-storming sessions to define bounded contexts and context maps, using entities, aggregates, and domain events to redefine the current architecture and aligning each domain's responsibilities to the present cognitive load of the team's resources, what we call DDD, domain-driven design. Based on this DDD work, we are achieving solid grounds to review our current architecture aligned with DDD building blocks and principles of modularity, high cohesion, low coupling, and abstraction information hiding, and in the end, reorg our tribe charts according to teams' domains and interaction modes so that we achieve low overhead of communication and faster decision making, faster and quality delivery, and allow our teams to have autonomy, sound mental health capacities, and expertise in the domains they are responsible for. In the end, we are aiming to scale and reorg our chart bottom-up, taking into account our business domain and vendor aggregate relations to set up our system architecture, which in turn shapes the team's landscape, which in turn builds an organizational structure that validates the principles of Conway's Law, to positively influence speed of delivery, quality of work, and autonomy, to then directly impact the software we are producing. And that's it. Thank you for listening. We'll see you next time. This is a production of Conway, Inc.