sunk thought’s - the playbook

“the playbook” is a mini-book delivered in 8 posts (the forward and seven chapters.) It’s the dinosaur sponge bath capsule of the organizational theory world!

A Scaling Rosetta Stone

Organizational Growth Patterns :: Application Scaling Patterns

Culture Enables Innovation (Conway’s Law)

Conway’s law states that “organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.”

Your operating structures define your architectures. If your structure cannot effectively scale, your software systems will encounter those same pains.

This is why organizations which try to implement architectures not suited to their operating structure almost always end up with some sort of Frankenstein’s Monster. By not embracing the underlying philosophy, you fail to reap the full benefits.

The Evolution of Yammer

Kris Gale was VP of Engineering at Yammer where he was a big inspiration to me and my engineering colleagues. After Yammer’s sale to Microsoft, he spoke a lot about being intentional in your operational design choices to maintain the benefits of being small (even as you grow.) “Organizational design is at the core of how you optimize your engineering resources’ potential throughput,” he would emphasise.

When Kris joined Yammer they had three engineers, Kris included. They had overlapping skills and individual specialties. Work was greasy-fast and progress easy. They could function in areas outside their specialty, thus everyone understood the entire system more wholly.

Yammer’s Tiered Monolith

“By 2010, because we weren’t rigid in our operational design, these specialties became our team structures, those teams owning their own codebases.”

This is when I myself joined Yammer. We grew by around 100 engineers within my first year. With this came more process overhard, less cross-functionality, teams became more siloed, and while we were fairly fast still, we were not as fast as we felt we needed be.

We had found ourselves in a Tiered Organization, where “the marginal increase in throughput gained by adding more engineers decreases over time because of the organization's operational overhead.”

At the same time, Yammer’s codebase was growing into a tiered monolith of its own. Scaling required booting up complete duplicates of the codebase, and in order to keep the database up and running our infrastructure team was juggling all sorts of sharding and replication nightmares to keep up with usage.

Yammer’s team went to work figuring out what else could be done to escape the scaling problems both the organization and application were experiencing.

Once again, Conway’s Law was to rear its head and show us how culture and application architecture follow one another.

Microservices

By 2012 Yammer was looking something like this operationally. They worked in autonomous Business Initiative teams, which pushed more context and decision making authority to the initiative team itself.

These teams were:

Cross functional - with all the needed elements to execute and redefine the initiative as needed.
Small in Number and Scope - with 2-10 members grouped together for a 2-10 week sprint.
Autonomous - with no external blockers or additional decision makers to report to outside of their initiative. Teams were fully empowered to achieve their goals by any means they agreed upon.

Even though Kubernetes and the massive microservice ecosystem we enjoy today didn’t yet exist, Yammer was also pretty early in terms of experimenting with the idea of running small independently scalable services in production.

By creating autonomous microservices (each providing a vital business utility) that aren’t dependent on other outside services to run, we can independently scale as that service’s load dictates.

We can also more easily iterate, test, and deploy changes to these services on the fly, without impacting our other microservices.

Event Streams

In the Event Streams application model, rather than have a central source of data/truth, what we get is a central log of events that your services can consume as needed, creating their own data stores, functionality, processes, and APIs which allow other services (inside and outside your organization) to interact with them.

We are also starting to see large organizations tinker with more flat and decentralized operating structures which act like an internal economy. What were “business initiatives” become more autonomous Internal Businesses. Your executive leadership acts more like Internal Venture Capital with services, vision, and philosophy it uses to fund projects and serve teams.

The Evolution

The evolutionary trend here is pretty striking. In each stage another layer of hierarchy is replaced with an increased number of “nodes” at the bottom. Information becomes more directly communicated. More importantly the autonomy and cross-functionality of each “node” increases in every stage.

The system is becoming less hierarchical, faster, and increasingly more decentralized with each progression.

Here’s a final analogy from Kris that might help you see why this is so effective.

Amdahl's Law, but for Organizations

In a parallel computing context, Amdahl’s law is used to predict the maximum improvement for a program as you add more parallel processing power to the execution.

Kris Gale described this thus, “It shows that the improvement you’ll get from performing a parallel computation is constrained by the serial portion of the computation.”

Under an organizational lens, your “up the chain” decision making overhead is like the serial portion of a parallel computation, capping any added resources’ overall effectiveness. The longer this serial decision making process is, the less effectively you can increase velocity by adding more contributors.

This serial overhead restricts your engineering throughput, as contributors find themselves “blocked on the serial part of the process.” he explained.

We achieve better results as an organization by making our “nodes” more autonomous, and by delegating more authority to decide how to get something accomplished.

The point?

“You shouldn’t be building a product, you should be building an organization to build your product.” - Kris Gale

Operational Requirements

In order to run a more and more decentralized operating structure, your organization must foster increasingly more effective:

Culture through which your vision, decision making process, and business priorities are evangelized
Trust that your team is capable and has aligned around your initiatives’ goals and/or vision
Scientific Product Teams which use data and customer insights to hypothesize and test ideas
Empowered Engineers who are able to make informed business decisions by aligning tightly on goals/needs vs solutions/implementations.

Sound familiar? By now it should...

Now, this is when you’re likely to say, “This all sounds well and good, but it’s not that easy. My business is already big, we already have lots of people, and processes, and customers, and work to do. We can’t just start all over.”

You’re absolutely correct. But, there is one more piece of my little introductory text to go.

In this final chapter we will reuse some of the key themes we’ve learned in parts 1-6, and talk about learning to put them into practice via an iterative, results-based, and deliberate methodology over time.

Basically, it’s time to “Product Manage Your Company’s Operating Structure!”

the playbook part 6: a scaling rosetta stone