Spotify engineering culture. How do they deal with growth issues?

Spotify grew tired of it and is trying to solve problems related to software development process. Luckily for us they are willing to share their conclusions.

Why do we need to introduce changes to our work routines?

How many times did you struggle with half-baked requirements which were thrown at you despite little to no consultation? How many times you couldn’t release your code or were stuck with some irritating tools because of dependencies to other teams? This is just a quantum of problems bothering average programmer every day. But can something be done about it? Well, we will not know unless we try, right? Spotify grew tired of it and is trying to solve these problems. Luckily for us they are willing to share their conclusions.

Team and work organization

We all know Scrum. It is so popular that some people tend to accept it as the only right way to do things. But is it always optimal? What if something does not really fit us, but changing that would be against Scrum rules? I guess Spotify guys answer to that question would be “Rules are good start, but break them when needed”. The meaning behind this sentence is that sticking to Agile principles worked better for them than keeping firm Scrum practices. They even came up with a new terminology which better express their ideas.

So Scrum Master was renamed to Agile Coach as it does not imply that he is someone who ensures that Scrum framework is followed. The new role could be described more like a servant leader rather than a process master, so team members are supported but not restrained and limited.

Now let’s look at Scrum team which in Spotify universe is called Autonomous Squad. So why autonomy was so important that it became a part of the name? Unlike many other companies which restrict their scrum teams options with fixed module dependencies, requirements and tools. Spotify sets for its squads only general goals and let them figure out how it should be done. Given that freedom development becomes faster as most decisions can be done locally and lower limitations are additional motivation for squad members.

But what when design of whole system has to be concluded? Who should do that? Well, we are used to (at least I am) that these demands are given to us from above in form of new API, maybe some messages or even something else. In the ideal world all we had to do is to implement them and be thankful that some system designer or architect defined them for us. But is having that someone always beneficial? Don’t get me wrong here, I am not saying that those people do their job wrong but rather that they are on different level in hierarchy and sometimes they slow or even stop communication between teams. Unnecessary dependencies introduced that way are harder to eliminate as they don’t affect designers and might be seen just as some additional work. I assume you know where I’m getting at already. Yes, in Spotify teams are internally and externally self-organizing. They collaborate with each other to find the best solution. Leaders job is to communicate to squads what problem needs to be solved and why. The goal here is to keep teams loosely coupled, but strongly aligned so they can accomplish long-term objectives without stepping on each other’s toes.

Despite of our best efforts some changes will impact several modules or systems and very likely procrastination of other squads will at some point block us. Based on my experience there rarely was agreement to modify another teams code. But at Spotify every component repo is not kept as precious treasure that only chosen can access. They prefer have thing in more open source manner where everyone can make amendments when owners have some more important things to do. Now, let’s imagine such situation where few teams work on same feature. Squad 1 is responsible for search engine and squad 2 works on list that displays results. This new feature requires some modification in function fillSearchResultsList but squad 2 which owns list is delaying this change.

class SearchResultsList {

    // ...
    public void fillSearchResultsList() {
        // ...
        someMagicSwtitch(false); // needs to be changed to "true"
        // ...
    }
    // ...
}

In many companies teams would be forced to wait and spam villains mail boxes with endless reminders.

But in Spotify squad 1 can just make a change and ask squad 2 for review. This approach speeds up work and helps to spread knowledge.

Company organization

Spotify tend to create communities of interest rather than formal structures. So Tribe could gather all the people who work at same place. Columns in Tribes are Squads and these were explained previously. Each row represents a Chapter which is created based on competency area like agile coaching, web development and so on. Also Chapter Leaders are formal managers for other members and they do not change while switching squads. Last thing are guilds that can be described as lightweight community of interest. They gather and share knowledge on specific area. Anyone can join or leave guild as these usually are represented as mailing lists, walls etc..

Releasing changes

To keep this well-oiled machine going every squad must be able to release their changes frequently, not worried too much about impact it may have on the rest of the system. Often and small releases have several advantages like more extensive testing of a new code, bugs are easier to locate and they don’t stop other changes from being released. To achieve that Spotify have completely changed its desktop client which previously was just one big monolithic application and now is basically web browser build on Chromium Embedded Framework. Breaking their client into dozen of microservices allowed them to scale up without drowning in release work.

Summary

What I have described in above article is just a part of innovative actions taking place in Spotify company. In my opinion it is great that they are willing to make mistakes and bear the cost for brighter tomorrow.  I think the lesson we should take out of this is to not bluntly accept things that gives us sleepless nights and try to do something about them. Of course some changes will be too expensive to be carried out, but with every new project there is a chance to turn the table. You can find more information on Spotify engineering culture at this link.

Sources

https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/

http://www.startuptalky.com/wp-content/uploads/2017/02/Company-Culture-Matters-Heres-Why3-950×509.png