By Dr. Robert B. Cohen, Senior Fellow, Economic Strategy Institute, November 9, 2015
(click here to download PDF)
Companies are overwhelmed by the number of transactions they must manage and the constant demand for new applications and services due to the glut of mobile communications and a rapid rise in data flows. To respond, some firms have developed rapid action teams to make their firms more productive and responsive.
This appears to permit such firms not only to derive important advantages from the new software-defined infrastructure that supports accelerating product and service development but also to create work teams or groups that meld skills that had formerly been isolated to perform specific tasks. So these firms are terminating jobs such as software developers or quality assurance employees. The new positions they are filling in teams or squads have superseded the old occupations. The new occupational designation for these employees is not clear, nor are the tasks clearly defined. Team and squad members may be called platform team engineers but this is not true of all cases where such positions exist.
These changes in the workforce are part of an experiment in agility that has begun over the past five years. As the use of virtualized infrastructure with software defined networking, cloud computing and containers becomes more widespread, it is providing firms with a chance not only to speed the creation of new products and services, but also to emphasize creativity and teamwork above task work.
This summary reviews the main organizational changes that are being tried at several companies and the upside and downside for these changes.
The main focus here is on three firms, Spotify, Supercell and Netflix. We are also exploring whether firms that are using containers/Docker and continuous service delivery are making similar organizational changes.
Spotify, Netflix, Supercell and the move to Team-Like Structures within Organizations
Spotify, “Swedish music streaming service offering digitally restricted streaming of selected music from a range of major and independent record labels” began life in 2011. It introduced a type of matrix organization built on a scaling model that included what it named “Squads, Tribes, Chapters, and Guilds” in 2011.
The structure works in the following way. There are various groups built on squads, work teams or scrums that work together. At a higher level, the squads are joined into tribes, chapters and guilds in a type of matrix structure. The matrix makes it possible for individuals in different squads and tribes to collaborate with each other because they work on similar types of tasks. This provides a way to connect employees across the different Spotify structures.
At the most basic level, Spotify has created squads that behave like small startup firms. A squad is like a scrum team with the “skills and tools needed to design, develop, test, and release to production.” They follow lean principles to design, build, test and tweak, and ship new products and services. Instead of being an anarchistic scrum, the squad has product owner who prioritizes the work of the squad. Functionally, squads have a “long-term mission such as building and improving the Android client, creating the Spotify radio experience, scaling the backend systems, or providing payment solutions.”
“Each squad must be empowered to make [its] own decisions, not only on features, but also on development model, infrastructure, and implementation. Every decision that has to be approved outside the team means a delay that slows development. Each dictated implementation or infrastructure decision means that a technology that doesn’t fit to the way the team works or something new that must be learned before the team can build.”
A squad “is responsible for a single feature or component. For example, there is a squad that is responsible for search, a squad responsible for the AB test infrastructure, etc. As each tribe is set up to be as autonomous as possible, each squad is also set up to be autonomous. In the context of a feature development team, this means that each team is a full-stack team. A full-stack team is responsible for both backend implementation as well as the user interface implementation, on all platforms.”
“A typical feature squad would have web service engineers, iOS, Android, web and desktop engineers as well as testers, an agile coach, a product owner and UX designer. With this staffing, the squad has everything they need to implement anything related to their feature. They don’t have to wait on another team to implement the pieces they need. They also have autonomy and local decision-making ability, so there are few impediments on their speed of execution.”4
Supercell’s cells are very similar to Spotify’s squads. Supercell is a Finnish gaming firm that now has offices around the world. Softbank is its main investor. Supercell’s biggest success is Clash of Clans, “a strategy game where players can construct and expand one's village, unlock successively more powerful warriors and defenses, raid and pillage resources from other villages, create and join Clans.”
At Supercell,6 each cell has five people and creates a single game. The CEO focuses on not micromanaging the cells, although the combination of cells is called a “supercell.” Supercell hires people with broad experience in gaming and gives them the freedom to create games. Individual cell members take responsibility for their own work.
Spotify’s squads are also very similar to the team structure that Netflix developed. At Netflix (see the figure below), the need to complete software projects rapidly resulted in the creation of teams that include individuals with different skills who collaborate to complete projects rapidly. The team structure avoids previous delays in the product creation process because there is no leader of a specific group, such as quality assurance, who must approve work before it moves ahead. In previous structures where occupational tasks were separated from each other, the approval process added a significant delay to the completion of new services and products.
Netflix created two types of teams. One, teams that use microservices, combine project management, software development, quality assurance, and the deployment of new services in a single team. In the other type of teams Netflix combines microservices development skills with networking skills in what it calls project teams that use monolithic delivery.
Netflix executives, including former Cloud Architect Adrian Cockcroft, explain that the new structure was needed to accelerate product development and also provided support for data analytics. The latter enabled Netflix to experiment with different offerings to keep customers pleased about the services they were receiving. So there was an acceleration in the creation process as well as greater sophistication in analytics. Comcast also recently underscored the fact that the goal of customer retention is a prime reason it adopted more software–defined infrastructure.
Spotify is the main firm that has moved the new team structure up to a well-defined series of higher levels. The next level up in the Spotify hierarchy is a Tribe, or a “collection of squads that work in related areas – such as the music player, or backend infrastructure.” So in Spotify’s engineering and product organization, the “Tribe is responsible for a set of related features or engineering functions.” One of the largest Tribes is the Infrastructure and Operations Tribe. Other Tribes include the Music Player Tribe that handles “importing audio from our label and distribution partners, storing and streaming the music, search, collection and playlists, artist pages, music metadata and the music knowledge graph that supports things like the above, but also ads, discover, radio and the like.”
The next level up in Spotify’s hierarchy begins to create a matrix over the previous organization. That is, the vertical part of Spotify’s matrix is that “people are grouped into stable co-located squads, where people with different skill sets collaborate and self-organize to deliver a great product.” The horizontal part of the matrix is “sharing knowledge, tools, and code.”
Chapters are part of the horizontal structure and act as “reporting and functional groupings” that cross a number of tribes. There are several types of chapters, i.e., people with the same skills who work within a Tribe, such as “the testing chapter, the web developer chapter or the backend chapter.” The chapter lead “is line manager for his chapter members, with all the traditional responsibilities such as developing people, setting salaries, etc.” The different chapters meet to discuss their areas of expertise and the challenges they face. In 2014, there were “three backend (services) development chapters, two front-end development chapters (including all the UI developers), a core library chapter, and a test chapter. These seven Chapters span eight different squads. Almost every chapter lead has reports in 2 squads, and a few of them have reports in three squads. Almost all chapter leads work within a squad in some capacity as well, either as developer or technical lead, and not necessarily within a squad that has members of their chapter.” In reality, Chapter members are not evenly distributed across the squads; “some squads have lots of web developers, some have none.”
Another part of the Spotify matrix is the Guild. The guild provides a way for Spotify to coordinate the efforts of its autonomous teams. It is a virtual “cross-tribe organization” focused on “different technical or interest areas and their membership is voluntary. The guilds serve as ways to promote cross-tribe collaboration and communication, especially around things like best practices.” There are “guilds for Web Development, Agile Practices, Leadership, Test Automation, etc. The guilds foster developer-to-developer communication.” The “Guild is a more organic and wide-reaching “community of interest”, a group of people that want to share knowledge, tools, code, and practices. Chapters are always local to a Tribe, while a guild usually cuts across the whole organization.”
Issues in using the matrix structure. There are a number of challenges in using such a matrix structure. The key one is coordination among the different functional groups within the firm, such as the Tribes, the chapters and the Guilds. There is also a separation of groups such as legal and marketing from the employees in the matrix structure. So coordination with these groups and the employees in the Tribes and the legal and marketing teams is critical. If coordination is lacking, legal and marketing can negate other efforts to instill agility in the firm.
In the matrix structure, improvements depend upon actions by team members. If there is no follow-through, changes may not occur rapidly or may not be communicated from one team to another. “Best practices and technologies do spread from team to team through avenues like guilds. Teams adopt these practices and technologies on their own schedule or pioneer new ways of working if it makes it easier for them to deliver value to our customers and then spread their learnings to the other teams.”
Spotify has a number of processes and structures to improve coordination. Guilds hold unconferences where all the web developers at Spotify meet and “discuss challenges and solutions within their field.” There is also an agile coach guild with coaches are spread throughout Spotify. These coaches are continuously sharing knowledge. They also meet “continuously and … regularly to collaborate on the high level organizational improvement areas, which [Spotify] tracks on an improvement board.”
Thoughts about how cross skilling jobs are linked to the new software defined architectures. This discussion identifies how firms have employed new software defined architectures to create teams that break down previously tightly defined occupational categories, such as software developers, to create team members that collaborate to develop new products and services.
While there is not a specific occupational designation for these team members, team work may become more common in the future because it produces final products and services more efficiently than the previous job structure. It also offers a better outlet for creativity and for team members to learn skills that are beyond their area of expertise. Here, we are using the term cross skilling to describe such jobs. This accurately describes the need for employees to broaden their technical skills and not remain in a specific, more isolate profession largely based upon the original job skills they possessed. Collaboration is an additional skill that is often the key to success in new team structures.