The results of companies migrating to heavy-weight scaled frameworks and off-the-shelf platforms

This is an account of the results obtained by companies
adopting a heavy-weight scaled Agile framework
instead of pursuing Agility at scale,
or migrating to a heavy-weight off-the-shelf software platform
instead of pursuing technical excellence.

Feels free to copy and distribute this post as you please. Get in touch to suggest any improvements.

Compare this account with How large successful companies achieved Aglity at scale.

Summary

Both FitBit and Volvo Cars, for about two years, went through an Agile transformation to scale Agile using a heavy-weight scaled agile framework.
YNAP and Co-Op Insurance both tried to migrate their strategic and core systems developed internally to a heavy-weight off-the-shelf platform.

Heavy-weight scaled agile framework here is intended, for example, as one with a guide of several hundred pages (instead of few tenths), that requires configuration and tailoring-down (instead of being simple and partially incomplete to fit loosely to leave room for adaptation and manoeuvre), with roles that increase the deep of the hierarchy and the hands-over (instead of flattening the hierarchy and supporting self-organisation).
Heavy-weight off-the-shelf platforms and out-of-the-box configurable white label products here is intended, for example, as overcomplicated and rigid third-party products that end-up requiring significant rewriting work, huge and complex data migration, and are hard and difficult to customise to fit the always changing and evolving market requirements.

For FitBit, YNAP and Co-Op Insurance this had a very tangible and negative impact on their business. No tangible impact can currently be measured at Volvo Cars (it is too early to say at the time this post is written).

FitBit and Volvo Cars case studies don’t show signs of a strong focus on tech excellence. The technical approach at YNAP and Co-Op was shattering.
None of the four companies shows any sign of alignment to the Agile mindset (that is considered a fundamental ingredient of any successful Agile adoptions), or any sign of autonomy of teams.

Ar the end of this post, there is also an account of a session at a summit that mislead attendees to believe that Microsoft adopted a heavy-weight scaled “agile” framework to successfully achieve Agility at scale. This is a testimony of the aggressive marketing of Scaled frameworks that go beyond genuine and authentic sharing of lessons and learnings. For the real story of how Microsoft pursued Agility at scale, follow the link at the beginning of this post. For another PR stunt from the aggressive marketing of Scaled frameworks, see the USAF notes following the link at the end with comments from experts.

These stories shared below provide good learnings about things that did not work well for these companies. Correlation does not imply causation, so use these learnings to inform and inspire experimentation in your company with an experiment-learn-adapt-evolve approach.

ANZ bank rolled out a ‘Scaled Agile’ approach across its Australian division

ANZ, Australia and New Zealand Banking Group Limited is an Australian multinational banking and financial services company headquartered in Melbourne, Victoria.

In 2016 SAFe was rolled out in one part of ANZ, says Sam Kline.
In an event sponsored by Scaled Agile Inc. and one of their premium implementation partners, the rollout has been listed among some of Australia’s most successful SAFe implementations.

A second and bigger initiative started in 2017. In 2017, CEO Shayne Elliott wanted to reshape the bank’s staid culture to give ANZ customers more and faster by introducing Agile. Shayne Elliott was taking inspiration from what technology companies from Amazon to Microsoft did and looking at what some banks such as ING and ABN Amro had been doing although with mixed results. He focused on processes implementing an approach for scaling Agile across its Australian division and focused on tools: through an enterprise agreement with Atlassian for the use of Jira and Confluence.

In a 2018 article by Michael Gibson, Senior Analytics Delivery Lead in ANZ, there is a reference to SAFe suggesting that there was still some influence from SAFe ideas at that time. In another 2018 article Jean Dieudonne, Product Owner and Business Owners Tribe at ANZ, mentions a copy-paste approach of elements of the Spotify model and a reference to McKinsey’s articles containing several misrepresentations of Agile.

To summarise, both initiatives aimed at scaling Agile instead of achieving Agility at scale. The first initiative relied on a proven recipe, and the second initiative relied on copying and tailoring solutions from other organisations instead of growing and evolving their own approach based on the specific needs, context, and circumstances of ANZ.
All things that nowadays Agile experts strongly advise against.

Despite promises that the 2017 initiative would lead to “agile” teams delivering a digital transformation to generate growth and underpin the bank’s residential mortgage business, five years later the bank’s technology systems still lag behind its biggest competitors. This may be inferred from the bank’s home loan approval systems becoming one of the slowest in Australia, and for that ANZ lost a substantial share of the $2 trillion mortgage market, its share price is down 17 per cent since Elliott became CEO seven years ago. 

Martin North, the founder of DFA Analytics, says “I don’t rate ANZ as an agile organisation. In fact, I would say that they’re quite sluggish in terms of some of the things that they’re doing.”

References
– https://www.cio.com/article/213835/anz-bank-goes-all-in-with-scaled-agile-approach.html
– https://www.arnnet.com.au/article/646310/anz-taps-atlassian-agile-push/
– https://www.smh.com.au/business/banking-and-finance/shayne-s-world-how-shayne-elliott-s-agile-anz-got-stuck-20220822-p5bbsz.html

FitBit scaling with a heavy-weight scaled framework, a case study

Fitbit Inc. is an American company founded in 2007 with its headquarter in San Francisco. It now has about 15 offices around the world. In 2019 Fitbit was reported to have sold more than 100 million devices and have 28 million users.

Between 2015 and 2016 FitBit adopted a heavy-weight scaled Agile framework to scale their Scrum effort. They started in 2015 with 12 Scrum teams, that in 2016 worked on the release of four new products. That is an average of four teams per product.

Damian Brown, Sr. Director of Program Management Office, describes the adoption of the scaled framework as a success story. He describes such success by highlighting these results:

• We were able to scale the process and still deliver
• Velocity increased 33 percent year over year
• 100% Delivery on Objectives
• Fitbit has grown significantly since we adopted the scaled framework
• In 2016, Fitbit, released four new products to the market that were positively received by consumers and shipped over 22 million devices.

The first three points seem to describe success in traditional predictive planning and process terms such as delivering on-time and on budget, etc.  They are not representative of a paradigm shift or an Agile mindset. 
A person working at FitBit confirmed the lack of focus on autonomy with teams focused on the delivery (IT seen as mere executors of requests passed to them ) instead of being empowered product teams.

The last two points are more troubling. Marked data and statistics available online seem to contradict both of them. The growth rate of the number of employees started to decline in 2016 and then got worse. This is hardly growth. In 2016 the stock price of FitBit did fall more than 75% and has yet to recover as today. This is hardly a success.

Analysts explain the downward spiral due to lower margins and increasing competition.  A series of earnings reported feature vast increases in spending and weak guidance. All these signs are congruent with other comments about a lack of innovation in FitBit.

The person working at FitBit told that they then had abandoned the SAFe adoption, and the person that had persuaded the company to try it had left the company as well. Still, the FitBit case study is listed as a sounding success on the SAFe website.

So far there are no tangible and long-lasting benefits for the business coming from the adoption of the scaled framework.
Furthermore, there has been any focus on teams’ autonomy and strong engineering culture.

References
– https://www.scaledagileframework.com/fitbit-case-study/
– https://www.scaledagile.com/case_study/fitbit/
– https://www.macrotrends.net/stocks/charts/FIT/fitbit/number-of-employees
– https://www.svpg.com/revenge-of-the-pmo/
– https://www.svpg.com/spotify-vs-fitbit/
– https://medium.com/@seandexter1/beware-safe-the-scaled-agile-framework-for-enterprise-an-unholy-incarnation-of-darkness-bf6819f6943f

Volvo Cars Agile transformation with a heavy-weight scaled framework, case studies

Volvo Cars was founded in 1927. It is headquartered in Gothenburg, Sweden. it has around 40,000 employees and produces around 700,000 cars per year.

In software development, it employs about 10,000 people with about 150 million lines of code per car. Nowadays cars are steadily becoming “computers on wheels”.

Volvo Cars went through 2 years and a half Agile transformation phase to scale Agile with a heavy-weight framework. That effort has been completed in December 2019. Anna Sandberg, Head of Continuous Improvement & Change at Product Creation, says Volvo Cars Agile journey will continue after this phase.

Volvo Cars for some years had a bottom-up Agile movement. There were individuals and software teams that had been attracted to Agile for many years.  But Volvo Cars hadn’t managed to scale those efforts. This Agile transformation had the intent to scale such effort across the software community, and the whole product range, both hardware and software. Volvo Cars felt the need to develop the software and hardware at the same time in an integrated fashion. Because they knew that things were going too slowly compared to changes in the automotive world.

Now just after the transformation is finished. It is too early to say if there are tangible and lasting benefits for the business, or if innovation/flexibility/resilience have improved. Or if this change will be sustainable and will stick.

In the transformation a huge amount of time and money have been spent on scaled Agile training: “We spent more than 150,000 manhours in training people in different aspects of Agile. We also trained around 2,000 leaders”. 

The interview with Anna Sandberg leads to assuming the absence of a strong focus on removing technical dependencies and technical debt to support the scale and the speed, instead the focus has been on accommodating existing technical dependencies:  “we can identify dependencies and see whether actions and resources match with commitments”. 

When asked about mindset, this is the language used in the answer: “We *need* to realize that doing Agile at scale requires … *protect* the principles … I don’t know how to do Agile at scale without some kind of framework and *hierarchy* that helps us to decide and prioritize.” 

The interview also shows through a focus on processes, and the hierarchy with a traditional top-down approach. While there are no signs of focus on the Agile mindset.

The lack of focus on the Agile mindset is highlighted also by two academic case studies from the Chalmers University Of Technology.

The first study is titled: The role of the First Line Manager in a Scaled Agile organization A case study at Volvo Cars Corporation. It reveals that Volvo Cars retains a very tall hierarchy even after the transformation. The study says Volvo Cars appears to have scaled Agile vertically by implementing Agile practices throughout the management hierarchy.

The same case study states that at Volvo Cars the Product Owner is not really considered part of the team. While the Scrum Master role is a less important role compared to the first line manager role (FLM as in the picture below) and it is fading or taking a smaller role in the organisation, with less authority and responsibilities, as a consequence of scaling Agile. These interpretations of the Scrum Master and Product Owner roles differ from the original interpretation in Scrum, and not necessarily for the better.

The case study adds that as a consequence of the transformation, the first line manager role (FLM) has not changed significantly and that its supervising role had only strengthened because of its position in “systems of external, hierarchical supervision”. Even so, the theory and results from the interviews suggest that Volvo let managers stay in the same position. 

The study also comments on the inflexibility of the scaled Agile framework adopted mentioning two disadvantages a) its inflexibility in the implementation,  and b) it potentially preventing from finding solutions outside of the framework.

The tall hierarchy, the lack of authority and consequently autonomy of SM, PO and the teams, are all clearly misaligned with an Agile mindset.

The title of the second case study from the Chalmers University Of Technology is: Large-scale agile transformation A case study of Volvo Cars’ transformation process. It highlights the perceived success factors of Volvo Cars Agile transformation, in the image below.

At the top of the list of success factors, there is the very old-school traditional RACI concept, while some of the key aspects of Agile, such as the incremental and experimental approach, are at the bottom of the list. That also suggests a lack of alignment with the Lean and Agile principles and the Agile mindset.

References
– http://publications.lib.chalmers.se/records/fulltext/257373/257373.pdf
– http://publications.lib.chalmers.se/records/fulltext/255549/255549.pdf
– https://www.forbes.com/sites/stevedenning/2020/01/26/how-volvo-embraces-agile-at-scale/#597b6f324cf0
– https://gdqassoc.com/wp-content/uploads/2017/01/GDQ-seminar-sep-2019_lucas_dela.pdf

Yoox Net-a-Porter migration to a heavy-weight off-the-shelf platform

In 2015 Yoox and Net-a-Porter merge created YOOX Net-a-Porter Group (YNAP), an Italian online fashion retailer. Yoox was founded in Milan in 2000. Net-a-Porter was founded in London in 2000.

Yoox’s concept is to buy up overstocked or unsold items from previous seasons and sell them online at discounted outlet prices. Net-a-Porter was developed from the concept of a magazine in website format where users could ‘click’ to buy while trying to source products online for a fashion shoot.

The group in 2018 had about 5000 employees.

Prior to the merge, both Yoox and Net-a-Porter were adopting Agile ways of working and trying to improve the technical quality of their platforms to support their growing number of customers and to match the evolving expectations of tech-savvy customers. At that time a strategic decision was taken to build a single technology platform that would be shared across all the group’s subsidiaries.

Prior to the merger a decision was made to bring in IBM as a partner, and Chief Technology Officer Alex Alexander, hired for his understanding of IBM’s software, was tasked with leading the transformation. The goal was to migrate the existing platform to IBM products such as: 
– IBM Sterling Order Management
– IBM WebSphere – Open Commerce Express
– etc.

Third-party off-the-shelf platforms like these come with hidden costs of configuration, customisation, and migration. They typically are monolithic and have their roots in old technologies and outdated philosophies, so the migrations tend to be similar to old-style big-bang waterfall type of initiatives, with all the well-known downsides that go with it. These overcomplicated and rigid third-party platforms make it harder and slower to keep up with continuous changes and improvements and regular technology upgrades that are a necessity for e-commerce businesses in a fast-moving market.

Other fash-tech companies have faced the same challenges while adopting similar third-party off-the-shelf old products (e.g. Oracle Retail, etc.) with help from large, slow, and traditional consulting companies. Bernstein analyst Luca Solca wrote: “Mega-brands — who dominate the luxury industry today — have little to gain long-term from feeding ‘winner-takes-all’ third-party distribution platforms,”

The project was ambitious in both scale and scope, but the full extent of the troubles at the luxury e-commerce platform, was not made clear until mid-May 2019. The trouble with digital transformation started more than three years ago and has cost hundreds of millions of euros. The growing costs stemming from the global technology and logistics platform migration were estimated at €200 million ($221 million) for the fiscal 2019 year alone. Sources close to the operations suggest that the company will end up spending far more than the €500 million it originally budgeted.

For 12 months after the launch, The Outnet (one of the online shops of the group) saw year-over-year sales decline every month. Richemont, which acquired the group in 2018, announced its financial results for the previous fiscal year as its weakest profit margin in more than a decade, taking analysts by surprise.

The new platform made it difficult also for shoppers to easily navigate the site and perform basic functions.

In December 2017 one New York-based shopper wrote on Complaintsboard.com “I don’t know how and why, but now it works more terrible than it used to be”,  “Now it’s absolutely uncomfortable to use it… I don’t understand what was the point of this nonsense.”

The re-platforming has sparked complaints also from senior executives. There has been a talent exodus within the group. In the past three years, at least 20 of the group’s senior executives have left. Many of their direct reports have departed as well. Complaints of mismanagement and scapegoating have eroded morale and affected the company culture.

In conclusion investors, customers and employees have faced tangible and lasting negative consequences from a digital transformation that tried to solve a complex problem by ignoring technical excellence and the engineering culture, outsourcing and delegating the responsibility in the search for the mythical silver bullet, damaging the company culture and alienating employees and customers.

References
– https://www.businessoffashion.com/articles/professional/yoox-net-a-porter-richemont-tech
– https://www.ibm.com/case-studies/yoox-net-porter-group
– https://internetretailing.net/themes/interview-alex-alexander-of-yoox-net-a-porter-on-innovation-18101

Co-Op Insurance migration to a heavy-weight off-the-shelf platform

Co-op Insurance with the trading name of Co-op Insurance Services Limited is based in Manchester and was founded in 1867.
It currently offers business, home, motor and pet insurance products.
Co-op Insurance is part of the Co-operative group that has more than 100,000 employees.

Co-op Insurance started a transformation programme to migrate from the legacy IT infrastructure to a new, modern insurance platform for powering the insurance company’s operations.
The solution pitched and sold by IBM consisted of an out-of-the-box configurable white label Insurer Suite.

In January 2015 Co-op Insurance and IBM signed a £155m Agile project intended for IBM to deliver what promised silver bullet.

After that brilliant sell from IBM, it rapidly emerged that the out-of-the-box configurable white label Insurer Suite silver bullet was a platform that had been written specifically for the US insurance market, not the UK insurance market, and so it had to be substantially rewritten and re-developed. The platform needed significant work to be fit for use in the UK market.
Additional problems were caused also by the data migration.

Unsurprisingly, 15 months after its deadline the contract had not been delivered. In December 2017 a lawsuit was filed by Co-Op Insurance against IBM.

In addition to the unrealistic promises made to close the sale, there have been many serious mistakes in applying an Agile methodology:

• IBM claimed that “it would not have resulted in any increase in the [Co-Op]’s profitability or revenues”.
  By contrast, Agile software development is about creating valuable Outcomes for the customers.
  
• Co-Op was supposed to write User Stories that IBM and the consultants brought in by IBM were to jointly turn into a usable piece of software.
  By contrast, in Agile User Stories are co-created by the client, the product/business and the tech people collaborating together and without hands-over.
  
• IBM vice president Steve Allen complained that the client Co-Op had introduced a new requirement at a late stage, and complained about a large volume of change requests.
  By contrast, the Agile methodology welcomes changing requirements, even late in development (see also the Agile triangle for info about how to deal with scope, deadline, and costs and see the Agile contracts about how to deal with the contractual aspects)
  
• The project was expected to be completed with a single finale release followed by a final user acceptance testing (UAT).
  By contrast, in Agile the UAT is automated while the software is written, and tested software is released frequently and continuously.
  
• IBM vice president Steve Allen complained that defects “incorrectly raised” because the “system is in fact performing as expected”.
  By contrast, in Agile the goal is to create solutions that solve the real problem of the users, and test the software against that goal even before against the system specifications.
  

In conclusion, to close the sale by promising a silver bullet, IBM largely minimised and underestimated the changes and the work requires to deliver the solution.
As it commonly happens in such situations, also the complexity, the effort, and the time required for the data migration have also been largely underestimated.
On top of this, for the project, IBM has adopted an Agile methodology even if they fundamentally misunderstood even the basic aspects and the essential elements of Agile Sofware Development.

References
– https://www.theregister.co.uk/2018/12/19/ibm_v_co_op_insurance_130m_it_revamp_lawsuit/
– https://www.theregister.co.uk/2020/03/02/ibm_co_op_insurance_project_cobalt_trial/
– https://www.theregister.co.uk/2020/01/22/co_op_insurance_v_ibm_high_court_trial_project_cobalt/
– https://www.theregister.co.uk/AMP/2020/03/17/ibm_sopra_steria_co_op_insurance_lawsuit/

Microsoft misleading SAFe case study

There are many public unanimous accounts from different sources about Microsoft journey to Agility at scale. They all agree that Microsoft has grown internally their ways of working without following or taking inspiration from any scaled framework.

At a summit of one of the scaled frameworks, one session misled some of the attendees to believe that Microsoft employed a scaled framework for its journey to Agility.

If the public accounts were not enough to prove wrong such a narrative, sources with direct knowledge of several teams at Microsoft confirmed that they do not use any scaled frameworks and did not use them to scale Agility in Microsoft.

In the whole Microsoft only one team from a small peripheral service division, Supply Chain Engineering (SCE), used a scaled framework.  It is not clear if that division has been acquired by Microsoft while already adopting practices from a scaled framework, or if the division used the autonomy granted to the teams to make such a decision. It is peculiar that the scale in such a department does not seem to call for a scaled framework.

In any case, it has been confirmed that  Microsoft doesn’t and did not use scaled frameworks for its Agile journey.

References
– https://2018.safesummit.com/sessions/microsoft/
– https://twitter.com/kennieg/status/1206256609044946947
– https://twitter.com/NicolasChaillan/status/1206379969972260864

Misleading Scaled Agile inc. SAFe claims in relation to U.S. Air Force

Since the publication of this report from USAF that lists the flaws of SAFe and advice against it (see here at the end of the post), Scaled Agile Inc offered to work for USAF to address the criticism. After that, a couple of publications seemed to imply that the comments have been addressed and SAFe was ready to be adopted by USAF.

Instead, Nicolas M. Chaillan clarified: “my memo on SAFe still stands regarding my take on SAFe with DevSecOps. Until the issues mentioned in the memo are fully addressed with a released update of SAFe, SAFe is still not recommended to be used in conjunction with DevSecOps.” (link).

And later in March 2021, Nicolas M. Chaillan clarified again that the previous criticisms still stand (link): “SAFe is still anti DevSecOps. Not allowed in my teams.”

I personally find the PR stunt above from Scaled Agile inc. troubling and it raises questions about their professional competence or integrity.

Thanks

Thanks to Carlo Beschi, and Corrado De Sanctis for suggesting me write this piece and for helping me find some of the sources.

Their feedback does not imply an implicit endorsement of this post.

You may also like:

– SAFe: a collection of comments from leading experts

– Video: The problems of scaled agile frameworks and SAFe

– How large successful companies achieved agility at scale

If you’ve made it this far then you’re probably interested also in:

– Executive summary for everyone considering investing in Agile: https://bit.ly/AgileInvestingExecutiveSummary
– Information for decision-makers considering the SAFe framework: https://bit.ly/SAFe4DecisionMakers
– Information for decision-makers considering large consultancy firms’ Agile offering: https://bit.ly/agileOffering4DecisionMakers

---