{"id":3522,"date":"2026-01-24T19:58:57","date_gmt":"2026-01-24T19:58:57","guid":{"rendered":"https:\/\/www.smharter.com\/blog\/?page_id=3522"},"modified":"2026-01-31T12:24:36","modified_gmt":"2026-01-31T12:24:36","slug":"the-software-engineering-paradox-fall-redemption","status":"publish","type":"page","link":"https:\/\/www.smharter.com\/blog\/the-software-engineering-paradox-fall-redemption\/","title":{"rendered":"The Software Engineering Paradox: Fall & Redemption"},"content":{"rendered":"\n
\n
\n
\n

\n\n\n\n

This is one account of the events, from a keen observer with firsthand experience and a sharp analytical ability. However, the whole story can only be told by listening to many perspectives. Consider this a starting point, the first spark for a campfire retelling.<\/em><\/p>\n\n\n\n

The Fall Of Software Engineering<\/strong><\/h2>\n\n\n\n

In the 1980s-1990s, the prevalent interpretation of Software Engineering was flawed:
– It was disconnected from real-world professional software development;
– It arranged software professionals hierarchically, with coding at the lowest rung;
– It emphasised predictive, prescriptive processes over human ingenuity and creativity.
These flaws led to ineffective outcomes: Software development processes became over-bloated, bureaucratic, slow, and unreliable.
The acceleration of technological innovation and economic growth of the 1990s triggered a backlash against Software Engineering.

<\/p>\n\n\n\n

The Emerging Alternative<\/strong><\/h2>\n\n\n\n

From the late 1990s, new lightweight methods gradually emerged as the alternative, arising from the trenches of real-world professional practice. In parallel, Software Development began to be framed as a craft rather than a formal engineering discipline.
In this context, the lightweight methods repeatedly demonstrated clear success where prevalent Software Engineering models failed.<\/p>\n\n\n\n

A self-organised community formed, and practitioners began to codify, document,  share, debate and peer-review their learnings, advancing the field to improve the state of affairs for everyone. This community of volunteers was driven by a collaborative, open spirit that also characterised the Open-Source movement of the time.<\/p>\n\n\n\n

Nonetheless, in rejecting traditional Software Engineering, the field lost disciplined, methodical foundations rooted in social and engineering sciences, as well as impartial peer-review mechanisms free from vested interests, personal biases, and affiliations.

<\/p>\n\n\n\n

The Paradox<\/strong><\/h2>\n\n\n\n

When lightweight methods emerged from the failures of Software Engineering, many of them drew inspiration from within Software Engineering itself, particularly from the pioneering work of Tom Gilb, an outlier among its mainstream contributors.
His work, unlike the 1980s-1990s Software Engineering approaches,  was:
– grounded in real professional practice and concerned with all the practical aspects necessary for a successful delivery;
– rooted in collaboration among people working together as a control system, sharing responsibilities for success;
– evolutionary (non-predictive), founded on engineering pragmatism, learning, comparative decision-making and ultimately feedback and control.

Tom Gilb was the first to formalise an (iterative, incremental and) evolutionary software development model, driven by early and continuous feedback. The core ideas have been published in his 1976 book Software Metrics<\/em>, and further elaborated in a series of articles between 1978 and 1985, including the ‘Evolutionary Delivery versus the “waterfall model\u201d<\/em>‘.  In 1988, he published the book ‘Principles of Software Engineering Management<\/em>‘,  the most influential early bridge between his pioneering work and what eventually became Agile.<\/p>\n\n\n\n

Another evolutionary software development model of the same era was Cleanroom from Harlan D. Mills, IBM Fellow, published in the 80s by the IBM Journal.

<\/p>\n\n\n\n

The Fall Of Agile Under The Weight Of Time And Success<\/strong><\/h2>\n\n\n\n

In the early 2020s, after a long period of success, the lightweight methods, collectively known as Agile, faced a loss of trust and popularity. At the same time, the community lost its ability to credibly codify, document, critically evaluate, and peer-review new learning, a capability necessary to advance the field.<\/p>\n\n\n\n

This decline occurred under the combined weight of time and success:
– the growth in popularity and consequent economic success led to crude economic exploitation that diluted quality;
– the rapid expansion of the community eroded the original ethos of openness, sharing and collaboration aimed at improving the state of affairs for everyone;
– a broader cultural shift favoured proprietary, commercial, individualistic, and personalistic values and polarised views;
– market forces and government initiatives, amplified by the strategic importance of the Tech sector, pushed toward the commoditisation of professions in Tech, resulting in overspecialised, over-standardised, narrowly defined roles and under-equipped professionals.<\/p>\n\n\n\n

A combination of these and other factors diluted what made Agile effective, enabled the proliferation of unproven and ineffective methods wrongly branded as “Agile”, and led to the discoordination of a community no longer capable of fulfilling its role in advancing professional software development.

<\/p>\n\n\n\n

Software Engineering Redemption<\/strong><\/h2>\n\n\n\n

With Agile and its community losing their role as “custodian” of professional software development, Software Engineering has a renewed chance to become relevant again, through modern (iterative and incremental) evolutionary models, supported by a new culture of software development that is:
– grounded in real professional practice and concerned with all the practical aspects necessary for a successful delivery;
– rooted in collaboration among people working together as a control system, sharing responsibilities for success;
– evolutionary (non-predictive), founded on engineering pragmatism, learning, comparative decision-making and ultimately feedback and control.<\/p>\n\n\n\n

This opportunity emerges from Tom Gilb’s Evolutionary Delivery<\/em>. His work inspired the lightweight methods and Agile, and has evolved through the years into Evolutionary Value Optimisation<\/em> (EVO), while retaining the valuable lessons worth preserving from Agile. EVO now closes the circle, becoming a credible answer to “what comes next after Agile” by bringing Software Engineering and System Engineering back to their original purposes, without the distortions and detrimental effects of the old culture of the 1980s-1990s.<\/p>\n\n\n\n

Evolutionary Value Optimisation<\/em> focuses on value optimisation, not merely software delivery. It includes Planguage<\/em>, a disciplined approach to quantification and measurement, forming the foundation of the feedback loops driving the evolutionary process model.
In this latest form, EVO represents both maturity and continuity. Its core pillars include:<\/p>\n\n\n\n

Pillar<\/th>Essence<\/th><\/tr><\/thead>
Quantified value<\/td>Make value explicit and measurable<\/td><\/tr>
Ends vs Means<\/td>Separate objectives from solutions<\/td><\/tr>
Value optimisation<\/td>Optimise value, not output<\/td><\/tr>
Evidence & measurement<\/td>Learn from reality<\/td><\/tr>
Prioritisation<\/td>Engineer trade-offs<\/td><\/tr>
Small increments<\/td>Limit risk, maximise learning<\/td><\/tr>
Clear roles<\/td>Prevent value dilution<\/td><\/tr>
Generality<\/td>EVO beyond software<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n



Do you want to know more about EVO?<\/p>\n\n\n\n