How to cross the transition gaps: ways to smooth the path

Recently I found myself talking with colleagues on transition management about mass psychology and successive groups of people. How the different mindsets give us insight on improving transition management. The concept of successive groups originates from marketing and popped up surprisingly in my research on collective intelligence. This article gives insight on lessons learned about successive groups and the development of an improved transition management method.

In marketing, successive groups are created to understand the mindset of different customers. In a book by Geoffrey Moore, the market chasm is expressed between the adopters and early majority. In my view the market chasm is specific to common innovation. Common innovation is the most occurring innovation, the adjective common is a reference to common good. I will introduce different stages of transition focusing on different types of innovation, each showing an increased gap at a different location. Before common innovation comes fundamental innovation with the increased gap between innovators and adopters. After common innovation comes prescriptive innovation with the increased gap between Majority and Laggards. The prescriptive innovation is the outcome of an experiment that elaborates the mobilisation across successive groups and the eventual gap creating the tragedy of the commons.

The experiment actually elaborates that we better not call the last group laggards. The laggards are oversimplified as a passive group during common innovation. Laggards have a passive aspect, but the experiments showed how the group can be an active opposition. The opposition also comes in two separate mindsets: the rejectors (13.5%) and reactors (2.5%). The innovators and reactors create agency at both ends of the curve. They can mobilize the adjacent group (adopters / rejectors), creating a cascading effect that will decide what the majority will do: transform by following the innovators or conserve by following the reactors.

In some cases rejecting change is in fact the appropriate thing to do. For example, climate change is one where we should think about conserving the world. Similar to cultural heritance, we do want to conserve some parts for future generations. Too often unique artifacts are lost because they get outdated, while not yet old enough to get recognized as cultural heritance. The challenge is not to be progressive or be conservative, but to have balance and grow our civilization for future generations. The problem with this challenge is we are blind to the future. A need exists to gain an outlook about things to come, like weather forecasts. A need exists to have an transition forecast to develop effective policy. To this end a phase before fundamental innovation is introduced to deal with potential innovation.

Each type of innovation is expressed as a transition phase: potential innovation during the premature phase, fundamental innovation during the incubation phase, common innovation during the growth phase and prescriptive innovation during the maturity phase. The name “premature stage” is a reference to the embryonic nature of the stage. It is an artificial stage that is focused on learning. The name “incubation stage” is easy, fundamental innovation is a practice mostly seen in university incubators and focused on investment. The name “growth stage”, is a reference to scaling up a company, market growth and focusing on profits. The name “maturity stage” refers to the existing markets and long standing organizations. It shows a shift from management of private organisations to governance of public organisations, with a focus on sustainability.

The focus of today’s transition management is on fundamental innovation by university incubators.It leads to a significant gap between innovators and adopters. The gap is the effect of the Collingridge dilemma which relates to a double-bind problem (drawing below). An information problem: impacts cannot be easily predicted until the technology is extensively developed and widely used. A power problem: control or change is difficult when the technology has become entrenched. The transition from one problem to the other is also a transition from innovators to adopters in respect to fundamental innovation. If the gap becomes too large it becomes the valley of death.

Notice the figure considers funding and not market share. It is the transition from premature phase to incubation phase. In other transition literature it is often visualized as a funnel. The funnel is the effect of convergent evolution turning the laboratory for research into a workshop for industry. The industrial funding on this drawing shows the early adopters who invest in the technology. In the drawing, the late adopters are referred to as the “innovation stage”.

Late adopters are business investors who get us to the actual first customers by the Minimum Viable Product (MVP). We could generalize and see the successive groups for fundamental innovation is about investment. In the growth phase, venture capital invests in scale, turning the startup into a scale-ups. The majority contains users who do invest time and energy to learn the fundamental innovation. For example in the 1980s-1990s the majority of western societies learned to work with personal computers, a tool with no president and they all needed to invest time and energy to learn this tool. During the maturity phase the investment is to improve efficiency, now the investment is again at the organisational level, transforming a large scale organisation by change management.

The tricky part is to see how innovation transforms, how learning turns in investment, turns in profits and turns in evolving the investment. The dynamic is metaphorically like a music canon, creating a cascade of different successive groups. As the fundamental innovation wave starts, it gets quickly followed up by the common innovation. This canon of successive groups will get concrete in the experiment. For now let me continue to introduce the dynamics in general. The change happens when fundamental Research shifts to fundamental Development (i.e. R&D) turns technological flexibility into market viability. It is the shift from investment to market value and the price being paid is to lose technological flexibility. This means the market chasm is partly created by decisions made during prototyping. By gathering customer experience, a failure can be exposed, but now we have a wicked problem.

A wicked problem is not understood until after the formulation of a solution. A design tool exists to solve the wicked problem and gain customer experience before prototyping: pretotyping. Pretotyping simulates as-if the technology exists, test users have no clue it is an illusion or play along, creating experience as if the prototyping would exist. The customer interacts and gains experience, creating data to make design choices before technological flexibility is lost. The technique has proven itself quite well. Most examples of pretotyping focus on the product. To see it also applied to a service, let me give the case of McDonald brothers. They explored how to deliver a hamburger from 30 min to 30 sec. By pretotyping the kitchen workflow on a tennis court:

The Founder ‘Speedy System’ Featurette (2017)

The case is particularly relevant to elaborate potential innovation as it gains customer experience without the need to include actual customers; it actually introduces an essential tool for another wicked problem: how to understand the mobilisation across the successive groups before a market introduction. This is a challenge to gain data for the growth phase. The process of dogfooding has been used to solve the wicked problem. Dogfooding is the practice of using one’s own products or services inside the company before going to the customer. Dogfooding simulates customer experience at scale by having employees be the adopters of the technology. Dogfooding has been proven by large tech companies. Organisations would roll out a solution to one division and learn how to scale it to all divisions i.e. creating an internal version of the successive groups in the market.

Pretotyping was the first tool for potential innovation creating data about the incubation phase. Dogfooding is a second tool creating data for the growth phase. In both cases a kind of wrapping happens to simulate how it eventually would work. Dogfooding happens by actual companies having actual products. It required more development to get dogfooding into the premature phase and apply it purely to learning (the potential innovation). Making students do R&D and teaching other students via peer-learning, created an effect similar to employee experience.

The challenge of the premature phase is to gain experience via actual measurements before any development is done. The customer experience requires actual customers, what gets simulated is the prototype. The employee experience is dogfooding customer experience, so again actual people now interacting with an actual MVP. What gets simulated are the market dynamics to develop an effective organisation for the MVP. With more details later on about the maturity phase, it will become clear why this stage is about civilians. It is a game of governance. In organisational governance frameworks are used, yet the civilian experience goes beyond one organisation, it is about living in an ecosystem. Let me finish the introduction by giving an overview.

Three evolutionary dynamics appear to act on the four stages. It reminded me metaphorically of a carburetor, so I’ve named the proposed method Carbureted Action Research (CAR). It has three parts: funnel, tube and horn. The funnel is the effect of convergent evolution. The early adopters are still in the funnel developing the prototype, while late adopters develop the MVP. The late adopters and early majority are in a tube. The tube is about accelerating growth. In an actual carburetor the tube is a low pressure high-velocity flow where fuel is mixed with air by the venturi. IN CAR, the venturi refers to venture capital i.e. money is the fuel.

The early majority comes in before the growth is saturated. Finally we get to divergent evolution, with the late majority. It begins by improving the product or service to niche markets. With technology using platforms (i.e. more explicit frameworks) it led to long-tail business models.

To increase efficiency, organisations start merging in consortiums who will optimise its inner workings creating the ecosystem’s dynamics. In tech we have seen organisational governance turning into industrial standard (e.g. IBM PC compatible). Governance, standards and infrastructure is historically more a game of public governance and politics. These past decades a mixture of public and private players are used to improve public services (i.e. electrical grid, public transportation, etc). In 2016 social media started seriously interfering with elections. The latest effect during the covid pandemic is still deploying, but the trend seems to continue.

To separate the growth phase and maturity phase more clearly, we can look at the difference between emerging markets and mature markets. Emerging markets and existing markets have a different effect on the market chasm. In emerging markets, the inclusion of customer experience can be enough to cross the market chasm. It is not enough for mature markets. To illustrate, consider the interesting mix today of the emerging market with smartphones in close proximity to the mature PC market. It led to a mix of Apple phones integrated in Microsoft 365. Interestingly because the use of Apple computers, in business settings, did not follow. Notice how this example has to do with Business to Business (B2B) in mature markets (i.e. ecosystem dynamics).

The resistance in mature markets can be understood by considering the percentage of successive groups i.e. the reactors (2,5%) and rejectors (13,5%). For an emerging market the group has no power. In a mature market they are the incumbents, holding the majority of market shares. This power dilemma is another wicked problem and it got solved by disruptive innovation. Disruptive innovation happens by growing an MVP with clear technical design advantage in a niche not challenging to the incumbents. It can sometimes take decades to grow inside the niche to resolve hard to scale problems, yet once the problem is solved the incumbents have no way to stop the innovation. Interestingly the same nonlinear transition dynamics is recognized in biological evolution and called exaptation. Birds are an interesting case, evolving feathers for mating and heat regulating, to gain a nonlinear transition to flight.

Disruptive innovations appear to be a hack from the past, today incumbents do know the danger and the business of venture capital has matured. Recent concepts like corporate ventures resolve the problem for the shareholders, but not for the employees. The early hack of disruptive innovation was a kind of piracy, robbing the incumbent of their power. Today their power seems less challenged, but disruptive innovation has increased. The masses are being robbed of their power and with it comes the tragedy of the commons. The current situation is not sustainable at all. A transformation from disruptive innovation into prescriptive innovation is required where the masses can prescribe the rules of the game to conserve the commons.

During my research I witnessed the growth of the commons and tragedy of the commons as a trophic cascade in a digital ecosystem. The growth of the commons was the effect of a startup culture aware of the fragile nature of the ecosystem. The tragedy happened as incumbent players entered the niche market not having the same culture. To understand the development of the commons, I interviewed most entrepreneurs and discovered self-organizing innovation. Understanding the collective intelligence in the ecosystem was fascinating. Getting insights into actual controlled experiments was a challenge.

In an unexpected turn of events we ended up focusing on personal-development, creating educational innovation. The fundamental innovation in education related to a wicked problem with the speed we see by innovation in software development: How to educate when the topic of the course is outdated by the end of the year ? It took me three runs (i.e. three years) between 2006–2010 to get to the solution. The students were the developers helping to co-create the solution. This setup was ideal to investigate the incubation phase.

The success of the course gave my department the confidence to set up a large-scale experiment and investigate the growth phase. It was also ideal to solve a crisis that had emerged with another course where a lack of managerial resources emerged. The one-year master’s course became unexpectedly popular with foreign students. To manage the course appropriately the professor required 10 teaching assistants, yet the department was that same two year in a transition. In the first year I was asked to join the other three course assistants and it was not enough to manage the course appropriately. The second year I would be the only assistant. It was a wicked management challenge and ideal for my research. By delegating some of the teaching tasks to the students, the students became teachers (i.e. employees) and it allowed a setup for a dogfooding experiment.

After the first year of preparation, we ran the course in 2011 with 400 students. The setup showed how the successive groups relate to the student abilities to learn. The innovators are autodidact students. The low number of 2.5% becomes big at scale. In the experiment we had 10 innovators. The 13.5% adopters (i.e. 54 students) are in the zone of unaided development, they don’t need a teacher, just enough examples. The examples were created by the innovators. The majority were in a zone of proximal development (ZDP) and by delegation of teaching tasks they got served too. The delegation of teaching tasks happened by including reviews. Each student had to upload a task and for each task put in the system, two reviews had to be made. The setup is an exponential curve for delegating teaching tasks.

Like I only supervised the first tasks, now I only supervised the first reviews. After I reviewed the innovators I took a step back and only focused on the evaluation of the reviews. Enough tasks existed for the adopters to get into action doing peer-review. Because I hold the end responsibility my reviews didn’t need evaluation. Once peer-review started, I still had to evaluate the reviews. This is when the canon gets active. I could identify good reviewers and asked them to become my assistants. I still needed to accept those evaluations, so the end responsibility stayed with me, but the bulk of feedback came from peers. To separate the different waves in the canon, a prefix is required: the dev-wave (short for developers) and mana-wave (short for managers, in this case teaching assistants).

The dev-wave would always be one successive group before the mama-wave. The cannon starts with mama-innovators and dev-adopters. The total group of 64 of the best students was big enough to find those 10 mana-innovators. All are still in a zone of unattended learning, so it ran smoothly. Moving to the next successive groups we get to: mana-adopters, still in a zone of unattended learning. The early majority (34%, so 128 students) only needed some assistants and many had talent to become teaching assistants. The group grew smoothly to 60 assistants (almost all mana-adopters). For a first trial this was a great success. With the late majority we saw the limits of the setup. The mana-early-majority needed some support and the dev-late-majority needed a lot of support. Support that was not foreseen during the design of the course and had to be developed on the go.

The solutions made were good enough to wrap up the course. A summary of improvements for a second trial was made that would create advanced tools to mediate issues that arise late in the setup. Sadly, the conditions didn’t allow a second trial. The experiment was able to fix an acute problem and it helped to finish my PhD. Of course the people managing the faculty had created other solutions to fix the problem beyond this one crisis year. Indeed, the set up was not to actually transform our education system. It was a temporally quick fix. Comparing the scale experiment with the basic educational experiment does give some view on the expected impact. The second run would have been a great improvement and a third run would have created a stable solution. Yet it would reach a boundary and the weak signals to understand a bigger solution got picked up.

The unexpected dynamics that arose during the experiment allowed more insights into the maturity phase. In retrospect it is clear we need a third wave about remediation by governors. Some weak signals did occur, I had some of the assistants come to me reporting about serious abuse and misinterpretations. What should have happened is a kind of harvesting governors from the bigger group of teaching assistants. Part of the reason an upper limit is expected to the scaled education relates to the absence of a common project. Now all 400 students only focus on personal development, not creating the appropriate conditions for a wave of governors of the commons. With a common project it would become possible to investigate the gap between the majority and the opposition. Let me elaborate on the gap and the opposition by taking a closer look at rejectors and reactors.

In case of a course, the rejectors would drop out, so let me turn to the dropout numbers. The actual number of students is 600–300 depending on how you approach the data set and it relates to early and late dropouts. We had about 600 registered students. 200 (~33%) dropped out without any interaction and 70 students (~12%) never uploaded work to the peer-learn platform. I found 30 students (5%) who did upload once or twice, yet didn’t become part of the active students.

The effect of the opposition became clear as 30 people stormed the dean’s office in an attempt to cancel the course. The relation between the 30 opposition and 30 late dropouts is unclear, as data is missing. Investigating the event, I found out that about 10 students had gathered after my first class to take action. Those 10 reactors have been able to mobilize some of the rejectors in an effective opposition. A more refined data gathering would be required to learn more about the dynamics of reactors. Still it seems clear this is the power transformation we know from politics. A serious experiment for the maturity phase requires to include dynamics to regulate

the power transformation.

The dropouts are not customers of education, but potential customers, notice how terminology of the premature phase enters, it indicates we have to shift perspective. As the dynamics become more about politics and power transition the potential customers are considered civilians. Now data is looked at per capita, not by market share. The dropouts and students who need remediation are a more passive response to a power game. The reactors play an active game and could mobilise the rejectors, creating an effective opposition if needed.

The scale to see such dynamics of the maturity phase clearly, would again require a factor 10 increase, which brings us for education in the range of Massive Online Open Courses (MOOC). Just consider the dropout rate of 50% in the experiment. From the perspective of a classic course the rate is high, for MOOC the rate is low. To quote a study on MOOC: “The average MOOC course is found to enroll around 43,000 students, 6.5% of whom complete the course” Notice this how the numbers of student affect the completion rate a class of: 40 students with 99% completion, 400 students with 50% completion and 43.000 students with 6.5% completion. It shows we have not yet figured out how to effectively delegate education to a platform.

Education is only an aspect of the transition, in total three layers are recognized (from PhD): a huge base of R&D, a middle layer of education and a top layer for power transition. A small R&D team in the premature phase triggers a mobilization in two directions. Horizontally the development continues, vertical the research activates education. The power transition relates to the Collingridge dilemma so only activates in the incubation phase as the data of the R&D, explored by education, gives policy a tool to do its own transformation, which needs to get integrated in later transition stages. The three layer enterprise architecture could explain the transition of the pillars of a university. In the article on Interversity I elaborate how I’m mostly just a witness to an evolutionary process affecting universities. Trying to make sense of the dynamics by creating these sketches.

The sketch shows how an initial R&D project could evolve across the phases, mobilising different layers. Education is the second layer, starting with a small group during the incubation phase, turning to the level of peer-learning in the growth phase and eventually reaching the size of MOOC in the maturity phase. The middle layer got well enough understood, the question was how it can relate to the other layers. How would the R&D develop, not by self-organising market dynamics, but by transition management. Also the question was how to integrate the power transformation (i.e. top layer)?

To include policy makers requires a project that is on the political agenda. It also requires a new premature tool. So far we had pretotyping, where the prototype was artificial and dogfooding where the customer was artificial. Now we need artificial civilians, which brings us to an embodied agent-based model (ABM). To play a kind of simcity game, but for real (i.e. serious gaming). ABM can be used for any kind of complex adaptive study. The focus of embodied ABM is to eventually get integrated by real sensory motorical coupling. Like building real-time traffic navigation. For such tools historical data can be used to forecast how traffic will behave, but as events happen (e.g accidents, roadworks, heavy rain, large scale cultural events) the forecast needs corrections. The feedback from actual sensory input on the road needs to get integrated to adapt to the actual situation.

The challenge is how to build transition forecast models using embodied ABM. One research project exploring the question is the NEON project: New Energy and Mobility Outlook for the Netherlands. As of June 2012 I became a postdoc researcher for NEON. My role is modest, as the Point of Contact (PoC) I serve the partners. The past month I talked to most external partners, interacted with the most PhD candidates and recently began talking to the Principal Investigators (PIs). I expected to spend a long time participating and servicing before seeing how to make the experience relevant to my research. Little did I know how quickly it would turn to actual research on the sketch made above.

Just as the agile-education had a group of 30+ students, now the NEON group has 30+ PhD candidates. In an abstract way the whole setup done with master students is re-emerging. The complexity does make everything more challenging, but it seems the right change as anticipated. This time the project is about transition and it goes deep into R&D, so not just potential innovation, but fundamental innovation too.

Unexpectedly, only after two months servicing the NEON group, a fundamental insight arose about resolving the tragedy of the commons. The complexity shows resilience that could “fix” the tragedy of the commons in a similar way as the open innovation got “fixed” by complexity: far from equilibrium we witness self-organisation. It happened during some of the partner meetings that required an answer to emerging tension, resulting in a reconfiguration, which only was possible thanks to the complexity of the project. The dynamic became possible as incumbents are part of the group. So incumbents cannot disrupt the ecosystem after incubation of the innovation, as they are involved from the beginning.

The NEON project is currently in the premature phase doing research and developing the embodied ABM. Weak signals indicate it might evolve as the sketch anticipated, yet as it goes with science in action, the actual feedback will decide how it all develops. So to wrap up this article, let’s zoom out and try to get some insights about the gap between majority and opposition.

The increased gap between majority versus opposition needs a name, let me propose the integral sinkhole. It is a power transition and the power game has been picked up by market dynamics as disruptive innovation. At the beginning, disruptive innovations rejuvenated the markets. Innovators used the integral sinkhole to their advantage to overthrow the incumbents. As incumbents start playing the disruptive game, the integral sinkhole shifts from the market to our society. If the incumbent collapses the market rejuvenates, the effect is positive for customers and negative for the incumbent shareholders. If society collapses, we all are shareholders, so only the negative effect is recognized.

The past decades incumbents have used the disruptive game to take over power at the level of nations by a divide, conquer and lobby mechanism. All dynamics increase the fragility of our society and make clear what the integral sinkhole is about (e.g. financial crises, climate crisis, heath crisis, etc). Most recently everyone has experienced the pandemic how fragile our systems are and how easily we can lose all freedom under the assumption it is for the common good. If you study the cynical game of power, you learn it is all about controlling the narrative. Controlling the narrative appears the first step in civilian experience, let me give an example.

In recent scandals related to the war on drugs narrative, it has become clear how cunning policymakers figured out a way to strongly come down on minorities, while creating the public illusion the methods were acting for the common good. In reality it was incumbents playing the disruptive innovation game on minorities. Disruption and scandals are places to recognize the system dynamics clearly, but the dynamics are not negative in themselves. In fact a lot of good speeches have built the foundations of our civilization by holding on to the narrative. Public speeches create a narrative about the future and it creates potential innovation. How to ensure the vision gets integrated appropriately is the bigger challenge.

As history shows us, the movement that arises can turn quite volatile and may end up in exactly the opposite as the initial intention of the narrative. Like communism was intent to resolve poverty, not increase it equally. From my narrow observations and limited experiments some weak signals do make me assume the problem lies with the canon of successive waves. The challenge will be to actually do experiments at the right scale with enough people involved. To finish this article let me go back to the peer-learn experiment and share my anticipation.

In the peer-learn experiment two waves got active: developers needing customer experience and managers needing employee experience. The system broke down as a third canon had to start in relation to governors needing civilian experience. A narrative is the potential innovation to get civilian experience and the fundamental innovation the NEON project is working on is to have an embodied ABM tool for policymakers to go beyond just the narrative and simulate civilian experience. It is interesting to see how NEON is currently in the premature phase as fundamental research, but how it has the potential to trigger the two directions expressed in the sketch above.

A recent meeting with the PhD candidates about how to integrate their teaching tasks in the research got both me and my postdoc colleague excited about the potential. Adding to it the fascinating talks with partners in relation to the cultivation of the commons and the PIs in relation to research on transition management. Of course many things can go wrong, but the tension that will arise is the feedback that will correct the sketch into an actual system architecture. If the NEON project will help us to gain more understanding about civilian experience is an active research question. What seems clear is we are at the most likely spot to find the weak signals and act upon them.