This prototyping activity demonstrated how an emerging learning technology could be used and integrated with existing infrastructure to allow the long-term tracking of learning of individuals and teams in Army training.
Manifested as an Integrated Organisational Learning System (IOLS) this project showed how the broader utilisation of analytics can: improve insights from training, inform what training is needed in the future at the individual level, and how more advanced data-science methods could predict future performance. Understanding how best to exploit capabilities like IOLS is critical to providing decision-makers with evidence-driven assessments needed to maximise the availability of qualified and experienced personnel cross Defence.
In Jan 19 we submitted an ‘Open Call’ to the Defence and Security Accelerator (DASA) to enable Cervus to demonstrate a Force Development concept called the Integrated Organisational Learning System (IOLS). In Jul 19 we were awarded £246K to enable us to develop IOLS by building and testing an application for our analytics engine called Hive™.
Facilitated through Dstl’s Transforming Training, Education and Preparation (TTEP) project, over 12 months, we developed and tested (with the End-User at the British Army Training Unit Kenya (BATUK)) a system and processes which could support the Army’s vision for Collective Training becoming an adaptive and responsive lessons loop.
The specific research questions used to guide this project were:
- How best can the training transformation programmes help maintain advantage through the optimisation of people, equipment and processes?
- What is the best simulation, learning technologies and data analytics methods available to the Army to measure its ability to adapt?
- What are the benefits of integrating individual and collective performance data to provide richer data sources into the Future Collective Training System (FCTS) lessons loop
We decided to adopt a more user-centric approach to prototyping called Lean Start-up. We normally use this approach when developing analytics tools as it is cost effective and ensures we build products that are easily adopted through a relentless customer perspective on design and functionality.
The Lean Start-up method considers experimentation to be more valuable than detailed planning that focuses on the entire proposition and business model, and importantly not just the technology solution.
Also, we are comfortable with this method as the Lean Start-up provides a scientific and procedural approach to creating and managing innovation and get a desired product to customers’ hands faster. The Lean Start-up method teaches you how to drive innovation – how to steer, when to turn, and when to persevere – and grow an idea with maximum acceleration. It is a principled approach to new product development.
Importantly, as a Small Medium Enterprise (SME) the most organic principle of any lean start-up is its ability to utilise resources in the most efficient way possible. Most start-ups don’t have the benefit of unlimited financial resources, so the lean business model encourages the controlled deployment of resources that we do have.
The fundamental activity of any start-up is to turn ideas into products, measure how customers respond, and then learn what to pivot or preserve. Instead of business plans, lean start-ups use a business model based on hypotheses that are tested rapidly. Data does not need to be complete before proceeding; it just needs to be enough. When customers do not react as desired, the start-up quickly adjusts to limit its losses and return to developing products consumers want. Failure is the rule, not the exception.
We conducted this project through a 3-stage sprint process, delivering outputs and improvements incrementally throughout the prototyping processculminiating in a field trial in Kenya with 3 PARA.
- We believe that IOLS would improve the assurance of preparation of force elements trained to meet readiness with improved articulation of the training risks/delta (both individual and collective).
- We believe that IOLS would enable employment of Integrated performance metrics combined with the means to collect and objectively analyse training data. This would specifically enable the articulation of risk and will enhance Divisional capabilities, responsiveness (reduced Notice To Effect) and contribute to resilience at the individual level if aggregated at the right level.
- We believe that IOLS would improve the contribution to future training delivery, capability development and wider WARDEV, acting as a capture instrument for a laboratory for adaptation through greater fidelity of data sources (to the individual level). Thus, enhancing the level of evidence for experimentation and adaptation, thereby increasing the effectiveness of the Army’s force preparation.
- We believe that IOLS would enhance the adaptability and flexibility of training delivery at the lowest level.
- We believe that IOLS would increase responsiveness in the design of individual and collective training.
- We believe that IOLS is well suited to soldier personalisation and as such would enrich the learning experience and empower subordinates to support adaptive organisational learning and change
Prioritised observations and findings are:
- We identified aspects of IOLS that would enhance the Collective Training Transformation Project (CTTP) Benefits as outlined in the FCTS Blueprint and to enhance the adaptive and responsive lessons loop.
- We created evidence to develop and inform future Training Measurement and Evaluation (TME) and FCTS Requirements
- We applied Experience Application Programme Interface (xAPI) in both the lab and field, and we believe that it is a suitable common data standard to enable the link between educational, individual and collective training data, although some modification will be required.
- We demonstrated descriptive analytics but not predictive or prescriptive analytics during this project owing to the size of the data set gathered.
- We identified that the Order of Battle (ORBAT) store is key in linking the enterprise data points to an individual and collective data.
- We identified a key dependency for this level of data to be accessible by TME providers in FCTS as being Ministry of Defence (MoD) policies on access to and use of personal information.
- We concluded that future FCTS dashboards need to be tested with end users and should be adaptable, but that this project provides a start point for this work.
- We identified that the Future Defence Learning Environment (DLE) requirements should enable the aggregation of individual data and integration into the Army Data Warehouse. In addition, FCTS should consider using DLE as the system’s Learning Records System (LRS) if it adopts an IOLS-like approach to TME.
Cervus gave an exemplary performance despite both some setback and challenge; they delivered against every aspect and exceeded every expectation. Moreover, they proved ambassadors for the partnering approach adopted by DASA – offering feedback to DASA/Dstl that I fervently hope will (and absolutely should) shape partnering arrangements between MOD and SMEs in the future. Ultimately, a highly successful project that has coupled supporting a risk-taking innovative UK company serve UK interests, and also offered to MOD empirically derived recommendations on how to successfully embrace Data Science / Machine Learning approaches to Defence training.”