7. Results – Calculations

Basics

Calculations derive a numeric value from team member votes using basic arithmetic. During an assessment, team members can choose from one of 5 Agility Stages for each Practice.

Each of these Agility Stages maps to a numeric value according to the number following the word in the above screenshot. For example, Blocked maps to 0 and Emerging maps to 2.
These numeric values are used to calculate an average on a team/group/organization level that can be again mapped back to the Agility Stages using the legend below.

Scenario

Consider a scenario where a team took one assessment and in that assessment there were 3 Practices, 3 team members, and 1 facilitator. Since the facilitator is there to facilitate and not to participate, the facilitator did not cast a vote for any of the practices. Therefore, there are only 3 votes per Practice to be considered for calculations.

Practice Votes
Learning 2, 4, 1
Team Stability 0, 0, 3
Agile Planning 2, 4, 3

Practice level

To get an agility stage for a given practice, the average of all votes received is calculated .

Practice Agility Stage (Average)
Learning 2.33
Team Stability 1
Agile Planning 3

These values are displayed as dots and trend lines in the Insights Canvas and the agility stage is available in the Insights Canvas drill down.

Overrides
During an assessment, there is an option to override the votes of the team and choose a single vote. In this case, the chosen override vote will replace each user’s vote for calculation purposes. For example, if 3 team members vote on a Practice and an override vote of Emerging is chosen, then those 3 original votes are converted into 3 votes of Emerging.

Group/Organization level
At the group and organization level, all individual votes are averaged. It doesn’t matter how team members are distributed across the teams or groups.

Variability

The variability for a given Practice is the standard deviation of all votes being considered.

Standard deviation range Variability (number of dots)
< 1 1
1 – < 1.49 2
1.49+ 3

For the group and organization level, all votes across all teams are considered.

Overall average

The overall average is the number that appears in the center of the Insights canvas. This number is calculated by averaging the scores from each Practice. So using the scenario above and averaging the nine votes gets a score of 2.11.

Note that this number is rounded to the nearest tenth for display in the middle of the Insights Canvas.

Group/Organization level
At the group and organization level, all votes across all entities are averaged.

Business Outcome level

The Business Outcome level is calculated the same way as the Overall Average except that only Practices that are linked to the Business Outcome are included. Below is a list of the three Practices and their linked Business Outcomes.

Practice Business Outcomes
Learning Employee Satisfaction, Innovation
Team Stability Time to Market, Customer Satisfaction, Employee Satisfaction, Innovation, Reliability, Responsiveness, Predictability
Agile Planning Time to Market, Customer Satisfaction, Responsiveness, Predictability

Also, remember the averages per Practice that were calculated. Here they are for reference:

Practice Agility Stage (Average)
Learning 2.33
Team Stability 1
Agile Planning 3

To calculate the Time to Market Business Outcome score, average all Practices linked to Time to Market, which is Team Stability and Agile Planning. The table below shows the results for some Business Outcomes.

Business Outcome Agility Stage (Average)
Time to Market (1+3)/2 = 2
Reliability 1

Group/Organization level
At the group and organization level, all votes across all entities are averaged.

Trends

Trend lines are the lines that connect the per-practice score dots on the Insights Canvas. These trend lines behave differently depending on which level of the organization is being viewed.

entity Level

  • A maximum of four trend lines are shown.
  • The trend lines are the date of the very first assessment taken and the dates of the three most recent assessments taken.
  • The trend lines show the latest score for each practice that is on or before the assessment date.

The first assessment trend line is always displayed so you are able to measure the amount of improvement seen since the first assessment baseline was established. The three most recent assessments allow you to see the most recent changes.

Examples

  1. A team took 2 assessments on 7/6/2020, 3/4/2020
    Trend lines: 7/6/2020, 3/4/2020
  2. A team four took assessments on 7/6/2020, 3/4/2020, 11/15/2019, and 6/4/2019.
    Trend lines: 7/6/2020, 3/4/2020, 11/15/2019, and 6/4/2019
  3. A team took six assessments: 7/6/2020, 7/5/2020, 7/4/2020, 7/3/2020, 7/2/2020, 7/1/2020.
    Trend lines: 7/6/2020, 7/5/2020, 7/4/2020, 7/1/2020

GROUP/ORGANIZATION Level

  • A maximum of four trend lines are shown.
  • The four trend lines are displayed/not displayed in the following order:
    • First Assessment – Include the first assessment taken for each entity, regardless of date the assessment was taken. This trend line is always labeled as ‘First Assessment’.
    • Last Assessment – Include the latest assessment taken for each entity, regardless of the date the assessment was taken. If this is the same data as ‘First Assessment’ then do not show the trend line. Label it as the latest calendar quarter a response was received (for example, 2020 Q3).
    • Previous Quarter Assessment – Include the latest assessment taken for each entity that has a latest date of the last day of the calendar quarter before Last Assessment. If this is the same data as ‘First Assessment’ or ‘Last Assessment’ then do not show the trend line. Label it as the latest calendar quarter a response was received.
    • Last Year Assessment – Include the latest assessment taken for each team that has a latest date of the last day of the previous calendar year before Last Assessment. If this is the same data as First Assessment or Last Assessment or Previous Quarter Assessment then do not show the trend line. Label it as the latest calendar quarter a response was received.

The First Assessment trend line is always displayed so you are able to measure the amount of improvement seen since the First Assessment baseline was established. The Last Assessment trend line shows the latest responses to know where you are today. The Previous Quarter and Last Year assessments provide a good representation of recent improvements.

Examples

  1. A group of 3 entities – each entity took one assessment on 3 different dates – 7/4/2020, 5/4/2020, 11/12/2019.
    Trend lines: First Assessment
  2. A group of 3 teams – two teams took one assessment – 7/4/2020, 5/4/2020, the other one team took two assessments – 7/3/2020, 5/12/2020.
    Trend lines: First Assessment, 2020 Q2, 2020 Q3
  3. A group of 3 teams – two teams took one assessment – 7/4/2020, 3/4/2020, one team took two assessments – 7/6/2020, 1/4/2020.
    Trend lines: First Assessment, 2020 Q1, 2020 Q3
  4. A group of 3 teams that took assessments for every quarter in 2020 and every quarter in 2019.
    Trend lines: First Assessment, 2019 Q4, 2020 Q3, 2020 Q4
  5. A group of 3 teams that took assessments for 2020 Q1 and every quarter in 2019.
    Trend lines: First Assessment, 2019 Q4, 2020 Q1