Tag: metrics

Posted on by

Bring Your Own Dashboard

For more information

Listen to Michael Schrage, research fellow with MIT Sloan School’s Initiative on the Digital Economy, describe the need for strategic KPIs, common pitfalls organizations encounter when grappling with new technologies and why co-creating dashboards in a workshop setting are fundamental to AI capability development.

The game

Read through your favorite Management publication today and you’re likely to find an article on Artificial Intelligence. One we particularly liked, Strategy For and With AI, clarifies the difference between using AI in your product or service (Strategy *For* AI) and harnessing AI to plan your strategy (Strategy *With* AI). The latter involves a tight coupling of KPIs, Data Governance and Decision rights that few companies can claim – the article recognizes Google, Uber, and GoDaddy. While these case studies may reveal an intimidating gap between their achievement and your organization’s progress, a discussion with article co-author Michael Schrage reveals simple steps any organization can take to move forward on this path.

AI with strategy works best with a strong foundation of established KPIs, mature Data Governance practices and clear Decision Rights
Object of Play

Most teams, regardless of size, can access data measuring their progress towards goals. Use this group activity to validate the strategic alignment of your KPIs, understand the relationships between them, and brainstorm tests you can perform to validate both.

Number of Players

5-15

Duration of Play

90 minutes

How to Play

OPEN

There are two options for opening this game:

Option A

If your participants have defined KPIs or OKRs which they currently measure:

  1. Ask them to bring a sample report or dashboard to the activity
  2. Once gathered in the room, ask each team or individual to briefly present their report, identifying:
    1. Which organizational strategies the report aligns to (OKR)
    2. Report KPIs (OKR)
    3. Other important reporting metrics
  3. Once all the reports have been presented, ask the teams to write down their KPIs and metrics, one per sticky note and put them up on the wall.

Option B

If your participants have less defined measurement and feedback infrastructure OR you’re looking to explore new measures and KPI’s

  1. Inform the players the purpose of the activity is to explore our strategy by creating a dashboard
  2. Write at the top of the whiteboard an organization-wide strategic goal
  3. Ask the players to take five minutes for an individual brainstorm: list all the customer behaviors impacting the strategic objective of your organization. For example,  a digital marketing team may be concerned with: customers signing-up for the newsletter, shoppers visiting your website, follow the brand on Twitter.
  4. At the end of the brainstorm, ask each player to put their sticky notes on the wall, quickly presenting them to the team one-by-one. 
  5. Once all the brainstormed ideas are on the wall, ask the group to organize them into themes. Let the themes emerge organically, i.e. don’t guide or direct their behavior. 
  6. Take 5 minutes to review each theme; ask for the players to briefly explain their thinking and insights. 
  7. For each theme ask the group to identify one or two KPIs that best measure the desired consumer behavior. 

EXPLORE

KPI relationship matrix
A KPI relationship matrix
  1. Let the players know you’re going to explore your KPIs by looking at their relationships to one another. 
  2. Set the board by creating a matrix of the KPIs identified in the Opening. 
  3. To play, the group determines the relationship between each set of KPIs: Direct if an increase in one would cause an increase in the other OR Indirect if an increase in one would cause a decrease in the other.
  4.  A group may choose to write down relationships individually at first and then call out their results on each item and criterion to create the tally. 
  5. Identification should be done quickly, as in a “gut” check.
  6. A discussion after the table has been completed may uncover uncertainties about strategy and KPI maximization. For example, Marketing’s attempt to maximize website visitors may negatively impact Sales’ conversion rate. 
    1. Some questions to prompt: 
      1. What do you notice about these relationships? 
      2. What KPIs should you consider adding? Removing?
      3. Are there instances where KPIs should be optimized instead of maximized?
      4. Are the representative strategies aligned? Do the KPIs indicate any conflicts? 
      5. What other organizations are implicated by these KPIs?
      6. What are we uncertain about? How might we test those uncertainties in the next week?

CLOSE

Based on your discussion perform a Start-Stop-Continue. Ask the group to consider the customer behaviors, KPI relationship mapping and subsequent discussion and individually brainstorm in these three categories:

  1. Start: What are things that we need to START doing?
  2. Stop: What are we currently doing that we can or should STOP?
  3. Continue: What are we doing now that works and should CONTINUE?

Have the individuals share their results.

STRATEGY

In our data-rich world, your strategy is what your KPIs say it is. Teams often try to maximize KPIs in the absence of understanding their impact. This exercise clarifies ripple effects strategies have on each other and surfaces considerations for when Optimization should trump Maximization. 

CREDITS

This game was inspired by David Kiron and Michael Schrage’s MIT Sloan Management Review article, Strategy For and With AI

COMPLEMENTARY GAMES