Core•6–10 hours

AI Use-Case Scoring Board

Build an AI use-case portfolio with evidence-based scoring across value, frequency, data readiness, capability fit, automation potential, failure impact, controls, and experimentability.

use-case discoveryportfolio scoringevidence analysisrisk framingprioritization

Scenario

Task

After workshops, the team has 20–40 AI ideas: summarization, document extraction, support, recommendations, agentic actions, and forecasting. The list has quickly become a contest for the most charismatic slide. Build a scoring board that separates evidence from assumptions, penalizes unknown failure impact, and prevents vendor demos from substituting for your own business case.

Step-by-step execution

1. Normalize use cases to one contract

Outcome: Ideas are compared as jobs to be done rather than as model or product names.

Tasks

→Describe the user/job and measurable pain
→Record the current workflow and non-AI baseline
→Define frequency/volume and the source of truth
→Separate the model task, workflow automation, and consequential action

Checks

✓The use case does not start with a provider or model name
✓Value has a business signal rather than “saves time” without a baseline
✓Action scope and failure impact are visible before scoring

2. Build scoring that does not reward uncertainty

Outcome: Unverified assumptions do not receive the highest score merely because data is missing.

Tasks

→Define 1–5 scales with anchor definitions
→Add evidence confidence and an uncertainty penalty
→Score capability fit and control burden separately
→Define hard blockers: prohibited data/action, unavailable ground truth, or unacceptable failure impact

Checks

✓Unknown does not equal a high score
✓A hard blocker cannot be arithmetically offset by a high value score
✓Vendor metrics are labeled as external context rather than internal evidence

3. Run an adversarial review of top candidates

Outcome: Priorities survive counterarguments and a transfer-risk check.

Tasks

→Create a failure pre-mortem for the top five
→Find a simpler deterministic alternative
→Check data and permission dependencies
→Assess evaluation feasibility and time-to-evidence

Checks

✓Every top candidate has a reason why AI is better than simpler automation
✓There is no hidden dependency on unavailable data or permissions
✓Testable forbidden outcomes are defined

4. Turn the ranking into an experiment queue

Outcome: The priority list has a next action, owner, and stop condition.

Tasks

→Define the smallest valid experiment
→Build a representative sample
→Record the baseline and target
→Add go/iterate/kill criteria and a re-score date

Checks

✓Every top candidate has a next evidence request
✓The kill criterion is defined before the experiment result
✓Re-score triggers include new data, an incident, or a provider/model change

Acceptance criteria

All use cases use the same problem/workflow/action contract
Every score has an evidence source or an explicit assumption label
Hard blockers are not hidden by a weighted average
Top candidates have a non-AI baseline, failure pre-mortem, and evaluation plan
The experiment queue includes an owner, sample, target, and go/iterate/kill criteria