Enterprise AI research

How to Test AI Operating Capability in One Workflow

·
5 min read
Amber Liu standing by a window in a white shirt

To understand whether AI can improve operations, choose a recurring workflow with clear boundaries and an observable result. For example, a client submits a service request, operations verifies the inputs, a team proposes a response, and the client confirms whether it is usable. The workflow has a start, an end, and a real recipient. That is enough to test whether quality and speed improve together.

Define a measurable pilot

Limit the pilot to one request type, one team, and a fixed observation period. Record the current completion time, first-pass acceptance, rework, human takeovers, and whether recipients use the output. Tell participants which requests are excluded, who grants access, and when the pilot must stop.

A workable target is to reduce the time from complete inputs to the first acceptable response, without reducing quality or increasing the client’s wait or the team’s review costs. The same measurement applies before and after the change.

Set handoff and takeover rules before starting

For input verification, require a checklist with sources and unresolved items. AI can extract details and flag contradictions. The responsible person verifies key fields and decides whether to request missing information or proceed. Once the checklist passes review, confirm the next team is authorized, available, and has what it needs.

Require human review for high-risk fields. Pause automation and notify the owner if the system cannot explain a source, loses authorization, or repeats the same error. Record what triggered the handover so the team can tell whether the problem was the data, method, or permissions.

Review the evidence behind average speed

Compare similar cases. Look separately at speed, acceptance, downstream use, and client outcomes. Review exceptions on their own. A rare but costly error can matter more than several routine time savings. If the observation window is too short, collect more evidence before deciding.

Connect the step, task, output, and version of the standard with identifiers. A reviewer should be able to find the basis for a decision and distinguish what the responsible person controlled from what depended on others. Use that distinction when reviewing performance.

Require repeatability before expanding

Record what worked in role descriptions, work instructions, evaluation rules, and AI skills. Then have another colleague handle a comparable case. Check whether they meet the same acceptance criteria without unwritten instructions from the original pilot team.

Expand to a similar workflow when delivery is stable, material risks are controlled, and others can repeat the work. If rework or costs rise, adjust or stop. The pilot can show what happened under its conditions; it cannot establish that the whole enterprise has rebuilt its operating capability.

Research reference: Andrew David W.Y., Rebuilding Enterprise Operating Capability with AI (exploratory V2.0).

Amber Liu
Stakeholder capital theory researcher
Pick one workflow the team already understands. Agree upfront on what counts as a usable result, when to stop, and how someone else will repeat the work.
Amber Liu · Research and consulting

frequently      
 asked questions

Does deploying more AI tools improve operating capability?

Check whether outputs passed review, whether the next team used them, and whether business results changed within the agreed observation period.

Which pilot decisions need human approval?

External commitments, material risks, and access changes should be decided by authorized people. Define who takes over and when the pilot must stop before starting.
Discuss a workflow with us