A goal of “adopting AI” is not enough to evaluate a project’s results. Pin down which step of which task AI will support, and check at a small scale what results users can accept and what operating conditions are needed.
01. Choose one unit of work
Pick candidates from tasks where you can define the result and who reviews it, such as document classification, question answering, drafting, and checking video events. Try to define the task’s input and output in one sentence.
Even in work that staff handle again and again, set up a separate review procedure for the parts with many exceptions or with decisions that someone must be accountable for. Decide which steps are processed automatically and which are handed to people.
02. Separate data to use from data to exclude
Check whether data that fits the purpose exists, and whether you have the permissions and conditions needed to use it. If it includes personal information or sensitive business material, first agree on what will be provided and how it will be handled.
In early stages, when real data is hard to use, you can review the workflow with anonymized material or a limited sample. Whether results from the sample hold in the real environment needs follow-up validation.
03. Agree on criteria for success and failure
Don’t look only at time saved. Also check the accuracy of results, their basis, the review workload, and exception handling. Agree with the people in charge on which results to accept and which need correcting.
Prepare not only inputs you expect AI to handle well but also cases with too little material and inputs in other formats. Record the limits you find and use them to set the scope of the next step.
What to prepare
- Representative inputs and expected results
- Conditions that require staff review
- A path back to the existing process when results fail or are held back
04. Design where people check the results
Decide who checks a summary or draft before it is sent outside or before it changes a business system. Consider screens where users can compare the original with the processed result and correct it.
If the setup calls work tools, make clear which functions are allowed and how far changes can go. Also agree with the operations staff on how approval history and execution results are recorded.
05. Decide the next step from the pilot results
The validation results should let you decide whether to scale up, refine, or stop. Review the follow-up scope, including further data preparation, user training, integration with existing systems, and operating costs.
Confirm that the results are good enough for users to actually work with, and expand the scope in stages. Also record the items to check again when the model or data changes.
That’s all you need before a consultation.
If you prepare the items below, we can align on scope more quickly in early discussions. Even if not everything is ready, we can start with your current situation.
- One task to validate and its owner
- Available materials and data scope
- Criteria for judging results and representative cases
- Review, approval, and exception-handling procedures
- Conditions for deciding to scale up or stop
KPS project guide · The specific technical setup and conditions depend on your requirements.
