Give your AI a test folder before giving it live work

If an AI draft promises a customer an appointment without checking the calendar, I would fail that test. Friendly wording would not change my decision.

That is a hypothetical example, but it gives us a useful place to start. Someone has to decide what makes an answer acceptable before the team starts using it.

I would bring a finished piece of work to that conversation. A request your team already handled, the information available at the time, and the answer someone approved. Use a sanitized copy or a fictional version that preserves the business problem. Keep anything confidential out of a tool that has not been approved to handle it.

Now you have something concrete to compare. You can see whether the AI understood the request, used the right information, and produced work your team could use. You can also see where a person still needs to step in.

For a small business getting started with AI, I would make this the first assignment. Build a small test folder before trying to roll the tool out across a department. Give the person who does the work a say in what belongs in that folder and what a good answer needs to contain.

Start with work you can judge

Choose a task with a clear finish. Drafting a reply to a routine customer question is easier to assess than an open request to improve customer service. Turning approved meeting notes into an internal action list gives you something specific to check. Asking for a better business strategy leaves much more room for people to disagree about what good looks like.

I would choose work familiar enough that someone can explain a bad answer. If nobody can say why the output is wrong, you will struggle to improve it or decide when to trust it.

Imagine a service business testing drafts of customer replies. A fictional customer asks whether an appointment can be moved. The team has an approved rescheduling policy, but the appointment calendar is not part of the test. A useful draft explains the next step without claiming that a new time has been reserved.

That distinction gives you a testable boundary. The AI may help write the reply. It cannot confirm a booking from information it does not have. Write that boundary down before looking at the result, so a friendly tone does not distract you from an unsupported promise.

Keep the answer separate

Inside the folder, save the request and the material the AI is allowed to use. In a separate file, save the approved answer or a short description of the required result. Keep that second file out of the prompt for the test.

An approved answer can help you explain your standards during training. For a useful comparison, though, the tool also needs to handle a request without being shown the answer it is supposed to produce. Otherwise you are checking how well it copies the example.

Make the input realistic. If employees normally receive a brief email, test with a brief email. Do not quietly turn it into a perfectly organized brief that nobody has time to prepare during the workday. Any preparation you do to make the input usable belongs in your assessment of the effort involved.

Include the relevant policy, form, or reference note. Use the version that applies to the example. If the task requires information that is missing, leave it missing and record what you expect the tool to do about that gap.

I would keep the first review simple. A short note beside the approved result can be enough: the draft must explain the next step, preserve the customer's request, and avoid promising availability. Those are decisions the person handling the work can check.

Include an awkward request

An easy example belongs in the folder. So does a request that would make an experienced employee pause.

For the fictional rescheduling task, the customer might ask for a time outside normal hours. Or the message might omit which appointment they mean. A separate example could contain conflicting instructions, such as asking to keep the original appointment while also requesting a replacement.

These are hypothetical test cases, not reported customer incidents. Their purpose is to make the expected behavior clear. Should the draft ask a question, explain the policy, or hand the request to someone who can decide?

I would write the expected response to uncertainty into the task instructions. For example: if the appointment cannot be identified from the supplied information, ask for the missing detail. Do not choose an appointment on the customer's behalf.

That is a more useful direction than telling the AI to be accurate. It names a situation and the action you want. It also gives the reviewer something to look for when the answer comes back.

Keep the exercise small enough to review carefully. You can add examples as you discover meaningful differences in the work. A folder full of near-identical requests is less useful for this purpose than a few cases that expose different decisions.

Mark the correction, then explain it

I would review the draft with the input and expected result open beside it. Mark the exact sentence or missing detail that would stop you from using it. Avoid giving the whole answer a vague label such as good or bad.

If the draft promises a booking, underline the promise. If it misses the customer's question about access to the building, note the omission. If the facts are correct but the wording sounds too formal, treat that as an editing issue. These problems call for different fixes.

A missing policy may mean the tool needs better reference material. A promise outside the task's authority may need a clearer boundary and a check before sending. Repetitive wording may need a short example of the tone you want.

Ask the employee reviewing the answer to explain the reason for the correction. The reason is often the part that needs to become an instruction. A replacement sentence alone may hide the business rule that made the original unacceptable.

Keep the first output as well as the corrected version. When you change the prompt or reference material, rerun the same case and compare. You want to know whether the specific issue improved and whether the change created a different problem elsewhere in the answer.

Count the work around the draft

I would keep counting the effort after the answer appears on screen. Include the effort of finding the source material, preparing the input, checking the result, making corrections, and moving the final version into the place where it will be used.

You can keep a simple record while testing. Note who prepared the request, who reviewed it, what they changed, and whether they could finish the task. If you measure time, use your own observations. Do not borrow a savings claim from a vendor demonstration and apply it to your business.

The comparison should use a similar task handled through your current process. An unusually tidy test request and a difficult live request are not a fair comparison. Record the differences if the examples cannot be matched closely.

Also pay attention to who carries the effort. If a draft becomes faster to produce but requires the owner to check every detail, that matters. The owner may still choose that arrangement for a sensitive task, but the decision should be visible.

At the end of the test, you should be able to describe what the tool helped with, where people had to intervene, and what remains uncertain. There is no need to turn a mixed result into a success story. A clear account of the remaining work is useful.

Make the handoff easy to follow

Before trying the task with live work, write down who reviews the draft and where an unresolved question goes. Give the reviewer the source material alongside the answer. A person should not have to search through a conversation history to find the policy they are being asked to verify.

For the rescheduling example, the draft could remain in a review queue until someone checks the calendar and confirms the correct response. A question outside the policy could go to the person who owns that decision. The test should show whether those handoffs are understandable.

I would keep the initial scope narrow. If you tested drafting, do not quietly add permission to send messages or change appointments. Those actions introduce different consequences and need their own checks. Expand the task deliberately when the earlier results justify another step.

Write a brief instruction for the person covering the work when the usual reviewer is away. They need to know what to check, what they may approve, and when to stop. If that explanation is hard to write, the workflow may still depend on knowledge that has never been made explicit.

Save the folder for the next change

Keep the test inputs, expected results, outputs, and correction notes together. Give each round a clear label that identifies the instructions and reference material used. That makes it possible to compare the same work after a change.

When you revise a policy or change the tool's setup, return to the folder. Run the familiar cases again. Add a case when a new kind of request reveals something the original test did not cover.

I would not take a clean test as proof that every future answer will be right. It is a practical way to avoid judging each change from a fresh demo with no record of what happened before. Your team can point to a specific request and explain what improved, what got worse, and what still needs attention.

If you want a useful starting point for AI this week, ask the person closest to a recurring task to choose a completed example and describe an acceptable answer. Sit with them while you compare the draft. Listen to the details they notice.

That conversation gives you the beginning of a working standard. Save it with the example. You can use it the next time someone asks whether the AI is ready for the job.