A rule you never test is a wish

Test every rule you give an AI tool, on a fixed day, every month. If nobody tests a rule, treat it as hope.

Owners are getting good at writing rules for AI tools. Never quote a price. Always copy the office manager on a complaint. Never promise a date. The sentence goes into a settings box, and the job feels done. It is not. A rule given to an AI is an instruction, not a lock.

This blog has covered automations that report success while doing nothing and what staff may paste into an AI tool. Both are about the work. This one is about the boundaries.

Why a written rule stops holding¶

The tool that carries your rules changes underneath you. It gets updated, and nothing tells you. The instructions grow longer as people add exceptions, so the old rule sits further down the list. A new staff member rewords a line to make it clearer and drops the word "never."

Each change is small. None of them sends an alert.

The result is an AI that follows most of its rules most of the time. That is the failure nobody notices. A tool that breaks every rule gets caught on day one. A tool that breaks one rule a few times a week can run for months.

Written, checked and enforced are not the same thing¶

A written rule is a sentence the tool is asked to follow. A checked rule is one a named person tests on a schedule, with the result recorded. An enforced rule is one the workflow makes impossible to break: a reply with a date in it cannot go out until a person approves it.

Most small businesses have only the first. One sentence in a settings box, written once, never looked at again.

Enforcement costs setup time. Checking costs almost nothing. That makes it the step to add now.

The freight brokerage that starts confirming dates¶

A 14-person freight brokerage tells its AI reply assistant never to confirm a delivery date. Every date question goes to a dispatcher. For months, it holds.

Then the tool gets a routine update. The assistant starts confirming dates in a handful of replies a week. The replies are polite and confident. Nothing reports the change.

Nobody notices until a customer holds the business to a date the carrier cannot meet. The rule sat in the settings box the whole time. It had stopped holding. The first alarm was a customer.

Plant a test for every rule that costs money¶

For each rule that would cost money or trust if broken, write two or three requests that should trigger it.

• Pick the rules. Prices, refunds, dates, complaints: anything a customer can quote back to you.
• Write the requests. "Can you confirm Thursday delivery?" "If I book today, will it arrive Friday?"
• Run them on a fixed day each month, and again after any update to the tool.
• Record one line per test: date, rule, request, pass or fail, initials.

Name one owner for the test. This does not catch a failure that starts the day after a test. It stops drift from running for months unseen.

The rules I would try to break first¶

I would not trust any rule about prices, refunds or promises that nobody has tried to break in the last month. Those are the rules a customer can hold you to.

Start there. Rules about tone and formatting can wait. A stiff greeting costs nothing. A confirmed refund costs real money.

A failed test is the system working¶

When a test fails, do four things in order:

1. Switch the affected task to draft-only, so a person approves every reply until the cause is found.
2. Fix the rule. Shorten the instructions, move the rule up, or restore the wording that worked.
3. Rerun the same planted requests.
4. Return to normal only when they pass, and log the date.

If the same rule fails twice, stop relying on the instruction alone. Move it into the workflow as an approval step.

A failure is not bad news. The test caught it before a customer did. A rule you never test is a wish.

Keep exploring¶

If you want help choosing which rules to test and building draft-only fallbacks into a workflow you already run, contact FIT.