riften
← All articles

Where AI is worth the effort.

Before buying an AI tool, sit beside the person who does the work.

Watch them prepare a client brief, answer an enquiry, or assemble a proposal. Notice what they copy between systems, what they look up twice, and where they stop to ask someone else. Those details are more useful than a list of things a model can do.

For a small team, a useful first project should remove a specific burden without creating another one. That means understanding the task well enough to know which parts to change and which parts to leave with the person responsible.

Follow one piece of work

Take a weekly client brief. Someone reads updates, checks whether they matter to the client, looks up earlier decisions, and writes a summary. The finished document makes this look like one task. In practice, it contains several.

Collecting sources and preparing a draft might be good candidates for assistance. Deciding what the client should do next calls for context, experience, and accountability. Start by separating those responsibilities.

A first version could gather material from an agreed set of sources, flag what has changed, and produce a draft with a link beside each factual claim. A person checks the sources, adds the advice, and approves what goes out. The system should leave gaps visible rather than fill them with plausible guesses.

This is an illustrative workflow, not a customer result. The same exercise applies to reports, proposals, and customer requests: follow one real example before deciding what to automate.

Keep the ordinary software

A model does not need to handle every step. If an enquiry arrives through a standard form, ordinary software can check required fields and put it in the right queue. A spreadsheet can calculate a price. A database can retrieve a record. Anthropic makes a similar recommendation in Building effective agents: begin with the simplest solution and add complexity only when it improves the outcome.

A model is worth testing where the material is less predictable: differently worded requests, inconsistent documents, or questions whose answers sit across several sources. It can interpret that material while simpler software handles calculations, permissions, and saving the result.

That division also makes mistakes easier to investigate. You should be able to tell whether the problem came from a missing source, a wrong interpretation, or a broken connection between systems.

Count the work after the draft

A fast first draft is only useful if it helps someone finish sooner. Include the time spent checking facts, correcting mistakes, and transferring the answer into the system where it belongs.

In the 2023 working paper Generative AI at Work, researchers studied 5,179 customer-support agents and measured productivity in issues resolved per hour. They found a 14% average increase with AI assistance, with much larger gains for less experienced workers. That is evidence from one setting, not a forecast for your business. It is also a useful example of measuring completed work.

Suppose a report takes thirty minutes today. An AI-assisted draft arrives in two minutes, but checking it takes another twenty-eight. There is no time saving yet. If the full process takes fifteen minutes and meets the same standard, there is something worth investigating. Those numbers are hypothetical; the comparison needs to come from your own work.

Test several completed examples, including the awkward ones. Hold some back while building so the final check includes cases the system has not been adjusted around. Record where it helps, where it fails, and how much review it needs. Anthropic’s guide to agent evaluations recommends starting with tasks drawn from real failures, then expanding the test set as the system matures.

Then account for setup, subscriptions, and maintenance. A task that happens every afternoon can justify a different investment from one that happens twice a year.

Leave someone in charge

Before connecting real records, decide who may access them and which services may receive them. An internal assistant should not make a restricted document available to everyone simply because it can search it.

Keep approval with a person for work that commits the business: sending advice, quoting a price, or promising a deadline. Give that person a way to see the source, correct the result, and complete the task when the tool is unavailable.

The NIST Generative AI Profile calls for clearly defined human oversight responsibilities. Someone also needs to own the system after launch. Sources go stale. Formats change. A workflow that was useful in its first week still needs attention six months later.

A good first project is small enough to inspect, useful enough to keep, and clear about who is responsible.

That is how we build with companies at Riften: understand the work, build a focused first version, and test whether it earns its place. Reach out ↗

References

  1. Building effective agentsAnthropic · December 19, 2024
  2. Generative AI at WorkErik Brynjolfsson, Danielle Li, and Lindsey R. Raymond · NBER Working Paper 31161 · 2023
  3. Demystifying evals for AI agentsAnthropic · January 9, 2026
  4. Generative Artificial Intelligence ProfileNIST AI 600-1 · July 2024