Build and test analytics for your first business domain.

What happens in Phase 2

Eight steps take the scoped plan through build, evaluation, pilot, and wider rollout. Your team learns to run the system throughout, with evaluation gates before users get access.

1. 01 · Confirm the focussed, high value domain

   We start with the focussed, high value domain and implementation plan from your Analytics Assessment & Build Plan, confirming its goals, owners, and related questions and metrics. This defines the domain-wide scope and priorities for the build.

2. 02 · Write golden questions

   Your domain expert and I select real questions covering a representative range of analyses and follow-up questions. We record the approved sources, periods, and answers your team trusts, including when the agent should clarify a request or decline access.

3. 03 · Build the evaluation pipeline

   I set up repeatable tests against those questions and expected answers so each agent version can be evaluated under representative employee roles. We agree the readiness criteria for the pilot, including answer quality, permissions, and known limits. Your team learns how to run the tests and interpret the results.

4. 04 · Build the semantic layer

   I implement the scoped warehouse changes, build and test the dbt models, and map business entities and metrics to the curated data model. I document the vendor-neutral business context and connect the agent to approved data and dashboards, checking what each role can open and which requests it must decline.

5. 05 · Enrich, evaluate, gate

   I refine the context and evaluate each change against the test set. Nothing reaches pilot users until it meets the agreed readiness criteria. We record known limits and keep anything that is not ready outside the pilot’s approved scope.

6. 06 · Pilot with a test group

   A small group tests representative analyses across the domain in its day-to-day work, with trusted reports still available. We compare answers with approved sources, test follow-up questions, flag failures, and refine the semantic layer and context before testing again.

7. 07 · Roll out to the wider team

   We widen access only when the pilot’s evaluation results and feedback meet the agreed rollout criteria. I train the people who will use and run the system, including how to check answers, recognize known limits, and escalate problems. Your team practices adding a question, tracing an incorrect answer, and testing the fix before handoff.

8. 08 · Iterate, then expand

   After rollout, each in-scope response gets an accuracy status, including ‘awaiting verification.’ Your reviewers check flagged responses and sample unflagged ones, trace errors to their causes, and add new test cases before retesting fixes. Your team keeps the models, vendor-neutral context, test cases, results, and a playbook for daily review and weekly reporting. Ongoing evaluation support is optional. When you are ready, Phase 3 adds another business domain through separately scoped work.