Government, Digital, Project delivery
Canada
Self-reported, expert-reviewed
8 min read
Canada.ca AI Answers: how the Government of Canada tested a public AI assistant
GP
Guru Prasad, Founder, AAAI. Published 2 November 2026.
Between June 2025 and February 2026 the Government of Canada trialled AI Answers, an assistant on Canada.ca that answers questions using only federal web content. It reports accuracy of 94%, 95% and 96.7% across its first three trials, measured by expert review of roughly a third of questions. User approval fell from 88% to 65% as harder topics were added, and public access is currently paused. The lasting lesson is the evaluation method, which any organisation running a public AI assistant can reuse.
In brief
- AI Answers gives short, plain-language answers sourced only from Government of Canada websites, with department-specific prompts for 10 institutions and a single cited source or next step.
- Across three trials covering 7,851 questions, the Canada.ca Experience Office reports accuracy of 94%, 95% and 96.7%, checked by experts from the departments that own the content. We found no independent evaluation.
- Approval fell from 88% to 77% to 65% as partner topics became more complex, and the service is not currently open to the public. The reusable part is how it was tested.
Plain English
AI Answers is a retrieval-based assistant: according to the Experience Office, it searches and downloads specific government pages at the moment a question is asked and answers from them, rather than relying on what a model remembers. Expert review here means staff from the department that owns the content judging whether an answer is correct and useful. A golden answer is the Experience Office's term for an answer rated fully correct and useful.
Scoring is more granular than a pass or fail. The service's published system card describes each sentence scoring 100, 80 or 0, the citation scored separately, and a weighted total of 75% for the sentences and 25% for the citation. Accuracy figures in this post are as reported by the Government of Canada, not measured by AAAI.
Australian context
Australia's equivalent expectations sit in two places. The Guidance for AI Adoption (National AI Centre, October 2025) includes a practice on testing and monitoring AI, and the Guidance for AI Adoption explainer covers what it asks of an organisation. For Commonwealth agencies, version 2.0 of the Digital Transformation Agency's Policy for the responsible use of AI in government, in effect from 15 December 2025, requires an accountable owner and an AI impact assessment for in-scope use cases.
What transfers from Canada is the method: sampling answers for expert review, scoring answers and citations separately, and reporting results per language and per topic. What does not transfer directly is the bilingual requirement, the federal-only scope of Canada.ca content, and the Canadian Digital Service's delivery context. The public-facing Q&A assistant is also a different risk class from the internal staff tools that dominate Australian government trials to date.
Sources: Canada.ca Experience Office, "AI Answers" blog post, 17 December 2025 (updated 16 June 2026); Canadian Digital Service, AI Answers trial post, 17 December 2025; AI Answers system card v1.3, August 2026; Government of Canada, AI Answers availability page, modified 16 April 2026; National AI Centre, Guidance for AI Adoption, October 2025; Digital Transformation Agency, Policy for the responsible use of AI in government v2.0, December 2025. Checked 2 November 2026.
The risk and the opportunity
The headline accuracy figures need careful reading. They are the government's own, based on expert review of a sample (994 of 2,866 questions in the second trial and 1,155 of 3,222 in the third, as reported), by staff of the departments whose content is being answered. The Experience Office also reports that follow-up questions in a session were 2% more likely to contain errors, that citations often pointed to topic pages rather than the exact destination, that accuracy was lower for the 19 non-official languages in the first trial, and that approval fell as harder partner topics were added. The sources are not fully consistent either: the Experience Office blog describes three trials and later says a wider 2026 roll-out is planned, while the service's system card records a fourth trial ending in February 2026 and no public access since. We found no independent audit and no published costs.
The opportunity is in how the service was built and tested. Reviewers came from the content-owning departments, and their ratings fed back into answer generation as examples, so evaluation improved the service rather than only reporting on it. Reviews also exposed content problems, such as outdated form-submission information at one department, which is a benefit that accrues to the content owners. The published safeguards are concrete: a personal-information check that blocks a question before it reaches the AI service, with blocked questions not sent or stored, a limit of three questions per session, a 260-character cap, answers limited to four sentences and a visible reminder that AI can make mistakes. Canada.ca's own design guidance says that notice does not substitute for doing the accuracy work.
Testing a public AI assistant: a four-step model
Restrict the scope and the sourcesAnswer only from content you own and can check, and decide in advance what is out of scope. AI Answers draws only on federal sites and excludes provincial, municipal, personal and legal questions.
Block what should never reach the modelFilter personal information, prompt injection and over-long or very short questions before the AI service is called, and cap the length of answers and sessions.
Measure with experts who own the contentReview a meaningful sample, score the answer and the citation separately, and report results per language and per topic, as Canada did. Trial 3 asked partner staff to review at least a quarter of questions on their own content.
Feed findings back, then release in stagesSend errors to the answer logic and the content owners, and publish results at each stage. The fall in approval from 88% to 65% was reported openly, which is what makes the other figures easier to trust.
This four-step model is AAAI's reading of the published trials, not a framework published by the Government of Canada.
What this means for you
Business analysts and project delivery leads
A chat assistant is easy to demonstrate and hard to specify. Canada's trials treat accuracy, citation quality, language coverage and user approval as separate measures with their own review process, which is the shape a good set of acceptance criteria takes.
Do this: write the evaluation plan into the requirements before build, including sample size, who reviews, how answers and citations are scored, and the threshold that blocks release.
Service delivery and digital leads in government and large organisations
The reviews improved the content as much as the assistant. When an assistant answers from your pages, its errors point back to unclear or outdated source material.
Do this: give each content owner a share of the review, and treat every wrong answer as a content ticket as well as a model problem.
Directors, risk and governance leads
The Canadian figures are strong but self-reported, and a headline accuracy rate can sit alongside falling user approval and a paused public service. Reported results are a starting point for questions, not proof of benefit.
Do this: when management presents an assistant's accuracy, ask who reviewed it, what share was reviewed, how errors were defined, and whether results are reported by topic and language.
Frequently asked
Is Canada's AI Answers service available to the public?
Not currently, as far as we could find. A Canada.ca page modified on 16 April 2026 says public access is restricted while infrastructure work takes place, and the service's system card (version 1.3, August 2026) records no public access since February 2026, with use available within the government network only. The project's repository says it is preparing for a public pilot.
How accurate is AI Answers?
The Canada.ca Experience Office reports accuracy of 94% in the first trial, 95% in the second and 96.7% in the third, based on expert review of a sample of questions by staff from partner departments. These are the government's own figures, and the reported user approval over the same period fell from 88% to 65%.
Has AI Answers been independently evaluated?
We found no independent audit or external evaluation. Accuracy was assessed by experts from government partner departments, with AI-assisted evaluation also used in the third trial, so the results should be read as reported by the service's owners.
Designing the tests and controls around an AI assistant is applied governance, and it is the kind of work AAAI's certifications are built around. If you are building baseline AI capability, AI Core is the place to start. If you are accountable for AI strategy and assurance, CAIS: Certified AI Strategist goes deeper.
Explore AI Core
Explore CAIS
