Skip to content
Novandri.

AI for Data Analysis Checklist

AI can query data, not decide what a number means. Read the checklist, a worked example, and why a fluent answer still needs provenance.

Rachmadwipa Novandri4 min read

AI assistance in data analysis is often measured by how quickly it produces an answer, which ignores whether the answer is right. Anthropic's use-case guides describe production patterns, and the data-curation entry covers the quality of the underlying material. The checklist starts with the question, not the tool.

Direct answer: Use AI to query and summarise data with a defined question, verified provenance, and a human check on consequential conclusions. The sources describe capability and data quality; the method is a synthesis.

Delta: This article adds a claim-status table and a worked example, separating documented capability from local decisions about verification.

The mechanism

The first item is the question. A vague request to "analyse this data" will produce a plausible summary of whatever the model notices first. A specific question — which cohort changed, and when — gives the tool a target and gives the analyst a way to judge the result. The use-case guides describe what models can do, not what your question is.

A five-step flow: state the question, identify the data and source, ask the tool, trace and verify the answer, then record the reasoning.
Conceptual: a fluent answer still needs provenance and a human check.

The second item is provenance. The data-curation entry describes ongoing management of datasets. An answer that cannot be traced to the records behind it is difficult to check, and a wrong answer with no provenance is harder to correct than no answer. Asking which rows produced a figure is a simple control.

The third item is verification. A model can compute correctly on the wrong table, or summarise a result without noticing an outlier that changes the conclusion. Checking a sample of the answer against the source data is the practical test. This is especially important when the conclusion will drive a decision, because the cost of a wrong conclusion is not the same as the cost of a slow one.

The sequence is: state the question, identify the data, ask the tool, verify the answer, then record the reasoning.

The verification belongs to the analyst, which is the boundary the checklist exists to preserve.

A worked example

Illustrative. An analyst asks a tool which segment drove a drop in retention.

Three checks decide. First, is the question specific about the segment, the period, and the definition of retention? Second, can the answer be traced to the rows that produced it, with a dated source? Third, does a sample of the result match the underlying data, especially where the conclusion is surprising?

The outcome is a verified answer with a recorded limit. The scenario is hypothetical; the method is the contribution.

Evidence status

ClaimStatusSourceVerifiedRecheck when
Models support defined production taskssource-reportedAnthropic use-case guides2026-11-18Capabilities change
Data needs ongoing managementsource-reportedWikipedia, Data curation2026-11-18Definition changes
Verify the answer against source datainferredDerived from data-quality practice2026-11-18Data changes

Why fluency is not correctness

The visual quality of the output has no relationship to its accuracy, and the gap is widest when the data is messy. Treating the output as a draft is the habit that keeps the tool useful.

A final consideration is that the analyst remains accountable for the conclusion. Handing a question to a tool does not transfer responsibility for the decision it informs, and a review process should make that clear. The person who signs off on a number should be able to explain where it came from.

The checklist also needs a recheck condition. A tool that was reliable on one dataset may behave differently on another, and a conclusion that depended on a specific source will not transfer automatically. Recording the dataset and the date keeps the result challengeable.

The question should be written down before the tool is used, so the answer can be judged against it later.

A verified answer should name the dataset version it came from, because a later revision can change the underlying numbers.

A conclusion should be recorded with its limits, so the next analyst knows what the answer did and did not establish.

What it cannot do

It cannot know which question matters. It does not replace understanding the data. And it cannot guarantee that a summary is complete.

The counterargument that a fast answer is better than none is fair for exploration and incomplete for a decision.

A decision test

  1. State the question.
  2. Identify the data and its source.
  3. Ask the tool.
  4. Trace the answer to rows.
  5. Verify a sample.
  6. Record the reasoning and recheck date.

For the data side, see Data Quality.

References & Further Reading