Conjoint Analysis Trade-Offs
Conjoint analysis measures stated preferences across attributes. Read the trade-offs, a worked example, and why a choice in a survey is not a purchase.
Conjoint analysis is often used to decide what to build, which overstates what a stated-choice survey can tell you. The Wikipedia entry describes measuring preferences across attributes, and AAPOR's standards cover wording and transparent reporting. The trade-offs concern the gap between a survey choice and a real decision.
Direct answer: Use conjoint analysis to understand relative preferences, then validate the result against real behaviour before committing to a product decision. The sources describe the method; the rule is a synthesis.
Delta: This article adds a claim-status table and a worked example, separating documented method from local judgements about design.
The mechanism
The first point is the task. The conjoint entry describes presenting combinations of attributes and asking for choices, which produces relative preferences under the conditions of the survey. A respondent who chooses a cheaper option in a survey may choose differently when paying, and the gap is a property of stated methods.

The second point is design. AAPOR's standards require neutral wording and transparent reporting, which applies to the attributes and levels offered. A design that omits a realistic option forces a choice that would not exist in the market, and the result will reflect the design.
The third point is validation. The output is a model of preferences, not a prediction of sales. Checking it against an observed choice, a pilot, or a real purchase is the step that turns an interesting result into a usable one.
The sequence is: define the decision, choose realistic attributes, field the survey neutrally, then validate. The choices are local, so the conclusions are inferred; the method is source-reported.
A worked example
Illustrative. A team uses conjoint analysis to choose the price of a new product.
Three checks decide. First, were the attributes and levels realistic, including the options customers actually have? Second, was the wording neutral, and was the sample reported? Third, does the result agree with a pilot or an observed purchase, or does it only describe a hypothetical choice?
The outcome is a preference estimate with a stated limit. The scenario is hypothetical; the method is the contribution.
Evidence status
| Claim | Status | Source | Verified | Recheck when |
|---|---|---|---|---|
| Conjoint measures preferences across attributes | source-reported | Wikipedia, Conjoint analysis | 2026-11-18 | Method changes |
| Surveys need neutral wording and reporting | source-reported | AAPOR standards | 2026-11-18 | Standards updated |
| Validate against observed behaviour | inferred | Derived from the gap | 2026-11-18 | Market changes |
Why a survey choice is not a purchase
A respondent in a survey faces no budget consequence, which changes the weight of price. The result is useful for comparing attributes, and weak as a forecast of sales, which is the distinction that a product decision has to respect.
A further consideration is that preferences are not stable. A result collected before a competitor changes its offer describes the old market, and a record of the fielding date is part of interpreting it. Reviewing the design when the market changes is part of the practice.
The method also has a cost in attention. A conjoint survey with many attributes is tiring, and respondents may simplify their choices near the end, which introduces a pattern that belongs to the instrument. Keeping the task short is a design decision that protects the data.
The third limitation is that a preference model cannot create a product. It ranks combinations that were shown to respondents, which means an option nobody thought to include cannot emerge.
A stated choice carries no budget consequence, which changes how respondents weigh price.
Preferences describe the market at the time of fielding, which is why the date belongs with the result.
The method ranks the combinations it showed, and a product nobody thought to include cannot emerge from it.
A short survey protects the data, since a tiring task produces simplified choices.
The result should be validated against an observed choice before it drives a product decision.
A relative preference is not a sales forecast, which is the boundary of the method.
The attributes shown define the boundary of what the study can learn.
A result should be dated, since it describes the market at the time of fielding.
A decision test
- Define the decision the study supports.
- Choose realistic attributes and levels.
- Word the task neutrally.
- Report the sample.
- Validate against observed behaviour.
- Record the result and recheck date.
For the survey standards, see Survey Design.
A counterargument
A reasonable critic would question the article's central prescription. The rule to "validate against real behaviour before committing" is marked inferred in its own evidence-status table, derived from "the gap" between stated and revealed choice, and neither cited source establishes it: Wikipedia describes the measurement of preferences, while AAPOR's standards concern neutral wording and transparent reporting, not predictive validity. The strongest claim is therefore a local judgement dressed as method discipline. There is also a tension in criticising the model because "an option nobody thought to include cannot emerge": the controlled option set is exactly what lets conjoint isolate attribute trade-offs, which observational purchase data cannot do, since sales are confounded by availability, habit, and marketing.
Insisting on purchase validation may discard the method's distinctive counterfactual leverage.
References & Further Reading
- Wikipedia contributors, 2026. Conjoint analysis - Wikipedia — Measuring preferences across product attributes.
- American Association for Public Opinion Research, 2026. AAPOR — The professional association for public opinion and survey research; its standards and resources underpin the survey-design discussion.