How much should conversational AI be allowed to change human choices?

Traditional advertising primarily involved distributing pre-made messages to a group. Conversational AI can adjust its next move in response to the user's expressed interests, counterarguments, and hesitations. In a series of 2024 studies by Matz et al., 30 out of 33 messages created by ChatGPT tailored to psychological traits moved in the intended direction, and 20 showed statistical significance. [1] The ability to mass-produce personalized messages from minimal input changes the constraints of scale.

However, effects are not uniform. While alignment with openness and extraversion was relatively consistent, it was unstable with agreeableness, and results varied across health behaviors, products, and political appeals. Many outcomes were based on message likability or willingness to pay, not actual purchases or long-term behavior. [1] We must retain which traits, objectives, and outcome metrics showed differences, rather than assuming 'personalization works.'

In 2025, Salvi et al. randomly assigned 900 U.S. participants to 12 conditions to engage in short, multi-turn discussions with either a human or GPT-4. GPT-4 given personal attributes was more persuasive in 64.4% of combinations where it did not match humans in persuasiveness, and the estimated odds of increased agreement with the interlocutor after the discussion were 81.2% higher than with a human. [2] This was not actual voting, but immediate agreement changes on 30 topics by Prolific participants.

A formal meta-analysis in 2025 integrated 7 studies directly comparing LLMs and humans, involving 17,422 people and 12 effect sizes, estimating a mean difference of Hedge's g=0.02, which was not significant. The heterogeneity between studies was large at I²=75.97%, and we cannot generalize the superiority of personalized GPT-4 in one experiment to conversational AI in general. [7] Exploratory moderators examining models, dialogue formats, and domains one by one were also not significant, with limited power due to the small number of studies.

Costello and colleagues’ 2024 article reported reduced beliefs following GPT-4 Turbo conversations tailored to participants’ claims and evidence. The September 2026 author correction deposit distinguishes 2,094 screened participants from 2,044 complete pretreatment/post-treatment cases in the headline analyses, comparing focal-conspiracy conversations with unrelated control conversations.[3]

The original abstract’s 2,190 cannot be treated as a count of people who all received rebuttal conversations. Science issued an Editorial Expression of Concern in June 2026; updated author materials do not establish publisher acceptance of a correction or removal of the concern.[8] This article does not treat effect sizes or persistence as definitive evidence.

This case illustrates a research question regarding dialogue formats that respond to individual counterarguments, but it does not currently provide evidence for the strength or durability of correction effects. [3][8] Moreover, the content being socially desirable does not prove autonomy. The question of who should choose correction goals in daily environments cannot be answered by the original study's design alone.

In practice, 'moving in the right direction' is not an unconditional objective. When supporting goals the user has chosen, such as quitting smoking, learning, or expense management, the owner of the objective differs from cases where a company seeks to increase sales, retention, or political support. In addition to the truthfulness of the content, we must assess who sets the goal, whether the user knows and accepts it, and whether it can be changed midway, as separate review items.

Eiroa-Solans and Inzlicht randomly assigned participants to a GPT-4 conversation structured around self-determination theory and motivational interviewing or to free conversation with untuned GPT-4. Immediately after 15 minutes, perceived importance, confidence, and readiness for prosocial change increased. Differences at 24 hours, in trait motivation, and in the charity-related cognitive-effort task were not statistically detected.[4]

Failure to detect a difference does not prove that no effect exists. The abstract reports N=237, whereas Methods and Table 1 list 234 participants in the intervention analysis and 203 at 24-hour follow-up.

This test shows that even support drawing out the user's own reasons does not guarantee behavior change from a single conversation. While the control being a GPT-4 conversation is useful, the target was limited to prosocial motivation, the intervention analysis included 234 participants and the 24-hour analysis 203. The authors explicitly state that the study was not sufficiently powered to establish the absence of an effect. [4] We must not interpret increased short-term self-reports as evidence of retention or performance.

In 2026, a study that has not been peer reviewed conducted two pre-registered experiments on 14,779 UK adults with 17,950 responses, finding that conversational AI increased signatures on an existing petition by 19.7 percentage points in one example. However, a correlation across conditions between attitude effects and behavioral effects was not statistically detected. [6] Because attention to the conversation was ensured in a paid survey and the actions (signing petitions or small donations) were low-cost, generalization to natural usage or high-cost, irreversible choices is reserved.

• Immediate Response: Record satisfaction, understanding, and emotion immediately after the conversation
• Retention: Check if attitudes and reasons persist over time
• Action: Measure execution corresponding to the objective, not just clicks
• Choice Quality: See if alternatives are understood and reasons can be explained
• Reversal: Measure the cost and time to return to the previous choice after reconsideration

A 2026 preprint by Salvi, Cuevas, and Ribeiro compared search ranking, placement in dialogue, and active recommendation by conversational AI among 2,012 people choosing e-books. When one-fifth of all products were randomly assigned as sponsored, 61.2% chose the sponsored product with actively persuasive conversational AI, significantly exceeding the 22.4% in the top-search ranking condition. [5]

With in-conversation warnings and Sponsored labels, sponsored choice was 55.5%, and its difference from 61.2% without disclosure was not statistically significant. This does not establish equivalence. The composite accuracy measure for identifying influenced products was 35.1% under explicit disclosure. With instructions to conceal persuasive intent, sponsored choice remained 40.7% while this detection accuracy was 9.5%.[5]

This accuracy is the mean participant score: the proportion of identified products that were truly sponsored, with zero assigned when no bias was perceived. A label and understanding the purpose of the conversation are distinct.

This is a single-session experiment that has not been peer reviewed, limited to English-speaking e-book users, with sponsored products assigned randomly rather than by fit to the user. Effects on high-cost products or ongoing relationships are unknown. [5] Nevertheless, the reason why simply adding a disclosure cannot be treated as controlled is clear. We must visualize who benefits, what was excluded from comparison, and how recommendation rankings and explanations changed for commercial purposes.

From here, we present operational proposals derived from confirmed evidence, not the research results themselves. The center of judgment is not 'did the person change?' Advice, education, and motivational interviewing also have influence. We examine whether the user knew the goal, had access to reasons and alternatives, and was influenced while able to refuse or withdraw.

In particular, it is high risk to infer psychological traits, hesitations, fatigue, isolation, or financial anxiety and use them for purposes unknown to the user. Problems do not disappear even if personalization accuracy is low. We must test both inappropriate treatment due to misestimation and exploitation of weaknesses due to accurate estimation, and make non-use of sensitive estimates the default path.

Missing even one of the seven conditions does not immediately constitute illegality or unethicality. However, as the concealment of purpose, substantial reduction of options, exploitation of vulnerability, and inability to withdraw overlap, the balance tips toward stopping it as manipulation. Increases in sales or retention do not offset this judgment in a total score.

• GOAL: Clarify the behavior to be changed and the entity setting that goal
• INTENT: Make the intent of advice, education, sales, advertising, or solicitation readable before and during the conversation
• DATA: Show inputs used for personalization, estimated traits, and sensitive information not used
• OPTIONS: Leave realistic alternatives outside recommendations and the option to do nothing with the same operational load
• PRESSURE: Prohibit and inspect the use of fear, time pressure, repetition, guilt, and isolation
• REVERSAL: Enable execution of consent withdrawal, purchase cancellation, notification stop, and history deletion within the conversation
• ACCOUNTABILITY: Enable auditing of objectives, prompts, candidate sets, ranking changes, and user refusals

Results such as keeping the product after guidance, satisfaction with the conversation, or trust in the recommendation do not prove autonomy. We measure whether the user makes the same choice after knowing the commercial purpose, whether judgment changes after seeing unpresented alternatives, and whether the reason for choice can be explained in connection with their own values. Satisfaction and understanding are separate columns.

In pilot tests, we separate standard support, personalized support, support including commercial objectives, and non-personalized controls. We align factual accuracy and available candidate sets across all conditions, changing only objectives, disclosure, data usage, and opportunities for counterargument in a planned manner. We pre-register attitudes, actions, choice reasons, detection, withdrawal, and regret, and do not pass based on a single convenient metric.

Even if the effect of personalization is small on average, it may move groups with specific anxieties or attributes significantly. We stratify by whether the user explicitly desired the goal, whether estimated traits were used, and whether the influence is irreversible, in addition to overall change. We do not treat vulnerable groups with insufficient samples as 'no difference,' but provide them via paths where personalization is stopped.

• Understanding: The proportion of users who can explain the AI's purpose, conflicts of interest, and usage data
• Choice Width: The proportion who could actually choose non-recommended candidates, hold, or exit
• Ownership of Reason: The proportion where the reason for choice matches the user's prior goal
• Reconsideration Stability: The proportion who maintained their choice even after full disclosure and presentation of alternatives
• Reversal Effectiveness: The number of operations and time to complete cancellation, stop, or deletion
• Pressure Indicators: Count of hasty expressions, repeated solicitation, guilt appeals, and re-solicitation after refusal
• Distribution: Differences by age, proficiency, distress, anxiety, and other pre-approved categories

Low-risk support is limited to goals input by the user, with presentation of reasons, alternatives, and an exit button as defaults. For corporate sales, contract renewals, political solicitation, or health/financial judgments, we do not leverage the intimacy of the same chat, but separate support functions from persuasive objectives. We do not repurpose psychological traits estimated from conversation history for sales optimization without separate user consent.

Stopping conditions are insufficient based on misinformation alone. Increases in users unable to explain intent, re-solicitation after refusal, hiding of alternatives, sharp increases in choice rates only among vulnerable groups, and mass withdrawal after disclosure are also reasons to stop. When the model version, prompt, sponsor, or candidate set changes, we do not carry over previous passing status but re-test.

The value of conversational AI is not in not moving people. It lies in supporting changes chosen by the user while maintaining room for understanding and reconsideration. Therefore, success is not maximizing conversion rates, but simultaneously meeting the objective and the lower bound of autonomy. If either falls below the lower bound, we weaken personalization, separate commercial objectives, or revert to human support.

• Support for User Goals: Personalize within the scope of explicit goals, allowing exit at any time
• Proposal for Organizational Goals: Disclose conflicts of interest first, and separate non-sponsored comparison paths
• High-Impact Domains: In health, finance, employment, and politics, disable persuasion via psychological estimation by default
• Re-evaluation Judgment: Require passing re-tests of understanding, choice width, and reversal, not just correction