Abstract diagram showing the behavioral path branching from context to perception, choice, and multiple outcomes in dark blue, blue, and gold
← All articles
8 MIN READ
Behavioral ScienceAI RiskCrisis Response

Increasing the number of warnings does not necessarily increase safety.

Given habituation to repeated warnings, false alarms, and ignoring due to irrelevant notifications, design a warning budget by risk, display variations, and mandatory stops.

QUICK ANSWER / 3 LINES

What this means in practice

  1. WHAT / CONCLUSION

    Repetition of the same warning reduces attention and response, and adding more does not necessarily increase safety.

  2. WHY / EVIDENCE

    Frequent, irrelevant warnings and false alarms can lead to ignoring, delay, or opting out.

  3. ACTION / PRACTICE

    Allocate warnings by risk for suppression, aggregation, variation, and mandatory stops, and measure appropriate avoidance actions.

Explore this article8 sections · key points & references

Attaching warnings to AI uncertainty, confidential information, hazardous operations, and policy violations is an intuitive countermeasure. However, if the same confirmation is presented every time and most pass without issue, users learn to close them reflexively. Warnings should be evaluated not by the number displayed, but by whether they regain attention when needed and lead to correct avoidance actions.

Assume that repetition reduces attention.

Vance et al. investigated habituation, where repeated exposure to security warnings reduces attention and response, using brain measurements, eye tracking, and experiments.[1] This is evidence that the mere presence of a warning does not guarantee continuous attention.

The research did not directly handle generative AI screens with general security warnings.[1] Nevertheless, the human factor of reduced attention from repeated stimuli applies to AI controls as well.

Do not assume that a user clicking implies understanding or consent. Distinguish between display time, gaze, content comprehension, chosen action, and subsequent outcomes.

Frequent low-risk warnings can lower the signal value of rare high-risk warnings. Do not use the same color, sound, or wording for all alerts.

Habituation is not solely due to user negligence; it is also the result of the system repeatedly issuing irrelevant warnings. Do not place responsibility solely on the screen user.

  • Exposure: How many times the same warning was seen
  • Attention: Whether the content was actually seen and understood
  • Action: Which option was chosen (stop, verify, correct)
  • Outcome: Whether the hazard was avoided and normal work maintained

Measure false alarms and the cry-wolf effect

Wickens et al. analyzed 495 air-traffic conflict alerts and classified 45% as false alerts. However, they found no evidence that centers with more false alerts had more nonresponses to true alerts or delayed response times, and controllers also showed anticipatory action before alerts. [3] Under these conditions, the hypothesized cry-wolf effect was not supported.

This counterevidence shows that false alerts do not always produce the same decline in response. Do not directly extrapolate the air-traffic observation to AI operations; measure responses to true alerts and false-alert rates separately in your own environment, including user expertise, anticipation before alerts, and how readily hazards can be visualized. [3]

Look not only at the false positive rate but also at the continuity of experiences where no danger occurred after a warning. Users update their trust based on their recent experience rather than statistical tables.

Also measure false negatives. Reducing the number of warnings to increase response rates is not an improvement if major hazards are missed.

Determine the acceptable ratio of false alarms and misses by impact level. Do not treat minor wording corrections with the same threshold as irreversible transfers.

Examine opting out from frequency and relevance

Sutton and Wood examined the relationship between the frequency, relevance, and content of emergency alerts and warning fatigue/opting out.[2] If there are many irrelevant alerts, users may disengage from important notifications.

While emergency alerts and corporate AI screens differ, the relevance to the recipient is a common design challenge that influences response.[2]

Narrow the warning targets by user, data, operation, permission, and region. While broadcasting to everyone is easier to implement, it lowers relevance.

Completely prohibiting opting out may cause users to move to other tools or unofficial channels. Make low-risk notifications adjustable, and show reasons and release conditions for high-risk stops.

Before labeling those who stop notifications as non-compliant, investigate past frequency, false alarms, and warnings that made action impossible. Provide a path for easy re-engagement after improvements.

  • Targeting: Is it relevant to that person, operation, or region?
  • Novelty: What has changed since the last time?
  • Actionability: Is there a specific action that can be taken now?
  • Mandatory nature: Differentiate notifications, confirmations, and stops by risk

Allocate warning budget by risk

List all alerts on screens, emails, chats, and logs to see the total volume a user receives for one task. If each team adds individually, the whole system becomes saturated.

Aggregate low-impact information into screen displays or post-verification, allocate medium-impact to explicit confirmation, and high-impact to mandatory stops and human approval. Do not distinguish by color alone; also change the action.

Place responsibility for decisions to remove warnings with designated managers. Do not weaken safety controls under the guise of fatigue countermeasures; replace them with upstream input restrictions or automatic blocking.

When multiple warnings overlap for one hazard, consolidate them into the single most actionable one. Resolve conflicts between legal, product, and security wording.

The warning budget is not a fixed number of items. Allocate based on risk distribution, user role, work time, and urgency, and re-evaluate misses and false alarms after changes.

Use display changes for content comprehension

Vance et al. explored the possibility that polymorphic displays, which change the appearance of warnings, alleviate habituation.[1] Visual changes can serve as a means to regain attention.

It is not necessary to change them drastically every time. If surprise becomes the goal, users will habituate to new displays, impairing accessibility and comprehension.

Use changes to indicate the type of hazard, the difference from the previous time, and the required action. Do not randomize decorations alone; maintain meaning and consistent visual roles.

Test for color blindness, screen readers, cognitive load, and mobile displays. Do not convey urgency solely by color or motion.

In display experiments, measure hazard identification, appropriate action, and accidental stops during normal work, not click rates. Do not adopt if attention is gathered but judgment does not improve.

Stop hazards upstream that cannot be warned about

Do not entrust irreversible operations or major confidential data leaks to a user's single moment of attention. Implement separation of duties, data blocking, dual approval, and pre-execution previews.

Warning fatigue can be alleviated not only by improving screens but also by reducing the hazardous operations that require warnings. Prioritize safe defaults and least privilege.

When AI executes automatically, processing may proceed before the final user sees the warning. Shift controls to agent permissions and tool-side constraints.

Do not unilaterally place responsibility on the user for ignoring warnings. If ignoring is predictable, the design side must choose stronger controls.

Clearly state in the approval form the residual risk acceptable by warnings and the risk not acceptable by warnings. Do not treat the presence of a display as a disclaimer.

  • Safe default: Disable hazardous operations by standard
  • Least privilege: Permit only for necessary targets and periods
  • Dual approval: Independently verify major operations
  • Automatic blocking: Stop the execution itself, not just warn

Change warnings by role and proficiency

Beginners may need background explanations, but for experts, differences and rationale are important. Do not repeat the same long text to everyone.

Experts are not necessarily exempt from warnings. In high-frequency tasks, automated operations increase, making them susceptible to habituation and time pressure.

Role-based displays are not to hide important information but to optimize understanding and action. Ensure full-text rationale is always accessible.

If workarounds to avoid warnings are shared, investigate business needs and false alarms before punishing individuals. Unofficial workarounds are a signal of poor design.

Education should not be an instruction to read warnings but practice with actual failure examples, stop rights, and release procedures. Do not weaken system-side controls after education.

Continuously audit appropriate responses and misses

Monitoring metrics should include not only display counts and click counts but also avoidance in true hazards, accidental stops during normal times, response time, opting out, and avoidance actions.

When response rates drop, distinguish between user attention decline and worsening warning accuracy. Check content, frequency, relevance, and business pressure.

Do not consider adding a warning after a major incident as sufficient for recurrence prevention. Limit the role of warnings from countermeasures including upstream constraints, detection, and recovery.

If models or business change, the warning baseline rate also changes. Check for increased false alarms with old thresholds and regress for each change.

A good warning does not frighten users. It is a control that identifies rare and important hazards, clarifies the action to take, and reduces unnecessary attention consumption.

Each quarter, the operations owner reviews avoidance of true hazards, false stops during normal work, response time, and opt-outs. If missed critical hazards increase, the team returns to upstream blocking controls before adding another warning; if false stops do not fall, it narrows the trigger conditions. A warning remains only where the required action and the condition for clearing it can be explained.

SOURCES / 03

References and interpretation

  1. [1]
    Tuning Out Security Warnings: A Longitudinal Examination of Habituation Through fMRI, Eye Tracking, and Field Experiments

    MIS Quarterly, 42(2), 355–380

    Finding
    Demonstrated habituation to repeated security warnings through multiple methods and explored the possibility that display changes alleviate habituation.
    Limitation
    The research on general security warnings does not directly handle generative AI uncertainty or execution confirmation screens.
  2. [2]
    Opting Out: Over-Alerting and Warning Fatigue in the Era of Wireless Emergency Alerts

    Journal of Contingencies and Crisis Management, 33, e70076

    Finding
    Examined the relationship between the frequency, relevance, and content of emergency alerts and warning fatigue/opting out.
    Limitation
    Does not directly evaluate corporate AI screen warnings or execution permissions in the context of emergency alerts.
  3. [3]
    False Alerts in Air Traffic Control Conflict Alerting System: Is There a ‘Cry Wolf’ Effect?

    Human Factors, 51(4), 446–462

    Finding
    Among 495 air-traffic conflict alerts, 45% were false, but the study found no increase in nonresponses to true alerts or delayed response times and did not support a cry-wolf effect in this setting.
    Limitation
    This observational study involved highly trained air traffic controllers and a specific alert system, with anticipatory action before alerts, so it cannot be directly generalized to consumer or enterprise AI warning environments.