An abstract diagram showing multiple knowledge clusters connected via golden validation points into a single operational network
← All articles
8 MIN READ
AI FairnessAlgorithmic AuditDecision-making

Procedures other than outcomes are required for fair AI judgments.

We design procedures that do not equate fairness with merely displaying explanations, but instead enable individuals to understand, present their circumstances, correct data, and proceed to human review and relief.

QUICK ANSWER / 3 LINES

What this means in practice

  1. WHAT / CONCLUSION

    Merely adding explanations does not make algorithmic judgments feel fair in all situations.

  2. WHY / EVIDENCE

    The opportunity to present circumstances and control over outcomes holds procedural value distinct from transparency.

  3. ACTION / PRACTICE

    We measure notification, understanding, voice, correction, review, and relief completion, and do not consider a low number of objections as success.

Explore this article8 sections · key points & references

The fairness of AI judgments cannot be fully assessed by adoption rates or error rates across attribute groups alone; for individuals, it matters what is decided, why it is so, whether they can present their circumstances, correct erroneous data, and have a person review the decision. Even if outcome distributions are identical, procedures without voice and relief create different disadvantages.

Separate distribution results from procedures

Statistical fairness measures differences in distribution or errors between groups. Procedural fairness addresses a different question: whether the decision process is consistent, mitigates bias, handles the individual's voice, and allows for correction. Neither replaces the other.

Binns et al. experimentally investigated people's perceptions of fairness by varying explanation methods and scenarios for algorithmic judgments, showing that the effect of explanations is not uniform and varies by judgment context and explanation type.[1]

Feeling fair is not proof that actual distribution is fair. Conversely, we do not attribute all dissatisfaction with adverse outcomes to procedural failure. We align perception with objective metrics.

Procedure is not just an explanation screen after the result is issued; it is a continuous path of data collection, input verification, decision, notification, objection, re-decision, and relief.

In high-stakes judgments, result audits and procedural audits are conducted by different responsible parties who cross-reference each other. Good average results do not offset the lack of individual relief.

  • Distribution: Who received benefits or harms?
  • Accuracy: Who experienced which errors?
  • Procedure: Could the individual understand, voice, and correct?
  • Relief: Were errors remedied within the deadline?

Do not treat explanations as a panacea

Explanations are important, but displaying lengthy technical documents limits procedural value if the individual cannot understand the reasoning or act on it.[1] We distinguish between the existence of an explanation and its usability.

We show not only model feature importance but also the data used, application rules, thresholds, decision-making entity, and modifiable information. Internal calculation explanations are not the same as the practical information needed for objections.

There is a concern that explanations may teach operational methods and make it easier to game the system. Rather than hiding reasons, we separate information necessary for individual relief from details for fraud prevention.

If reason codes are too standardized, they do not reflect individual circumstances. Conversely, free-text alone cannot be audited for consistency. We combine structured reasons with specific grounds.

In understanding verification, we measure whether the individual can explain what they can correct and what to do next, rather than counting views or notification opens as success.

Provide opportunities for voice and control over outcomes

Lee et al. showed in an experiment on algorithmic resource allocation that fairness perception relates not only to transparency but also to participants having some control over the outcomes.[2] Showing information and having one's voice reflected are different.

Opportunities for voice are not just free-text fields; circumstances must be submittable before the decision, read by someone, reflected in the judgment, or accompanied by a reason if not reflected.

Outcome control has limits. Individuals cannot freely change scores; instead, correcting erroneous data, submitting exceptional evidence, selecting alternatives, and requesting human review are formalized as rights.

If collected voices do not influence decisions, participation becomes performative. We track from submission to processing and decision changes, auditing not just reflection rates but also valid reasons for rejection.

If only those who voice opinions benefit, disparities in utilization capacity arise. We provide multiple channels addressing language, disability, time, and digital access.

  • Pre-voice: Submit circumstances and evidence before the decision
  • Data correction: Verify and correct source information and derived values
  • Exception application: Review circumstances not handled by standard rules
  • Human review: An authorized person reviews independently

Address differences in fairness perception by context

In hiring, lending, insurance, education, healthcare, and content delivery, the required explanations, speed, expertise, and individual participation differ. We do not transfer procedures validated for low-stakes recommendations to high-stakes rights judgments.

Starke et al.'s systematic review organized research on fairness perception in algorithmic judgments, showing that procedure, outcome, individual differences, and context are complexly related.[3] No single explanation format is universally superior.

The degree of automation also matters. When AI proposes and humans decide, we verify whether the human substantially re-evaluated or merely ratified. Human signatures alone do not humanize the procedure.

Reasons individuals desire human judgment include not only empathy but also understanding of exceptions, negotiation, and access to responsible parties. We decompose which functions are needed.

Expectations of fairness vary by culture and institution. We do not determine acceptance across all regions based on experiments in a single country but adjust procedures according to local legal systems and user participation.

Make objections effective

Even with an objection button, relief remains weak if reasons are unclear, evidence cannot be submitted, and the same model simply re-judges. We separate filing, acceptance, investigation, re-decision, and correction.

We consider methods where reviewers do not see the original AI score to reduce anchoring to the initial judgment and independently verify necessary information.

We set deadlines and allow individuals to check progress and responsibility. In high-stakes decisions, we design pauses if adverse effects continue during review.

We audit whether future scores or services deteriorate due to filing an objection. The absence of retaliation is confirmed not just by terms of service but by subsequent outcomes for the group of filers.

We do not use a low number of filings as evidence of high quality. Discovery, understanding, cost, fear, and resignation lower filing numbers, so we conduct simulated usage and interviews.

Measure until relief is completed

Even if an error is acknowledged, relief is not complete if source data, downstream systems, billing, and records are not fixed. We trace correction propagation via data lineage.

If harm has already occurred, relief may require not only re-judgment but also refunds, re-provision of opportunities, record deletion, and notification of relevant parties. We designate compensation responsibility and deadlines.

When using objection data for model improvement, we verify individual rights and confidentiality. We do not unconditionally feed information from those seeking relief back into retraining.

Relief metrics include time to finalization, overturning, downstream correction, individual notification, and recurrence, rather than just acceptance rates. We do not formally reject objections for the sake of apparent speed.

If objections regarding the same cause recur, we do not end with individual responses but identify the impact set and re-review past judgments.

Audit disparities in procedure utilization

If only those who can use explanations and objections can improve outcomes, disparities in relief capacity become new unfairness. We audit language, readability, disability, cost, and time.

We provide channels for agents, supporters, paper, phone, and in-person interactions. However, we protect individual consent and privacy to ensure proxy use does not create new risks.

For those unable to submit complex evidence, the organization presents held data and assists in verification. We do not require individuals to prove system errors from scratch.

We examine notification understanding, objection initiation, completion, overturning, and time required by group. If specific groups cannot reach relief even with good overall averages, we improve.

Even if automation speeds up processing, we do not reduce the capacity of the objection window. The expertise of people who understand exceptions becomes more important after standard processing is automated.

Integrate procedures into change management

Changes to models, thresholds, data, explanations, objection screens, and review systems constitute changes to the entire procedure. We do not approve based solely on model accuracy regression.

We compare outcome differences, understanding, voice, correction, objections, and relief before and after changes, checking whether understanding or discoverability has decreased even if objections have reduced.

Before production, we conduct simulated usage including erroneous data, adverse outcomes, language differences, disabilities, and urgency. We test relief paths from end to end, not just success paths.

Responsibility tables include decision-makers, data owners, explanation accountable parties, reviewers, and relief executors. We do not fragment the individual's interface even when using external vendors.

Procedural fairness is not public relations to make individuals accept AI judgments. It is creating measurable processes for actionable rights that allow individuals to participate in decisions, correct errors, and recover harms.

The monthly procedural-SLA record stores the service version, decision type, and affected group, with maxima for the time and group disparity in notice, appeal, correction, and remedy. A severe adverse-decision process that exceeds a maximum stops automation and switches to independent human review and a temporary appeal channel. Automation returns only after an independent auditor confirms remedy for pending cases and process correction and the remedy owner reapproves it.

SOURCES / 03

References and interpretation

  1. [1]
    ‘It's Reducing a Human Being to a Percentage’: Perceptions of Justice in Algorithmic Decisions

    Proceedings of CHI 2018

    Finding
    Multiple experiments varied explanation methods and scenarios for algorithmic judgments, demonstrating that fairness perception differs by explanation and context.
    Limitation
    Cognitive evaluation in constructed scenarios does not directly measure actual long-term objections, relief, or legal fairness.
  2. [2]
    Procedural Justice in Algorithmic Fairness: Leveraging Transparency and Outcome Control for Fair Algorithmic Mediation

    Proceedings of the ACM on Human-Computer Interaction, 3(CSCW)

    Finding
    An experiment on algorithmic resource allocation examined the impact of transparency and outcome control on fairness perception.
    Limitation
    Experimental allocation tasks do not demonstrate actual relief effects in high-stakes institutions like hiring, lending, or healthcare.
  3. [3]
    Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

    Big Data & Society, 9(2)

    Finding
    A systematic review of empirical research on fairness perception in algorithmic judgments organized the relationship between procedure, outcome, context, and individual differences.
    Limitation
    A systematic review of heterogeneous studies does not provide single causal effects of specific procedural interventions or legal conclusions.