INSIGHTS / RESEARCH NOTES
We document research on AI, data, behavioral science, and organizations, including methods worth testing in practice. Each note shares not only conclusions, but also hypotheses, observations, and limitations.
69 RESEARCH NOTES
3 / 6 PAGE
Do not deploy based solely on the overall score of translation benchmarks; determine the scope of provision by analyzing culture-dependent items, native-speaker evaluations, worst-case performance per language, and relief pathways.
Read article
Do not equate the productivity of generated code with safety; instead, treat vulnerabilities, user safety perception, static and dynamic testing, and independent review as gates for adopting high-impact code.
Read article
Rather than prompting guesses solely based on accuracy, design the boundaries between answering, confirming, and stopping by accounting for semantic uncertainty, unanswerable questions, and the losses from incorrect answers versus abstention.
Read article
We examine the inclusion of the public test set in training data from multiple perspectives, including data duplication, completion behavior, and example order, to isolate suspicious scores from decision-making.
Read article
We establish boundaries to return to human evaluation by auditing not only the agreement rate with human evaluators but also through reversal of answer order, redundancy, self-preference, and difficult cases.
Read article
For owners approving long-document AI, this article compares full text, chunking, and retrieval under changing positions, task types, and noise, then sets an automation boundary from worst-case quality and total verification cost.
Read article
We organize work design by separating allocation, monitoring, evaluation, and reward by function, and setting minimum standards for autonomy, connectivity pressure, anxiety, and the ability to appeal, in addition to efficiency.
Read article
We present a correction protocol anchored to primary sources, integrating the potential of personalized correction dialogues, the LLM's own epistemic limits, the persistence of effects after warnings, and the decay of effectiveness.
Read article
We design an operation that distinguishes the quality of AI-generated empathetic text from the recipient's perceived understanding, connection, and source preference, while safeguarding human editorial responsibility and recipient choice.
Read article
This article demonstrates an audit design that distinguishes between the fluency of Chain-of-Thought and internal judgment reproduction, focusing on answer accuracy, causal fidelity, hidden cues, and separated execution logs.
Read article
We design an evidentiary procedure that distinguishes AI text detectors from generation-time watermarks and includes paraphrase resistance, unknown models, group-based false positives, the author's version history, and avenues for appeal.
Read article
Instead of uniformly prohibiting the reuse of synthetic data, we design learning data operations that separate the replacement, accumulation, and selection of real data to preserve rare cases and diversity.
Read article
