Topical authority
AI Bias, Data Equity & Black Communities
Most conversations about AI bias start at the model. This one starts earlier — at the archive. A model can only be as representative as the documents it was trained on, and enormous portions of the Black press were never digitized, were digitized badly, or were lost before anyone thought to scan them.
Bias begins in the corpus, not the model
When a language model answers a question about a Black neighborhood, a Black newspaper, or a Black religious tradition, it is reconstructing from whatever text about that subject survived into machine-readable form. If the surviving text is thin, hostile, or produced entirely by outsiders, the model inherits that shape. No amount of downstream fine-tuning fully repairs a missing source.
This is not an abstraction in the Living Archive Series. Restoring a nineteenth- and twentieth-century American newspaper means confronting pages where the OCR output is unusable, where microfilm degraded before capture, and where the only surviving run sits in a single institution that never funded digitization. Every page not recovered is a page no model will ever read.
What OCR failure actually costs
Optical character recognition performs worst on exactly the material that most needs recovery: tight-set community broadsheets, poor-contrast microfilm, non-standard typography, and vernacular spellings that spellcheck-style correction quietly overwrites. The correction step is where bias compounds — an automated proofreader trained on standard English will 'fix' dialect, names, and place references into something that never existed.
The practical remedy is a human-supervised pipeline: machine transcription for speed, targeted model-assisted correction with a domain glossary, and a human reader who knows the community the paper served. Speed without that last reader produces clean text that is confidently wrong.
Repair is a data problem before it is a policy problem
Policy work on algorithmic equity tends to focus on auditing outputs. Auditing matters, but it is downstream. The higher-leverage intervention is funding and executing digitization of under-represented archives — because that is the only action that changes what future models can know.
The same logic applies to any organization deploying AI on its own records. Before asking whether a model is fair, ask what your record set actually contains, who created it, who was excluded from creating it, and what your ingestion process silently discards.
Frequently asked
- What causes AI bias against Black communities?
- Primarily the training corpus. Large models learn from digitized text, and the Black press, Black religious records, and Black community documents were digitized late, partially, or not at all. When those sources are missing, a model reconstructs Black history from outside accounts, so the bias is baked in before any model architecture choice is made.
- Can AI bias be fixed by fine-tuning?
- Only partially. Fine-tuning can adjust tone, refusal behavior, and surface stereotyping, but it cannot supply facts that were never in the training data. If the underlying documents do not exist in machine-readable form, the gap persists. Repair requires digitizing and correcting the missing archive.
- Why does OCR quality matter for AI fairness?
- Because OCR determines whether a historical document enters the machine-readable record at all. Community newspapers, tight-set broadsheets, and degraded microfilm produce the worst OCR results, and automated correction trained on standard English frequently overwrites names, dialect, and place references — producing text that reads cleanly and is factually wrong.
- What should an organization audit first before deploying AI?
- Its own record set. Establish what the documents contain, who produced them, who was structurally excluded from producing them, and what the ingestion pipeline discards. Output auditing without corpus auditing measures the symptom rather than the cause.
