Engineering Notes

BiasShield started from a straightforward question: what would moderation look like if users, rather than platforms alone, could decide how harmful content is handled while browsing?

Online misogyny and sexualised content already cause measurable harm, and deepfakes make that problem worse. Existing automated moderation also introduces gendered bias by over-flagging benign images of women while still failing to shield users from harmful content.

The project explores a user-controlled alternative: identify, audit, and optionally blur misogynistic or deepfake content, while still keeping the decision-making transparent and reversible for the person browsing.

The extension segments webpages into text blocks and image context elements, then sends compressed representations to the backend for analysis. The backend returns predictions, and the extension applies user-configurable actions directly on the page.

In practice, that means users can enable or disable detectors, blur flagged content, reveal it on click, inspect an activity log for the browsing session, and review dashboard analytics about what the system flagged and where.

BiasShield combines a text classifier for misogynistic discourse with an image classifier that has two heads: one for misogyny detection and one for deepfake detection.

The paper builds on several datasets, including Reddit posts and comments, human image datasets, fashion imagery, misogynistic memes, and deepfake images. The goal was not just raw classification performance, but a pipeline that could reason across text, overlaid text, and visual context.

The text classifier uses DistilBERT fine-tuned on Reddit data with focal loss to address class imbalance. For image misogyny detection, neutral human fashion images were mixed with misogynistic memes to decouple the mere presence of women from misogynistic classification. A separate deepfake head flags synthetic imagery.

BiasShield reframes moderation as a user-controlled process rather than a platform-enforced one. Transparency is central: the system shows what was flagged, why it was flagged, and what options the user has next.

On the Reddit test set, the text classifier achieved an F1 of 0.65, precision of 0.96, and recall of 0.49. The gap between precision and recall mattered: the model was strong when it committed to a harmful prediction, but subtle, stereotype-based, or ironic misogyny remained hard.

Examples like “The way every man feels when a woman is driving” show why this is difficult. These statements often rely on stereotype and implication rather than explicit abusive language.

The ablation study confirmed that misogyny detection is fundamentally a language-driven multimodal task. The image-only variant achieved only 22.1% F1, while configurations that included overlaid text and generated context consistently performed better.

A Variational Autoencoder was initially used to normalise and compress visual representations across heterogeneous web images, but this discarded useful discriminative detail. The best configuration removed the VAE and reached 94.9% F1.

The fairness analysis used benign human images split into female-target and non-female-target groups. Because all of these images were non-misogynistic, any positive prediction counted as a false positive.

Before image regularisation, 34.5% of female-targeted images were incorrectly flagged as misogynistic, compared with 1% of non-female-targeted images. That is a fairness gap of 33.5%.

A final fine-tuning stage on neutral images reduced false positives to 0% in both groups. The trade-off was small but visible: macro-F1 on the meme test set fell from 0.931 to 0.919, while misogynistic recall dropped from 0.913 to 0.875.

That trade-off was worth surfacing. If a system is meant to help women navigate harmful content online, it cannot simultaneously reinforce the bias that equates the presence of women with sexualisation or harm.

The biggest lesson was that performance numbers alone are not a complete story. Building a system like this means balancing model quality, fairness, user control, explainability, and interface design at the same time.

BiasShield also made clear that misogyny detection is deeply contextual. Overlaid text, surrounding language, image content, and user intent all matter. Neither text nor image signals are sufficient in isolation.

On the engineering side, the project gave me experience shipping a browser extension frontend, a FastAPI inference backend, a multimodal evaluation workflow, and an academic paper that ties the whole system together.