As online platforms increasingly rely on shared technical systems to detect and remove harmful content, the human rights implications of those systems are becoming clearer, and more urgent. Tools such as hash and signal databases play a vital role in addressing serious online harms, including terrorist and violent extremist content and child sexual abuse material (CSAM). Yet as governments expand regulatory oversight and pressure platforms to act faster and more broadly, these mechanisms can also create new risks for freedom of expression, privacy, and due process.
In October, the Global Network Initiative (GNI) convened the second session in its learning series examining the role of human rights due diligence in content moderation. This series aims to deepen collective understanding of how different models of content governance can impact the rights to freedom of expression and privacy, particularly as governments around the world become more active in regulating online spaces. As with all GNI activities, these discussions are held under GNI’s policies, including our antitrust compliance policy and code of conduct.
Hash databases – and broadly signal sharing – is becoming a staple in industry efforts to combat terrorist content, violent extremism, and CSAM. But their widespread use raises complex questions about fairness, accountability, and rights protection.
The October call focused on the use of hashed and signal databases in content moderation, exploring how these technologies are evolving, what risks they introduce, and how meaningful safeguards and oversight can ensure their operation aligns with international human rights standards. This blog provides an overview of key takeaways from the learning call.1
At their core, hash databases allow platforms to share unique digital fingerprints (hashes) of known harmful content, without exposing private user data. In theory, this enables more consistent and efficient removal of illicit content. Yet the industry is swiftly moving into signal sharing, which can encompass URLs, usernames, and other identifiers beyond simple image or video hashes. This expansion promises greater coverage of harmful activity, such as human trafficking or large-scale scams, but it also deepens the human rights stakes.
Signal-sharing systems heighten risks to privacy, raise the potential for misidentification, and expand the range of content that could be swept into enforcement actions. Because definitions of “terrorism” or “violent extremism” are often inconsistent across jurisdictions, there is real danger of scope creep, where signals used for one purpose gradually extend to others less closely justified. Moreover, once content is removed, it may be irrecoverable, even when it has documentary or evidentiary value.
These risks are not evenly distributed. Users from historically marginalized or surveilled communities – such as Muslims, LGBTQ+ individuals, or minority language groups – are more likely to suffer from wrongful enforcement. Smaller platforms, particularly those without robust moderation capacity, face a distinct challenge: they have less capacity to review materials related to shared signals, which could exacerbate the risks.
Hash and signal-sharing initiatives such as GIFCT and the Tech Coalition have taken steps to embed human rights safeguards into their work. They have conducted human rights due diligence, engaged civil society through advisory bodies, and committed to transparency and accountability through regular reporting and independent oversight. These measures aim to ensure narrow, legitimate use and prevent misuse, providing a foundation for continued improvement as the field evolves.
Throughout the call, participants emphasized that human rights due diligence must not be an occasional exercise but a continuous, embedded practice. Companies and database operators should proactively assess risks, engage civil society experts, and build in auditability, transparency, and pathways for remedy.
Meaningful safeguards might include limiting data shared or stored, preventing bulk downloads, automatically flagging anomalous behavior, and requiring human review for high-risk signals. Oversight mechanisms should be independent, with stakeholder participation from affected communities, researchers, and human rights bodies. Transparency reporting must go beyond aggregate numbers, offering insight into how decisions are made, how errors are handled, and under what conditions government requests are accepted or rejected.
Equally vital is creating redress mechanisms: individuals should be able to challenge wrongful takedowns or account suspensions and have their content re-evaluated. Feedback loops, where flagged signals can be contested and revised, help databases evolve and self-correct over time.
Participants emphasized that moderation practices must be grounded in international human rights law, particularly Articles 19 and 20 of the International Covenant on Civil and Political Rights (ICCPR), which require that any restriction on expression meet the tests of legitimacy, legality, necessity, and proportionality. These principles are essential to ensuring that moderation systems and hash databases do not impose arbitrary or excessive restrictions on speech. Yet in practice, the absence of clear definitions, consistent standards, or meaningful due process makes these tests difficult to uphold.
The discussion also touched on the growing interest in applying signal-sharing models beyond the technology sector. Some financial services institutions, for example, have begun exploring ways to contribute signals related to online fraud or child exploitation. While cross-sector collaboration can strengthen responses to complex harms such as sextortion, it also introduces new challenges. Many of these industries lack the established frameworks and human rights experience that have developed within the technology sector through initiatives such as GNI and GIFCT.
One example is the Irish Central Bank’s role as a trusted flagger under the EU’s Digital Services Act. As a trusted flagger, the Central Bank of Ireland can report potentially illegal content (e.g. fraudulent services or scams) that must be given priority by platforms, and acted upon without undue delay. This means that, in contexts like signal sharing, when such entities supply signals tied to illegal content in their area of expertise, those signals carry more weight, faster processing, greater legal obligation by platforms, but also bring risks if signals are not well-curated, definitions are fuzzy, or oversight is insufficient. Strong collaboration among companies, civil society, and governments can help ensure that new entrants learn from existing best practices and avoid repeating early mistakes. At the same time, the diversity of sectors, languages, and operational cultures involved means that developing shared standards for transparency and accountability will require careful coordination and sustained dialogue.
Speakers warned of increasing government interest in these databases, whether through regulation or direct access requests. While cooperation between industry, government, and civil society can play a constructive role in addressing online harms, government involvement must be carefully managed to prevent politicization or censorship. Overly broad or unclear legal definitions risk transforming tools designed for safety into instruments of repression. Several participants stressed that clear governance frameworks and procedural safeguards are necessary to prevent such outcomes.
The discussion underscored that these challenges do not have easy answers. As one participant reflected, while there are real human rights risks associated with hash and signal sharing, there are – likely larger – human rights risks in not using them at all, particularly when it comes to protecting children and preventing violence. The goal, therefore, is not to reject these tools but to ensure they are implemented responsibly, transparently, and with meaningful oversight.
Without proper safeguards, databases created to counter violent extremism or child exploitation could be repurposed for censorship or surveillance, undermining the principles of legality, necessity, and proportionality that international human rights law requires.
The challenge is compounded by the fact that these systems often operate in highly automated or semi-automated modes, leaving little room for human review or accountability. In turn, users whose content or accounts are affected may receive no explanation or recourse.
The discussion underscored a persistent tension between the serious human rights risks posed by hash and signal systems and the reality that refusing to use such tools is not a straightforward alternative, especially as platforms face mounting pressure to address child abuse, terrorism, and other grave harms. The path forward lies not in rejecting these tools, but in building them responsibly, transparently, and with accountability baked in.
This learning call formed part of GNI’s ongoing effort to deepen understanding of how human rights principles can be effectively applied to evolving content moderation practices. GNI will continue to convene members and experts to identify practical ways companies can strengthen transparency, accountability, oversight, and remedy across the stack.
Note: The views expressed in this post do not reflect the positions or opinions of the organizations or representatives who participated in the learning call.
More content on: HRDD.