Who remains responsible when an algorithm shapes your life?

 

The growing use of data and algorithms marks a fundamental shift from contextual human judgement towards automated decision-making based on computational logic and statistical probability. Algorithmic systems now influence public services, law enforcement, employment, lending and information provision. Through prediction, risk profiling and classification, they increasingly determine how people are assessed and what opportunities or services they receive.

Algorithmic governance concerns systems that support or determine decisions about identifiable persons, particularly in public administration and essential private services. The decisive issue is not whether a model is labelled automated or intelligent, but how data, policy objectives, procurement choices, model design, interfaces, professional judgement, and legal remedies combine to shape an outcome. Governance must therefore cover the complete institutional chain rather than treating the technical model as an isolated object. The preceding analysis addressed the infrastructural and agenda-setting power of platforms. The narrower question here is how an institution should govern a decision chain in which data, models, interfaces, professional judgment, and legal remedies jointly shape an individual outcome[1]

Although these systems are often presented as neutral, they contain normative assumptions about relevant data, acceptable risks, social categories and desirable outcomes. Their reliance on historical data can reproduce existing inequalities[2], while their complexity often makes their operation difficult to understand. Automation also enables decisions to be made rapidly and at great scale, magnifying both benefits and errors. Algorithmic governance must therefore be understood as a form of institutional power rather than merely as a technical instrument[3].

Accountability must cover the complete decision chain[4]: the choice to automate, the definition of the policy objective, procurement, data selection, model design, threshold setting, interface design, organizational use, human review, communication, and remedy. A technically accurate model can still produce unlawful or unjust outcomes when the surrounding institutional process is defective.

Transparency and explainability are essential to democratic and legal control. People affected by an algorithmic decision should receive a meaningful explanation of how and why it was reached. Systems should be accompanied by accessible documentation covering their purposes, data, design choices, assumptions and limitations. Appropriate access should also be provided to regulators, researchers and, where possible, the public. Complete openness may conflict with privacy, security or legitimate commercial confidentiality, but controlled transparency must remain sufficient for independent scrutiny and effective challenges.

Explanation, contestability, and remedy are distinct requirements[5]. An explanation makes the basis and limits of a decision intelligible; contestability allows affected persons to introduce contrary facts, challenge assumptions, and obtain reconsideration; and remedy supplies an authoritative response capable of changing the outcome, compensating harm, or suspending the system[6]. Disclosure without these additional capacities does not make algorithmic power correctable.

Clear responsibility is equally important[7]. Organizations using algorithms must remain accountable for their results and may not shift responsibility onto the technology itself. Legal frameworks should specify who is liable when automated systems cause discrimination, exclusion, erroneous decisions or other harm. Independent regulators need sufficient technical expertise, investigative powers and resources to obtain information, conduct audits[8], enforce standards and impose sanctions. Without identifiable responsibility and enforceable liability, transparency alone cannot make algorithmic power correctable.

Protection against bias and discrimination requires intervention throughout a system’s entire lifecycle. Mandatory impact assessments should evaluate likely consequences for different social groups before deployment[9]. Because models, data and social circumstances change, continuous monitoring is necessary after implementation. When discriminatory effects arise, institutions must be able to alter datasets or models, suspend the system or terminate its use. Affected individuals must also have access to appeal, redress and compensation. This prevents historical inequality from becoming automated and institutionally entrenched.

Human control remains indispensable, particularly when decisions affect fundamental rights, livelihood, liberty, credit, welfare or access to essential services. People should have the right to obtain meaningful reconsideration by a competent human decision-maker capable of considering context, proportionality[10] and exceptional circumstances. Certain decisions should never be fully automated. Hybrid models can use algorithms to analyze data and formulate recommendations while leaving final judgement and responsibility with human actors. Human review is meaningful only when the reviewer is competent, independent enough to question the system, informed about its relevant limitations, given sufficient time and contextual information, authorized to depart from the recommendation, and required to provide reasons[11]. Merely confirming an automated output or selecting from options predetermined by the system does not constitute effective human control.

Algorithmic governance must be embedded within existing constitutional and democratic institutions. Its use should remain subject to legality, proportionality, legal certainty, equality and non-discrimination. Independent supervisory bodies should combine legal authority with technical competence, while the public use of AI should be subject to parliamentary oversight, public accountability and democratic deliberation. Decisions about where and how algorithms are deployed cannot be treated as internal technical matters when they substantially affect society.

Safeguards should increase with the severity, scale, opacity, and irreversibility of the possible harm. Low-impact administrative support may require documentation and periodic review; high-impact systems require prior rights assessments, independent testing, traceable human responsibility, notification, and accessible appeal. Uses that are incompatible with dignity, equality, or meaningful individual assessment should be prohibited rather than merely audited.

Several structural tensions nevertheless cannot be eliminated completely. Automation can improve speed, consistency and efficiency, yet justice often requires contextual sensitivity and individualized consideration. Advanced machine-learning models may be intrinsically difficult to explain, while full disclosure can conflict with privacy[12], security and intellectual property. More data may improve accuracy and oversight but can simultaneously enable surveillance and diminish autonomy. Historical bias cannot always be removed through technical adjustments because it reflects deeper social structures. Public regulators may also lack the knowledge and resources available to the private organizations they supervise, creating persistent information asymmetries.

Public attitudes introduce another tension. Excessive distrust may prevent socially valuable applications, whereas uncritical belief in technological objectivity can weaken oversight. Legitimate governance therefore requires informed public dialogue about the purposes, limits and consequences of algorithmic systems. Efficiency and innovation must continually be balanced against autonomy, justice, privacy and democratic legitimacy.

Because complete transparency and control are unattainable, algorithmic governance must remain adaptive and reflexive. Systems require continuing evaluation, institutional learning and revision as technologies and social effects evolve. The objective is not to eliminate algorithms from decision-making, but to ensure that they remain understandable, contestable and correctable. In a democratic legal order, algorithmic systems may support but should not displace the reasoned exercise of public responsibility.

Would you like to explore how a relational understanding of humanity can transform democracy, law, the economy, and public institutions? Click the link below to read the full book, Becoming Human Together, and discover a new framework for building a more humane, democratic, and correctable society within ecological limits.

https://www.researchgate.net/publication/413458836_Becoming_Human_Together_Rethinking_Democracy_Power_and_Institutions_for_a_Complex_and_Changing_World

---

[1] European Union, Regulation (EU) 2024/1689 laying down harmonized rules on artificial intelligence, consolidated text of 27 July 2026, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:02024R1689-20260727.

[2] Solon Barocas and Andrew D. Selbst, ‘Big Data’s Disparate Impact,’ California Law Review 104, no. 3 (2016): 671–732; Virginia Eubanks, Automating Inequality (New York: St. Martin’s Press, 2018).

[3] Karen Yeung, ‘Algorithmic Regulation: A Critical Interrogation,’ Regulation & Governance 12, no. 4 (2018): 505–523, https://doi.org/10.1111/rego.12158; Mireille Hildebrandt, Smart Technologies and the End(s) of Law (Cheltenham: Edward Elgar, 2015).

[4] Andrew D. Selbst et al., ‘Fairness and Abstraction in Sociotechnical Systems,’ Proceedings of FAT* 2019 (2019): 59–68, https://doi.org/10.1145/3287560.3287598; NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (2023), https://doi.org/10.6028/NIST.AI.100-1.

[5] Danielle Keats Citron, ‘Technological Due Process,’ Washington University Law Review 85, no. 6 (2008): 1249–1313; Joshua A. Kroll et al., ‘Accountable Algorithms,’ University of Pennsylvania Law Review 165, no. 3 (2017): 633–705.

[6] General Data Protection Regulation, Regulation (EU) 2016/679, arts. 13–15 and 22; Court of Justice of the European Union, SCHUFA Holding (Scoring), Case C-634/21, Judgment of 7 December 2023, ECLI:EU:C:2023:957.

[7] OECD, Recommendation of the Council on Artificial Intelligence, OECD/LEGAL/0449, amended 2024; Mark Bovens, ‘Analysing and Assessing Accountability,’ European Law Journal 13, no. 4 (2007): 447–468, https://doi.org/10.1111/j.1467-9299.2007.00378.x.

[8] Inioluwa Deborah Raji et al., ‘Closing the AI Accountability Gap,’ Proceedings of FAT* 2020 (2020): 33–44, https://doi.org/10.1145/3351095.3372873; Wieringa, ‘What to Account for When Accounting for Algorithms,’ FAT* 2020, 1–18, https://doi.org/10.1145/3351095.3372833.

[9] Ada Lovelace Institute, AI Now Institute, and Open Government Partnership, Algorithmic Accountability for the Public Sector (2021); Council of Europe, Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, CETS No. 225 (2024).

[10] European Union Agency for Fundamental Rights, Getting the Future Right: Artificial Intelligence and Fundamental Rights (Luxembourg: Publications Office, 2020); Council of Europe Convention 225 (2024).

[11] Ben Green, ‘The Flaws of Policies Requiring Human Oversight of Government Algorithms,’ Computer Law & Security Review 45 (2022): 105681, https://doi.org/10.1016/j.clsr.2022.105681; EU AI Act, art. 14.

[12] Julie E. Cohen, Between Truth and Power (New York: Oxford University Press, 2019); Helen Nissenbaum, Privacy in Context (Stanford, CA: Stanford University Press, 2010).