•  
  •  
 
Vanderbilt Journal of Entertainment & Technology Law

Doa A. Elyounes

First Page

451

Abstract

The goal of this Article is to unpack the “human in the loop” requirement in the process of automation. It will analyze the impact of automation on street-level bureaucrats and lay out the steps policy makers need to take into account to ensure that meaningful human discretion is maintained. This issue is examined by comparing two algorithms related to the use of automation to detect and investigate fraud in welfare benefits. The first algorithm is used by Michigan’s Unemployment Insurance Agency for detecting and investigating unemployment fraud. This is a draconian algorithm with the ability to automatically decide to cut an individual’s benefits and collect debts. The second algorithm is used in the Netherlands by the Ministry of Social Affairs and Employment to detect different types of welfare fraud. It aids human fraud examiners and it automates only the process of data collection and analysis.

This Article concludes that both algorithms could do more to maintain meaningful discretion. In Michigan, automation has left little room for street-level bureaucrats to apply discretion. Thus, this Article suggests that the algorithm be limited to a few segments of the unemployment fraud detection and investigation process. In contrast, the Netherlands’ algorithm allows street-level bureaucrats greater discretion. This discretion is also more meaningful because the human in the loop has a well-defined decision-making role. However, since the algorithm is the de facto authority on who will be investigated, more steps should be taken to ensure that key decisions are overseen by humans. It is important to note that the lack of human agency was only one reason for the failures of the algorithms. Poor technical design and the sociopolitical context that the algorithms operated in were also responsible. The failures stemming from all three reasons are further explored in this Article. Although these cases demonstrate the importance of keeping a human in the loop in an automated process, questions such as what the role of the human should be and how to design the human-algorithm interaction have not received sufficient attention in academic literature. This Article sheds light on these issues.

Download

Share

COinS