Code Critique: Chimeria: Gatekeeper by Fox Harrell

Comments

• JamalSRussell

  January 2018

  The initial thought that comes to mind when I look at the code and play through the game is that the act of passing isn't really reducible to a binary Turing Test of Sylvann/Not-Sylvann (or really even Turing Computability in general). While I appreciate what has been done here, it actually works to obscure the lived and embodied everyday aspects of passing in favor of the discrete moments in which the individual decisions about how to pass are made (in short, passing is a constant interfacing with and negotiation of the Bourdieuian habitus, something that code and Turing Machines in general have real difficulty simulating).

  It makes me think of David Golumbia's description of computation as a cultural logic organized around an impulse toward explaining human social experience through computational processes, which is a dangerous road to go down when attempting to make the experiences of Blacks (or other people of color...or women...or the lower classes) legible to those who have not had said experiences. It ignores the fact that the embodied negotiation I discuss above is always a part of being a minority.

  Now, this isn't meant to be any type of vicious critique of what Harrell and the ICE Lab (boy, is that an ominous acronym in this day and age...) have done in putting this game together; I quite appreciate what they have done here in fact. But it raises real questions (some of which you've already asked) regarding the ability of certain game genres to properly reflect the social experience of minorities (modes of social realism, to crib from Alexander Galloway's Gaming). One could even go further and ask whether any software would be able to do so without encountering the problems I've articulated here.
• markcmarino

  January 2018

  @JamalSRussell

  The initial thought that comes to mind when I look at the code and play through the game is that the act of passing isn't really reducible to a binary Turing Test of Sylvann/Not-Sylvann (or really even Turing Computability in general).

  I agree that there is something in these intentional performative choices that seems a bit reductive. And I also see the point you are making that this code renders discrete and binary something that is not, something in a lived, embodied experience, although I don't think that is the only interpretation.

  In Harrell's simulation (thought experiment?), it is the sum total of actions (and here I think of Butler's sense of an iterated identity) that leads the person to be read as more or less Sylvann. These performative acts are read and accrue, sediment, to use Butler's term. Of course, in this binary world -- or world of races conceived as binary (which may only exist in the mind of the gatekeeper)-- the scale runs only between Sylvann and Brushwood. That reductive model does not capture the complexity of life.

  And yet, I think this instantiation of the Turing Test can also be viewed as a critique of the kinds of Shibboleth used in digital realms, for even though profiling systems may base their readings of humans on much more complex data sets, at some point the data is reducible to discrete factors and the categories they sort humans into can be, like census boxes, fairly rigid and reductive -- as they participate in the production of these racial and ethnic categories.

  So, I agree that this code is incomplete and could not serve to represent the lived experience "the embodied negotiation I discuss above is always a part of being a minority." But I think that this artistic work could be present an opportunity to reflect about how that lived experience is modeled in discrete digital ways in systems of control, surveillance, et cetera.
• JamalSRussell

  January 2018 edited January 2018

  @markcmarino said:
  And yet, I think this instantiation of the Turing Test can also be viewed as a critique of the kinds of Shibboleth used in digital realms, for even though profiling systems may base their readings of humans on much more complex data sets, at some point the data is reducible to discrete factors and the categories they sort humans into can be, like census boxes, fairly rigid and reductive -- as they participate in the production of these racial and ethnic categories.

  So, I agree that this code is incomplete and could not serve to represent the lived experience "the embodied negotiation I discuss above is always a part of being a minority." But I think that this artistic work could be present an opportunity to reflect about how that lived experience is modeled in discrete digital ways in systems of control, surveillance, et cetera.

  This part of your comment actually reminds me of something I had wanted to post (and had discussed quite extensively with @jeremydouglass) during the first week of the WG. When you brought up the "BURN_BABY_BURN" segment of the AGC code and began to unpack its possible implications, my mind immediately went to its possible relationship to the controversy surrounding the use of COMPAS scores to predict recidivism throughout Florida's judicial system. I'm definitely in agreement with what you're saying here, and how the incompleteness of the code compels one to question what effects algorithmic governance can have on the lived minority experience (or how outside observers perceive said experience).
• rogerwhitson

  January 2018 edited January 2018

  Very interesting discussion. I'm fascinated by the fact that -- currently -- such recidivism scores cannot be challenged in court. At least that is the claim made in this review of Cathy O'Neil's Weapons of Math Destruction.

  But grave legal issues are implicated when these models are used in determining a criminal sentence (a practice that is currently used in nine states, according to a 2016 ProPublica report, and which seems to be gaining in popularity). The U.S. Constitution guarantees that a criminal defendant can confront witnesses and challenge the evidence presented against him or her. But O’Neil explains that recidivism risk scores, unlike witness testimony, cannot be recorded and challenged in court, but instead “are tucked away in algorithms, intelligible only to a tiny elite.” Moreover, risk scores can be based on information that would not be admissible in a court proceeding, such as the criminal background of a defendant’s friends and family members, or the crime rate in his or her neighborhood.

  In the case of Harrell's game, the situation can still be "read" as subject to human error and also to the same review processes as any choices made by human authorities. But given that recidivism algorithms are often proprietary, they don't currently have the same opportunities for review. And they are also often assumed to be objective by judges, since they are algorithmic.

  But perhaps we are also seeing a modeling of the O'Neil's problem with recidivism algorithms in the act of reading the code behind Fox's game. The guard is a black box. We know certain things will appeal to him and allow us to pass, but we don't always know which choice is the best one. But by looking at the coding of these decision trees, there's another chance for a kind of evidential challenge.
• jmjafrx

  February 2018

  @JamalSRussell writes:

  While I appreciate what has been done here, it actually works to obscure the lived and embodied everyday aspects of passing in favor of the discrete moments in which the individual decisions about how to pass are made (in short, passing is a constant interfacing with and negotiation of the Bourdieuian habitus, something that code and Turing Machines in general have real difficulty simulating).

  Thanks for this. It helps me put into words what I've been struggling with in this (really great--thanks @markcmarino !) code example all week. How can code (or games like this) capture that embodied experience? And for me, so much of the embodied experience is about racialized terror and state violence. So how can a game capture that state of terror (and hopefully critique it)?

  So that takes me back to this, sort of the basic narrative here in the XML tree:

  <narrative-element>
              <type>entrance</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
          <narrative-element>
              <type>motivation</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
          <narrative-element>
              <type>group-evaluation</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
          <narrative-element>
              <type>naturalization-trajectory-evaluation</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
  

  (And if I'm wrong on how I'm reading the code in what I say next, let me know) So it looks like the game of the Sylvan trying to get past the gatekeeper is broken down into four narrative elements: entrance (or encounter with the gatekeeper), motivation (having to prove yourself to the gatekeeper), the gatekeeper's eval, and "naturalization-trajectory-evaluation" which is essentially the gatekeeper stamping your passport or sending you away. It structures the user experience so that they are always in the wrong, essentially proving themselves worthy from gatekeeper's perspective--so how the gatekeeper feels about you has supremacy in this narrative. And, of course, the use of naturalization here is a charged word to use.

  If I gave this to a class and then limited myself and them also only to four narrative elements, could I flip this? Would it work to have something like this:

  <narrative-element>
              <type>rupture</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
          <narrative-element>
              <type>shapeshift</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
          <narrative-element>
              <type>dark-sousveillance</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
          <narrative-element>
              <type>the-people-could-fly</type>
              <min>1</min>
              <max>1</max>
          </narrative-element>
  

  content tags for the rupture clauses would offer storylines of possibility for the borders and the violence of borders and explain why the user is at the gate ("Your sister is sick and you need to be with her")

  shapeshift clauses would offer ways that the user has had to change themselves or suggest what they have gone through to meet the arbitrary border criteria ("You colored your hair purple because Brushwoods have purple hair") or ("This is your third time at the border because your pass keeps getting rejected; each time you've paid a fine") etc. There could even be an added money factor where the game shows in the corner how much you've already spent on even arriving at the gate.

  dark-sousveillance clauses would be similar to group-evaluation clauses but instead riff on Simone Browne's articulation of POC resistance to surveillance that comes in the form of dark sousveillance--or rendering yourself visible in particular ways for the purpose of avoiding surveillance. It'd be great if content like "Guard approves" captured that dark sousveillance as resistance -- "Guard, fooled by purple hair, smiles." Or "Guard does not hear Sylvan accent."

  the-people-could-fly clauses -- which is a riff on black diasporic vernacular stories of enslaved who flew back to Africa in resistance and also in homecoming and community reformation -- would replace the somewhat neoliberal use of naturalization-trajectory-evaluation in the code.

  And so on...this is just a riff and a brainstorm, but code like this not only gestures to the vernacular and lived reality of people encountering borders, experiencing state terror of displacement or even the microaggressions of gatekeeping in the code itself, but also force us to think about what a game would look like that captured the feeling of terror but also that evervescence of community possibility that is the experince of les damnés de la terre.
• markcmarino

  February 2018

  @jmjafrx I love to see this critique coming in the form of new code.

  I wonder if part of the problem that @JamalSRussell was having with this code also arises out of the discrete nature of the interactions. Not only is passing here a conscious act along a spectrum between two binaries, it is also being measured one action at a time. And, though this performance only partially moves the read identity, the exchange itself is reduced to a discrete transaction.

  What would happen if the script were flipped even further:
  The player writes an open answer that is then read by the gatekeeper (a la Facade)
  The player has choices about whether or not to stay at the gate.
  The gatekeeper's rating system changes over time due to interactions.
  The gatekeeper's response can be read by the person at the gate.
  The player can choose to inhabit an identity that is neither Sylvann nor Brushwood
  The player can choose to cross through the gate with a provisional status

  I still want to think about how the player's interaction could be represented not as chosen but as a combination of passive and active choices. What might that look like?

  Another model of this code might allow the person to choose the strategy rather than the manifestation.

  How else might we write back to this code? Imagine it. Talk through it? Of course that does not diminish Fox's code, which has inspired these adaptations.
• JamalSRussell

  February 2018

  @jmjafrx The question of what kind of game can properly articulate the issues of embodied racial experience (or whether a game can do this at all) is a tough one, and one that I struggled to answer as I wrote my first response to this thread. My initial answer was that it would have to be some kind of Sims hack (SimBlackness or something to that extent; inevitably problematic in its execution), but I figure that it would run into some of the same issues that Gatekeeper runs into.

  I think that the code you've written moves in the direction of a solution, but my initial thought would be to say that looking for a "solution" may not be the way to frame any act of activist or creative coding. Part of media theoretical work is attending to the gaps between the operations of media objects and how those objects are in dialogue with larger cultural discourses (a problem of interfaciality in both the most material and most abstract sense). If we understand code's inability to model the embodied nature of passing as a feature of code/algorithms/Turing Machines instead of a bug to be programmed out, then the issue at hand is not to make a perfect game, but to make something that gets us close, and then let the questions and answers of more traditional humanistic work get us the rest of the way there (as we have been doing here and in other threads throughout the working group).

  @markcmarino That's one hundred percent the issue, which I admit may just stem as much from having lived (almost) 28 years as a Black man than from any particular academic interest.

  I love the suggestions you make, specifically because they would work to move away from the extremely discrete and binarized nature of how passing is represented in the game. Not because the act of passing doesn't reduce to a binary choice sometimes (the low-level and most dangerous decisions reduce to binaries more often that we'd like to admit), but because those binaries are nested within larger complexes of, as you term them, passive and active choices (and that's not even getting into issues such as colorism, for instance). As you say, this isn't to tear Gatekeeper down, but to think about how one might extend some of the work he's doing by writing code and creating an interface/gameworld that might fill some of the gaps that Harrell's work leaves unexplored.