Question: Our article advocates for the close reading of ancillary deep learning code. Although this code does not do any deep learning work per se, it still contributes to the language generation process (e.g. decoding the model's raw output, content moderation, data preparation). We did give some thought to how to do the work of extending a CCS analysis to the weights and the model and arrived at this conclusion: "Even more important to the running of the original GPT2 model are the weights, which must be downloaded via “download model.py” in order for the model to execute. Thus, one could envision a future CCS scholar, or even ordinary user, attempting to read GPT-2 without the weights and finding the analytical exercise to be radically limited, akin to excavating a machine without an energy source from the graveyard of 'dead media.'" Nearly three years on from the composition of our article, in the context of the API-ification of LLMs, we can build upon Evan’s earlier question and ask not only should we/could we, but also how can we read what we termed “core” deep learning code?

Note: This article was written in 2021.

Here's the original abstract, with key words and thoughts italicized.

"Reverse Engineering the Gendered Design of Amazon’s Alexa: Methods in Testing Closed-Source Code in Grey and Black Box Systems"

​This article examines the gendered design of Amazon Alexa’s voice-driven capabilities, or, “skills,” in order to better understand how Alexa, as an AI assistant, mirrors traditionally feminized labour and sociocultural expectations. While Alexa’s code is closed source — meaning that the code is not available to be viewed, copied, or edited — certain features of the code architecture may be identified through methods akin to reverse engineering and black box testing. This article will examine what is available of Alexa’s code — the official software developer console through the Alexa Skills Kit, code samples and snippets of official Amazon-developed skills on Github, and the code of an unofficial, third-party user-developed skill on Github — in order to demonstrate that Alexa is designed to be female presenting, and that, as a consequence, expectations of gendered labour and behaviour have been built into the code and user experiences of various Alexa skills. In doing so, this article offers methods in critical code studies toward analyzing code to which we do not have access. It also provides a better understanding of the inherently gendered design of AI that is designated for care, assistance, and menial labour, outlining ways in which these design choices may affect and influence user behaviours.

*

Here are some discussion questions to get us started:

1. How do humanities- and social sciences-based theories and methods augment CCS? For instance, in this article, I describe Anne Balsamo's method of "hermeneutic reverse engineering," through which I used methods of close reading and discourse analysis to analyze Alexa's responses, the Alexa skills kit console, and some of the code used to build Alexa skills.

2. What other examples of closed-source and proprietary code and software do you think would benefit from a mixed method approach to reverse engineering their design?

3. What does CCS lend to analyses of gendered, racialized, and class-based biases in code and software?

4. Comparisons are sometimes made between language-based AI such as Alexa and ChatGPT, but in addition to these AI systems stemming from different types of NLP design, they also create different user expectations about what they should be used for and how users are expected to interact with each interface. Discuss.

Abstract

Our article highlights the increasing importance of code in computational literary analysis and the lack of recognition and visibility code as scientific output currently receives in scholarly publishing. We argue that bespoke code (i.e. purpose built analytical code intended for single or infrequent use) should be considered a fundamental part of scholarly output. As such, a more overt inclusion of code in the scholarly discourse is warranted. An unanswered question is what a proper methodology for this would be.

As an experimental contribution to developing such a methodology we introduce the concept of “defactoring”. We propose defactoring as a technique for critically reading and evaluating code used in humanities research. Defactoring involves closely reading, thoroughly commenting, and potentially reorganizing source code to create a narrative around its function. This technique is used to critically engage with code, peer review code, understand computational processes, and teach computational methods.

We describe a case study where we have analyzed in this way the code associated with a publication in literary studies by Ted Underwood and Jordan Sellers (“The Longue Durée of Literary Prestige”). Based on our case study experience we question the separation between scholarly publications and code, and we advocate breaking down these boundaries to create a more robust scholarly dialogue. Linking code to theoretical exposition can enhance scholarly discourse and invites further exploration of the relationship between literary interpretation and computational methodology.

Finally, we also reflect on the challenges we met in publishing an article that combines theoretical discussion with defactored code, and we highlight the gap between scholarly argument and case-study material that is enforced by current academic publishing platforms. We suggest that there is a need for academic genre conventions for publishing bespoke code and we proposes the idea of a notebook-centric scholarly publication that integrates code and interpretation seamlessly.

Questions

I have been wondering again off late (but this is years old unease tbh) why scholars and scientists seem so indifferent to the quality of the code and algorithms they routinely apply in research. To me it seems that the technical and methodological quality of code applied in any analytical fashion should be interrogated quite rigorously. Any tiny error in that code may completely invalidate any finding. However, all our quality control processes (peer review, metrics, academic crediting, institutional evaluation – flawed as they may be by themselves) are almost exclusively aimed at the final outcome of research: the publication. One could argue that a thorough discussion of the code applied should be part of any research paper and should therefore have been scrutinized during peer review. But this is almost never the case. Only a facile methodological abstraction is presented on paper, and there is the assumption that this expresses what the code actually does. Arguably in many cases tailor made algorithms will not exactly do as the author will have us believe. (To substantiate this at least a little bit: a colleague of mine who is a research software engineer, almost completely refuses to use any Python libraries for statistical analysis on account of her finding them almost all flawed tot lesser or greater extent in mathematical precision.) Many reasons have been forwarded why we as researchers do not engage with code in some peer review fashion: lack of technical skills and knowledge, the impossibility of adding yet another infeasible task to the academic process, lack of resources, misplaced trust in perceived impartiality and mathematical correctness of code, assumption that quality control of code is being fully covered by RSEs (Research Software Engineers). All of these explanations are part of the problem. However, I cannot escape the impression that we mostly hesitate at taking intellectual responsibility because we acknowledge the sheer insurmountable effort involved with organizing and executing code peer review, all the while being rather poignantly aware that we are methodologically falling short. Will code peer review by in a nascent state perpetually, because it presents us with too many inconvenient truths?

Question: The bulk of the article is about the evolution of code and the way this evolution was motivated by the suppression of state, in order to make code a better epistemic object. It ends on a bit of a dismal note: with machine learning algorithms, we get a peculiar epistemic inversion, where the state of the machine (all the learned weights on the neural network, for example) holds pretty much all the interesting parts, but state is precisely what has been strategically ignored in order to develop the very powerful epistemic tool that is code. The big million dollar question: now that we've spent 70 years developing our epistemic tools in the exact opposite direction, can we make heads or tails of all of these weights? If we construe critical code studies very narrowly, this is not a critical code studies question—weights are not code, they're not written by humans, and they're not even really read by humans (yet!). But on the other hand, this thing that we do—look at a technical symbolic language and extract a more complete meaning than a compiler does, than many coders are aware of, etc.—this thing looks kinda similar to the problem of making sense of a trained system. Further, trying to read weights looks a lot like Mark Marino's original proposal for CCS: software criticism is great, but we can't ignore the source code -> LLM criticism or training data criticism is great, but we can't ignore the weights.

So my big question for the group is: what now? When the weights and other state data might be more important than the code, what is the role of reading code? Should we/could we expand code studies methods to look at the state of machine learning algorithms, the weights or other data that make a given algorithm work?

Read the paper here

Questions:

1. This paper sets out a new research programme in digital humanities and critical code studies related to exploring concept formation and how rather than purely discursive, extra-discursive prescriptive elements from software (and para-code) can be mobilised to stabilise it. What other examples of similar approaches are seen in the literature or in technical work?
2. Tracing as a method in critical code studies seems to me to have great potential for traversing the code-domain and social (the socio-code) in order to understand the importance of the interrelation of the two. What other tools can we bring to examining the socio-code for this type of code/concept analysis that moves between social and algorithmic levels of description? (Here I am thinking of techniques such as Entity Relationship Diagrams, UML, and other techniques).
3. One of the problems with this method is that it requires the management of multiple levels of discourse and code and their interdependence. As this was a relatively scoped analysis the difficulty of juggling the levels was relatively straightforward, but it would be good to be able to use some form of digital asset management, code management tool, IDE or other technique to assist with the process. Perhaps something like NVivo might assist with this? Do people have experience of managing multiple, inter-layered and inter-textual document analysis that might help with this?
4. The Concept Lab https://concept-lab.lib.cam.ac.uk (University of Cambridge) studied the architectures of conceptual forms in discourse. Are people familiar with this approach to concept mapping and how can approaches such of these be incorporated into critical code studies?
5. This approach of code/concept critique are clearly much more easily undertaken in open source work, but what are the methods we can use in proprietary systems?
6. Does this approach result in a tendency towards a pragmatic mode of analysis within a specific problem situation, and which may just reflect an instrumental approach to software rather than genuine insight? How does one connect these case studies to more generalisable situations, perhaps to questions of common sense assumptions and/or embedded values and norms?
7. There are obvious links to questions raised by the notions of explainability and interpretability in software, AI and automated decision systems. Does critical code studies have implications for the debates over explainability vs interpretability and what might the assumptions built into explainability mean for critical code studies?

In late fall 1965, John Kemeny wrote a 239-line BASIC program called FTBALL***. Along with his colleague Thomas Kurtz and a few work-study students at Dartmouth College, Kemeny had developed the BASIC programming language and Dartmouth Time-Sharing System (DTSS). BASIC and DTSS represented perhaps the earliest successful attempt at “computer programming for all,” combining English-language vocabulary, simple yet robust instructions, and near-realtime access to a mainframe computer. This article takes a closer look at FTBALL as a crucial program in the history of “programming for all” while gesturing to the tension between a conception of “all” and FTBALL’s context in an elite, all-male college in the mid-1960s. I put FTBALL in a historical, cultural, gendered context of “programming for all” as well as the historical context of programming language development, timesharing technology, and the hardware and financial arrangements necessary to support this kind of playful, interactive program in 1965. I begin with a short history of BASIC’s early development, compare FTBALL with other early games and sports games, then move into the hardware and technical details that enabled the code before finally reading FTBALL’s code in detail. Using methods from critical code studies (Marino 2020), I point to specific innovations of BASIC at the time and outline the program flow of FTBALL. This history and code reading of BASIC FTBALL provides something of interest to computing historians, critical code studies practitioners, and games scholars and aficionados.

The FTBALL*** code is in an appendix to the article, and also available on Dartmouth emeritus professor Peter Doyle’s website and the Vintage Basic website.

Here, I want to focus on the idea of witness marks in code. I’ll explain what they are and give a long preamble to ask:
• What witness marks do you see in the code that you study?
• Who are the witnesses to the construction of your code, and how might you connect with them to learn more about the processes of the code’s composition?

Witness marks

Horologists restoring ancient clocks note the little marks and dents in the clock’s workings to see the path of previous restorers, undo the damage they might have done, and replace the missing screws or tiny gears that make the clock function. Those little bits of human evidence in the machine are called witness marks. (I learned about these from the S-Town podcast, which features a horologist and opens up with a description of witness marks.)

One feature of early BASIC’s line numbers is that by looking at the line numbers you can often glean the process of composition from the code itself. BASIC Convention calls for line numbers by 10s, like so:

10 PRINT "Hello World!"
20 GOTO 10

But if I decide later that I don’t want in infinite loop in my code, or if I want to add something else to the message, I might revise my code like so:

5 LET N = 0
10 PRINT "Hello World!"
12 PRINT "This is Annette."
14 LET N = N + 1
20 IF N < 2 THEN GOTO 10
25 END 

That way, my code will terminate after printing my introductory message twice.

A code sleuth analyzing this program later, without me around, would be able to surmise from the in-between-10s line numbers that I added those lines in later versions of the code. The line numbers in early BASIC mattered a lot because GOTOs would point to them. Not such a big deal for my little Hello World, but for a program of any length, it would have been a giant pain to renumber the lines. Moreover, interface for code revision didn’t give random access to the code—you couldn’t just see the whole program and line edit it like I’m doing now with my text in my word processor. Instead, you used the I/O console to add lines to the code in the computer’s very limited memory. You could overwrite previous lines of code or add new ones, but it was too difficult to renumber lines. So, these odd-numbered lines serve as witness marks for the construction of the code, and clues to those of us who come afterward, trying to understand how it works.

In reading the code for FTBALL***, written one Sunday by John Kemeny—one of the co-inventors of BASIC—I wanted to see how it worked, but also how it was composed. I can guess, for instance, that the PLACE KICK was added later in the session because of the line numbers:

5001 PRINT "PLACE KICK"
5005 LET F = -1
5006 IF R > .5 THEN 5010
5007 PRINT "KICK IS BLOCKED***"
5008 LET Y = -5
5009 GOTO 1480

So I can see a bit about how Kemeny’s mind worked in composing this program, which is pretty cool.

Witnesses

John Kemeny passed away in 1992, long before I did any research on BASIC, but years after I’d dabbled in BASIC myself, sitting at my Commodore 64 as a kid in the mid-80s. BASIC was everywhere in the mid-80s, as the CCS classic 10PRINT notes. My research on BASIC was, in part, a way to connect with my own early history with computing. But it also connected me with Kemeny, who was an early advocate of coding literacy, a former assistant to Einstein, and by all accounts a genius and a great guy.

It also connected me to Thomas Kurtz, the other inventor of BASIC, and an amazing person as well. Kurtz let me interview him in 2017, wrote me memos on how BASIC was put together, and then, at age 95, once this DHQ article was published, approved my take on it. He was a witness to both the construction of BASIC and my research on it 60 years later. I also emailed with Stephen Garland and John McGeachie, who were part of the Dartmouth undergraduate team working on ALGOL, BASIC and DTSS in the mid-1960s, for some details and clarifications about BASIC. So, to augment the witness marks I saw in the code, I also reached out to witnesses of it so I could understand the environment in which Kemeny wrote the code. Both the witnesses and the witness marks were key to my reading of BASIC FTBALL***.

Questions

This is a long preamble to ask:
• What witness marks do you see in the code that you study?
• Who are the witnesses to the construction of your code, and how might you connect with them to learn more about the processes of the code’s composition?

The Less Humble Programmer investigates the aesthetics of esolangs (esoteric programming languages) as drawing from / challenging / reacting to two competing aesthetic impulses in mainstream computing and computer science:

• The first is, borrowing from Dijkstra, Humbleness. This is the neutral voice of code that downplays personal style. It values clarity above all else, including, for the most part, efficiency. This is code that is friendly to corporate coding, easily understood by others. It is how we, for the most part, are taught to code, even creative coders.

• The second is conciseness and cleverness. This is the view of "elegance" drawn from mathematics, valuing of the shortest expression that says or does the most. It prizes conciseness and (sometimes) efficiency over clarity. It is familiar from other hacker arts (e.g. demoscene) and was commonplace in early computing when extreme efficiency was necessary.

In Wendy Chun's Sorcery and Source Codes, she discusses the move in mainstream coding from the second aesthetic to the first, when programming became professionalized and corporatized. But the impulse to show technical wizardry through clever and unreadable code never went away; it simply finds other avenues for expression.

An Esolang Example

Consider brainfuck, the language that infamously recreates the Turing machine offering only eight commands to the programmer, each represented by a punctuation mark. The + adds one to a memory cell. < and > let us navigate through memory cells. The square brackets loop until the memory cell pointed to is at zero.

Brainfuck has an absolute refusal of Humbleness; there is no neutral way to express anything in the language and there are no standards to adhere to. One must make a choice even for the most trivial of code.

If we want to print 46 to the screen, we could have 46 +s followed by . (the command to print):

++++++++++++++++++++++++++++++++++++++++++++++.

This is not especially readable, as it is too many +s to recognize at a glance.

Then there is the multiplication pattern: one cell starts with a value and counts down, each time adding a second value to the neighboring memory cell, effectively multiplying the first by the second via a loop of addition. It multiplies 5 by 9 and then adds 1:

+++++[>+++++++++<-]>+.

And here is the shortest (known) brainfuck algorithm to print 46:

-[+>+[+<]>+]>.

It is extremely short, has an appealing rhythm of +s and brackets, but is mostly unreadable without playing out a complex algorithm in ones head. It is a loop within a loop and gets to 46 by walking back and forth across six memory cells. It wraps past zero (in brainfuck, zero minus 1 is 255) many times. It is efficient in number of characters, but is the slowest in execution.

These alternate solutions show how brainfuck may appear chaotic, but it prizes the re-assertion of order within a seemingly chaotic space. It is chaos to be tamed. The most celebrated approach will be the shortest or the most efficient.

Questions

Esolangs have matured as a medium since 1993, when brainfuck appeared, and deal with a wider set of concerns. I've been thinking lately about how current esolangs build on these foundational aesthetics. The language Unnecessary (2005), for instance, refuses any form of computation: its only valid program is the one that doesn't exist. This is an example of the kind of idea art that allows no space for technical wizardry, by taking formal play with language design to its most extreme, yet feels indebted to the early esolangs in that choice. I wonder how other current esolangs engage (or do not engage) with these aesthetics of early esolangs.

Also, how is this thinking reflected in code poetry? Code poetry is one of the other code forms (along with esolangs and demos) where one has the freedom to break from clarity. At least some code poets, (such as the recent book ./code –– poetry, poke fun at the mathematical idealism of code -- in their case via the (much hated) golden spiral.

Question: In the article, I suggest that Mark C. Marino's conception of code legibility requires interrogation, because it is informed by a "dubious premise that programming languages, including natural language programming languages, should strive for (or can only function using) a one-to-one equivalence between named operations and methods and what those operations and methods do" (para. 21). What are peoples thoughts about these different perspectives on code legibility?

Question: The article suggests that, in the cause of a general code literacy, CCS should "study, explicate the potential of, and advocate for programming languages that strive to emulate 'natural languages' so that code literacy does not require a highly specialized literacy far removed from the common literacy that most people possess" (para. 26). What are the potential pros and cons of such a suggested approach?

Abstract: Lillian-Yvonne Bertram's 2019 book of poetry is titled Travesty Generator in reference to Hugh Kenner and Joseph O'Rourke's Pascal program to “fabricate pseudo-text” by producing text such that each n-length string of characters in the output occurs at the same frequency as in the source text. Whereas for Kenner and O'Rourke, labeling their work a “travesty” is a hyperbolic tease or a literary burlesque, for Bertram, the travesty is the political reality of racism in America. For each of the works Travesty Generator, Bertram uses the generators of computer poetry to critique, resist, and replace narratives of oppression and to make explicit and specific what is elsewhere algorithmically insidious and ambivalent. In “Counternarratives”, Bertram presents sentences, fragments, and ellipses that begin ambiguously but gradually resolve point clearly to the moment of Trayvon Martin's killing. The poem that opens the book, “three_last_words”, is at a functional level a near-echo of the program in Nick Montfort's “I AM THAT I AM”, which is itself a version or adaptation of Brion Gysin's permutation poem of the same title. But Bertram’s poem has one important functional difference in that Bertram's version retains and concatenates the entire working result. With this modification, the memory required to produce all permutations of the phrase, “I can’t breathe”, is sufficiently greater than the storage available most computers, so the poem will end in a crashed runtime or a frozen computer--metaphorically reenacting and memorializing Eric Garner’s death. Lillian-Yvonne Bertram's Travesty Generator is a challenging, haunting, and important achievement of computational literature, and in this essay, I expand my reading of this book to dig more broadly and deeply into how specific poems work to better appreciate the collection's contribution to the field of digital poetry.

Question:

Looking back at this essay that first began as a CCSWG post four years ago, I am interested in how Bertram's work (including Travesty Generator and more recent projects) continues to explore computational creativity at the edges of LLM. In a recent If, Then talking about their new chapbook, A Black Story May Contain Sensitive Content, Lillian-Yvonne spoke about the value of "small" and "bespoke" language models and used the phrase "creative research" to frame their project working with a corpus of writing by Gwendolyn Brooks. All this has me thinking about the value of (let's say) artisanal computational poetics as an alternative to LLMs. I have some ideas, but I'll pose this is a question: How and why do tiny language models, "code as text" poetics, and new work built on these legacies critique or resist the operations of large-language models?

Abstract:

A code-critical close reading of two related works by Daniel C. Howe. The artist's Automatype is an installation that visualizes and sonifies minimal-distance paths between English words and thus explores a possibility space that is relatively familiar to western readers, not only readers of English but also readers of any language which uses Latin letters to compose the orthographic word-level elements of its writing system [Howe 2012-16a]. In Radical of the Vertical Heart 忄 (RotVH) Howe engages with commensurate explorations in certain possibility spaces of the Chinese writing system and of the language’s lexicon. Translinguistically these spaces and, as it were, orthographic architectures, are structured in radically different ways. A comparative close reading of the two works will bring us into productive discursive relationship not only with distinct and code-critically significant programming strategies, but also with under-appreciated comparative linguistic concepts having implications for the theory of writing systems, of text, and of language as such. Throughout, questions concerning the aestheticization of this kind of computational exploration and visualization may also be addressed. His website is programmatology.com.

David Berry, Jason Boyd, Kevin Brock, Evan Buswell, John Cayley, Lai-Tze Fan, Zach Mann, Daniel Temkin, Annette Vee, Zach Whalen, Joris Van Zundert

The two special issues of Digital Humanities Quarterly, one published in 2023 and the other forthcoming, focussing on Critical Code Studies represent a watershed moment. For while we have published essays in electronic book review and other venues, these collections mark the first set of scholarly explorations of code gathered in issues dedicated to that topic. Most of the authors have published in more than one working group, and some of the content has been developed from discussion threads in working groups. We are grateful to DHQ and all the authors, editors, and peer respondents who have made these two issues possible.

For this week, we have assembled a group of authors from these two issues to discuss with us their reflections on the articles and what they mean for CCS. We have also set up discussion threads for the articles. (See list below) You will notice the diversity of the code objects and approaches reflects the wide ranging applications of Critical Code Studies.

Here is a link to the first of the special issues.

Discussion Questions

For the authors:
What did we authors learn from each other’s articles?
What connections do we see between our articles in methodology, discoveries, or theoretical approaches?

From all of our participants:
What’s one article that catches your interest and why?

And our cohosts have further questions!

Join also the individual threads by David Berry, Jason Boyd, Kevin Brock, Evan Buswell, John Cayley, Lai-Tze Fan, Zach Mann, Daniel Temkin, Annette Vee, Rita Raley and Minh Hua, Zach Whalen, Joris Van Zundert