Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Categories
In this Discussion
Cybernetics and “Command and Control” in kubectl (2022 Code Critique)
Code: kubectl - Kubernetes command line interface
Author: Google
Date: Current open source framework.
Github: https://github.com/kubernetes/kubernetes
“Kubectl” or “Kube Control” is the command-line interface for Kubernetes, or K8S, the server orchestration framework originally developed by Google. It is used to manage software principally in the cloud, where it can scale up software applications based upon a specification or manifest, as well as via monitoring the live expenditure of computing resources. The command-line interface works on top of these deployment definitions, much like the Unix command-line allows a user to interrogate the state of a runtime system.
K8S’ name and its history at Google are problematic though: “Kubernetes” is the root word or variant of the root from which we get the "Cybernetics" of Norbert Wiener, and an early name of the platform was "Borg", based upon the Star Trek aliens who assimilate/colonize other alien civilizations and boldly announce to them that "resistance is futile”. In addition, the architecture of the platform demarcates "master" nodes that run within a "control pane" and "worker nodes" that host applications and run them. The fundamental unit of the nodes is the “pod,” which is a collection of application container environments running code. Especially with the name of the command-line interface suffixed with -ctl, which is an abbreviation for “control” the K8S platform seems to be a direct embrace of the discipline of cybernetics as “command and control”.
That said, K8S is powerful, and could be seen to facilitate exploratory programming, coding ateliers, and Marxian dreams of allowing proletarian developers to instantly instantiate a technological and productive base that determines how political and cultural bodies are represented. In particular, two powerful features of Kubernetes/kubectl are namespaces and labels, allowing a developer to organize/orchestrate server resources, as the so-called helmsperson.
Labels more specifically allow search-engine-like queries on these resources, so that a developer could summon all components running their app via just a few commands—despite the fact that this app would be built from nodes of worker pods in a vast homogeneous sea of other worker pods.
For example, labels can be used to group pods by application name or release type as demonstrated by Marko Lukša in his Manning guide to K8S, but the labels can be anything. With kubectl, I can create a YAML manifest:
apiVersion: v1
kind: Pod
metadata:
name: myApp-cloud-001
labels:
app: picturebook
rel: beta
spec:
containers:
- name: myApp-Container
image: myApp-w-nodeJS
ports:
- containerPort: 8080
...
And I then can create a cloud cluster, replicating nodes with pods by passing the manifest file above as an argument:
kubectl apply -f pod.myapp.yaml
And then, after K8S has spun up my app and distributed it potentially to multiple pods all automatically, I can summon these pods with a command like the following, specifying which ones I want to fetch via the labels I've defined:
kubectl get po -L app,rel
Or:
kubectl get pods -l app=taxes, rel=beta
Or another query using kubectl:
kubectl get pods -l 'app in (picturebook, taxes)' -L app
These searchable labels exemplify inaugural Cybernetics, as “labelling” has been at the center of development of programming after World War II—whenever we speak of variables, identifiers, keywords and code. Because first generation cybernetics formalizes symbolic language within the same cultural and disciplinary space as Lacanian psychoanalysis, so a kubectl label is a use of language becoming absolute and formal and integrated into societal controls. In fact, K8S labels embody applied programming, but also command and control, since the entire lifecycle of their applications is automated through selectors (like in the queries above) on these labels. Note too that, scaling up, scaling down, and restarting pods and containers, a given application never terminates—in true Borg fashion.
At the birth of the Unix computer, if a command to list the file contents of a single computer made the human-computer interface operative, so kubectl has multiplied this system interrogation for thousands of machines simultaneously—as one giant distributed machine. The “power” of the command-line is propagated to kubectl, which can also wrap the operating system command-line in serverless instructions that produce a command result while bypassing runtime systems altogether in some cases**. In effect, in the figure of the software platform K8S, the metaphor of command and control in its specific post-war flavor has been resurrected. But there are many hidden costs of this magic-seeming orchestration across thousands of machines, whether environmental or related to the denial of death and materiality.
As a result, the wealth of critiques of cybernetics, even those from its own age, must be applied to cloud computing. Deleuze and Guattari are helpful here, in offering better metaphors for technology: the notion of the machinic is a critique of the Lacanian signifier-signified relation assumed by the Western labelling subject—while not abandoning technological metaphors. Gilbert Simondon’s work is relevant here too: in the notions of individuation and mechanology, value of technological artifacts in their “aseity” is deferred until interested actors, readers, and cultural laborers productively articulate their interconnections.
As it turns out, one of the lesser known features of K8S is what is called its affinity and anti-affinity definitions, which can be added to a deployment specification or manifest. An affinity definition is a set of numeric values that specify the probability of hardware and system conditions by which containers and pods should group themselves together and reform pod instances when one or more of them has been temporarily terminated to be replaced by other pods. No doubt, this feature is a tool of automation still ultimately tied to command and control. But could the general notion of affinity rather than a helmsperson offer a way out of current K8S metaphors?
Questions
- What should Kubernetes/kubectl be called instead? :-)
- What would Kubernetes look like if it were built from the ground up on affinities, which would be its raison d’être? What if corporate technology were forced to build its systems on “affinity machines”?
- What are some ways “always running” systems are problematic? Does Kubernetes enforce a status quo in its highly programmatically deterministic specifications and manifests?
** For example, a command to look up a DNS setting on a resource:
kubectl run -it srvlookup --image=tutum/dnsutils --rm --restart=Never -- dig SRV taxes.default.svc.cluster.local, after Lukša.
The rm removes the server image, but the command still produces an output.
Comments
Kubernetes has a huge barrier to entry for non-server/unix people. I am an experienced software engineer and architect and I find the set up and maintenance (from reading the documentation nd playing around with it) largely incomprehensible. I have used it in the past and I know it can be really good when set up correctly but it suffers from a lot of the assumed type of technical experience that a lot of open source tools do. I wrote about this issue under the p5.js discussion, but it basically comes down to an assumed history of knowledge and experience that not everyone has, particularly those who come from minority and non-tech-normative pathways. So I would argue that this barrier to entry and this assumed background does a lot to enforce the status quo in the Kubernetes environment and community, which then bubbles through into the specifications and manifests as well as the general processes and focuses of the tools.
I highlight again I actually really like the functionality of Kubernetes and I think it has amazing potential to provide contained portable server functionality for creative projects but I do think the barriers to entry are high.
@annatito, thanks for your comments. Certainly Kubernetes has a huge barrier to entry. At my work for a non-profit, I need to migrate a project from Docker CE to Kubernetes, and so am starting to learn more about the platform, without having tons of experience with it just yet.
But even without being a seasoned veteran in this platform, the motivation for my code critique stems from a strong disbelief concerning how the project renders itself within the history of computing and culture. It seems totally unreflexive to basically name a software platform, unabashedly “cybernetics”, which is pretty much what calling it “Kubernetes” does. I don’t think the systems that Google builds qualify as the same watershed moment as Wiener’s cybernetics. It seems arrogant to name a software platform in this way.
So thanks for the forewarning to me that working with this platform could be difficult. But I think that purely from the way the platform originated and works necessitates a targeted cultural critique. I say this at the same time that the platform does offer possibilities for pushing the boundaries of cultural critique as one goes about the daily labor of software development.
I think formulating this critique of the cultural aspects of the platform can equip us to begin to do work with it — or so I hope. I wonder if using elements in the platform like the affinity definitions can allow us to reshape how Kubernetes is being used. This undoubtedly will butt up against the knowledge requirements the platform assumes, but maybe it would be worth it for the creative applications to which you alluded.
Can you explain what you mean by this?
In terms of affinities: the point of k8s is to capture the essentials of a system architecture (at a particular level of abstraction), while avoiding cluttering the description with incidental detail.
So whilst a concrete cluster may comprise a set of physical nodes, the idea underpinning k8s is that we can write a description of an architecture and various kubernetes controllers will attempt to get the system to converge on an instance of the specified model.
But as @annatito points out, even the transcription of a pretty trivial picture like this into yaml:
... ends up with a pretty complex set of yaml documents. (That's where a higher-level tool like
helmenters the picture - abstracting patterns like this again, and producing a higher-level component that's "opinionated" - that is, enshrining some decisions in code - and leaving other values open to the instantiating user to select. The result is a higher-level component that parameterised by fewer values. It's still not trivial to instantiate, however.)In this picture there are a bunch of concrete details which really don't matter: the IP addresses of internal endpoints, etc; or the specific hardware hosts that particular pods are brought up on. As it converges, k8s picks concrete values (eg, picking particular nodes to instantiate pods on) to satisfy a set of constraints. Those constraints might be tacit (eg, there might be some default anti-affinity constraints in a topology-aware scheduler) or they might be explicit - coming from the affinity constraints, amongst others.
The solutions found to those constraints tend to be greedy (that is, each placement decision made independently*).
Putting affinity first and foremost would lead to more clutter in yaml descriptions - clutter that's arguably unnecessary in order to get the job done. (yaml isn't a great language to describe these pictures in the first place.) An increase in clutter means a concomitant raise of the barrier to entry. How would you suggest this be avoided?
(* The reason for this is that general packing problems are hard, in a specific technical sense - k8s can typically find "good enough" solutions where it's not massively resource-constrained.)
The point of these placement constraints is pretty mundane: the goal is typically to avoid replicated software components in a system from occupying the same piece of hardware, or the same TOR switch, or the same power supply, etc; or to colocate corresponding components in order to reduce transit costs (be those latency or other).
It came across like you have a broader notion of what "affinity" might mean, aside from this technical one. Is that the case? Can you explain that a little further?
Hi @jang, thank you for your comments. I DO have a broader, non-technical notion of "affinity," stemming from the base critique of Kubernetes as replaying command and control. Affinity and anti-affinity definitions stand out to me as less like command and control in the classic cybernetic vein. The notion seems intensely philosophical, whereas the use of affinity and anti-affinity in K8S is probably in fact quite practical: to automate the selection of nodes and pods given all the material constraints you mention, i.e. the TOR switch, power supplies.
So I accept that in practice, more affinity would wreck Kubernetes specs and orchestration technically, but I was envisioning an aesthetic use of technology that could very well result in inefficient use of the platform while creating a set of artistic or literary facts that could be documented and represented in a humanities context.
When I think of affinity as a philosophical concept, one of the first discursive fixtures that pops to mind is the notion of "affect" and the numerous discussions of this in contemporary philosophy. "Affinity" is of the same caliber as "affect". Historically, I think of Goethe and Diderot as examples of literary and philosophical writers who explore how things as varied as romantic actors or chemical components have affinity. And with affinity, there is a major component of chance.
In practice, Kubernetes mitigates the chance of affinity, but still implies that it is at play by using affinity/anti-affinity as a construct. So for me, this becomes an impetus to imagine the K8S platform as a humanities tool. For instance, an application with several components could be designed, i.e. for textual or visual processing, but let's say historical figures were attached to each component as performatively designated agents of this processing based upon particular contributions of these components. On top of this, for Kubernetes, we could design a spec (or use Helm, Terraform, etc.), which would result in a number of different arrangements of nodes/pods that could be logged or that the system would track. The logs or some other record could be presented in a humanities context to show which nodes "bonded" vis-à-vis their affinities.
Of course, the implications for this for a corporate setting are terrible for business and the bottom line, and it may not even be doable actually, or make sense, or properly understand the uses of K8S. The notion that we could base the entire Internet cloud on affinities would then stand only as a critique of the corporate bottom line and of the societal controls of cybernetics and neo-cybernetics.
I'm relatively new to Kubernetes, so this aesthetic use is completely speculative, but it is what has occurred to me in considering how to culturally critique the platform for the reasons I do so above. I sincerely appreciate yours and @annatito's technical experience with K8S, which helps to tease out what I had envisioned!
Thank you for this code critique, @gregorybringman! I've often thought about an analysis of k8s along these lines, and I'm very glad to see it. I've used k8s, often in an attempt to understand why a pod "kept crashing" or why a particular request was not being parsed correctly - i.e. I've peeked into pods via kubectl to follow the logs. Occasionally, it was necessary to "gracefully restart" a pod - an "interesting" euphemism for killing a worker.
But I also wanted to suggest some avenues into thinking about affinity. As an anthropologist, the concept of affinity immediately takes me to kinship theory. Affinity is, of course, the relationship between in-laws. However, recent approaches to kinship have seriously reshaped the notion of affinity by looking at the role of other-than-human being (spirits, animals, plants, machines) in the making of "human" kinship. Here, Amazonian ethnography was crucial - and so, the philosophy of Deleuze and Guattari. Take this example, for instance, of a paper by Eduardo Viveiros de Castro:
the displacement of the analytical focus from an intra-specific horizon to an inter-specific one; from a human economy of desire — world-historical, racial, sociopolitical rather than familial or Œdipal desire, but human nonetheless — to an trans-specific economy of affects, which ignores the natural order of genera and species and its limitative syntheses, disjunctively including ‘us’ (all and any of ‘us’) in the plane of immanence.
Here, affinity is an alliance between human and non-human forms, which are "disjunctively" related. To me, that is a good description of a programmer, a being that is neither human nor machinic, but a disjunctive combination of both.
Perhaps we could use such notion of affinity to think about k8s aesthetically? If a pod/worker is the machinic aspect of this relation, could we become the human that are allied to it? What kind of aesthetic form would that alliance produce? What king of artifacts could emerge?
Hope that makes sense - and thanks for the critique!
(A link to Viveiros de Castro's paper, if you're interested)
@code_anth, thank you for bringing up the role of the non-human in any aesthetics of affinity, and thank you for the article by Viveiros de Castro. Great to see such a thorough and detailed application of Deleuze and Guattari to contemporary anthropology.
As you demonstrate, it is important to re-emphasize Deleuze (and Guattari) in this encounter with K8S, and I think the possibilities for linking humans, programmers and machines via discursive networks needs their thought, and their work helps us avoid anthropomorphizing K8S nodes and pods. No doubt, my example of historical figures being attached to K8S components dramatizes how an affinities aesthetics would play out in K8S, but a more purely machinic, or machinic allied with the human - way of thinking about it really multiplies possibilities.
Actors both human and non-human perform or present affect, and affinity is the encounter of two or more affective actors. The machinic entails an affective/productive actor reaching out transversally to other actors, and these actors are "all and any of us". So, given that affinity seems central to Deleuzian/Guattarian critique, it also seems central to a critique of cybernetics, which is maybe why I was struck by the construct of affinity/anti-affinity hiding in a software platform with neo-cybernetic branding like K8S!
As far as the artifacts that the human/non-human alliance would produce, I wonder if the programmer would have to be regarded as another "thing" by the non-human. K8S logs could have a trace of the programmer in things like variables, names and other identifiers chosen. It would also be important to not negate the productive aspects of the non-human K8S node/pod, when the programmer might present logs and other artifacts in a humanities, literary, or exhibition context. The programmer and the non-human would be co-producers.
I'm not sure how the ACTUAL affinity definitions in K8S could technically accomplish all this while still scaling and not breaking, or at least not cluttering the specs, given the comments of @jang and @annatito, but it seems apparent, now, that the notion of affinity/alliance is expressed by K8S and its programmers in other ways. Thanks again for your comments!