Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Categories
In this Discussion
Code Critique: "Who Are You In 1917 Russia?" and arbitrariness in political compass tests
Title: Political Compass of the Revolution
Authors: Alexander Reznik,Dmitry Golubovsky,Paul Richardson
Language: HTML, CSS, Javascript
Software requirement: Any modern browser
Link to quiz: https://arzamas.academy/materials/1269
First launched in 2001, the Political Compass quiz has made an indelible mark on online political discussion, popularising a two-axis political spectrum and launching thousands of memes. In the quiz, the participant answers strongly agree, agree, disagree, strongly disagree on a number of questions, each answer having a hidden point rank which moves them up and down two scales of authoritarian/libertarian and left/right. The outcome is a point on a four-quadrant chart.
The trouble with this format is that even if it were possible to word and present the questions in a non-leading way (something the original political compass actively avoids doing), the point values would still necessarily be arbitrary. This can be seen in the code below. As the Political Compass's source is hidden, I have presented one of it's successor quizes instead. This Political Compass of the Revolution has the same structure as its forebear. However rather than trying (and failing) to reflect an ideology for all people, places and times, it focuses purely on specific factions in a specific time and place (namely, the main political groupings in Russia in 1917). In this way it is more informative than the original Political Compass (the specific positions on the chart actually reflect real positions), and it is able to tell you which faction your answers are closest to, and what those factions answers would be.
Here's a relevant section of the code, which shows the questions and the point values for their answers:
questions: [
{
id: 1,
title: 'Private property is a natural right and a vital economic institution.',
answersWeight: [16, 8, 0, -8, -16],
axis: 'x'
},
{
id: 2,
title: 'The agrarian question can only be resolved through socialization of all land. Land should become the property of the people as a whole and should not be bought and sold!',
answersWeight: [-32, -16, 0, 16, 32],
axis: 'x'
},
{
id: 3,
title: 'I favor swift nationalization of the key sectors of industry.',
answersWeight: [-14, -7, 0, 7, 14],
axis: 'x'
},
{
id: 4,
title: 'A universal shortening of the workday to eight hours is essential.',
answersWeight: [-14, -7, 0, 7, 14],
axis: 'x'
},
{
id: 5,
title: 'The workers must be given control of industrial enterprises.',
answersWeight: [-14, -7, 0, 7, 14],
axis: 'x'
},
{
id: 6,
title: 'Elections should be universal, direct, fair and secret.',
answersWeight: [6, 3, 0, -3, -6],
axis: 'y'
},
{
id: 7,
title: 'The State, as a matter of principle, does not have a right to limit freedom of speech or assembly!',
answersWeight: [8, 4, 0, -4, -8],
axis: 'y'
},
{
id: 8,
title: 'Women should receive rights equal to those of men!',
answersWeight: [6, 3, 0, -3, -6],
axis: 'y'
},
{
id: 9,
title: 'The Church should be separated from the State!',
answersWeight: [6, 3, 0, -3, -6],
axis: 'y'
},
{
id: 10,
title: 'The death penalty is a vital means of punishment during current wartime conditions.',
answersWeight: [-6, -3, 0, 3, 6],
axis: 'y'
},
{
id: 11,
title: 'The war with Germany, Austro-Hungary, Bulgaria and the Ottoman Empire has been going on for three years. This war must be swiftly ended through a democratic peace without annexations or levies!',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 12,
title: 'It is vital to defend the Revolution, but not to carry out attacks!',
answersWeight: [-2, -1, 0, 1, 2],
axis: 'y'
},
{
id: 13,
title: 'We should honor our responsibilities to our allies. War until a victorious end!',
answersWeight: [-4, -2, 0, 2, 4],
axis: 'y'
},
{
id: 14,
title: 'Democratization of the army is good. There is no need for military insignia, officers should be elected, they must be equal in rights with soldiers.',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 15,
title: 'Military dictatorship is the best way to get out of the revolutionary crisis.',
answersWeight: [-6, -3, 0, 3, 6],
axis: 'y'
},
{
id: 16,
title: 'I support the Provisional Government!',
answersWeight: [-4, -2, 0, 2, 4],
axis: 'y'
},
{
id: 17,
title: 'The present state of the Revolution demands a lengthy period of agreement between all democratic forces.',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 18,
title: 'I am for the swift convocation of the Constituent Assembly!',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 19,
title: 'The Soviets of Worker, Soldier and Peasant Deputies should take over the reigns of state power!',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 20,
title: 'The city Dumas (legislatures) are the proper mechanism for self-government, we don’t need any other.',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 21,
title: 'The working class is the driving force of the Revolution.',
answersWeight: [-14, -7, 0, 7, 14],
axis: 'x'
},
{
id: 22,
title: 'The State should be swiftly liquidated and replaced by the people’s self-government.',
answersWeight: [24, 12, 0, -12, -24],
axis: 'y'
},
{
id: 23,
title: 'A federal state structure is the best option for multinational Russia.',
answersWeight: [8, 4, 0, -4, -8],
axis: 'y'
},
{
id: 24,
title: 'Russia should continue to be a major power, retaining its continental and overseas colonies, ruled from the center.',
answersWeight: [-6, -3, 0, 3, 6],
axis: 'y'
},
{
id: 25,
title: 'A multiparty system is capable of serving to stabilize democratic power.',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 26,
title: 'Russia’s political system should develop in the direction of parliamentarianism.',
answersWeight: [4, 2, 0, -2, -4],
axis: 'y'
},
{
id: 27,
title: 'Our end goal is socialism!',
answersWeight: [-14, -7, 0, 7, 14],
axis: 'x'
},
],
There's presumably some rationale why strongly agreeing to question 23 should move you 8 points along the Y axis while strongly agreeing to 25 moves you half as much. Even if we allow that there is an art to judging the comparative ideological significance of any given opinion, the exact numbers chosen cannot escape being somewhat arbitrary.
Note that the purpose of the quiz is achieved without reference to the axis position: at the end of the quiz it judges how close your answers are to the answers of other historical actors, and that can be done just by comparing your answer with theirs: the exact point values of these answers don't matter. In this way, this version of the political compass points the way towards more meaningful ideology tests.
Comments
Thanks for sharing this. What a great example of a piece of (hidden) code shaping a large public political discourse over time.
Just a few notes on the original code as a research problem. The original Political Compass has a Wayback Machine record from 2001, its first year online -- although that only preserves the client-side question set and interface, not the server-side method of computing results.
In 2014, Alex Chandel did a Python reimplementation of the original Political Compass question set -- although the source of the scoring method isn't clear.
In 2019 Justin Bodnar did a Python reverse-engineering of the Political Compass with computed question weights correlated to its reported results.
Thinking about this is almost grim! It's basically code deciding which values are most important to which groups of people and 'gently nudging' them to one side or the other. It's polarizing. As you point out @JoeyJones, the difference in point value is what really disturbs me about this. A quiz like this can seem neutral on the outside, but look at question 22! Why should that decide someone's alignment so heavily? Given that humans sort of naturally fall into factions, this seems to want to pre-determine people's groups without real reason. Perhaps I'm being a bit to cynical about this, but something about this particular code really rubs me the wrong way.
@KalilaShapiro:
That's an interesting question, and in truth probably reflects the reality of this situation. The alternative question might be, "why should someone's alignment cause this score to have such a heavy weight?" I don't think that's happening here, but...
I'm reminded of this: https://fivethirtyeight.com/features/the-scoring-for-the-decathlon-and-heptathlon-favors-running-over-throwing/ (and similar articles) - the relevance of which I'll try to explain.
I was first exposed to this idea by an Open University stats lecture that was televised, I guess, sometime back in the 1970s (so I can't find a citation, alas), where the presentation began with the decathlon scoring system and did a principal component analysis - by hand, naturally :-). The result was a handful of eigenvectors with corresponding eigenvalues that pretty much corresponded to performance in speed, strength, and what you might term skill (for the technical disciplines).
They then asked if there was a "fairer" way to construct a scoring system that would encourage balanced training by athletes. Although these are possible to construct, there's a problem with them: it's impossible for spectators to easily tell how well an athlete is doing at any critical juncture, compared to systems that award points linearly combined into a final score. To paraphrase: the tension is lacking when you need tables and a slide-rule.
So whilst I don't doubt that your instinct about "which came first" in the example above is accurate, it would in principal be possible for the software to reflect a more principled analysis. You could ask people to place themselves along various axes, ask them the questions given, and then reverse-engineer a best fit of weights for those questions.
There's a related problem today, as AI "decision support" tools become more widely distributed. It's a laudable goal that a decision taken by an AI tool should be understandable to the person whose life it's just changed (approving loans, for instance). However, "understandable" seems to be interpreted as meaning, related to identifiable features with simple labels, which produce a score combined linearly. Most of these tools don't work like that - exceptions being things like this piece of software.
It seems like what is being asked for, in that last case, is the moral equivalent of this kind of software. Would you find it more or less unsettling were the scoring functions and their relation to the output more opaque to inspection?
This also assumes that some form of accuracy / being understood is actually what people desire from a model. Many people want surveys to express their values, not reveal their values... and in the case of collective surveys, groups often attempt to game the system to make a survey a normative statement of popular opinion, approaching it like a sport more than a scientific measurement. How would we want to place ourselves, and how accurately do we individually wish to be read?
There are related problems in recommendation engines. Which do people want more from a recommendation engine -- something that predicts things that they will actually like, or something that recommends things that conform with how they identify or how they present -- how they are publicly performing their preferences? Sometimes this relates to privacy -- people find certain kinds of recommendations offensive, or humiliating, or invasions of privacy. Just as algorithms have accidentally guessed that someone is pregnant and "out" them[1][2], algorithmic recommendations of all kinds can be tied to performative self and identity, particularly when a lot of recommendations are for entertainment and leisure activities and goods that are . A suggested film, song, medicine, or coffee mug could potentially impute the recommendee as being trans / libertarian / vegan / pregnant / a My Little Pony fan / a gun owner / Muslim / terminally ill ... or almost anything that people might choose to express differently (or not at all) in different social contexts, which is potentially almost everything... and yet, in different cases the stakes in this information leaking are dramatically different -- ranging from minor embarrassment to potentially life-threatening.
There are long-established methodological traditions in psychology, sociology, and public opinion polling to tackle these kinds of problems -- although sometimes with different goals to produce different forms of knowledge. We even have popular concepts around how some of these goals are negotiated and what they should be -- for example, many people know what "push-polling" is. Where this becomes interesting as we move from the Political Compass to more complex software algorithms is not just the complexity, but if the goals -- the kind of knowledge that the software is supposed to be producing -- have shifted over time or not, and why.
Duhigg, Charles. "How companies learn your secrets." The New York Times 16.2 (2012): 1-16. ↩︎
Hill, Kashmir. "How Target figured out a teen girl was pregnant before her father did." Forbes, Inc (2012). ↩︎
@jeremydouglass I think you make a good point about the expressive use of these kinds of quizes. You can see this in the fact that for each question, it's possible to agree or disagree strongly and its up to the user to decide whether their feelings constitute a strong or weak agreement. People who take a lot of these sort of quizes know that they'll get a more definitive outcome if they take stronger stances, so the same person may get different responses to the same quiz depending on how they present their own views without those views actually changing.
Also thanks for sharing the estimated values of the original political compass. They demonstrate exactly the point I was making. Take the following two questions which have the same point value:
There are several obvious problems here:
1. It completely arbitrary that a belief in astrology is considered exactly as authoritarian as a belief that morality requires religion.
2. It is a pure assumption of the poll that a belief in astrology or religious basis of morality is considered to be an indicator of an authoritarian mindset.
3. It is very likely that many religious people are also not believers in astrology, which would mean the questions may tend to cancel one another out. This tendency towards self-cancelling along with leading questions may go some way to explaining how nearly everyone who takes the test ends up in the bottom left quadrant, whereas nearly all political actors described by the test fall in the top right quadrant.
@KalilaShapiro, I think the question about the liquidisation of the state being given such a huge point cost is less unsettling in the specific context of the quiz: this was a major question over which different factions disagreed, and indeed was the chief dividing line between the anarchist and socialist factions at the time. So for this narrow quiz I think that the weighting may be justified as a reflection of the actually existing polarization of the conflict. But you are correct to see that the weightings in the broader political compass are selected on ideological grounds and where people fall on the chart may have an impact on how they feel about themselves and others.
@jang, I can see the appeal but I don't think reverse-engineering weights will necessarily be a meaningful task. For one, people who take tests may not have a clear enough idea of their own ideological commitments to weigh them against each other, different people will have different judgements over what is important, and so the average of all these differences won't necessarily reflect any deeper 'true' weightings. Ultimately, a quiz approach which is more like bird-spotting or mushroom identification is, I think, a more fruitful direction for political quizes: what features of belief does a person actually have and what ideology does that reflect.