docs/chapter1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 1: Glitch and Visual Ethics\n",
    "\n",
    "I played the cello growing up. For a couple years in high school, I was at a school too far away for it to be practical to schlep my cello to orchestra class. As a result, I played on a cello that belonged to the district when I was at school, and on my own cello at home. The cello at school was not in very good shape: it was missing an endpin, so I had to hold it up with my lower legs as I played. A particular note in the middle of the register would buzz when plucked, and I never quite figured out what made it do that. The sound was always dull, and if I bowed too hard, its strings would rattle unpleasantly.\n",
    "\n",
    "As a crowning detail, if you peered into the cavity of the cello, you could see the Roman numeral 13 painted in dripping red letters. \n",
    "\n",
    "Playing between these two instruments made me see cello playing in a new light. I didn't play *on* a cello— I played *with* a cello. On both instruments, I was able to create decent music, but the experience was completely different. Playing the cello at home was like collaborating with a good friend. We both knew what each other wanted, and our goals aligned. Playing on cello $XIII$ required pleading, bargaining, negotiating, and deceiving. I had to change my technique to get it to do what I wanted, as well as change my expectations, to match my playing to its desires.\n",
    "\n",
    "This experience is likely familiar to anyone who practices a craft involving some tool. In taking a photograph, a photographer makes decisions such as framing, shutter speed, and aperture. Many other decisions are made by the camera and its manufacturers. From deciding which shutter speeds and apertures to support, to how the pattern of red, green, and blue light sensors will be laid out inside the camera, to even seemingly obvious things like the shape of the sensor, camera makers decide a lot. In editing, a similar thing happens: the photographer will decide on what tweaks they want to make to exposure, contrast, white balance, and so on. But the specifics of these algorithms are written by someone else.\n",
    "\n",
    "This is not to say that these forces are antagonistic. Far from it: a major component of photography is learning to work collaboratively with these tools. Cameras are built to cater to the desires of photographers. This is especially apparent in point-and-shoot cameras and phone cameras, where most technical decisions are made by the designers of the camera hardware and software.\n",
    "\n",
    "In *The Burden of Visual Truth*, Julianne Newton lays out a typology for the visual ethics of photography. The typology focuses on three perspectives of analysis: \"the subject, the photographer, \\[and\\] the viewer\" [(Newton, 139)](https://www.worldcat.org/title/burden-of-visual-truth-the-role-of-photojournalism-in-mediating-reality/oclc/1086530191). Here, I build off that work, and propose a new set of archetypes: the user, the device, and the developer. The *user* is any person who works directly to create an image. Examples of users include photographers, retouchers, editors, and workers in a darkroom or print shop. All of these users interact with some form of technology: the *device*. Devices include cameras, darkroom equipment, computers, computer software such as Photoshop, storage formats such as discs or hard drives, and printers. These devices are created by humans, though usually not by the users themselves. These *developers* include inventors, technicians, engineers, and programmers behind any devices. A photograph could not be created without users, devices, and developers, and all three play a role in the appearance and impact of images. All three roles deserve scrutiny within the field of visual ethics.\n",
    "\n",
    "<img src=\"img/model.png\">\n",
    "\n",
    "Many categories of Newton's typology can be applied to the interaction between user, device, and developer. For instance, many photo editing programs, such as Photoshop, give users the option to \"dodge\" and \"burn\" images. It is no secret that these programs do not literally hold a shadow over photo paper as it is exposed to a projection, as dodging or burning was done traditionally. However, these effects engage in *visual theater*, which Newton defines as images \"created by photographers and subjects in a kind of conscious or unconscious ritual or performance\" (143). Users editing the images with the dodge and burn tools interact with the device as if they were dodging or burning a photograph, and the device responds as if it was a physical photograph being dodged or burned. Both the developers and users are aware of the artificiality of this interaction, and interact in this performance.\n",
    "\n",
    "Visual clichés, which Newton defines as \"behavior that has become trite or stereotypical\" (143) are also present in interactions between user and device for the creation of photographs. As an example, consider the craze of digital filters to emulate VHS distortion. Harry Frank, a designer of a digital VHS distortion emulator, described striking a balance between achieving accuracy and playing to expectations. \"A lot of the time, people are quite young, applying these retro filters or styles, and have sort of an idea of what they're supposed to look like\" [(Scott, 1:23)](https://youtu.be/h6RRNNztN5o?t=83). In this way, the interaction between the developers and the users of the software passes from mere visual theater (creating the look of a distorted VHS tape, while both parties are aware that a VHS tape was not directly used), to visual cliché (creating a filter that plays to a stereotype of \"retro\" effects).\n",
    "\n",
    "Through techniques called computational photography, devices can play a more active and prescriptive role in photography. Computational photography is a technique used by recent smartphone cameras, in which the camera takes several source images— up to 15 in the case of the Google Pixel 3— instead of just one. These images are captured before, during, and after the pressing of the button. The camera combines these images, creating a more detailed image than the sensor would be able to produce otherwise. In some modes, this merely provides higher dynamic range and greater stabilization. However, in the \"Top Shot\" mode, \"a machine learning algorithm suggests other frames from the image stream which may have captured preferable moments according to a normative aesthetics. This includes detecting when a subject’s eyes are open rather than blinking, when a subject is facing towards the camera, or when faces are not obscured by foreground objects\" [(Taffel, 6)](https://doi.org/10.1177/0163443720939449),\n",
    "\n",
    "Some smartphone cameras also use facial recognition algorithms for automatic facial retouching, dubbed \"beauty mode\" by smartphone manufacturers. \"Facial retouching involves the camera recognising human faces and applying an algorithm to ‘smooth’ skin, through adding blur and removing chrominance or luminance differences that suggest ‘blemishes’ such as spots, freckles, blackheads or scars.\" In major smartphones by Google and Apple, facial retouching is the default setting (Taffel, 13). For some users, this retouching could be a visual gift, hiding blemishes that they do not wish to appear in pictures. For others, it could function as visual theft, which Newton defines as \"taking something against the will of the observed, and often without the person’s knowledge.\" The difficulty, or even impossibility, of turning off the filter means any details considered blemishes are robbed from the user, without their knowledge or assent. In prioritizing the unblemished face, the developers of these algorithms boldly support certain beauty standards. Whether or not we believe they are developed ethically, these devices are clearly an active force, modifying the image outside of the user's control.\n",
    "\n",
    "<table>\n",
    "    <tr>\n",
    "    <td><img src=\"img/beautymode_off.png\">Image taken with a Samsung Galaxy S5, without beauty mode</td>\n",
    "    <td><img src=\"img/beautymode_on.png\">Image taken with a Samsung Galaxy S5, with beauty mode</td>\n",
    "</tr></table>\n",
    "\n",
    "Most importantly for the study of visual ethics, developers can act against the intentions of the user. This is not a new phenomenon. As glitch artist Rosa Menkman writes, the \"de-facto ideal standard that has been in play since the early part of the twentieth century for most analogue photo labs has... been positively biased toward white skin tones, which naturally have a high level of reflectivity. As a result it was not only difficult to capture darker and blacker skin tones, but it also proved impossible to capture two highly contrasting skin tones within the same shot\" [(Menkman, \"Behind White Shadows\", 6)](https://beyondresolution.nyc3.digitaloceanspaces.com/Rosa%20Menkman_Beyond%20Resolution_2020.pdf). Not only are these test images generally of white people, they are generally young white women: in fact, one commonly used test image, \"Lena,\" is cropped from a *Playboy* centerfold. By using these images for color calibration, film processing itself erases the details of dark skin. This bias has continued into the digital era, with image compression algorithms irreversibly discarding more color information in the ranges of darker skin tones.\n",
    "\n",
    "The history and conventions of photography cover up the ethical concerns between devices and users. \"While media developers design their technologies in order that the user will forget about the presence of the medium, following the ideal logic of transparent immediacy, in reality, the complexity of the user’s inherently aesthetic and perceptual responses to the human computer interface requires a more nuanced approach\" [(*Glitch Moment(um)*, 29)](https://networkcultures.org/_uploads/NN%234_RosaMenkman.pdf). By \"systematically concealing nonhuman agency,\" photography conventions have historically led \"to conflation between human and technical modes of vision. This conflation is the outcome of social conventions, whereby technical parameters such as shutter speed are typically delimited in order to produce naturalistic images\" (Taffel, 10). Many genres of photography, such as photojournalism and family photos, expect conformity to established tropes of naturalistic presentation. Of course, devices both old and new make the processing of images less objective than we might believe. Believing in the objectivity of image processing could lead people to internalize the white supremacy inherent in image encoding as fact.\n",
    "\n",
    "Drawing attention to the unreliability of image processing can be done not just through academic study of visual ethics, but also through art. One such genre of art is *glitch art*. Adapting a definition from Menkman, a glitch is an interaction with technology that is \"a break from a procedural flow\" (*Glitch Moment(um)*, 34). This break can be accidental, or intentionally brought on by a user by deliberately using a device for something at the margins of, or outside of, its intended purpose. \n",
    "\n",
    "Images produced through glitches, or that appear to be produced through glitches, and that provoke aesthetic or emotional reactions, are works of glitch art. A distinction can be made between intentional and accidental glitches. Following Caroly Kane, I chose instead to spend time \"less with the origins or procurement of a glitch, and more with the unique contextual choices and critical and creative effects accomplished through final results\" [(Kane, 16)](https://doi.org/10.1525/luminos.83). In much of glitch art, the user intentionally misuses the device, in a way that they know can create artistic results. The results can still be surprising, however, so a clear line cannot be drawn between discovery and creation.\n",
    "\n",
    "Glitch art draws attention to the unreliability of transmission and processing of images. Crucial to the definition of glitches is the \"departure from our expected result\" [(Moradi et. al, 8)](https://www.worldcat.org/title/glitch-designing-imperfection/oclc/213315037). Glitches exist within a cultural context: without expectations of what \"normal\" digital images look like, glitches would have nothing to depart from, and wouldn't have an impact. Menkman writes that for her and other glitch artists, \"it has become a personal matter to break the assured informatic flows of media\" (*The Glitch Moment(um)*, 33). Due to photographic conventions, these assured information flows involve an implicit promise to display a \"true to life\" image. Not only does glitch art not show what the eye sees: as this quote shows, it tries to break that promise, drawing it into question not just for the particular image, but for images as a whole."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}