Train with an Emoji avatar to manipulate facial expressions and thwart the controlling potential of emotional recognition devices.
Today, digital mapping, street view and machine learning technologies enable us to navigate cities over time, making self-evident the aesthetic change cities are subject to. Since 2005 several companies, such as Google, Microsoft, Baidu and Kakao (Daum), started to develop systems to digitally map urban and rural areas across the world, using various types of vehicles and cameras to create 360° views, creating a growing visual urban archive. Wandering through a city in these ‘street view’ environments, users can explore through the years different visual documentation1.
As these urban archives get more extensive each year, new technologies also enable us to do more than just ‘look back’. Computer vision, a field that is concerned with using computers to gain a high-level understanding of images, is rapidly becoming more and more sophisticated2. Alongside this, developing technologies such as generative adversarial networks (GAN’s), are able to generate new images based on existing datasets, making it possible to speculate on, and visualize, the future of urban growth or decline.
Aesthetics of Exclusion is an ongoing research project that looks into the aesthetics of gentrification in cities through digital and physical wandering. From July to August 2018, we walked through Seoul in South Korea physically and with Daum street view, computer vision and GPS tracking. Here we present an insight into our findings thought Blindspot, Beach Umbrella, and Bias.
When walking or navigating through a digital street view archive of Seoul, street vendors, markets and pojangmacha’s (temporary food carts) fill the streets, creating an indispensable part of the urban fabric. These specific typologies are linked to the informal economy, which has decreased significantly during the last decade and will continue to grow smaller in the future. Several socio-economic processes, such as gentrification3, strict law enforcement and rapid cultural change, make it hard for vendors to survive. Many foresee a bleak future as it seems only those who have the capital to invest in government issued street vending units or approved food trucks will survive. In the future, will we only be able to remember Seoul’s informal economy through the lens of visual archives?
While in Seoul, we physically and digitally zoomed into a neighbourhood that proved the rapid shift in urban infrastructure, 익선동 / Ikseon-dong. The neighbourhood is small and mainly built from traditional Hanok houses4 which used to be inhabited by working class residents5. We found that Daum street view images of the neighbourhood were recorded for the last time in 2013, just before the start of the gentrification process, creating a blindspot in the virtual city. Visiting in 2018 felt like wandering through a live instagram feed: shoppers looked for glitzy necklaces, while Rihanna blasted from speakers and foodies roamed searching for the perfect latte macchiato. By experimenting with combining different datasets visually (hashtags from instagram, with street view images of 2013 and recordings of 2018) we hoped to established new insights into the neighbourhood.
“…This population of objects… is no other, when we come to think about it… than that observed in countless natural species, species which man has successfully inventoried…”
파라솔 Beach Umbrella
Beach umbrella’s provide shelter from rain, create shadow in Seoul’s humid climate and also create temporary spaces of exchange. We found them to be an interesting symbol of Seoul’s informal economy, and a phenomenon often overlooked. We trained object recognition software to detect umbrella’s across Seoul through images and street view, making it possible to experience their movement and evolution through time. By tracing beach umbrella’s with object recognition, could computer vision become a kind of fortune teller, predicting the future of urban growth in the city?
“We live by object time: by this I mean that we live at the pace of objects, live to the rhythm of their ceaseless succession. Today, it is we who watch them as they are born, grow to maturity and die (…)”
It is always important to analyse the creator of an algorithm and how they research6. While walking through Seoul, all the data from my mobile GPS location was collected, then all street view images that correspond with the coordinates visited in the city were automatically downloaded through an API and, using computer vision, sorted by visual similarity.
This process raised some interesting open-ended questions. Could we start to see the ‘filter bubbles’7 that are described within an online context, also existing in the way that we move through a city? Do these ‘clouds’ of streets and the interiors we consume as foreigners, completely differ for various groups of people who inhabit the same city? The data also shows an absence of movement in a large part of Seoul. Are designers equipped to train algorithms to look at a city when their view and experience is limited?__
Aesthetics of Exclusion is an ongoing design research project by Sjoerd ter Borg, researcher Thomas Smits, visual researcher Mark Jan van Tellingen and Dr Melvin Wevers, PhD at Royal Academy of Sciences, Digital Humanities Lab. The works in this visual essay were developed in seven weeks, during the Summer Sessions at Art Center Nabi in Seoul, South Korea, in collaboration with V2_ in Rotterdam and supported by the Creative Industries Fund (NL).
파라솔 Beach Umbrella directed by Sjoerd ter Borg & Mark Jan van Tellingen. Computer vision programming by Jorrit Schaap. Sound design by Mark IJzerman, with a narration by #I5IMQ4O, based on two books by Jean Baudrillard The System of Objects and The Consumer Society written in 1968 and Myths and Structures in 1970. Blindspot programmed by Jorrit Schaap. Bias programmed by Dr Melvin Wevers and Jorrit Schaap.
파라솔 Beach Umbrella will be screened at the Frankenstein Exhibition at Baltan Laboratories during Dutch Design Week in Eindhoven, Netherlands, from the 20 – 28 October, 2018.
Sjoerd ter Borg is an artist and designer based in Amsterdam, whose research focuses on the transformations of cities. Through the use of design, fiction and technology he uncovers hidden layers within an urban context, showing forgotten histories, societal developments or even alternate realities. He holds a Bachelor in Political Science from the University of Amsterdam and Masters from the Sandberg Instituut (Vacant NL).