Interfaces ←

Amazon Go

Ray Wu / 2018
Yale School of Architecture

Amazon Go is Amazon’s newest entry into the brick-mortar retailing along with its recent acquisition of Whole Foods. Amazon’s dominance in the field of logistics, as well as its AWS cloud services, has primarily operated behind the scene of our quotidian life, and has slowly revolutionized the online retail industry through the combination of its transportation and logistical services. The deployment of vast supply and sorting floors inhabited by swarms of automated Kiva robots is highly illustrative of Amazon’s innovation. This implementation of automation technologies successfully minimizes the logistical friction between the retailer and the consumer, leading to growth. Discernibly, this logistical lubrication is not without consequences, for despite growth and innovation, the loss of jobs and displacement of physical retailers have been points of criticism [1]. Yet the more fundamental change which Amazon’s logistical revolution has brought is the further acceleration to the space compression of globalized economy and its adjustment of our perception of material consumption. The same automation approach has been taken with Amazon Go — reducing frictions associated with physical retail experience. Yet this is not without caveats of how, and to what extent this frictionless gratification would bring to the surface.

The Run-Through

To achieve this so-called seamless retail shopping experience, many fundamental hardware as well as soft upgrades have been made to the traditional convenience store typology. Amazon calls this Just Walk Out Technology, [2] a set of innovative technologies that enable the seamlessness. Before the customer enters the store, he or she must have a smartphone with internet connection, the amazon go mobile application installed and a registered amazon account tied in with a valid form of payment, in most cases a credit card. Upon finishing the installation and launching of the mobile application, it would provide a quick ten slide animated tutorial [3] of how to shop at the cashier-less store:

1. Here, Just Walk Out is emphasized again, urging you to start the tutorial.
2. Step-by-step explanation of the need for swiping of the app on the scanners of the entry gates to enter.

3. How to let your let your friend and family join you on the shopping experience, noting the need for the account holder to enter last.

4. Informs you of the dynamics of your action, i.e. putting an item back on the shelf would link to the status of item in your virtual shopping cart on your phone.

5. Again, reiterating and emphasizing your interaction with the items of the shelf would register and reflected in your account with a replay button for the animation as to visual instruct this logic.

6. Informs you the same actions will register on family and friends you have let in at the gate.

7. With a large red sign, the animation warns you not to hand or take items to and from other shoppers, complicating the status of item in virtual carts.

8. Reassures to you that once you have everything you do need you are able to leave without checkouts and lines.

9. After leaving the store, an automated receipt will inform you of your purchase and deduct resulting amounts from the provided payment method.

10. Confirms you are now ready ready to shop at the Amazon Go store, “Seriously, you can go!”

The Technology

The Just Walk Out Technology consists of three separate technologies to realize this cashier-free shopping experience: Computer vision, deep learning algorithms, and sensor fusion. [4] Computer vision is the ability for acquiring, processing and analyzing digital images. Deep learning algorithms allow for the system to grow and change upon collection of new data. Sensor fusion is the integration and calculation of data collected from various sensors to determine position and orientation information. But how are these deployed and integrated to allow for the automation of cashier-free shopping?

Upon scanning your phone to enter the store, cameras capture and track the movement of the human body by mapping through imaging and depth sensors. The identity of the shopper is matched with tagged mapping based on his or her movement. Multiple camera and sensor combinations are distributed in the overhead space, in customordered flat black rectangular volumes. The image and depth data of the camera sensors acquire are compiled and integrated in real time, producing a three-dimensional simulation of all movement within the store space. This integration of real time image acquisition, processing and analysis from hundreds of camera requires significant computer processing capacity and occupies most of their flat rectangular body. The processing is further carried out offsite in Amazon’s AWS data centers. The ability to identify and map accordingly the movement of multiple human bodies are based on the analysis of shifts in RGB pixels values and makes use of the deep learning algorithms. Deep learning algorithms adapt and change according to every unit of data provided, learning through the accumulation of iterations under similar conditions. Its cascading structure of the of these algorithms employs the outcome of previous calculations as input for the next, gradually increasing its accuracy at recognizing and tracking humans and their movements. The algorithm then registers the proximity of the human to any merchandise located on the shelves as well as the presence and removal of an item from the shelf. Upon reaching a proximity threshold, the camera respond by taking photos of the hand in proximity to the merchandise and analyzes the pixels to define whether the item has been removed — merchandise follows the movement of hand; or if the merchandise has been put back onto the shelf, the detachment of the merchandise’s pixels relative the pixels of the human hand. To increase the accuracy of this determination process, sensor fusion provides an alternative set of data collected from the shelves to cross reference data provided by the camera. Although it is uncertain to what these sensors actually constitute due to Amazon’s secrecy, they are most likely located in the physical attachment on the shelves per Amazon’s patent application: ”[0033] In addition to camera, other input devices, such as pressure sensors, infrared sensors, a scale, load cells. A volume displacement sensor, a light curtain, etc., may be utilized with the implementations described herein. For example, a pressure sensor and/or scale may be used to detect when an item is added and/or picked from inventory locations.” [5] The significance is that all three types of technology introduced to form the Just Walk Out Technology, along with the smartphone terminal, are synchronized in realtime. The operation and integration of separate technologies in synchronicity is perhaps most exemplary of Internet of Things, but at a much greater intensity of data collection, transmission, processing reflecting actions of humans and positions of merchandise is highly logistical in nature.

The Store

The introduction of these technologies into the typology of conventional retail space inevitably demands the conforming of architecture to the the technology. In the particularities of Amazon Go, the array of hundreds of camera overhead aims at the complete, unobstructed imaging coverage of spaces below to ensure the accuracy of the deployed technology. The resulting layout of the space is defined by the maximizing of an unobstructed open space by pushing retail shelving to the perimeter of the architectural plan. However, the open plan differentiates itself from conventional retail plans with rows of shelving occupying the main central volume as means to increase the number of merchandise that could be exposed to customers. Its narrow path width between the shelvings would pose as a challenge to the camera system implemented in Amazon Go, as the cameras must be able to capture the interaction of the human hand with that of the merchandise, from multiple angles as to maintain an acceptable level of accuracy. To accommodate the traditional layout of shelves would perhaps require the undesirable placement of multiple camera systems from below.

The Precedents

A closer study of precedences of small retail spaces of similar typology reveals the highly intertwined history between spatial layout and surveillance. Early precedences of convenience stores date back to the general stores, or general merchandise stores, emerged during the 18th and 19th century catering to small populace and townships. Original general stores most often featured the placement of merchandise shelve behind counters, preventing direct access and retrieval of items by customers. In comparison to Amazon Go, this dated form of store layout can be characterized as high friction, requiring the customer to interact with the store clerk to gain physical access to the items. The layout configuration is nonetheless logical given the focus on security and the lack of surveillance. Another relevant precedence is the Piggly Wiggly grocery stores, featuring a turnstile entrance and a dictated, meandering circulation route flanked by shelved merchandises, finally to arrive at the manned cashier and the exit turnstile beyond. In this configuration, customers are directly engaged with the merchandise and need for surveillance is deferred to the rigid spatial layout. However, this scheme is susceptible to congestion and flexibility of movement which have been resolved in more modern layouts. Modern layouts’ attempt at maximizing surface area of display shelving was made possible by the introduction of surveillance technology in the form of a camera. In these retail spaces, the deterring effects of surveillance camera effectively liberates floor space from the need for rigid spatial control through architectural elements. In the spaces of Amazon Go, the immense number of cameras no longer deter, but actively police spaces as to allow the excision of cashiers. In this this form of spatial control, visibility replaces physicality, but nonetheless occupies space. Inadvertently, the perimetric layout of Amazon Go exchanges surface area for expediency.

The Social Contract

The gradual change from physical forms of control to a form of deterrence is, on one hand, enabled by the technological innovations, but fundamentally a shift in social contract. Counters and turnstiles in earlier retail spaces are examples of physical barriers. They are apparent, visible and are devoid of the need for social contract. The introduction of surveillance camera into retail spaces effectively defers the function of physical barriers to that of visibility. While visibility does bring about immediate ramifications but its presence insinuates the likelihood of future ramifications. Thus a social contract is formed by which most, if not all, parties subscribe to.

The Privacy

Under the guise of Just Walk Out Technology, the neologized trifecta of computer vision, deep learning algorithm and sensor fusion have the obvious capacity for facial recognition and abuse of data collection. Real-time monitoring and processing of image data can easily switched to identify individuals. Data collection further increases the chances of profiling and its monetization by any entity. While Amazon denies of collection and retention of facial data, the technologies have the capability and can be easily appropriated by third parties. Another facet of this ensemble of technologies is the necessity of collection of data for training and growth of its algorithms. While consumers enjoy the convenience of cashier-less shopping, the technology simultaneously also consumes data generated by the shoppers to further itself. In this sense, the users are no different than the merchandise on the shelves, inventory to be used. [6]

The Contingency

Though privacy concerns for new forms of technology are relatively prevalent, Amazon has the scale and capacity for both national and global deployment of Just Walk Out Technology. And it plans to do so. [7]

Currently, the lack of detailed regulations for these new technologies greatly increases the difficulty of enforcement, or worse, the unfeasibility of enforcement. [8] Regulations will be amended with time, but the precarity with Amazon Go is its likelihood to be the first instance of theses technologies to intrude on quotidian spaces. In the same way surveillance cameras were able trigger the social contract by deferring physical barriers to visibility, Amazon Go and its technologies defers all obstructions.

The fear is in the conditioning, establishing, acceptance, and desensitization to these forms of technology to permit its permeation to all space, given the potential. Amazon Go is nevertheless the exchange of convenience for voluntary participation and subscription to a regime, a regime in which we renounce our privacy.

The Alternative

Between the quiet filing of original patents for the what would constitute the Just Walk Out Technology in 2015, which were first speculated to be implemented in Amazon’s warehouses, to the announcement of Amazon Go’s existence in late 2016, to the unveiling of its physical store to the public in early this year, analogous retail models have already sprung up elsewhere. One of which is the startup Bingbox, aiming for a smaller, less investment intensive cashier-less retail ‘box,’ which also happens to be mobile. In contrast to the camera loaded system of Amazon, Bingbox integrates WeChat, for identification, payment as as well as facial recognition at the checkout. Customers gains access to the shop by scanning a QR code with their already logged in WeChat application, once inside, customers are free to handle merchandises. The checkout process is not as sophisticated as the Amazon Go’s Just Walk Out Technology — by scanning the barcodes of the purchasing items on the self-checkout machine, with the option to pay with digital wallets of either WeChat or Alipay. To exit, a camera identifies and matches the items held by the customer with the purchase record. Contrary to Amazon, which suppresses and denies the use of facial-recognition technology [9], BingBox actively publicizes its adoption in identifying unrecognized users within the store. By mid-2017, BingoBox has already opened their physical store and by last count [10], over 300 stores deployed in China. BingBox’s swift proliferation is possible due to their significantly lower initial investment, lower operation cost, lack of need for real estate, a franchise business model, and its adoption of existing platforms such as WeChat.

Though both Amazon Go and BingoBox are cashier-less, their approaches are evidently different. Amazon Go’s size, technology, and design of experience fundamentally revolve around a fast turnover rate of its customers. Its aim and opportunity are one and the same — to minimize friction at the very end of the logistical chain. Spatially, customers experience both minimal physical and visual obstructions from the time of entry to their fulfillment. What Amazon has achieved for virtual shopping is replicated for the walk-in experience. The cashier-less design of BingoBox, on the other hand, puts emphasis on the recuperation of labor costs by a mix of technology, self-service, and physical barriers. In this sense, BingoBox is inherently highly reminiscent of the design of Piggly Wiggly store, less the cashier, replaced with self-service checkout. BingoBox thus operates within the existing set of social contracts, augmented by technology.

The gradual, yet surely, expansion of Amazon Go stores to every major metropolitan area is likely unavoidable. The contingence remains to how and to what extent will this retail innovation modify our social contract, and our acceptance, or even desensitization, to the pervasive technologies of computer vision, deep learning algorithms, and sensor fusion. Potential exist, in their endless applicability in creating responsive, interactive, sentient environments and architectural spaces, but so do threats of their abuse.

1. Swartz, Jon. "Amazon Is Creating 100,000 U.S. Jobs, but at What Cost?" USA Today.
January 15, 2017. Accessed May 09, 2018.

2. "Amazon Go" Amazon. Accessed May 09, 2018.

3. Du, Meiling. "Examining the User Experience of Amazon Go Shopping - Just Walk Out." Prototypr. January 23, 2018. Accessed May 09, 2018.

4. Carroll, James. "Computer Vision and Deep Learning Technology at the Heart of Amazon Go." Deep Learning and Artificial Neural Networks Improve Machine Vision Applications. January 13, 2017. Accessed May 09, 2018.

5. Puerini, Gianna Lise, Dillip Kumar, and Steven Kessel. TRANSTONING TEMIS FROMA MATERALS HANDLING FACILITY. US Patent US2015/0012396A1, filed September 24, 2014, and issued January 8, 2015.

6. Referring to Heidegger’s concept of standing reserves.

7. Reisinger, Don. "Here Are the Next Cities to Get Amazon Go Cashier-Less Stores." Fortune. Accessed May 09, 2018.

8. Menosky, Alexandra. “Walk Out Technology: The Need to Amend Section 5 of the Federal Trade Commission Act to Protect Consumer Privacy and Promote Corporate Transparency.” Pittsburgh Journal of Technology Law and Policy, vol. 17, no. 1, June 2017, pp. 35–52., doi:10.5195/tlp.2017.200.

9. McBeath, Bill. "Amazon Go and the Future of Sentient Buildings: An Analysis." Supply Chain Brain. Accessed May 09, 2018.

10. Billward, Steven. "In China, Amazon’s ‘store of the Future’ Is Already Open." Tech in Asia - Connecting Asia's Startup Ecosystem. Accessed May 09, 2018. china-version-amazon-go-bingobox-funding.