Share

Apple’s “Liquid Glass” design language, the company’s first major visual overhaul in a decade, has been met with a tidal wave of criticism since its debut. From designers and accessibility experts to the average user, the consensus seems to be that Apple has committed a cardinal sin of design: prioritizing aesthetic flair over fundamental usability.

On the surface, these criticisms are both valid and necessary. The most forceful complaint targets the design’s profound accessibility and legibility problems. The pervasive use of transparency, blur, and distortion often leaves text unreadable against shifting, low-contrast backgrounds. As product designer Allan Yu noted, “It’s hard to read some of it… they made it too transparent” (Rogers and Ashworth). This isn’t merely an annoyance; for users with visual impairments, it can be a significant barrier, violating core principles of inclusive design.

Beyond legibility, critics point to the increased cognitive load. An interface should, as human-centered design pioneer Don Norman argues, make it easy to understand what actions are possible and how to perform them (Norman 14). Yet, by creating a "busy" and inconsistent experience, Liquid Glass often forces users to expend mental energy just to parse the screen.

However, this is where the common critique, while fair, may be missing the forest for the trees. I argue that to dismiss Liquid Glass as a mere stylistic blunder is to misunderstand its aim, as detailed by Apple's own designers. It is, as they state, 'the start of a new chapter in...interaction design'." This is not the usual Apple fluff. It is not simply a new coat of paint; it is Apple’s attempt to define a new digital material.

The Deeper Ambition: From Quantum Paper to Liquid Glass

To grasp what Apple is attempting, we must first look back to Google's Material Design. More than just a visual style, Material Design was a robust ontological statement. It answered a fundamental question: "What is the material that our software is made out of?" (“Making Material Design”). The answer was “Quantum Paper”, a material with a consistent, interpretable set of rules. Elements stacked predictably, and it cast shadows that implied physical depth. Google deliberately constrained what its Quantum Paper could do to make it learnable.

Without such constraints, a digital screen is a world where anything can happen, which is often disorienting and unpredictable. By reintroducing laws to the UI universe, Google made interactions more intuitive.

Apple's Liquid Glass is picking up this same challenge 10 years later by defining a fundamentally different digital material for their own devices’ UX. Where Google's Material Design was built on the metaphors of layered paper with shadowed and lifted surfaces, Apple has chosen something fundamentally fluid. This 'new digital meta-material' is defined by 'Lensing', where it 'dynamically bends, shapes, and concentrates light' to create layering. Its motion has an 'inherent gel-like flexibility' and responds to touch by 'flexing and energizing with light' (“Meet Liquid Glass - WWDC25 - Videos”). Instead of just flowing, it 'dynamically morphs' between controls as context changes." Like Google, this metaphor is a philosophical statement about what our digital elements are made of and how they should behave.

The Goal of Good Design: From Conscious Effort to Second Nature

The first time you drove a car, every action was a conscious, calculated decision: the precise pressure on the pedals, engaging the blinker, the deliberate movement of the gear shift. This is an exhausting, high-friction state of thought. Now, as an experienced driver, you navigate your daily commute almost automatically. The complex symphony of actions becomes second nature, an effortless flow. An experienced driver can even arrive at their destination with no specific memory of the journey, having operated in a state of pure intuition.

This transformation from deliberate effort to instinctive action is the holy grail of interface design. It’s what Nobel laureate Daniel Kahneman distinguishes as the shift from “cold cognition”—slow, analytical, and mentally taxing—to “hot cognition”—fast, automatic, and intuitive (Kahneman 23). A successful user interface, like a familiar car, should allow us to operate entirely in that effortless state of hot cognition. We aren’t consciously analyzing the screen; we are simply doing.

So, how does a design achieve this state? It builds a predictable world by tapping into a deep mental process that theorists Gilles Fauconnier and Mark Turner call Conceptual Blending. This is the powerful act of integrating inputs from different, often clashing, domains to create a new, coherent reality in our minds. We often speak in metaphors such as describing life as a physical journey where the future is ahead, the past is behind, and the road is paved with obstacles. But Fauconnier and Turner argue it’s not just our language that ends up as metaphors, but the entirety of our interactions with the world

The classic example that made personal computing possible was the "desktop metaphor." To achieve this, designers blended two distinct conceptual inputs. The first was the physical world of an office, complete with its folders, files, and a trashcan. They merged this familiar space with a second, abstract input: the world of computer commands like save, find, and print. The result was an entirely new, understandable reality—the graphical user interface, with its own emergent rules (Fauconnier and Turner 23). This blend worked because it gave us a familiar foundation for interacting with the unknown. Google’s Material Design followed this tradition by blending the properties of physical paper with digital hierarchy and morphability, and Apple's Liquid Glass is the next chapter in this ontological project.

And it is precisely here, in the pursuit of hot cognition, that the current version of Liquid Glass falters.

Apple’s designers state that legibility was a "central consideration", and they built in a sophisticated system of adaptive behaviors to support it; shadows become more prominent over text, and floating titles are kept clear by "scroll edge effects" that dissolve content underneath (“Human Interface Guidelines”). And yet, despite these intentions, the default implementation’s inconsistencies can still disrupt the formation of an intuitive conceptual model. This disruption forces the user out of the desired "hot cognition" state and back into the deliberate, problem-solving "cold cognition.” We are forced to stop and consciously puzzle over the interface, breaking the very state of intuitive being its design aims to create.

As Don Norman emphasizes, good design is always iterative. The real-world accessibility issues and clunky behavior are not signs of a terminal failure, but the visible, messy, and essential work of an unfinished step forward (Norman 234). Apple's immense challenge is to continue this iterative process, tuning the design until the blend becomes coherent enough to keep us firmly within that zone of intuitive action.

Reintroducing Laws to the UI Universe

Apple’s endeavor, even with its current flaws, is a significant statement: it is an attempt to reintroduce laws to the UI universe. Users too often naviagte a digital world of chaos, where every app can feel like learning a new language with its own arbitrary rules for animation and interaction. UI expectations follow trends rather than rules. Apple's move is an effort to establish constraints that make behavior legible, learnable, and embodied, building a predictable framework that goes beyond just how things look. These are not just implied constraints; Apple provides explicit rules for developers, such as reserving Liquid Glass for the 'navigation layer that floats above the content' and to 'always avoid glass on glass' to prevent a cluttered interface.(“Human Interface Guidelines”). These laws create user expectations that over time will lend to the development of hot cognition. This is where the power of Liquid Glass lies; not in it’s look, but in its behavior.

How frosted or transparent the glass elements are seem less important than the way the glass behaves — how it refracts light, how it morphs and moves, how it feels.

This is where the long-term impact could be immense. With major players like Apple and Google now both focused on defining the physics of their digital worlds, we start to see the possibility of a true convergence. This does not mean that all interfaces will look the same; rather, it means that a set of shared principles and cross-platform expectations could emerge. The rules for how digital materials behave might begin to influence interfaces across all ecosystems, from Microsoft's platforms to Meta's immersive realities, leading to a sort of unspoken constitution for digital interaction.

Imagine a baseline consistency that makes all our digital experiences more intuitive because there's an underlying logic everyone adheres to. Achieving such a standard will be a monumental task. As Norman points out, standardization is a laborious process, often taking decades and involving difficult political and economic compromises (Norman 248). However, the fact that Apple is now seriously engaging with this philosophy is a powerful signal.

Despite its rocky start, Liquid Glass should be viewed as foundational groundwork. It is an ambitious, if flawed, philosophical statement about what digital materials can be, aiming for a future where our interactions are more predictable and built on understandable rules. This shift from arbitrary pixels to a tangible, understandable digital environment is a massive potential benefit for usability. The question now is not whether the first draft is perfect, but how these evolving digital materials will ultimately shape our reality in the years to come.

And here, I believe, Mr. Kastrenakes is even more right than he realises.

Subscribe now

Share

Works Cited

Fauconnier, Gilles, and Mark Turner. The Way We Think: Conceptual Blending and The Mind's Hidden Complexities. Basic Books, 2002.

“Human Interface Guidelines.” Apple Developer, https://developer.apple.com/design/human-interface-guidelines. Accessed 25 July 2025.

Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2013.

“Making Material Design.” YouTube, 28 May 2015,

Accessed 25 July 2025.

“Meet Liquid Glass - WWDC25 - Videos.” Apple Developer, https://developer.apple.com/videos/play/wwdc2025/219/. Accessed 25 July 2025.

Norman, Donald A. The Design of Everyday Things: Revised and Expanded Edition. Basic Books, 2013.

Rogers, Reece, and Boone Ashworth. “'Beautiful' and 'Hard to Read': Designers React to Apple's Liquid Glass Update.” WIRED, 9 June 2025, https://www.wired.com/story/designers-react-to-apple-liquid-glass/. Accessed 25 July 2025.

The Verge. “Diving into Apple's Liquid Glass.” Youtube, The Vergecast, 25 July 2025,

Elishalucero Ontologicaluidesign

537KB ∙ PDF file

Download

Download

Ontological UI Design: Expanding the Moral Obligations of Our Design Process

Abstract

In this paper we explore the philosophical theory of Ontological Design, and the moral consequences that arise from it, in the domain of applied UI/UX design for post-WIMP GUI based devices. First, we ground the philosophical theory into a manipulatable paradigm by establishing the commensurability it shares with ‘Conceptual Blending’, a powerful model for cognition from Gilles Fauconnier and Mark Turner. Second, we will explore how ontological design has been implicitly practiced by forming a reliance on hot cognition, and ways we can more thoughtfully embrace this design choice. Lastly, we explore how designing for a reliance on hot cognition can lead us astray by assuming it as a default value. This research hopes to expand our understanding of the moral obligations we have when designing for the user interfaces that increasingly dominate the way in which we interact with the world.

Keywords

ontological design, post-WIMP, conceptual blending theory, hot cognition, conceptual model, generative power

1. Introduction

Ontological Design, revitalized by philosopher Anne-Marie Willis in 2015, is a philosophical approach to the design of tools and spaces which centres the consequential effects their design and use ends up having on individual and social cognition. A tool is designed for a particular purpose within a particular context, and this purpose in turn constrains the way we pervade through the world by way of a recursive feedback loop which “...adds up to a double movement — we design our world, while our world acts back on us and designs us” (Willis 80).

I explore this theory within UI/UX design by expanding on how it is already approached implicitly in the centering of some values, while also advocating for areas where it has not been approached critically, and, I argue, it should. I carry this out first by grounding the philosophical theory into a scientific domain by bringing light to the commensurability it shares with the notion of conceptual blending found in cognitive science. Conceptual blending, as we will see, is a poignant model for how human cognition, in the form of meaning creation, complex reasoning, and imagination, has advanced so rapidly through the evolution of our species. Once the broad, sweeping consequences promoted by ontological design have been placed within a proper system which can be deconstructed and manipulated, I will then begin to explore how this theory should be practically applied within UI/UX design. This notion of ‘should be’, I should state here in the beginning, carries a political aim; throughout this paper, and primarily in the latter half, my focus will be arguing that applied UI/UX design has sometimes progressed by centering values to the detriment of individual or social cognition. Sometimes this occurs implicitly out of ignorance or naivety, and sometimes this occurs explicitly by bad actors in the discipline. In either case, what has occurred is a lack of appropriately centering the right cognitive and cultural values that we ought to design for.

My attempt at centering the application of Ontological Design into UI/UX design takes shape through two primary sections; these are dubbed

“The Mission of Hot Cognition” and “The Problem of Hot Cognition” respectively.

In the former section I will unravel how modern, post-WIMP UI design has largely been carried out up till now, focusing on the engagement of system 1 functional networks in order to make tools effortless to use. In this section I will (1) elucidate how this one value focus is an implicit, and often too-narrow form of ontological design being carried out, and (2) discuss possible experimental paradigms in order to better determine which UI elements, practices, or principles that emerge out of conceptual blending, are best fit for forming such a reliance on hot cognition.

In the latter section I will explore the emergent issues of solely pursuing the mission of hot cognition in our design practices. The section will focus on two problems in particular, regarding (1) the exploitation of social media users through advertiser-based revenue models, and (2) the growing privacy and security concerns related to the unknown external connections our digital devices maintain. Though these two issues are not exhaustive of the problems of hot cognition, they hope to serve as the tipping point in a larger unconcealment on the importance of more thoughtful ontological design practices.

1.1 Ontological Design as an Inescapable

Politics

In order to talk about the implications for ontological design we need to talk more about what exactly the theory entails, how it works, and how it can be manipulated. It is a theory “... concerned with the nature and of the agency of design, which understands it as a subject-decentred practice” (Willis 81). That is to say, it works to dissolve the distinction between object and observer, between the designer and that which is designed. In this way, the designer, the process of designing, the designed object itself, and the context in which it was designed, all act to constrain how an agent may use the tool to pervade through, and understand, the world.

As an example, consider a jug of juice and its alternative container, the single serve juice box. What is the difference in the essential nature of these two tools? They both allow you to drink the same liquid. The way they allow you to do this, however, are fundamentally distinct. A jug is designed for communal gathering, and for being situated in a particular place whereas a juice box is individualistic, single use, and designed for on-thego efficiency. Hypothetically, how would people in a world with only jugs be different than those in a world with only juice boxes? I stress hypothetically because we must keep in mind that “...the juice box does not by itself design an entire way of life — the same kind of story could be told about any item in any designated environmental milieu. The point, however, is that there is no outside to this designing of things — material and immaterial.” (Willis 89). The consequences that arise from this designing of things, however, are the concern of ontological design; what cognitive and cultural values are we perpetuating or dissolving in the design of our tools?

The foundation for this claim lies with Heidegger and his concept of the hermeneutic circle. The hermeneutic circle is a philosophical model for meaning construction which relies on the central act of interpretation. Such interpretation relies on the feedback loop between context, an agent’s assumptions and values, and the tools they use to perform such interpretation. As we design or utilize new tools, our interpretations adapt and in order to situate our longstanding worldview with new information. Heidegger believes that it is impossible to escape this interpretive act since, as human beings, we are completely entrapped within the realm of our previous understandings. Lastly, I want to stress the political aim that I am extracting from this theory. As AnneMarie Willis states, arguments for ontological designing “...carry with [them] a politics...” by way of advocating for particular values to be accommodated for when designing (Willis 81). Through the rest of this paper we will be using this theory to make a stance, inevitably political, on how we should approach UI/UX design; what kinds of questions do we ask, and what problems should we really be trying to solve.

1.2 Conceptual Blending: A route to Bringing

Ontological Design into the domain of Cognitive Science

Ontological Design carries sweeping implications for how our tools and spaces can end up ‘designing’ us, but does this concept have any teeth? Conceptual blending, a powerful model for human cognition from Miles Turner and Giles Fauconnier, I argue, shares a lot of commensurability with that of Ontological Designing.

Acting as an evolution of ‘conceptual metaphor theory’ from Lakoff and Johnson, Conceptual Blending establishes relationships between multiple inputs in separate source domains in order to create a ‘blended’ domain of shared attributes and emergent structures. Emergent Structures are elements which did not belong to any of the original source domains yet arise inside the blended space due to the unique relationships established from the inputs and the context in which those inputs formed their relationships; Figure 1 helps show a general mapping of this schema. Fauconnier and Turner, in The Way We Think, demonstrate how this blending action occurs through most of human cognition including meaning formation, complex reasoning, language development, toolmaking, and even in constructing unrealities.

For example, in our domain of UI/UX design, the most pervasive result of conceptual blending has undoubtedly been the “desktop metaphor”, which is still being fashioned and refashioned (i.e. used as an input space in a new blend) in modern UIs. This blend is so successful because new users could “immediately use it...from their existing knowledge of office work, interpersonal commands, pointing, and choosing from lists” (Fauconnier & Turner 23). By sourcing elements from the various inputs of office spaces and hierarchical menus, and then integrating it into a unique mental activity, this blended space ends up producing what some designers refer to as “generative power”, or, the ability to produce even more complex future blending due to the nature of the emergent structures. This generative power is precisely why mental activity is not simply “...an elaborate conscious analogy but, rather, an integrated form with its own coherent structure and properties.” (Fauconnier & Turner 23) That is to say, when working on a UI employing this blend, we are not literally operating within the metaphorical constraints of a ‘Desktop’. This is why some actions, such as dragging a disc icon to the recycling bin to eject it, do not make sense when considered strictly from the Desktop domain but are coherent and intuitive within the blended space of the desktop GUI-based machine.

Once we grasp onto these blends and begin to use them for constructing future blends we become entrenched in them; that is to say, we become unable to think outside of them. This is what Heidegger meant by being trapped in the interpretative act. Further, a blended space is analogous with Heidegger’s “World”, in which he states “World is not equivalent to ‘planet earth’ nor to ‘all that exists’...instead world, or rather, worlds, are always circumscribed, situated, and multiple. But this does not mean worlds are entirely individualised, purely subjectivised spaces of perceptual dwelling” (Willis 84). Many blends are biologically or culturally ingrained through childhood development so are not “entirely individualised” and yet need to be uniquely manipulated in a subjective context by each individual.

(Figure 1. from Agarwal: Blending schema example of category

extension. A complex number is both a number and a point)

1.3 Material Anchors

In toolmaking, features of objects act as material anchors which compress the vitalrelationships between source domains. This sort of designing process is almost always being performed in the background, unconsciously. This is akin to ontological designing occurring

“...whether the perceiving subject (who is the subject of, as in subjected to, the designed) is aware of it or not” (Willis 90). But we can also step back and attempt to make it explicit. The notion of material anchors helps us clarify what Heidegger means by his intertwining of the relationship between building and dwelling. One generally assumes that you must build a space before you can dwell inside it; However, as Ann-Marie Willis summarizes, “Heidegger refuses this saying that ‘to build is in itself already to dwell” (Willis 85). What this means is that the planning of actions is intertwined with the performing of those actions. Anne-Marie Willis uses the task of weeding her garden as an example. She knows, while sitting at work, that she must do this once she arrives home.

In thinking about it, she has a multi-sensational representation of this task in her mind: A visualization of the lawn, equipment location in the garage, anticipatory loathing of the scorching sun beating down on her, a mental tracing of the path she’ll take, etc. This mental representation is more than a daydream, it is a form of “...thinking which prefigures doing and is a designing of the task and of my time...” (Willis 87).

This same notion of pre-planning is present in conceptual blending. In The Way We Think, Turner and Fauconnier talk of the construction of a gothic cathedral with the purpose of focusing the diffuse atmosphere of “sacrality” into a particular location. As such “...the cultural and perceptual experience of the cathedral...must conduce the community to activate a blend that gives them a sense of the sacred” (Fauconnier & Turner 207). When the architect considers what the building might look like he finds himself searching for that unfocused force of sacrality through imagination and memory. He sketches a blueprint from this imagination which acts as a focusing of this force before the church is actually made. In this way, “the cathedral is developed as a conceptual structure in the blend before it has an accurate material anchor to support that blend” (Fauconnier & Turner 209). Once this is crafted, the material anchors, such as candles, confessionals, special activities, vestments, etc. can begin to support and communicate this conceptual structure to the community.

Understanding and deconstructing this preplanning and establishment of material anchors in order to promote the right emotions, customs, or thought process is at the heart of ontological design; it is what Ann Marie Willis calls allowing ontological design to perform its “unconcealment”. The remainder of this paper is focused on performing this unconcealment for our modern UI/UX design practices. In the next sections I will explore (1) how we are implicitly performing conceptual blending already, for better or worse, in the mission of hot cognition, and (2) the ways in which this mission often causes us to engage in a more harmful concealment. As Turner and Fauconnier state, “Computer engineers and designers are well aware of the issues of metaphor in the design of interfaces...but the role of unconscious blending in the design of interfaces, and the construction of metaphor they use and develop, has gone unnoticed” (Fauconnier & Turner 33). Well, let’s notice it.

1.4 Scope

The scope of UI/UX design we could discuss here is potentially endless; we can discuss lower level abstract HCI models or high level UI elements, a narrow range of devices or the entire gamut of GUI based tools which nowadays includes Fridges, thermostats, and dishwashers. To keep the length and quality of this paper manageable I am limiting the scope to post-WIMP, internet connected devices focused at the high OS level. Post-WIMP interfaces are Graphical UIs which attempt to go beyond the classic menu and pointer paradigm. In this way, I will avoid talking about input methods which can vary between devices using the same OS (touchscreen vs mouse & keyboard). I will likewise avoid talking about solutions for individual applications and focus on features that can be implemented OS wide. Any lessons that can be extracted from this paper I hope can apply somewhat universally to the broad range of “smart” devices which increasingly pervade our lives, from smartphones and tablets to smartwindows and fridges. When framing potential solutions, I will do so using the two most pervasive post-wimp OS’s in place today: Google’s Android and Apple’s iOS.

2 The Mission of Hot Cognition

Don Norman, in The Design of Everyday Things, partakes in an implicit form of ontological design. He cares about how to utilize this feedback loop in order to provide the most intuitive and easy to use design of everyday tools around us (such as doorknobs and faucets). To accomplish this, the cognitive value he centres is that of a reliance on ‘hot cognition’. Hot cognition is a resource-light and automatic mode of human thinking commonly associated with a downregulation of the prefrontal cortex and increased activity of the Anterior Cingulate Cortex (Slingerland 33). When we rely on hot cognition we are often depending on our habits and instincts rather than conscious deliberation. This is accomplished, for Norman, by establishing proper conceptual models of the devices we work with; A conceptual model being the ‘story’ we form about how something works or behaves based on our experience (Norman 75). Conceptual models are critical in establishing what we expect our tools to do. By extending existing conceptual models onto new tools or creating new, but intuitive, models, individuals are less likely to expend more cognitive resources in utilizing a device by engaging in cold cognition.

In terms of conceptual blending, a conceptual model is the blended space’s organizing frame that is constructed and the integration network at play in constructing it. Fauconnier and Turner talk of four different kinds of integration networks that work in different ways to form a blended space; simplex, singlescoped, double-scoped, and mirror networks. For this paper, we will discuss the first three. It is important to note that these networks, while feeling quite different, “...are not four separate species that exhaust the world of blends, but, instead, are prominent points that stand out on a continuous landscape.” (Fauconnier & Turner 139)

2.1 Types of Integration Networks

The major difference between these networks is how they manage to synthesize relationships from the various inputs into an organizing frame within the blended space. A frame is a conventional and schematic organization of knowledge such as “the office space” within the desktop metaphor. Simplex networks take a frame from one input, often one that has become biologically or culturally entrenched, and maps elements from the second input directly into that frame within the blended space. Single-scoped networks similarly take the organizational frame of one input and integrate it within the blended space, however, in this case there is a frame within the second input that is not projected into the blend. Double-scoped networks, in contrast to both of these, utilize an organizing frame in the blended space that has an emergent structure of its own between the differing organizing frames found in the original inputs; “...in such networks, both organizing frames make central contributions to the blend, and their sharp differences offer the possibility of rich clashes” (Fauconnier & Turner 131). Due to this, the conceptual model learned from the blended space, in contrast to simplex and single-scoped networks, is often uniquely creative.

Norman’s program of hot cognition, I argue, implicitly relies on attempting to constrain the types of blending that occur when learning to use a new tool, discouraging a need for the user to perform double-scoped blending in favour of simplex or single-scoped blending setups. This is because double-scoped networks inherently require the user to construct a new organizing frame rather than rely on projecting one that’s already complete from the inputs. The inherent complexity in the latter case requires the user to engage in cold cognition at least up to the point of establishing the correct frame. To solidify this point, Norman incorporates a quote from Alfred North Whitehead which says “It is a profoundly erroneous truism...that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them” (Norman 101). This is not to say that Norman’s program outright bans the use of double-scoped integration networks, rather, it is just to make the point that this program implicitly avoids them, when possible, in favour of reducing the complexity of the system in order to promote “two of the most important characteristics of design...discoverability and understanding.” (Norman 3).

When we look at the Desktop metaphor, however, we see that it demonstrates a deep double-scope network.

“The computer desktop interface is a double-scoped network. The two principal inputs have different organizing frames: the frame of office work with folders, files, and trashcans, on the one hand, and the frame of traditional computer commands on the other. The frame in the blend draws from the frame of office work--- throwing trash away, opening files--- as well as from the frame of traditional computer commands----”find”, “replace”, “save”, “print.”. Part of the imaginative achievement here is in finding frames that, however different, can both contribute to the blended activity in ways that are compatible. Throwing things in the trash and printing do not clash, although they do not belong in the same

frame.” (Fauconnier & Turner 131)

Fauconnier and Turner suggest that double-scoped integration is the “crucial capacity needed for thought and language'' (Fauconnier 1). This calls into question the ultimate effectiveness of the Norman approach to the mission of hot cognition. Might it in some cases be better to create a tool which is harder to learn by depending on a double-scoped integration network, but, once learned, offers a greater generative power than it otherwise would have had the tool been forced to conform to a single-scoped or simplex blend? In the next section I begin to talk about a design language presented by Google as an example of the power we can tap into with our tools when we embrace double-scoped blending setups.

2.2 Material Design: A modern-day exemplar for double-scoped integration networks

To discuss double-scoped conceptual blending within UI design, I situate Material Design, a design language from Google and utilized in their major OS platform ‘Android’, as a research point. Material Design rose out of a fundamental question the developing team asked, “What is the Material that our software is made out of?” (Google Design). Rather than simply treating the screen as a static canvas on which boundless information can be displayed, they went beyond this by asking what properties this canvas should have and in what ways those properties should be conveyed. The answer came in the form of ‘Digital Paper’ and is a direct result of double scoped blending setups. Digital Paper is a unique organizing frame which sources independent elements both from the frame of physical ink and paper (shadows, colour, tangibility) and the frame of digital hierarchy (containers, viewscreens, scrolling). This frame has emergent elements such as the ability for a tangible canvas to reshape itself according to a naive digital physics. The emergent frame of digital paper, in theory, not only advances the generative power of the desktop metaphor in a profoundly creative way, but once learned, also provides the user a better conceptual model for how information is presented and organized across the UI. Through the frame of digital paper, the canvas of information that exists has now been given structure and laws to conform to; it becomes more predictable to the user as the metaphor is given more depth. Google’s Material Design project is one of the most concerted design efforts of any major tech company. Their goal, akin to Don Norman, is to create a universal design language (conceptual model) that pervades across platforms and devices. In this way, Google is implicitly considering implications of ontological design. Without constraint, a digital screen has infinite potential in the way it can create, display, and organize information. This metaphor of digital paper allows us to constrain this potential and set up proper expectations for it. Unlike Norman, however, immediate simplicity, which depends on preserving an organizing frame from an original input, is not the forefront strategy to obtain this conceptual model.

Seeing the potential of double-

scoped blending in forming more versatile and indepth conceptual models leads us to ask, how do we select the appropriate types of blend to set up in our designing of User Interfaces? In Evaluating metaphor reification in tangible interfaces, Celentano & Dubois attempt to characterize the relationship between metaphors and their implementation in Tangible User Interfaces (TUI’s). They seek to “...allow designers to evaluate to what extent a tangible interface is a reification of the metaphor, i.e.... to what extent the implementation of a (metaphorical) interface matches the metaphor” (Celentano 232). They do so by positing three properties which should take precedence when selecting metaphors: coherence, coverage, and compliance. However, limitations exist in this approach as they “...formalize the relations between a metaphor and an interface...but do not address the quality of the metaphor itself” (Celentano 245). Addressing the quality of the metaphor is difficult to accomplish in any objective manner since, through the lens of ontological design, it depends on the cognitive or cultural value a designer wishes to centre in their tool design. Further, while Celentano & Dubois make reference to conceptual metaphor theory in their application, they reduce the importance of background, environmental properties as “...part of the aesthetics....used to complete the perception of the system state...but have no role in fulfilling the specifications of the digital application” (Celentano 240). This approach, then, seems to ignore how such “contour objects”, as they refer to them, are often inextricably linked to which elements from an input space are selectively projected to the blended space when running a blend. Our design process should address the quality of the metaphor by focusing on this value of hot cognition while also embracing the impact contour objects have on the overall blend.

Before diving into the experiments of this section, I should make note of one other paper written by Jetter et al. This paper is unique, and worth the read, in that it explicitly approaches the design of interactive post-WIMP interfaces by a model crafted through conceptual blending. They build an experimental TUI based on four derived domains: individual interaction, social communication, workflow, and physical environment. Unlike Celentano & Dubois, this paper centers the physical environment, or ‘contour objects’, as a constructor of the blended space.

Though informative, the research faces limitations pertinent to this paper as well. Primarily, it lacks a

proper empirical testing apparatus in favour of a proof-of-concept UI. Second, the focus on TUI’s, much like the Celentano & Dubois’ article, rely too strictly on literal or skeuomorphic metaphors. This is because of their approach to the “power versus reality tradeoff” posited by Jacob et al, in which the goal “...is to preserve the user’s perception of familiar and natural interaction with the physical or social realm....by creating post-WIMP UIs that carefully blend our familiar reality with the novel powers of computation” (Jetter 1147). Much like the Don Norman program of hot cognition, this approach conservatively favours single-scoped and simplex integration networks in order to decrease complexity whilst increasing understanding and discoverability. That is to say, they set up a system of metaphor reification based on coherence while avoiding a double-scoped network's ability to expand what we may potentially qualify as coherent.

2.3 Experiment Proposals for guiding DoubleScoped Blend’s impact on hot cognition

In theory, we want to set up a blend which provides the most generative power of the metaphor while also setting it up in such a way that it is easy to establish an intuitive organizing frame in the blended space. That is to say, we want to continue along the mission of hot cognition while embracing a more explicit, conscientious focus on the double-scoped integration networks a user builds into their hot cognition.

In order to determine if a double-scoped blending setup, or any conceptual blending setup, is successful in forming a reliance on hot cognition, we should be able to empirically test the differences in the conceptual model it provides and the efficiency that comes about from that particular conceptual model. In this paper I propose two experiments in order to help determine whether a particular UI design element or principle, which was formed through an explicit conceptual blending practice, promotes a proper reliance on hot cognition in this way. The two experiments measure (1) and (2) respectively: (1) Does the UI element or design principle effectively communicate a conceptual model that would otherwise not be present? And (2), If (1) is true, does the formation of the conceptual model contribute to a more efficient ‘pre-planning’ of actions, i.e. does it better integrate the usability of it into hot cognition.

Point (1) follows the Don Norman program of creating a conceptual model that does not depart much from a user’s “knowledge in the head” while also promoting the unique abilities of the tool in question (Norman 74). Testing this domain will allow us to understand what UI elements or design principles are easiest to learn, or easiest to integrate with the models we already hold. Unlike Don Norman’s process however, this apparatus attempts to more conscientiously embrace the testing of conceptual models produced by double-scoped integration networks. The paradigm in (2) is more concerned with whether these conceptual models allow us to engage with our tools not just with more conceptual depth, but also with better efficiency. The idea of preplanning actions comes from Heidegger's hermeneutics we mentioned before.

2.4 Material Design in the Experimental

Paradigm

My reason for introducing Google’s Material Design is not solely based on its implementation of complex double-scoped blending, but also because the Android OS it originated on makes it a prime foundation on to which one could test these two questions in an experimental paradigm. Android, as opposed to its post-WIMP market competitor iOS, is open source and virtually hardware independent. Android can be run on any major chipset platform (x86 or ARM) either natively or by emulation through the Android Virtual Device (AVD). Further, Android can be emulated using versions prior to elements of Material Design being implemented (API > 21) and after (API <= 21). Lastly, as an experimental framework for designers, Google has a dedicated webpage with full-scale documentation for what the various components and principles of Material Design are along with how best to implement them (Design). This documentation is ever-changing, even including a beta section for developers to experiment with and provide feedback (Secord). All of this, I argue, makes it a prime foundation for testing implementations of conceptual blending for UI design, particularly blends formed from doublescope integration networks.

It should be noted that a limitation of using the AVD in this way is that it constrains the experimental design to only being able to test conceptual blends that arise from Material Design based UI elements or principles, and would be difficult to generalise outside of that model. The scope of this section, however, is principally focused on demonstrating the need and power of testing such blends which I believe the Material Design model allows us to do.

2.5 Experiment 1: How Effective is a Particular Blend in Informing Conceptual Model?

This first proposition is an attempt to measure to what ‘depth’ a particular UI element or overarching design principle can be used to inform a conceptual model. Does the blend of digital paper get adequately carried across by the individual elements and laws it calls for? In this first experiment I will focus on the Material Design principle of “Continuity”, shown in Figure 2. This principle states that “while a surface is expanding, a significant number of elements should remain visible during the transition” (Choreography). In other words, all content should be derived from an existing place within a preserved context.

Both a control and experimental group will use a ‘phone’ application on an AVD. In both groups the applications provide the same information and layout to organise it: However, the experimental group will get unique animations in their version which use Material Design API’s and focus on the principle of Continuity. In this example the animations of the profile picture, name, and header information explicitly preserve one instance of the data from one screen to the next. The control group, in contrast, will get the same information, but the context that information exists in will not be visually preserved; that is to say, the information should seem like it is reappearing in a new instance.

(Figure 2 from Eric Azares: Stills from a Material

Design transition animation example)

Both groups will be asked to perform a series of tasks, such as editing the information of a

contact. Afterwards, they’ll be asked a series of open-ended questions to describe how they conceived of the information and navigated through it; questions such as “What happened when you selected a contact from the list?” or “Can you describe how the information you saw appeared on the screen?” To support that the experimental group would have formed a more in-depth conceptual model, we would expect members of this group to provide some subjective indication by keywords or phrases such as “When I select the contact, the profile picture enlarges from the

thumbnail and moves towards the top”, or “The phone number moves into a column on the next screen where I can then edit it”. In contrast, we would expect the control group to give relatively more simplistic answers that are focused on the information and task itself rather than the relationship of the information and its context. We might expect them to say “Their phone number was editable after I selected the contact from the list” when asked the same question. It is important to phrase questions in a way that does not lead groups into particular, confined answers or keywords while also being broad enough to allow the experimental group to provide more conceptual detail which the control group would, presumably, be unable to. We should reinforce that in the case of double scoped blending, responses in the experimental group should indicate that they are selectively projecting, and dealing with clashes, from the two or more competing frames the blend sets up for. In the case of the Continuity principle we are specifically looking at the interplay between frames such as “Visual Hierarchy”, “Context Preservation”, “Containers”, and even “Animations”. These frames are not being integrated all at once of course, but, rather, all exist in a complex feedback loop of conceptual integration during the use of the UI.

The obvious limitation in this approach is that the principle of Continuity is in itself a complex blend built off more primitive blends; As we can see in the animation of Figure 2, multiple different UI elements are coming together at once to inform the model in this proposed test. Further, the principle of Continuity is but one input domain for the more complex principle of “Choreography”. I am not going to delve into how each of these pieces connect---that is what Google’s documentation is for---but I hope this serves the point that these principles exist in a complex, multi-chain feedback loop just as conceptual blending entails. Fortunately, Google’s documentation provides full-scale examples, and “Do and Don’t” animations showing the best methods in practice. These can all be subject to this sort of experimental design. The issue is, of course, that the more particular or minute the element, the harder it would be to get a significant difference in the subjective reports between groups. Some individual elements do not inform equally well when separated from the whole of the overarching principle it intends to convey.

2.6 Experiment 2: Tracking How a Better

Formed Conceptual Model Affects Efficiency

In the next experiments I want to focus determining to what extent establishing an informed conceptual model allows a user to more efficiently pre-plan their actions. Like the first experiment, an experimental group would be set up with a Material Design AVD whereas the control group would run an AVD with an Android version lacking any Material APIs.

Material Design stresses an importance on consistency; universal UI elements that commit the same predictable function or behavior amongst various applications or platforms. For example, it is recommended that all messaging apps have a floating action button (F.A.B.) on the main screen which allows the immediate initiation of creative functions. Further, within this F.A.B, an immediate presentation of recently engaged with items should be contained within it. Lastly, commands of commitment (such as sending a message) should be performed via a right-aligned button on the designated ‘ActionBar’. Figure 3 helps to demonstrate how these different elements come together on a general email application.

Do users who have been regularly exposed to a consistent conceptual model such as this carry out tasks faster than a control when exposed to a novel program or task?

(Figure 3. Consistent UI elements among similar application uses)

This experiment requires the preliminary gathering of survey data to screen the selection of control and experimental groups. These questions would screen a participant's daily device and application usage in order to separate them into a group with a pre-existing conceptual model for material design and a control without previous exposure. Questions would include:

Questions

Possible Answers:

Q1. Do you have daily access to a smartphone or tablet?

• To control for non post-

WIMP users

“Yes, No, I’m not sure”

Q2. Circle which mobile operating systems are you’re most familiar with using

• To separate Material Design OS users from non

iOS (iPhone),

Phone,

Sure

Possible Answers:

Android, Windows Other, Not

Q3. What is the primary email application you use on your smartphone to check email?

• To isolate users who may use a

Material Design application on a non-Material Design OS.

1. 2.

3.

4. 5.

6.

7.

Gmail Outlook on iPhone Outlook on

Android

Apple Email

Inbox by

Google

Yahoo

Email

Samsung

Email

8. Other

9. I do not use email

10. I’m not sure

For example, users who choose ‘Android’ for Q2 along with either 1, 3, 5, or 7 for Q5 would be candidates for group 1 (herein referred to as the familiar group) as all these applications incorporate the three elements of Material Design noted in the previous paragraph on the Android OS. Users who select 2, 4, or 6, for Q5, on the other hand, would be candidates for the control (herein referred to as the unfamiliar group).

There are two application shells that will be used for this example experiment, Material Email (ME) and Non-Material Email (NM-E). Material Email will be a basic list email application which includes the three material design attributes discussed earlier: the F.A.B, ActionBar, and Recents Initiation (as shown in figure 3). These attributes have been centered in for this example since (1) they are all consistently present in the email applications used by those in the familiar group, and (2) these UI elements have been in place since the earliest stages of Material Design. NMEmail will have intentionally contrasting properties with more general design choices commonly still commonly used today such as:

• “Create message” button incorporated with other selective actions across a bottom panel (akin to many first-party iOS applications) • Contacts list must be instantiated into a separate view by clicking a secondary button • The send button is located on the bottom of the screen in the same panel of buttons as before

Both groups will be given a series of tasks to perform using applications which do not contain any aspects of Material Design so as not to prime for a particular conceptual model before testing. After repeated sets of this, the researcher will then ask both participant groups to open the designated Material Email application, asking them to send an email to a pre-created user in the contact list. While they are carrying this task out, an internal timer and screen recorder will be running upon application launch; It will track and record timestamps for every tap response (or click if using a computer). When the final sent button is clicked, marking task completion, the data is then recorded as a single trial. Both groups will then repeat this task within the non-material Email app. The results between the speed at which the familiar group and unfamiliar group carried out the task will be compared in each application. If there is an effect on efficiency from having a more informed conceptual model, then we expect there to be a significant difference in the speed with which the familiar group carried out the assigned task in the material Email app. We would also expect the data between each group to not differ significantly, if at all, in the non-material Email app. If the nonmaterial Email results do carry a significant difference, then this would entail that the experimental conditions were not appropriate for testing conceptual model familiarity and that the comparison data for material Email has been confounded. Tracking the intermediate clicks in application use allows us to not consider results that show users completing the task with extended moments of inactivity.

The experiment above attempts to unravel the effectiveness of having formed long-term conceptual models by separating participant groups based on how familiar they are with Material Design based UIs. This could be modified in an alternate version in order to examine a conceptual model’s effectiveness as learned in the short-term. In this case, you would only screen for participants who have had little to no previous exposure with Material Design. In the first step of performing randomized tasks at the instruction of the researcher, one group would solely use an application with designated Material UI elements or principles whereas the other would use solely non-material counterparts. This allows one group to form a conceptual model for particular UI principles while the control does not get this opportunity. Then, after some repetition of these tasks, users in both groups would be given a novel material application with the Material APIs and asked to perform the same task which is recorded and time-stamped at each button click. Effectiveness of the conceptual model for providing efficiency over a short-term learning period is measured in the same way as the previous experiment.

The data-collecting phase of this experiment could also be modified by giving participants a task and asking them to talk their way through how they would attempt to carry this task out without actually having them do it. We might expect that the familiar group more accurately, and more consistently (between participants), describe the correct steps to carry out, even in the face of a novel task. Measuring the effectiveness of a conceptual model in this way allows us to more explicitly understand how such models affect the extent to which a user can ‘prefigure’ their actions before carrying them out.

This experiment maintains the same limitation as Experiment 1; Namely, that the example brings together multiple independent material UI elements in testing rather than isolating for the effects of one at a time. Again, it should be encouraged to manipulate how these elements inform each other in the overall conceptual model when testing. A further limitation in this design is in how it attempts to quantify efficiency; we may not apprehend any significant difference at all in task-completion time between groups. In this case, we might ask whether the goal of maximizing a reliance on hot cognition is valuable at all for the particular task being carried out. Perhaps a designer’s focus could be better served on a cognitive or cultural value at odds with the mission of hot cognition.

2.7 Accessibility

Before we leave this section on the mission of hot cognition, I want to briefly speak to how Accessibility features factor into this discussion. The Desktop Metaphor is visually contingent, thus, gives blind or low vision users an inherent disadvantage at being able to form and utilize the proper conceptual model a UI provides. As a consequence, Material Design is even more dependent on visual interaction due to its heavy use of lighting, depth, colors, and animations to carry across the Digital Paper & Ink organizing frame. This is an ontological design problem, I argue, because the notion of equal access to the generative power given by a conceptual metaphor for any particular UI comes as an afterthought to promoting hot cognition for non-visually impaired users. This segregation of the user base is a cultural value we ought to design against.

An insightful paper from Kris Van Hess and Jan Engelen attempts to construct an interface rendering framework which provides equivalent representations of a UI element through a multimodal user interface. They do so by constructing their framework in order to retain conceptual collaboration, i.e. “the act of working...with others on a joint project with communication...within the context of the modality-independent user interface interaction semantics” (Hess 354). I find that facilitating this collaboration between impaired and non-impaired users is of profound importance when trying to equalize and desegregate how tools empower each type of user. Hess et al. focus on developing an abstract, low-level framework which renders the UI within each modality separately based on syntactic or lexical information presented at a conceptual level. That is to say, using their PUIR framework, non-visual modalities are intentionally not built by interpreting and translating visual information through an external accessibility feature; Figures 4 and 5 show the differences in these structures.

(Fig 4 from Hees: Accessibility features tend to be additive and read information from a visual domain, leading to an inherent loss of information through its translation)

(Fig. 5 from Hees: Rendering information in different modalities using equivalent representations)

To briefly talk about this at a higher level, platforms such as Material Design ought to focus on ways they can create multi-modal representations through complex blending. While I will not discuss concrete solutions here, two modalities which I believe have not been given significant attention are Sound & Haptic Feedback; OS developers should begin asking questions regarding how we can build in-depth conceptual models through these modalities. Apple has made significant strides in the haptic feedback mechanisms for their major smart-phones, creating pressure-sensitive screens and variable vibration feedback mechanisms. Unfortunately, Apple has used these as luxury high-end features to increase the generative power of visually based UIs, often ending up as failed experiments (e.g. 3D Touch) rather than tapping into the power these new modalities could provide to disabled users. We could create areas of a screen that provide different levels or patterns of vibrational feedback depending on what kind of information is presented or what action is performed.

In The Design of Everyday Things, Don

Norman mentions that,

“Many devices simply beep and burp. These are not naturalistic sounds; they do not convey hidden information... Real, natural sound is as essential as visual information...reflecting the complex interaction of natural objects: the way one part moves against another; the material of which the parts are made—hollow or solid, metal or wood, soft or hard, rough or smooth... Experienced mechanics can diagnose the condition of machinery just by listening...When sounds are generated artificially, if intelligently created using a rich auditory spectrum, with care to provide the subtle cues that are informative without being annoying, they can be as useful as sounds in the real world” (Norman 156).

A whole world of sound could be created and experimented with in order to inform users of what actions they are taking through volume, pitch, tonal shifting, depth (surround sound), or complexity. To their credit, Google has been recently focusing on sound design within their material design documentation, but this focus is still in its infancy and the concern has not been extended appropriately towards disability access (Sound Choreography). The possibilities are endless for providing visually impaired users a properly in-depth conceptual model in which to facilitate equal collaboration with their nonvisually impaired peers. We first need to centre the value in our design work, however, in order to achieve such a goal. If we only ever design for users with disabilities using additive features which rely on an inherent loss of information through translation, we implicitly reinforce an inequality of information for disabled people. This, I argue, should not be the way we approach UI/UX design.

(Fig 6: Material Design Sound Choreography Example)

3. The Problem of Hot Cognition

So far, we have discussed implicit ontological design practices in the formation of a reliance on hot cognition. We have seen how this mission may constrain itself too harshly in creating easy to learn tools by ignoring the generative power that can come about through more complex, harder to learn, double-scoped integration networks. We have also seen how this mission may be geared unequally to a particular subset of users when we focus solely on visual-based conceptual metaphors. In this 2^nd section I want to shift from trying to better the mission of hot cognition towards trying to caution against it. Often, designing for a reliance on hot cognition ends up harming users by either reducing or exploiting cognitive ability, or by disempowering users as agents in the world by restricting how they can utilize tools. In this section I will briefly outline two examples of where I believe this occurs quite prominently today: (1) through exploitation of user attentional capacity by way of advertiser based revenue models, and (2) through growing privacy concerns related to the unknown external connections our digital devices maintain at any given moment. Though these two issues are not exhaustive of the problems of hot cognition, I intend them to serve as the tipping point in a larger unconcealment on the importance of ontological design practices that centre more than simply one particular cognitive value.

3.1 Painting the Issue: How Advertiser Based

Attention Models exploit Hot Cognition

Social media companies sell user attention to advertisers; this is the primary way in which they turn profit. These platforms need user attention to be sustained on their platform for long-periods of time while also requiring that their attention acts fleetingly towards any individual piece of content they may encounter; This encourages users to cycle through an influx of new content, and with it, targeted advertisements. The nature of this sets up a recursive system of exploitation in which social media platforms become intrinsically motivated to be as engaging and addictive as possible for the end user, including content creators and content consumers. These platforms employ systematically discrete tactics to reinforce this, such as the algorithmic suppression of content with external links which would encourage the user to leave the application, even briefly. This, I argue, indicates a systematic exploitation of a user’s reliance on hot cognition; the designers for these platforms have no motive for empowering the user with a UI feature that would give them an opportunity to decrease or cease engagement with the platform, especially as the engagement becomes more ingrained in their habits. This not only harms the end users through attention exploitation, but it also harms content creation as a whole; creators are coerced to constrain their content in order to appeal to the algorithmic rules of increasing engagement. The emergence of platforms such as TikTok and Vine, which are built on the premise of short 15second to 2-minute videos, are arguably a direct result of this exploitative feedback loop. Content creators notice that the algorithm will promote content which does not take them out of the application, does not force their attention too long, or is made with the use of platform specific tool (e.g. Instagram filters). At a certain point, the amount of content that conforms to these algorithmic rules dominate over content that does not since content creators are intuitively seeking to maximize engagement. This, then, leads to the creation of platforms, such as TikTok, which are able to capitalize on the manufactured popularity of these types of content (short, humour, meme videos). In this case the motives of the platform, content creators, and end users have all informed each-other in a way that could have, and probably should have, been otherwise.

I use the term manufactured popularity purposefully to act as a derivative of the Chomskian notion of “manufacturing consent” in the sense that media networks, through structural filters, serve the interests of those who finance or control them. One of the most prominent filters in which this occurs is through advertising.

“...major corporations which are parts of even bigger conglomerates...have a product which they sell to a market. The market is advertisers — that is, other businesses. What keeps the media functioning is not the audience. They make money from their advertisers...So they’re trying to sell a good product...which raises advertising rates...That means that they want to adjust their audience to the more elite and affluent audience...So what you have is institutions...that are selling relatively privileged audiences to other businesses.

Well, what point of view would you expect to come out of this? I mean without any further assumptions, what you’d predict is that what comes out is a picture of the world...that satisfies the needs and the interests and the perceptions of the sellers, the buyers, and

the product.” (Chomsky)

In regards to this type of exploitation within the domain of today’s social media platforms, Chamath Palihapitiya, a former Facebook executive and current CEO of Social

Capital said, “…the short-term, dopamine-driven feedback loops that we have created are destroying how society works...it is eroding the core foundation of how people behave by and between each other...Bad actors can now manipulate large swaths of people [and]...if you feed the beast, the beast will destroy you. If you push back against the beast you have a chance to reign it in” (Stanford). It is an even more powerful sentiment, I think, when a venture capitalist billionaire, who has more power than most, and also played a part in the creation of these tools, becomes concerned himself with how they fundamentally exploit and disempower users.

This is a large systemic issue with no onefix solution; Arguably the exploitation stems from a structurally capitalist system which puts profits over people, but such discourse is out of the scope of this paper. The approach that ontological design provides, however, can help us push back against the beast. Palihapitiya calls for a hard break of these tools, which may not be feasible due to how ingrained they are in modern society. But we can attempt to moderate the harmful effects and empower users against these forces when possible. Then, the question is, what tools can we give users at the OS level so that they at least have the opportunity of protecting themselves from such exploitation?

3.2 Digital Wellbeing: Empowerment Through

App Timers

In searching for a way to empower users against attention exploitation, one feature becoming increasingly common on post-WIMP OS platforms are App Timers. This feature allows users to cap their use-time of a particular app per twenty-four-hour period. I wanted to explore people’s social media habits and whether or not they’re using app timer’s, so I ran an informal survey distributed publicly through peers and the peers of peers on platforms such as Facebook, Twitter, and Reddit. This data has been gathered informally and should not, by itself, be used to make any generalizations. However, it is commensurable with more statistically valid data reported by Roffarello & Russis in their article focused on understanding the benefits and issues with App Timers and other “Digital Wellbeing” features. By aggregating review data and distributing their own Digital Wellbeing application for testing, they saw that, while these apps appear helpful for users who are trying to control unhealthy behaviours or limiting the amount of distraction during short-term goalachievement, the current available options, whether first party or third, come with limitations. These limitations are mainly in the lack of long-term habit formation, context sensitivity, and social support features. The remainder of this section will focus on the former two issues.

In my informal survey, with a sample size of 72, over 90% of the participants reported that, when bored, they will check a social media feed either “Almost always” or “Often”. While a bit more distributed, a majority of ~63% of participants indicated that they would also distract themselves with a social media app when they hit a moment of difficulty engaging with a difficult task utilizing cold cognition. Even though a majority accounted for doing these actions, only 47 (65%) knew about the app timer feature available on the device they have daily access to. Of those who knew of app timers, only 25 (53%) individuals knew how to access them on their device. At the forefront we see a motive for trying to better inform users about the nature and utility of these features when they interact with their devices; users should not have to intentionally seek these features out or inadvertently learn about them. Fixing this, however, does not fix the drawbacks that come from using the feature.

In my survey, only 12 people (16%) indicated ever using an app timer, with 9 of them indicating it was at least “Somewhat Helpful”. This is not a very generalizable audience and the data should not be held to any statistical relevance on its own. In the proceeding survey questions, however, I asked participants, open-endedly, why or why not they indicated app timers helped them in their goal. Many of the responses indicated that these app timers were not restrictive enough. One response indicated it was unhelpful because “… [iOS] screen time allows you the opportunity to extend your use on the app, which means I always just turn it off and keep using [the] app” (Lucero). 7 of the 12 participants who had used app timers provided the same sentiment. This parallels data from Roffarello & Russis such that “[An] interesting theme that emerges from the reviews is that users want restrictive solutions, since permissive, ignorable, and unrestrictive tools are useless to reduce phone addiction...A considerable number of users, in particular, (N = 76, 7%), point out that digital wellbeing apps are often bypassable in some ways” (Roffarello 6).

Part of the problem, I argue, is the lack of foreground reinforcement for the app timer. Dan Ariely, in the domain of cheating prevention, presents the notion of “Fudge Factors”. Ariely claims that we all have a personal fudge factor, i.e.

“…a level of cheating we cannot go over… but can still benefit from as long as it doesn’t change our impressions about ourselves” (Ariely 38). He argues that people are more likely to cheat when they have an easier time dealing with the cognitive dissonance of their actions. This can occur when the agent believes that peers are also cheating, that the agent isn’t likely to get caught, or if they feel morally justified in cheating. Ariely found that putting an anti-cheating honour code at the beginning of an assessment, which the agent was required to sign before taking it, reduced cheating or even eliminated it in some cases. The idea here is that “...our willingness and tendency to cheat could be diminished if we are given reminders of ethical standards'' (Ariely 40). I propose that perhaps this can be extended into allowing users to better abide by the personal restrictions they set for themselves. Rather than setting app timers in the background and then forgetting about them until the timer elapses during an inevitably inconvenient moment, why not have the app timer countdown shown within the app itself, or perhaps unobtrusively in the status bar? This would help serve as a foreground reminder for the restrictions the user has attempted to set for themselves.

It is commonly noted that users will extend app timers after they expire. As such, it may be useful to remind users of any occurring patterns, notifying them when they repeatedly extend timers. Further, Roffarello & Russis point out that a major limitation in current app timer implementations is their lack of tools to help users form healthier longterm habits. They state that “...contemporary digital wellbeing apps are mainly designed to break existing habits...totally lack[ing] other fundamental aspects of the habit formation process” (Roffarello 11). By informing users not only of their app usage, but also of their goals and the patterns they develop against those goals, it may act as a small step in promoting more explicitly desired habits.

Another common issue facing the adoption of app-timers is the integration of messaging features in social media apps. Platforms such as Instagram and Twitter have direct messaging platforms in which users can engage in one-on-one or group chats. Those who use these features find that app-timers also restrict their ability to communicate with someone inadvertently when the timer elapses when they only want to disable access particular parts of the app. The best solution might be to disassociate these features at the app-level, allowing the system-wide app timer feature to restrict access to some features of the app, but not all. Roffarello and Russis already note that “... participants were more interested in acting at the app-level, rather than setting interventions at the phone-level” (Roffarello 9). In this case, allowing users to restrict specific features of an app may help them resist removing the app based restriction altogether. However, a limitation of this increase in control is that this would require an API and hook-in setup at the system level which social media platforms likely would not have any motive to abide by if given the ability; OS Developers would likely need to set API requirements for an app to be eligible to be posted in the designated OS Marketplace or “App Store”.

An easier to implement solution may be to allow the user to engage app access restrictions only at certain times of the day or when in certain locations. Rofarello and Russis find that current app intervention tools do not allow control of set restrictions based on context such as location, time of day, or task-completion priority. Perhaps the user may have a “Homework” profile they can engage from the status bar of their phone which, when activated, restricts a selection of apps until the user disables the profile. This more specific event-based restriction would be another way to allow users more control from the deleterious effects of multitasking.

Furthering the issue of long-term habit formation, A literature review from Kaitlyn and Elder establishes a pattern of data that fits the informal survey data I gathered, whereby a majority of University students engage in “mediamultitasking” during lectures, assignments, and other course work. This increasing tendency among youth has been associated with reduced GPA and comprehension performance on task with distractor paradigms. This data gathered from this literature review is consistent with the notion that, whether beneficial or not, newer generations of students are increasingly relying on task-switching while carrying out tasks. Kaitlyn and Elder use their paper to examine to what extent this increasingly ingrained habit is harmful to critical thinking performance, and in what ways, if any, it may actually help promote task-switching efficiency. To do this, they compare two conflicting theories on attention dubbed the “trained attention hypothesis” and the “scattered attention hypothesis” respectively. In the trained attention hypothesis, “Frequent media-multitasking could positively affect cognitive control…promoting mental flexibility that enables high-level efficiency”. In contrast, the scattered attention hypothesis entails that “Long-term media multitasking may lead to disrupted cognitive control in which the individual gravitates towards the preferred task rather than maintaining focus…” ( May 3). While most of the data Kaitlyn and Elder gather supports the scattered attention hypothesis, they did find, in the context of comprehension on a reading with distractor task, that “the role of media multitasking...is dependent on self-regulation and self-awareness. Students who are particularly metacognitive…recognized deficits in comprehension upon returning to the primary task and subsequently re-read portions of the article after interruption” (May 11). If this is the case, in what ways could we help promote such meta-cognitive awareness, reducing the deleterious effects of scattered attention and potentially promote effects outlined in the trained attention hypothesis?

A limitation in this paper is a struggle in offering solutions for tasks that are not completed with a post-WIMP device since work being completed off the device itself would seem to require some sort of external tracking of a user’s task engagement. Any solution in this vein, I believe, would struggle to escape the harmful effects this may have on a user’s privacy (talked about in the next section). However, I could imagine avenues for solutions when considering work being done from the same device that acts as a potential for distraction. Similar to the issue of context sensitivity, we could give the user control through profiles that, upon activation, set restrictions and reminders in order to encourage the competition of a singular task. The profile could even be disabled automatically once a task is completed. While tracking user actions within an application to determine task-completion progress is also a privacy risk, such risk can be better managed by restricting the data collection to the device itself.

To focus specifically on training metacognition, I propose solutions that assist the user in the efficient reengagement of a task they became distracted from. For instance, if a user was reading a textbook in the iOS Books app and switched away briefly to check a Twitter feed, upon returning to iBooks it may be beneficial to produce cues, such as a brief highlighting of the passage they left off at, in order to reduce the effort or time it takes the user in reorienting back to where they left off in the original task. This type of feature should be feasible to implement as well given the advancement of external sensors and machine learning capabilities in consumer products today. For example, the iBooks app on iOS already has the ability of tracking reading speed over time by averaging the amount of page turns. Further, many smart-devices carry IR and RGB camera sensors for eye and face tracking either as a security feature or for camera auto-focus. Therefore, I don’t find it outlandish that these tools can be configured to assist users in reacclimating their attention to the appropriate task. This also carries privacy risks in regard to external data collection. However, the scope of that data collection is better restricted to tracking direct engagement with the device itself.

These may not be the perfect solutions we end up converging on, but the point, I hope, stands that when we focus our design efforts to consider what cognitive consequences we might inevitably train for, we may come to find better suited features for our long-term habits.

3.3 Data Exploitation and Civil Liberties:

External Connections Our Devices Maintain Unbeknownst to Us

The last argument I want to explore is in regard to privacy risks and data exploitation concerns in using post-WIMP internet connected devices. The risk of data exploitation is pervasive, and we do not have the time to review the complete literature here. Briefly, smart-device data is incessantly collected, unbeknownst to the user, by ISP’s, cellular companies, and any company with a data server and the proper app-based permissions granted by the user. All internet connected devices are equipped with globally unique, hardwired identifiers which allows collected data to be attached to a particular device. This point is important because companies, and even governments, will claim that they only collect anonymous meta-data in mass which is not individualized. However, this notion of anonymity doesn’t hold water; Even a device that only has access to Wi-Fi can map the strength of nearby access points as a proxy for location. Furthermore, until 2018, the U.S. heavily employed the thirdparty doctrine----a law which states that citizens who give their data voluntarily to a third party (such as ISP’s) do not have ownership and right to privacy of that data---in order to construct systemic mass surveillance that is only now beginning to be rolled back. This issue, in the U.S., has been further compounded post-9/11 with an expansion of these powers through the PATRIOT Act.

For the scope of this paper I am going to focus briefly on solutions at the OS level involving permission systems and transparency to external connections. However, the extent to which location and usage data is able to be exploited by service providers and, by extension, government itself is important to mention here since, ultimately, these are systemic issues that need a reshaping of cultural attitude and government. While empowering users through OS-level features can help to mitigate harm, what our design practices truly need to do is properly centre how our designing, up till now, has helped create the harm that we are now trying to mitigate.

3.4 Finding Empowerment in a Reformation of

OS Permission Models

For individual applications on a user’s device, the extent to how much, and what, data can be collected tends to be dependent on (1) accepting privacy policy and terms of agreement documentation when opening the application along with (2) an OS-wide permission system which allows the user restrict access to what forms of data the application may access, such as “File Storage”, “Camera”, or “Microphone”. These systems, however, do not do nearly enough to empower the user in their right to privacy; they semi-arbitrarily group large swaths of data collection into a single permission, and they do almost nothing to enforce usability of an app when it doesn’t have the requested permissions granted to it. What is required is a full-scale reformation on how permission systems work and a fundamentally new conceptual model for how a user approaches and understands the external connections that exist on a device. Former CIA subcontractor and whistleblower Edward Snowden, in talking about his novel Permanent Record, frames this issue in a profound way. Rather than regurgitating his remarks I think it is pertinent to quote in length:

“…the central problem with smart phone use today is you have no idea what the hell it's doing at any given time…iOS unfortunately makes it impossible to see what kind of network connections are constantly made on the device and to mediate them…We need to be able to make these intelligent decisions, not just on an app by app basis but a connection by connection basis: you want to connect to Facebook's content servers, you want to be able to message a friend,…download a photograph or whatever, but you don't want it to be able to talk to an ad server…an analytics server that is monitoring your behavior…Facebook crams their garbage...and you don't even know what's happening because you can't see it…This is the problem;…there is an industry that is built on keeping this invisible… [and] we need to make the activities of our devices...more visible and understandable to the average person and then give them control over it.

If you could see [on your phone]...a little green icon [representing] your handset…and...all these little spokes coming off of it [representing] every app, or host, that your phone is talking to right now…and you can see [that] every three seconds your phone is checking into Facebook. [If] you could just poke that app and then--- BOOM—your phone is not talking to Facebook anymore; Facebook's not allowed, you would do that, we would all…press that button.

[However] that button does not exist right

now…neither of them [Google & Apple]

allow that button to exist. In fact, they actively interfere with it because they say it's a security risk…because we don't trust [users] would make the right decisions, we think it's too complicated for people…there's too many connections being made. Well, if you think people can't understand…if you think there's too much complexity in there, [then] it needs to be simplified…The story of our lifetimes is how intentionally, by design, a number of institutions…realised it was in their mutual interest to conceal their data collection activities to increase the breadth and depth of their sensor networks” (PowerfulJRE).

OS Developers argue that the reason these systems have not been properly implemented is because they would be too complicated for users to learn; that to show the sheer amount of external connections to a device at any one-point would overwhelm the user and reductively make the problem worse. This argument, as Snowden remarks, is an admission of the problem itself. The argument that the exploitation of users is occurring because we are unable to create an intuitive conceptual model to empower them is, at the risk of sounding uncharitable, a blatant lie that undermines what the job of a UI/UX Designer should be. It is a clearly misguided centering of hot cognition over a user’s intrinsic right to privacy and protection from exploitation. That is to say, we are allowing a strict adherence to the mission of hot cognition to act as justification for an unethical concealment in the inequality of available information.

It may be argued that many users rely on app-based features that they do not know necessarily use background collection. If users were given full control over their devices on a connection by connection basis then they may not end up taking advantage of features that are potentially useful to them. What this sounds like, however, is merely an educational issue, and one that would be fruitful for UI/UX researchers to tackle. If you want users to truly take full advantage of such features, then one should seek to prove that these tools can do it through noninvasive means. If a feature cannot avoid data collection, then the application should be transparent about what data is being collected, and in what contexts, so that the user can opt-in only once they are properly informed. The solution is not found by concealing the harmful consequences of using such features by ensuring the UI is built in such a way that a user cannot form a proper conceptual model for how it maintains its connections to the world.

4. Conclusion

In his book, Don Norman includes a section titled “The Moral Obligations to Design”; In it, he states,

“...That design affects society is hardly news to designers... But the conscious manipulation of society has severe drawbacks, not the least of which is the fact that not everyone agrees on the appropriate goals. Design, therefore, takes on political significance. In Western cultures, design has reflected the capitalistic importance of the marketplace...In the consumer economy... usability is not the primary criterion in the marketing of home and office appliances...We are surrounded with objects of desire, not objects of use” (Norman 291)

Don Norman shows that he is fundamentally aware of the inherent political nature in design, how it requires the designer to consider the individual and cultural values we instill in a tool. However, what has been missed in this approach is not simply that the importance of the marketplace has reduced our capacity to design objects of use, but, also, we have let it reduce our scope of ‘use’ with an often too-narrow constraint on hot cognition.

Ontological Design is not an instruction manual for how we should design. Rather, it is an acknowledgment of the political, cultural, and cognitive ramifications of the agency we give to our designing process. Ultimately, what I hope to have done in this paper is fruitfully extend what our moral obligations are in the designing of user interfaces.

In the next decade we are going to witness an insurmountable leap in the number of post-WIMP UIs we interact with, and in the hardware capabilities they will provide us. From smartphones to smart-windows, fridges, dishwashers, and voice assistants, the trial and error of the UIs embedded in these devices are going to inform how we, as users, conceive of and are able to interact with the world. As the trial and error converges on some universal principles, we should work to ensure such trials are set up with the right intentions, with the proper values in mind, so that our errors do not lead us too far from that core. It does not mean we will always get it right, but it changes, for the better, what is considered a success and a failure.

Before engaging in the mission of hot cognition, we ought to more explicitly ask why we are doing so. And, as we have seen, we should not let the answer beg the question, framing the argument around efficiency, low skill floors and ease of use as the default value. When considering our rationale for what values we choose to centre, we ought to keep in mind how the ramifications from design values we chose in the past are now urging us to change course. This is a deeply complex issue; one in which every section of this paper could be afforded numerous papers of its own. However, I hope this paper has served to advocate the importance of critically thinking through which cognitive and cultural values we wish to perpetuate through our UI design practices.

References

Agarwal, Vivek, and Prince Arora. “Conceptual Blending.” Conceptual Blending,

Department of Computer Science and Engineering at IIT Kanpur, www.cse.iitk.ac.in/users/se367/10/present ation_local/Conceptual%20BlendingSE367.html

Ariely, Dan. The (Honest) Truth about Dishonesty: How We Lie to Everyone - Especially Ourselves. HarperCollins, 2013.

Celentano, Augusto, and Emmanuel Dubois. “Evaluating Metaphor Reification in Tangible Interfaces.” Journal on

Multimodal User Interfaces, vol. 9, no. 3, 2015, pp. 231–252., doi:10.1007/s12193015-0198-z.

Chomsky, Noam. “Excerpts from Manufacturing Consent.” Excerpts from Manufacturing

Consent, Noam Chomsky Interviewed by Various Interviewers, chomsky.info/1992____02/.

“Choreography.” Material Design, Google, material.io/design/motion/choreography.ht ml.

“Sound Choreography.” Material Design, Google, https://material.io/design/sound/soundchoreography.html.

“Design.” Material Design, Google, https://material.io/design.

Fauconnier, Gilles, and Mark Turner. The Way We Think: Conceptual Blending and the Mind's Hidden Complexities. Basic Books, 2003.

Fauconnier, Gilles. “Conceptual Blending.” The

Department of Cognitive Science at UCSD, The Encyclopedia of the Social and Behavioral Sciences, www.cogsci.ucsd.edu/~faucon/BEIJING/b lending.pdf.

Google Design. “Making Material Design” YouTube, 28 May. 2015,

Hees, Kris Van, and Jan Engelen. “Equivalent

Representations of Multimodal User

Interfaces.” Universal Access in the

Information Society, vol. 12, no. 4, 2012, pp. 339–368., doi:10.1007/s10209-0120282-z.

Jetter, Hans-Christian, et al. “Blended Interaction: Understanding Natural Human–Computer

Interaction in Post-WIMP Interactive

Spaces.” Personal and Ubiquitous Computing, vol. 18, no. 5, 2013, pp. 1139– 1158., doi:10.1007/s00779-013-0725-4.

Lucero, Elisha. “Social Media and Digital Well-

Being (Informal Survey).” Google Forms, Google, 2020, docs.google.com/forms/d/1kUMxESvrW2 DpaAm7THtOqteDoqSkgg_NsLt_bdE3g NE/edit.

May, Kaitlyn E., and Anastasia D. Elder.

“Efficient, Helpful, or Distracting? A Literature Review of Media Multitasking in Relation to Academic Performance.” International Journal of Educational Technology in Higher Education, vol. 15, no. 1, 2018, doi:10.1186/s41239-0180096-z.

Norman, Don. The Design of Everyday Things: Revised and Expanded Edition. Basic Books, 2013.

PowerfulJRE. “Joe Rogan Experience #1368 - Edward Snowden” YouTube, 23 Oct. 2019, http://y2u.be/efs3QRr8LWw.

Roffarello, Alberto Monge, and Luigi De Russis. “The Race Towards Digital Wellbeing.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19, May 2019, doi:10.1145/3290605.3300616.

Secord, Adrian. “What ‘Beta’ Means for Material

Design Guidance.” Medium, Google Design, 14 Feb. 2020, medium.com/google-design/what-betameans-for-material-design-guidance10c5739f47a9.

Slingerland, Edward G. Trying Not to Try: Ancient China, Modern Science, and the Power of Spontaneity. Broadway Books, 2015.

Stanford Graduate School of Business. “Chamath Palihapitiya, Founder and CEO Social

Capital, on Money as an Instrument of Change” YouTube, 13 Nov. 2017,