What do Emojis Stand for? Notes for a Semiotic View on the Digital Relationship between Written and Spoken Communication Practices

By Andrea Ferretti | The integration of digital messaging applications into the most diverse practices of everyday life drives a rethinking of the relationship between bodies, orality, and writing within human communication. Communicative cues such as tone of voice, gestures, facial expressions, etc. find in emojis a way of translating themselves through a new kind of informal writing. Under this lens, some significant aspects of the merging of emojis and alphabetic writing can be investigated. This article reflects on the semiotic features of this translation and on the encyclopedic skills that the use and comprehension of emojis require. Special emphasis is placed on the linguistically mediated nature of emojis meaning. Despite their apparent grounding and iconicity, they can only be explained through the filter of linguistic, cultural, and socio-pragmatic coordinates. In short, the paper provides an understanding of emojis as a semiotic code, which contributes to characterizing digital communication. The semiotic code of emojis creates a space for collective reflection and creativity surrounding the expression of emotions, mutual social evaluation, social relationships, etc. Despite the fragmentation and the exploitative dynamics typical of digital environments, the use of this semiotic code will be depicted as an ongoing collective game, concerned with the replacement of individual bodies, even in the most mundane and informal communicative exchanges.

Emojis in the Context of Digital Mourning A Twitter-based Analysis of Communication about ›Angel Babies‹

By Christina Margrit Siever | Emojis have become an indispensable part of digital communication. There are undoubtedly cultural differences and individual preferences in their use, but the utilization of emojis naturally also depends on the communication situation and the topic. The present article is about emojis that are applied in the context of digital mourning on Twitter for so-called ›angel babies‹ (German: »Sternenkinder«), i.e., children who die (shortly) before, during, or after birth, and sometimes also later. The article analyzes the extent to which emojis are used in mourning processes in addition to verbally expressed grief for a deceased child and what function they have in this context. In particular, it will be analyzed what multimodal communication with emojis looks like, i.e., whether emojis are part of the message or rather have an illustrative character. In addition, the question is explored whether there are specific emojis for digital mourning communication and to what extent symbols that can be interpreted in religious terms can also be found (for example angels, praying hands, or candles). The facial signs used will also be examined; for example, the extent to which crying and sad emoticons are used to express grief and empathy. Furthermore, it will be discussed whether the heart as a symbol of love is also as present in grief communication on Twitter – as observed elsewhere in digital communication. The data basis for the analysis is a corpus of around 8,351 German-language tweets containing the sequence of characters »sternenkind« (angel baby).

Digital Stickers in Japanese LINE Communication

By Michaela Oberwinkler | This study examines the usage of digital stickers in Japanese LINE communication by analyzing 764 cases in authentic data. Digital stickers are often described as emojis, just larger in size. I argue, however, that stickers differ from emojis in that they are more expressive and fulfill more functions as a result of their ability to perform a distinct speech act on their own, such as intensifying a text message, softening a request, or serving as decoration to indicate one’s positive attitude. Additionally, the analysis of sticker usage among university students brought to light that the majority of stickers are sent independently, i.e., without an accompanying text message, thus revealing a way of communicating visually without words. Moreover, further examination of textual features and gender differences showed that female students used more animal stickers than male students, that men used fewer stickers with an integrated text when communicating with women than with other men, and that women used fewer criticizing stickers than men. Overall, the analysis of the stickers actually employed indicates that sticker usage combines many cultural features that are closely connected to the Japanese way of communicating.

»I’m so Pogged I’ve Got Pog-Juice Seeping out of My Eyes!« The Affective and Communal Language of Emoji on Twitch and Discord

By Marcel Lemmes | The present article explores the affective and communal dimensions of emoji as semiotic resources in digital communication. From a media studies perspective, the author analyzes the usage of emoji on the live streaming platform Twitch and the community chat platform Discord. In exploring the specific affordances these media platforms provide, a comprehensive framework for examining the usage of emoji within these and related contexts is established. The framework takes into consideration emoji’s pictorial qualities, their role as signs, and their intersemiotic embeddedness within digital platforms. Additionally, the article emphasizes the importance of cultural and contextual knowledge in understanding emoji usage. By integrating these elements, the article aims to shed light on the multifaceted nature of emoji and their significance in fostering affective and communal interactions within online communities. It also points toward a broader transdisciplinary perspective that could further enrich an understanding of the social/communal functions of emoji, such as research on internet memes and fandom.

Generative Imagery as Media Form and Research Field: Introduction to a New Paradigm

By Lukas R.A. Wilde | This introduction examines whether generative imagery represents a new paradigm for image production and an emerging research field. It explores a humanities approach to machine learning-based image generation and questions posed by media studies. Rather than focusing on radical shifts in media history, it emphasizes continuities and connections. It highlights the unique aspects of generative imagery compared to photography, painting, and earlier computer-generated imagery. The ’new paradigm‘ is based on emergent or stochastic features, the interplay between immediacy-oriented and hypermediacy-oriented forms of realism, and a novel text-image relationship grounded in human language. The survey then discusses the conditions under which generative imagery should be seen as a distinct media form rather than a new technology. It suggests viewing it as a mediation within evolving socio-technological configurations that reshape agency and subject positions in contemporary media cultures, particularly between human and non-human actors. To understand the cultural distinctness, the essay proposes examining the establishment, attribution, and negotiation of cultural ‘protocols‘ within existing and emerging media forms.

AI Image Media through the Lens of Art and Media History

By Lev Manovich | I’ve been using computer tools for art and design since 1984 and have already seen a few major visual media revolutions, including the development of desktop media software and photorealistic 3D computer graphics and animation, the rise of the web after, and later social media sites and advances in computational photography. The new AI ‘generative media’ revolution appears to be as significant as any of them. Indeed, it is possible that it is as significant as the invention of photography in the nineteenth century or the adoption of linear perspective in western art in the sixteenth. In what follows, I will discuss four aspects of AI image media that I believe are particularly significant or novel. To better understand these aspects, I situate this media within the context of visual media and human visual arts history, ranging from cave paintings to 3D computer graphics.

Generative AI and the Collective Imaginary The Technology-Guided Social Imagination in AI-Imagenesis

By Andreas Ervik | This paper explores generative AI images as new media, focusing on the questions of what these images depict, how image generation occurs, and how AI impacts the imaginary. It reflects on other forms of image production and identifies AI images as radically new, distinct from traditional methods as they lack light or brushstroke registration. However, they draw from the remains of other production forms, relying on connections between images and words as well as other forms of images as training data. AI image generators function as search engines, allowing users to enter prompts and explore the virtual potential of the latent space. Agency in AI image generation is shared between the program, platform holder, and users‘ prompts. Generative AI creates a social form of images, relying on human-created training datasets and shared on social networks. It gives rise to a ‚machinic imaginary,‘ characterized by techniques, styles, and fantasies from earlier media production. AI-generated images become part of the existing collective media imaginary. As discourse on AI images focuses on their future capabilities, the AI imaginary is filled with dreams of technological progress.

Dumb Meaning: Machine Learning and Artificial Semantics

By Hannes Bajohr | The ongoing debate around machine learning focuses on ‘big’ terms like intentionality, consciousness, and intelligence; the philosophical challenge lies in more nuanced concepts. This contribution explores a limited type of meaning called “dumb meaning.” Traditionally, computers were seen as handling only syntax, their semantic abilities being limited by the “symbol grounding problem.” Since they operate with mere symbols lacking any indexical relation to the world, their understanding is restricted to empty signifiers whose meaning is ‘parasitically’ dependent on a human interpreter. This was true for classic or symbolic AI. With subsymbolic AI and neural nets, however, an artificial semantics seems possible that operates below meaning proper. I explore this limited semantics brought about by the correlation of data types by looking at two examples: the implicit knowledge of large language models and the indexical meaning of multimodal AI such as DALL·E 2.