Emoji research and creation
Emoji have become a fundamental way that people communicate digitally, particularly along with their written language. They also often draw on the
visual conventions already established in comics and manga around the world. We have both carried out psycholinguistic research on emoji and how they function, and have used our research to contribute to the creation of new emoji.
Emoji research
We have carried out research on the psychology and linguistics of emoji, particularly in ongoing collaboration with
Ben Weissman (Rensselaer Polytechnic Institute). This work has focused on how emoji convey meaning, and how they interact in multimodal interactions with written language. Our work has shown that people incorporate acquire the meanings of emoji with relative strengths. If emoji substitute for words in sentences, they are readily integrated into the grammar. However, emoji do not seem to be combined into sequences using a grammar of their own, and the way emoji combine into linear sequences is somewhat unnatural for graphic communication. Here’s some links to our research:
- Weissman, Ben, Jan Engelen, Elise Baas and Neil Cohn (2023), The Lexicon of Emoji? Conventionality Modulates Processing of Emoji. Cognitive Science, 47: e13275. (Journal)
- Cohn, Neil, Jan Engelen, and Joost Schilperoord. 2019. The grammar of emoji? Constraints on communicative pictorial sequencing. Cognitive Research: Principles and Implications. 4(1):33. (Online version & PDF)
- Cohn, Neil, Tim Roijackers, Robin Schaap, and Jan Engelen. 2018. Are emoji a poor substitute for words? Sentence processing with emoji substitutions. In T.T. Rogers, M. Rau, X. Zhu, & C. W. Kalish (Eds.), Proceedings of the 40th Annual Conference of the Cognitive Science Society. (pp. 1524-1529) Austin, TX: Cognitive Science Society. (PDF, Poster version)
Additional work is connected to emoji, such as the “upfixes” that float above characters heads like hearts or lightbulbs. This work has focused on the constraints on upfix structure, or how combining elements creates non-literal meanings:
- Cohn, Neil, Beena Murthy, and Tom Foulsham. 2016. Meaning above the head: Combinatorial constraints on the visual vocabulary of comics. Journal of Cognitive Psychology. 28(5): 559-574. (PDF)
