Multimedia, emotions, and learning experience




emotional design, motivated cognition, mental models, multimedia learning, learner experience


This paper considers emotions and learning through multimedia learning, mediated message processing, and affective computing. In multimedia learning, the emotional design includes all relevant features from the visual design, so emotional and motivational features are given through visual appeal. The Limited Capacity Model of Motivated Mediated Message Processing describes emotions as a product of motivated cognition. It explains how to create the message that will most likely fulfil its communication purpose. Through affective computing, adaptive and multimodal learning, aim is to reduce the gap between instructional message design, and the learner's cognitive and emotional mental model. The paper aims to outline emotions and learning considering multimedia learning, motivated mediated message processing, and affective computing as three perspectives in creating a learning experience.


Bradley, M. M., Codispoti, M., Cuthbert, B. N. & Lang, P. J. (2001). Emotion and motivation I: Defensive and appetitive reactions in picture processing. Emotion, 1(3), 276-298.

Bucciarelli M. & Cutica I. (2012). Mental Models in Improving Learning. In N. M. Seel (Eds.), Encyclopedia of the Sciences of Learning. Springer, Boston, MA.

Cacioppo, J. T., Gardner, W. L. & Berntson, G. G. (1999). The affect system has parallel and integrative processing components: Form follows function. Journal of Personality and Social Psychology, 76, 839–855.

Chung, S., Cheon, J. & Lee, K-W. (2015). Emotion and multimedia learning: an investigation of the effects of valence and arousal on different modalities in an instructional animation. Instructional Science, 43, 545–559.

Chung, S., & Cheon, J. (2020). Emotional design of multimedia learning using background images with motivational cues. Journal of Computer Assisted Learning, 36(6), 922-932.

Daily, S. B., James, M. T., Cherry, D., Porter, J. J. III, Darnell, S. S., Isaac, J., & Roy, T. (2017). Affective computing: Historical foundations, current applications, and future trends. In M. Jeon (Ed.), Emotions and affect in human factors and human-computer interaction (pp. 213–231). Elsevier Academic Press.

Desmet, P. M. A. (2012) Faces of product pleasure: 25 positive emotions in human-product interactions. International Journal of Design, 6(2), 1-29.

Fisher, J. T., Keene, R. H. & Weber, R. (2018). The limited capacity model of motivated mediated message processing: taking stock of the past. Annals of the International Communication Association 42(4), 1-21.

Fox, J. R. (2004). A signal detection analysis of audio/video redundancy effects in television news video. Communication Research, 31(5), 524-536.

Goleman, D. (1995). Emotional Intelligence: Why It Can Matter More than IQ. Bantam Books, New York. NY.

Guo, J., Zou T., Peng D. (2018). Dynamic Influence of Emotional States on Novel Word Learning. Frontiers in Psychology, 9:537, 1-12.

Hasan, M. A., Noor, N. F. M., Rahman, S. S. B. A., Rahman, M. M. (2020) "The Transition From Intelligent to Affective Tutoring System: A Review and Open Issues," in IEEE Access, 8, pp. 204612-204638.

Hassenzahl, M. & Tractinsky, N. (2006). User experience - a research agenda. Behaviour & Information Technology, 25(2), 91-97.

Hassenzahl, M., Burmester, M., & Beu, A. (2001). Engineering joy. IEEE Software, 1 and 2, 70–76.

Hede, A. J. (2002). An integrated model of multimedia effects on learning. Journal of Educational Multimedia and Hypermedia, 11, 177-191.

Heidig, S., Müller, J., Reichelt, M. (2015). Emotional design in multimedia learning: Differentiation on relevant design features and their effects on emotions and learning. Computers in Human Behavior, 44, 81-95.

Izard C. E. (2007). Basic emotions, natural kinds, emotion schemas, and a new paradigm. Perspectives on Psychological Science, 2, 260–268.

Jih, H. J. & Reeves, T. C. (1992). Mental models: A research focus for interactive learning systems. Educational Technology Research and Development, 40(3), 39–53.

Kapros, E., Koutsombogera, M. (2018). Introduction: User Experience in and for Learning. In E. Kapros & M. Koutsombogera (Eds.), Designing for the User Experience in Learning Systems. Human–Computer Interaction Series. Springer, Cham.

Lang, A. (2000). The limited capacity model of mediated message processing. Journal of Communication, 50(1), 46–70.

Lang, A. (2006a). Motivated cognition (LC4MP): The influence of appetitive and aversive activation on the processing of video games. In P. Messarsis & L. Humphries (Eds.), Digital media: Transformation in human communication (pp. 237–256). Peter Lang, New York, NY.

Lang, A. (2006b). Using the limited capacity model of motivated mediated message processing to design effective cancer communication messages. Journal of Communication, 56(1), S57–S80.

Lang, A. (2017). Limited capacity model of motivated mediated message processing (LC4MP). In The International Encyclopedia of Media Effects (pp. 1–9). Hoboken, NJ, USA: John Wiley & Sons, Inc.

Lang, A., Sanders-Jackson, A., Wang, Z., Rubenking, B. (2013). Motivated message processing: How motivational activation influences resource allocation, encoding, and storage of TV messages. Motivation and Emotion, 37, 508–517.

Lang, A. (2014). Dynamic Human-Centered Communication Systems Theory. The Information Society: an international journal, 30(1), 60-70.

Lee, S., & Lang, A. (2015). Redefining media content and structure in terms of available resources: Toward a dynamic human-centric theory of communication. Communication Research, 42(5), 599–625.

Leontidis, M., Halatsis, C. & Grigoriadou, M. (2011). Using an affective multimedia learning framework for distance learning to motivate the learner effectively. International Journal of Learning Technology, 6(3), 223-250.

Li, J., Luo, C., Zhang, Q. et al. Can emotional design really evoke emotion in multimedia learning?. International Journal of Educational Technology in Higher Education, 17, 24 (2020).

Magner, U., Schwonke, R., Aleven, V., Popescu, O., Renkl, A. (2014). Triggering situational interest by decorative illustrations both fosters and hinders learning in computer-based learning environments. Learning and Instruction, 29, 141-152.

Mayer, R. E. (2001). Multimedia Learning. Cambridge, UK: Cambridge University Press.

Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.

Mayer, R. E. (2011). Applying the science of learning. Upper Saddle River, NJ: Pearson.

Mayer, R. E. (2014a). Incorporating motivation into multimedia learning. Learning and Instruction, 29, 171-173.

Mayer, R. E. (2014b). Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In R. E. Mayer (Ed.),The Cambridge handbook of multimedia learning (pp. 345–368). Cambridge University Press.

Mayer, R. E. (2019). Taking a new look at seductive details. Applied Cognitive Psychology, 33(1), 139–141.

Mayer, R. E. (2020). Multimedia learning (3rd ed.). New York: Cambridge University Press.

Mayer, R. E. & Estrella, G. (2014). Benefits of emotional design in multimedia instruction. Learning and Instruction, 33, 12–18.

Mehrabian, A. & Russell, J. A. (1974). An approach to environmental psychology. Cambridge, Massachusetts: MIT Press.

Mikheeva, M., Schneider, S., Beege M., & Rey, G.D. (2021). The influence of affective decorative pictures on learning statistics online. Human Behavior and Emerging Technologies, 1–12.

Moreno, R. (2006). Does the modality principle hold for different media? A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning, 22, 149-158.

Moreno, R. & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309-326.

Münchow, H., Mengelkamp, C., Bannert, M. (2017). The Better You Feel the Better You Learn: Do Warm Colours and Rounded Shapes Enhance Learning Outcome in Multimedia Learning?. Education Research International, 17, Article ID 2148139, 15 pages,

Newton, P.M., Najabat-Lattif, H. F., Santiago, G. & Salvi, A. (2021). The Learning Styles Neuromyth Is Still Thriving in Medical Education. Frontiers in Human Neuroscience, 15:708540.

Norman, D. A. (2004). Emotional design. New York: Basic Books.

Norman, D. A. (1988). The psychology of everyday things. Basic Books.

Norris, C. J., Gollan, J., Berntson, G. G. & Cacioppo, J. T. (2010). The current status of research on the structure of evaluative space. Biological Psychology, 84, 422–436.

Park, B., Moreno, R., Seufert, T., & Brünken, R. (2011). Does cognitive load moderate the seductive details effect? A multimedia study. Computers in Human. Behavior, 27(1), 5-10.

Park B., Flowerday T., Brünken R. (2015). Cognitive and affective effects of seductive details in multimedia learning. Computers in Human Behavior, 267–278.

Park, B., Plass, J., & Brünken, R. (2014). Cognitive and affective processes in multimedia learning. Learning and Instruction, 29, 125-127.

Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(3), 315-341.

Petrovica, S., Anohina-Naumeca, A., Ekenel, H.K. (2017). Emotion Recognition in Affective Tutoring Systems: Collection of Ground-truth Data. Procedia Computer Science, 104, 437-444.

Picard, R. W. (2003). Affective computing: challenges. International Journal of Human-Computer Studies, 59(1-2), 55–64.

Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Y. Tettegah & M. Gartmeier (Eds.), Emotions, technology, design, and learning (p. 131–161). Elsevier Academic Press.

Plass, J. L., Heidig, S., Hayward, E. O., Homer, B. D., & Um, E. (2014). Emotional design in multimedia learning: Effects of shape and color on affect and learning. Learning and Instruction, 29, 128-140.

Plass, J., & Hovey, C. (2021). The Emotional Design Principle in Multimedia Learning. In R. Mayer & L. Fiorella (Eds.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 324-336). Cambridge: Cambridge University Press. doi:10.1017/9781108894333.034

Rodrigues, R. & Silva, M. (2022). Emotional Design in Multimedia Learning: Systematic Review. In N. Martins & D. Brandão (Eds.), Advances in Design and Digital Communication II. DIGICOM 2021. Springer Series in Design and Innovation, vol. 19. Springer, Cham.

Rubin, D. C. & Talarico J. M. (2009). A comparison of dimensional models of emotion: evidence from emotions, prototypical events, autobiographical memories, and words. Memory, 17(8):802-808.

Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178.

Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172.

Scharinger, C. (2022). Effects of decorative pictures on mental processing demands and learning: An EEG and eye-tracking study, Acta Psychologica, 231, 103798.

Scheiter, K., Fillisch, B., Krebs, M. C., Leber, J., Ploetzner, R., Renkl, A., Schmidt, H., Schüler, A. & Zimmermann, G. (2017). How to design adaptive information environments to support self-regulated learning with multimedia. In J. Buder & F. W. Hesse (Eds.), Informational environments. Effects of use, effective designs (pp. 203-223). Springer, Cham.

Schneider, S., Nebel, S. & Rey, G. D. (2016). Decorative pictures and emotional design in multimedia learning. Learning and Instruction, 44, 65-73.

Schneider, S., Nebel, S., Pradel, S. & Rey, G. D. (2015). Introducing the familiarity mechanism: A unified explanatory approach for the personalization effect and the examination of youth slang in multimedia learning. Computers in Human Behavior, 43, 129-138.

Schneider, S., Nebel, S., Beege, M. & Rey, G. D. (2018). Anthropomorphism in decorative pictures: Benefit or harm for learning? Journal of Educational Psychology, 110(2), 218–232.

Schneider, S., Wirzberger, M., & Rey, G. D. (2019). The moderating role of arousal on the seductive detail effect in a multimedia learning setting. Applied Cognitive Psychology, 33, 71–84.

Scheiter K., Schüler A. & Eitel A. (2017). Learning from Multimedia: Cognitive Processes and Instructional Support. In S. Schwan & U. Cress (Eds.), The Psychology of Digital Learning. Springer, Cham.

Silvennoinen, J.M., Jokinen, Jussi P. P. (2016). Appraisals of Salient Visual Elements in Web Page Design. Advances in Human-Computer Interaction, vol. 2016, Article ID 3676704, 14 pages, 2016.

Stark, L., Brünken, R., & Park, B. (2018). Emotional text design in multimedia learning: A mixed-methods study using eye tracking. Computers & Education, 120, 185–196.

Tiberti, S., Chirico A., La Rocca G. & Riva G. (2017). Developing Emotional Design: Emotions as Cognitive Processes and their Role in the Design of Interactive Technologies. Frontiers in Psychology, 8, 1773.

Tislar, K. L. & Steelman, K. S. (2021). Inconsistent seduction: Addressing confounds and methodological issues in the study of the seductive detail effect. Brain and Behavior. 11(9):e2322, 1-15.

Um, E., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in multimedia learning. Journal of Educational Psychology, 104(2), 485-498.

Wang, F., Li, W., Mayer, R. E., & Liu, H. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110(2), 250–268.

Worsley M. (2018). Multimodal learning analytics’ past, present, and, potential futures. CEUR Workshop Proceedings, 2163.






Peer Review Articles