Multimedia, emotions, and learning experience
DOI:
https://doi.org/10.1344/der.2023.44.76-82Keywords:
emotional design, motivated cognition, mental models, multimedia learning, learner experienceAbstract
This paper considers emotions and learning through multimedia learning, mediated message processing, and affective computing. In multimedia learning, the emotional design includes all relevant features from the visual design, so emotional and motivational features are given through visual appeal. The Limited Capacity Model of Motivated Mediated Message Processing describes emotions as a product of motivated cognition. It explains how to create the message that will most likely fulfil its communication purpose. Through affective computing, adaptive and multimodal learning, aim is to reduce the gap between instructional message design, and the learner's cognitive and emotional mental model. The paper aims to outline emotions and learning considering multimedia learning, motivated mediated message processing, and affective computing as three perspectives in creating a learning experience.
References
Bradley, M. M., Codispoti, M., Cuthbert, B. N. & Lang, P. J. (2001). Emotion and motivation I: Defensive and appetitive reactions in picture processing. Emotion, 1(3), 276-298. http://dx.doi.org/10.1037/1528-3542.1.3.276
Bucciarelli M. & Cutica I. (2012). Mental Models in Improving Learning. In N. M. Seel (Eds.), Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_669
Cacioppo, J. T., Gardner, W. L. & Berntson, G. G. (1999). The affect system has parallel and integrative processing components: Form follows function. Journal of Personality and Social Psychology, 76, 839–855. https://doi.org/10.1037/0022-3514.76.5.839
Chung, S., Cheon, J. & Lee, K-W. (2015). Emotion and multimedia learning: an investigation of the effects of valence and arousal on different modalities in an instructional animation. Instructional Science, 43, 545–559. https://doi.org/10.1007/s11251-015-9352-y
Chung, S., & Cheon, J. (2020). Emotional design of multimedia learning using background images with motivational cues. Journal of Computer Assisted Learning, 36(6), 922-932. https://doi.org/10.1111/jcal.12450
Daily, S. B., James, M. T., Cherry, D., Porter, J. J. III, Darnell, S. S., Isaac, J., & Roy, T. (2017). Affective computing: Historical foundations, current applications, and future trends. In M. Jeon (Ed.), Emotions and affect in human factors and human-computer interaction (pp. 213–231). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-801851-4.00009-4)
Desmet, P. M. A. (2012) Faces of product pleasure: 25 positive emotions in human-product interactions. International Journal of Design, 6(2), 1-29.
Fisher, J. T., Keene, R. H. & Weber, R. (2018). The limited capacity model of motivated mediated message processing: taking stock of the past. Annals of the International Communication Association 42(4), 1-21. https://doi.org/10.1080/23808985.2018.1534552
Fox, J. R. (2004). A signal detection analysis of audio/video redundancy effects in television news video. Communication Research, 31(5), 524-536. https://doi.org/10.1177/0093650204267931
Goleman, D. (1995). Emotional Intelligence: Why It Can Matter More than IQ. Bantam Books, New York. NY.
Guo, J., Zou T., Peng D. (2018). Dynamic Influence of Emotional States on Novel Word Learning. Frontiers in Psychology, 9:537, 1-12. https://doi.org/10.3389/fpsyg.2018.00537
Hasan, M. A., Noor, N. F. M., Rahman, S. S. B. A., Rahman, M. M. (2020) "The Transition From Intelligent to Affective Tutoring System: A Review and Open Issues," in IEEE Access, 8, pp. 204612-204638. https://doi.org/10.1109/ACCESS.2020.3036990
Hassenzahl, M. & Tractinsky, N. (2006). User experience - a research agenda. Behaviour & Information Technology, 25(2), 91-97. https://doi.org/10.1080/01449290500330331
Hassenzahl, M., Burmester, M., & Beu, A. (2001). Engineering joy. IEEE Software, 1 and 2, 70–76. https://doi.org/10.1109/52.903170
Hede, A. J. (2002). An integrated model of multimedia effects on learning. Journal of Educational Multimedia and Hypermedia, 11, 177-191.
Heidig, S., Müller, J., Reichelt, M. (2015). Emotional design in multimedia learning: Differentiation on relevant design features and their effects on emotions and learning. Computers in Human Behavior, 44, 81-95. https://doi.org/10.1016/j.chb.2014.11.009
Izard C. E. (2007). Basic emotions, natural kinds, emotion schemas, and a new paradigm. Perspectives on Psychological Science, 2, 260–268.
Jih, H. J. & Reeves, T. C. (1992). Mental models: A research focus for interactive learning systems. Educational Technology Research and Development, 40(3), 39–53. https://doi.org/10.1007/BF02296841
Kapros, E., Koutsombogera, M. (2018). Introduction: User Experience in and for Learning. In E. Kapros & M. Koutsombogera (Eds.), Designing for the User Experience in Learning Systems. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-94794-5_1
Lang, A. (2000). The limited capacity model of mediated message processing. Journal of Communication, 50(1), 46–70. https://doi.org/10.1111/j.1460-2466.2000.tb02833.x
Lang, A. (2006a). Motivated cognition (LC4MP): The influence of appetitive and aversive activation on the processing of video games. In P. Messarsis & L. Humphries (Eds.), Digital media: Transformation in human communication (pp. 237–256). Peter Lang, New York, NY.
Lang, A. (2006b). Using the limited capacity model of motivated mediated message processing to design effective cancer communication messages. Journal of Communication, 56(1), S57–S80. https://doi.org/10.1111/j.1460-2466.2006.00283.x
Lang, A. (2017). Limited capacity model of motivated mediated message processing (LC4MP). In The International Encyclopedia of Media Effects (pp. 1–9). Hoboken, NJ, USA: John Wiley & Sons, Inc.
Lang, A., Sanders-Jackson, A., Wang, Z., Rubenking, B. (2013). Motivated message processing: How motivational activation influences resource allocation, encoding, and storage of TV messages. Motivation and Emotion, 37, 508–517. https://doi.org/10.1007/s11031-012-9329-y
Lang, A. (2014). Dynamic Human-Centered Communication Systems Theory. The Information Society: an international journal, 30(1), 60-70. https://doi.org/10.1080/01972243.2013.856364
Lee, S., & Lang, A. (2015). Redefining media content and structure in terms of available resources: Toward a dynamic human-centric theory of communication. Communication Research, 42(5), 599–625. https://doi.org/10.1177/0093650213488416
Leontidis, M., Halatsis, C. & Grigoriadou, M. (2011). Using an affective multimedia learning framework for distance learning to motivate the learner effectively. International Journal of Learning Technology, 6(3), 223-250. https://doi.org/10.1504/IJLT.2011.043578
Li, J., Luo, C., Zhang, Q. et al. Can emotional design really evoke emotion in multimedia learning?. International Journal of Educational Technology in Higher Education, 17, 24 (2020). https://doi.org/10.1186/s41239-020-00198-y
Magner, U., Schwonke, R., Aleven, V., Popescu, O., Renkl, A. (2014). Triggering situational interest by decorative illustrations both fosters and hinders learning in computer-based learning environments. Learning and Instruction, 29, 141-152. https://doi.org/10.1016/j.learninstruc.2012.07.002
Mayer, R. E. (2001). Multimedia Learning. Cambridge, UK: Cambridge University Press.
Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.
Mayer, R. E. (2011). Applying the science of learning. Upper Saddle River, NJ: Pearson.
Mayer, R. E. (2014a). Incorporating motivation into multimedia learning. Learning and Instruction, 29, 171-173. https://doi.org/10.1016/j.learninstruc.2013.04.003
Mayer, R. E. (2014b). Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In R. E. Mayer (Ed.),The Cambridge handbook of multimedia learning (pp. 345–368). Cambridge University Press. https://doi.org/10.1017/CBO9781139547369.017
Mayer, R. E. (2019). Taking a new look at seductive details. Applied Cognitive Psychology, 33(1), 139–141. https://doi.org/10.1002/acp.3503
Mayer, R. E. (2020). Multimedia learning (3rd ed.). New York: Cambridge University Press.
Mayer, R. E. & Estrella, G. (2014). Benefits of emotional design in multimedia instruction. Learning and Instruction, 33, 12–18. https://doi.org/10.1016/j.learninstruc.2014.02.004
Mehrabian, A. & Russell, J. A. (1974). An approach to environmental psychology. Cambridge, Massachusetts: MIT Press.
Mikheeva, M., Schneider, S., Beege M., & Rey, G.D. (2021). The influence of affective decorative pictures on learning statistics online. Human Behavior and Emerging Technologies, 1–12. https://doi.org/10.1002/hbe2.250
Moreno, R. (2006). Does the modality principle hold for different media? A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning, 22, 149-158. https://doi.org/10.1111/j.1365-2729.2006.00170.x
Moreno, R. & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309-326. https://doi.org/10.1007/s10648-007-9047-2
Münchow, H., Mengelkamp, C., Bannert, M. (2017). The Better You Feel the Better You Learn: Do Warm Colours and Rounded Shapes Enhance Learning Outcome in Multimedia Learning?. Education Research International, 17, Article ID 2148139, 15 pages, https://doi.org/10.1155/2017/2148139.
Newton, P.M., Najabat-Lattif, H. F., Santiago, G. & Salvi, A. (2021). The Learning Styles Neuromyth Is Still Thriving in Medical Education. Frontiers in Human Neuroscience, 15:708540. https://doi.org/10.3389/fnhum.2021.708540
Norman, D. A. (2004). Emotional design. New York: Basic Books.
Norman, D. A. (1988). The psychology of everyday things. Basic Books.
Norris, C. J., Gollan, J., Berntson, G. G. & Cacioppo, J. T. (2010). The current status of research on the structure of evaluative space. Biological Psychology, 84, 422–436. https://doi.org/10.1016/j.biopsycho.2010.03.011
Park, B., Moreno, R., Seufert, T., & Brünken, R. (2011). Does cognitive load moderate the seductive details effect? A multimedia study. Computers in Human. Behavior, 27(1), 5-10. https://doi.org/10.1016/j.chb.2010.05.006
Park B., Flowerday T., Brünken R. (2015). Cognitive and affective effects of seductive details in multimedia learning. Computers in Human Behavior, 267–278. https://doi.org/10.1016/j.chb.2014.10.061
Park, B., Plass, J., & Brünken, R. (2014). Cognitive and affective processes in multimedia learning. Learning and Instruction, 29, 125-127. https://doi.org/10.1016/j.learninstruc.2013.05.005
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(3), 315-341. https://doi.org/10.1007/s10648-006-9029-9
Petrovica, S., Anohina-Naumeca, A., Ekenel, H.K. (2017). Emotion Recognition in Affective Tutoring Systems: Collection of Ground-truth Data. Procedia Computer Science, 104, 437-444. https://doi.org/10.1016/j.procs.2017.01.157
Picard, R. W. (2003). Affective computing: challenges. International Journal of Human-Computer Studies, 59(1-2), 55–64. https://doi.org/10.1016/S1071-5819(03)00052-1
Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Y. Tettegah & M. Gartmeier (Eds.), Emotions, technology, design, and learning (p. 131–161). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-801856-9.00007-4
Plass, J. L., Heidig, S., Hayward, E. O., Homer, B. D., & Um, E. (2014). Emotional design in multimedia learning: Effects of shape and color on affect and learning. Learning and Instruction, 29, 128-140. https://doi.org/10.1016/j.learninstruc.2013.02.006
Plass, J., & Hovey, C. (2021). The Emotional Design Principle in Multimedia Learning. In R. Mayer & L. Fiorella (Eds.), The Cambridge Handbook of Multimedia Learning (Cambridge Handbooks in Psychology, pp. 324-336). Cambridge: Cambridge University Press. doi:10.1017/9781108894333.034
Rodrigues, R. & Silva, M. (2022). Emotional Design in Multimedia Learning: Systematic Review. In N. Martins & D. Brandão (Eds.), Advances in Design and Digital Communication II. DIGICOM 2021. Springer Series in Design and Innovation, vol. 19. Springer, Cham. https://doi.org/10.1007/978-3-030-89735-2_19
Rubin, D. C. & Talarico J. M. (2009). A comparison of dimensional models of emotion: evidence from emotions, prototypical events, autobiographical memories, and words. Memory, 17(8):802-808. https://doi.org/10.1080/09658210903130764
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714
Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. https://doi.org/10.1037/0033-295X.110.1.145
Scharinger, C. (2022). Effects of decorative pictures on mental processing demands and learning: An EEG and eye-tracking study, Acta Psychologica, 231, 103798. https://doi.org/10.1016/j.actpsy.2022.103798
Scheiter, K., Fillisch, B., Krebs, M. C., Leber, J., Ploetzner, R., Renkl, A., Schmidt, H., Schüler, A. & Zimmermann, G. (2017). How to design adaptive information environments to support self-regulated learning with multimedia. In J. Buder & F. W. Hesse (Eds.), Informational environments. Effects of use, effective designs (pp. 203-223). Springer, Cham. https://doi.org/10.1007/978-3-319-49077-9_1
Schneider, S., Nebel, S. & Rey, G. D. (2016). Decorative pictures and emotional design in multimedia learning. Learning and Instruction, 44, 65-73.
Schneider, S., Nebel, S., Pradel, S. & Rey, G. D. (2015). Introducing the familiarity mechanism: A unified explanatory approach for the personalization effect and the examination of youth slang in multimedia learning. Computers in Human Behavior, 43, 129-138. https://doi.org/10.1016/j.chb.2014.10.052
Schneider, S., Nebel, S., Beege, M. & Rey, G. D. (2018). Anthropomorphism in decorative pictures: Benefit or harm for learning? Journal of Educational Psychology, 110(2), 218–232. https://doi.org/10.1037/edu0000207
Schneider, S., Wirzberger, M., & Rey, G. D. (2019). The moderating role of arousal on the seductive detail effect in a multimedia learning setting. Applied Cognitive Psychology, 33, 71–84. https://doi.org/10.1002/acp.3473
Scheiter K., Schüler A. & Eitel A. (2017). Learning from Multimedia: Cognitive Processes and Instructional Support. In S. Schwan & U. Cress (Eds.), The Psychology of Digital Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-49077-9_1
Silvennoinen, J.M., Jokinen, Jussi P. P. (2016). Appraisals of Salient Visual Elements in Web Page Design. Advances in Human-Computer Interaction, vol. 2016, Article ID 3676704, 14 pages, 2016. https://doi.org/10.1155/2016/3676704
Stark, L., Brünken, R., & Park, B. (2018). Emotional text design in multimedia learning: A mixed-methods study using eye tracking. Computers & Education, 120, 185–196. https://doi.org/10.1016/j.compedu.2018.02.003
Tiberti, S., Chirico A., La Rocca G. & Riva G. (2017). Developing Emotional Design: Emotions as Cognitive Processes and their Role in the Design of Interactive Technologies. Frontiers in Psychology, 8, 1773. https://doi.org/10.3389/fpsyg.2017.01773
Tislar, K. L. & Steelman, K. S. (2021). Inconsistent seduction: Addressing confounds and methodological issues in the study of the seductive detail effect. Brain and Behavior. 11(9):e2322, 1-15. https://doi.org/10.1002/brb3.2322
Um, E., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in multimedia learning. Journal of Educational Psychology, 104(2), 485-498. https://doi.org/10.1037/a0026609
Wang, F., Li, W., Mayer, R. E., & Liu, H. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110(2), 250–268. https://doi.org/10.1037/edu0000221
Worsley M. (2018). Multimodal learning analytics’ past, present, and, potential futures. CEUR Workshop Proceedings, 2163.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Tomislava Lauc, Sanja Kisicek, Danijela Unic
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The authors who publish in this journal agree to the following terms:
- Authors retain copyright and grant the journal the right of first publication.
- The texts published in Digital Education Review, DER, are under a license Attribution-Noncommercial-No Derivative Works 4,0 Spain, of Creative Commons. All the conditions of use in: Creative Commons,
- In order to mention the works, you must give credit to the authors and to this Journal.
- Digital Education Review, DER, does not accept any responsibility for the points of view and statements made by the authors in their work.
