ISSN: 2013-2255

Analysing the Deep Learning Strategies Questionnaire: a case study with higher education students in Catalonia

Ludmila Martins a https://orcid.org/0000-0002-9527-4295

Antoni Ruiz-Bueno b https://orcid.org/0000-0001-9651-3633

Universitat de Barcelona, Spain.

a Departament de Didàctica i Organització Educativa, Facultat d’Educació. Psg. Vall d’Hebron, 171 – Edifici Llevant, 2a planta. 08035 Barcelona, Spain. ludmila.martins@ub.edu

b Departament de Mètodes d’Investigació i Diagnòstic en Educació, Facultat d’Educació.

Research article. Received: 23/05/2024. Revised: 25/09/2024. Accepted: 29/10/2024. Published: 02/01/2025.

Abstract

INTRODUCTION. Finding an accurate instrument to measure self-regulated learning in higher education is a complex task. In 2021, Panadero and colleagues developed the Deep Learning Strategies Questionnaire (DLS-Q) to assess the learning strategies used by students. In this study, we aimed to assess the evidence of validity of the DLS-Q in the Catalan language. Additionally, we explored whether students’ beliefs about the subject and their previous experience with peer-assessment processes are related to their self-regulated learning skills.

METHOD. A total of 475 higher education students from various year groups and faculties at a university in Catalonia participated in this study.

RESULTS. We found a significant positive correlation between all dimensions of deep-learning strategies and the different expectations and beliefs expressed by students at the beginning of the academic year.

DISCUSSION. Since we decided to eliminate three items from the original version due to their low score, we propose a four-dimensional structure for a 27-item version of the questionnaire. The low-scoring items are consistent with the results of previous research in different educational contexts. We discuss possible explanations related to teaching practices and lecturers’ roles as feedback providers.

Keywords

self-regulated learning, learning strategy, higher education, questionnaire, validation.

Recommended reference

Martins, L., & Ruiz-Bueno, A. (2025). Analysing the Deep Learning Strategies Questionnaire: a case study with higher education students in Catalonia. REIRE Revista d’Innovació i Recerca en Educació, 18(1), 1–19. https://doi.org/10.1344/reire.46895

© 2025 The authors. This is an open access article distributed under the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. To view a copy of this license, visit

https://creativecommons.org/licenses/by/4.0/

Títol (català)

Anàlisi del Qüestionari d’Estratègies d’Aprenentatge Profund: un estudi de cas amb estudiants d’educació superior a Catalunya

Resum

INTRODUCCIÓ. Trobar un instrument precís per mesurar l’aprenentatge autoregulat en l’educació superior és una tasca complexa. El 2021, Panadero i altres col·laboradors van desenvolupar el Deep Learning Strategies Questionnaire (DLS-Q) per avaluar les estratègies d’aprenentatge utilitzades pels estudiants. En aquest estudi pretenem comprovar l’evidència de validesa de l’instrument DLS-Q en llengua catalana. A més, explorem si les creences dels estudiants sobre el tema i l’experiència prèvia en processos d’avaluació entre iguals tenen relació amb les seves habilitats d’aprenentatge d’autoregulació.

MÈTODE. Hi van participar 475 estudiants d’educació superior de diversos cursos i facultats d’una universitat catalana.

RESULTATS. Vam trobar que totes les dimensions de les estratègies d’aprenentatge profund mostren una correlació significativa i positiva amb les diferents expectatives o creences que els estudiants expressen a l’inici del curs.

DISCUSSIÓ. Atès que vam decidir eliminar tres ítems de la versió original a causa de la seva baixa puntuació, proposem una estructura de quatre dimensions per a la versió del qüestionari de 27 ítems. Els ítems amb baixa puntuació són concordants amb els resultats d’investigacions prèvies en diferents contextos educatius. Parlem de possibles explicacions vinculades a les retroaccions (feedback) que proveeixen les pràctiques docents i el paper del professorat.

Paraules clau

aprenentatge autoregulat, estratègia d’aprenentatge, educació superior, qüestionari, validació

Título (castellano)

Análisis del Cuestionario de Estrategias de Aprendizaje Profundo: un estudio de caso con estudiantes de educación superior en Cataluña

Resumen

INTRODUCCIÓN. Encontrar un instrumento preciso para medir el aprendizaje autorregulado en la educación superior es una tarea compleja. En 2021, Panadero y colaboradores desarrollaron el Deep Learning Strategies Questionnaire (DLS-Q) para evaluar las estrategias de aprendizaje utilizadas por los estudiantes. En este estudio pretendemos comprobar la evidencia de validez del instrumento DLS-Q en lengua catalana. Además, exploramos si las creencias de los estudiantes sobre la asignatura y la experiencia previa en procesos de evaluación entre iguales tienen relación con sus habilidades de autorregulación del aprendizaje.

MÉTODO. Participaron 475 estudiantes de educación superior de varios cursos de diversas facultades de una universidad catalana.

RESULTADOS. Encontramos que todas las dimensiones de las estrategias de aprendizaje profundo muestran una correlación positiva significativa con las diferentes expectativas o creencias que los estudiantes expresan al inicio del curso.

DISCUSIÓN. Como decidimos eliminar tres ítems de la versión original debido a su baja puntuación, proporcionamos una estructura de cuatro dimensiones para la versión del cuestionario de 27 ítems. Los ítems con baja puntuación concuerdan con los resultados de investigaciones anteriores en diferentes contextos educativos. Se discuten posibles explicaciones relacionadas con las prácticas docentes y el papel del profesorado como proveedor de feedback.

Palabras clave

aprendizaje autorregulado, estrategia de aprendizaje, educación superior, cuestionario, validación

1 Introduction

1.1 Self-regulated learning

The Council of the European Union’s 2018 recommendation on key competences for lifelong learning (2018) highlights learning to learn as one of the key competences for supporting training, learning and participation in society throughout life. According to this recommendation, “Personal, social and learning to learn competence is the ability to reflect upon oneself, effectively manage time and information, work with others in a constructive way, remain resilient and manage one’s own learning and career” (ST/9009/2018/INIT, p. 10). This requires students to be able to self-regulate their learning and, to that end, education must contribute to their development (Zimmerman, 2002).

Various models have been proposed to explain the development of self-regulated learning. While these models offer different approaches, they all concur that self-regulated learning is a cyclical process (Panadero, 2017). In accordance with the cyclical model proposed by Zimmerman (2000), which consists of three phases (forethought, performance and self-reflection), self-regulation can be defined as self-generated thoughts, feelings and goal-oriented behaviours (Zimmerman, 2001).

According to Winne (2018), self-regulated learners are students who actively strive to improve their own learning process by reflecting on its potential and evaluating their own performance. This definition implicitly draws on information processing theory, as the model proposed by this author describes the cognitive process undertaken by students while performing a task (Panadero, 2017). In Winne’s words, “At each phase of self-regulated learning, learners identify, process, and act on information” (2018, p. 12).

Closely related to this processing and acting on information, García-Pérez et al. (2020) suggested that any regulatory action students take to complete a task or comprehend content can be considered a learning strategy. In addition, the same authors argued that it is crucial to consider the context when exploring the complex relationship between learning strategies and self-regulation skills (García-Pérez et al., 2020).

In line with Pintrich (2000), self-regulation strategies are related to motivational aspects such as task value. Task value (Eccles, 2005) can be understood as the importance or perceived value placed on completing a task. Previous research has highlighted the positive relationship between this variable and self-regulated learning. For example, Li and Zheng (2018) observed a correlation between self-regulated learning and different types of task value, in particular causal relationships with utility and intrinsic value. In a similar vein, other studies have revealed the predictive nature of task value on self-regulation (Lawanto et al., 2014). Moreover, Ghasemi and Dowlatabadi (2018) pointed out that task value is a good predictor of the learning and self-regulation strategies used by students, while Lee et al. (2020) revealed the importance of task value in self-regulatory processes, thereby highlighting the importance of providing support for such purposes.

Regarding educational interventions designed to enhance self-regulated learning, several authors (e.g. Butler & Winne, 1995; Nicol & Macfarlane-Dick, 2006; Panadero et al., 2017) have highlighted the role of feedback practices to foster students’ self-regulation. On this topic, Hattie and Timperley (2007) warned that the effectiveness of feedback for promoting learning could depend on the type of feedback. Similarly, Lipnevich and Smith (2009) reported different effects depending on whether feedback is descriptive or evaluative, while Theobald and Bellhäuser (2022) reported that a feedback intervention improved students’ self-regulated learning, but that this varied depending on the content of the feedback.

1.2 Peer assessment for self-regulated learning

It is well known that assessment influences the learning process. In this regard, peer assessment is considered crucial to assessment for learning (Stančić, 2021). Specifically, peer-assessment practices provide opportunities to develop strategies related to self-regulated learning (Clayton Bernard & Kermarrec, 2022).

Peer assessment can be understood as a situation in which the level of quality or value of a task, product or performance is considered and specified by an equal (Tooping, 2021). Thus, feedback is closely related to peer assessment. As mentioned by Ibarra-Sáiz et al. (2020), “In peer assessment, the role of feedback is crucial” (p. 140).

As mentioned above, several authors have already suggested that peer-feedback activities provide an opportunity to enhance students’ self-regulated learning (Prompan & Piamsai, 2024) and that peer assessment creates the ideal environment for fostering metacognition and reflecting on one’s own learning process (Clayton Bernard & Kermarrec, 2022). To have an effect on learning, however, feedback must be translated into action (Wu & Schunn, 2020). In this regard, several factors that influence and/or mediate learning and peer assessment through feedback have been identified. One fundamental factor is the quality of feedback (Zhang & Schunn, 2023). The model proposed by Winstone and Carless (2019), which outlines seven principles of good feedback, presents the criteria that feedback must meet to support the development of students’ self-regulation. Indeed, “in the context of peer assessment, it is essential for students to understand what quality feedback involves” (Ibarra-Sáiz et al., 2020, p. 140).

Peer assessment can be an effective method for developing self-regulated learning (Schünemann et al., 2017). The processes of internalizing criteria, applying them to a peer’s work and interpreting feedback from a peer are valuable strategies for enhancing self-regulated learning (Muñoz et al., 2020). The bidirectional relationship between feedback and self-regulated learning has been widely explored (Panadero et al., 2018). Zhang and Schunn (2023) demonstrated the relationship between peer review and self-regulation as a cyclical process, given that peer review requires students to put into practice feedback loops that are essential for self-regulated learning processes (see p. 4).

1.3 Measuring self-regulated learning

To evaluate the need for and success of interventions, reliable instruments for measuring self-regulation are essential (Alonso-Tapia et al., 2014; Panadero et al., 2021). However, as Winne and Perry (2000) warned, “[…] it is plain that many facets of self-regulated learning (SRL) are not readily observable. Therefore, one challenge in studying SRL is to find ways to document its components” (p. 534). Subsequent studies have addressed the challenges of measuring self-regulation for several reasons (Alonso-Tapia et al., 2014; Boekaerts & Corno, 2005; García-Pérez et al., 2020; Panadero et al., 2016; Winne & Perry, 2000).

In a review of self-regulated learning models, Panadero (2017) summarized the instruments and measurement methods used in six different models. In a more recent publication, Gajda et al. (2022) adapted and validated a self-regulation scale for the Polish context. These latter authors also presented self-reporting instruments to measure self-regulation in an adolescent population, classified by domain (see pp. 3-5).

In light of the complexity of identifying an accurate tool for measuring self-regulated learning in higher education, Panadero et al. (2021) developed a questionnaire focused on the learning strategies used by students (Deep Learning Strategies Questionnaire – DLS-Q). The results of the analysis conducted by these authors revealed the factors that can influence the use of deep-learning strategies. They reported, among other findings, the influence of goal orientations and the indirect effect of self-efficacy. In summary, the study revealed that “the higher the value in the deep learning strategies the more the students regulate their learning strategies and achieve a deeper processing of new information” (Panadero et al., 2021, p. 8). To the best of our knowledge, a recent study with Ecuadorian students is the only research to have analysed the validity and reliability of DLS-Q (Yaguarema et al., 2022). This study resulted in four factors, like the original version, but identified some inconsistencies in the item loading across factors. Therefore, the study is relevant for interpreting the results of the analysis and for contexts in which DLS-Q is applied.

In this study, we aimed to determine whether the structure of DLS-Q is suitable for verifying the instrument’s validity in the Catalan language, given that it is currently available only in Spanish and English. In addition, we sought to contribute to the evidence supporting the questionnaire’s usefulness since, to the best of our knowledge, it has rarely been used in other studies. Our other objectives were to explore whether students’ beliefs about the subject and their previous experiences with peer-assessment processes were linked to their self-regulated learning skills. In this context, we posed the following research questions:

RQ1. What is the internal structure of DLS-Q in Catalan like and to what extent is it reliable?

RQ2. Do students’ beliefs about the subject before the beginning of the academic year bear any relation to their self-regulated learning strategies?

RQ3. Do students’ previous experiences with peer-assessment processes bear any relation to their self-regulated learning strategies?

2 Method

This was a transversal study based on a questionnaire and a quantitative methodology (Guárdia Olmos et al., 2008).

2.1 Study background

This study was developed in the context of an R&D project “Analysis of the effects of feedback supported by digital monitoring technologies on generic competences (e-FeedSkill)”. The project was designed to analyse the effects of peer feedback in the development of self-regulated learning strategies among higher education students. Moreover, it aimed to explore whether monitoring technologies have differential effects on the development of these skills. To achieve these objectives, the following components were designed and developed as part of the project: i) a didactic sequence based on a peer-feedback process, ii) a virtual tutor or chatbot to support the adoption of self-regulation strategies, and iii) a learner dashboard for providing feedback based on learning analytics.

The e-FeedSkill project was implemented with different year groups within several degrees at the Universitat de Barcelona (University of Barcelona). Since the project used a quasi-experimental methodology, the groups within each subject were divided into control and experimental groups. The students in the second group had access to monitoring technologies (a tutor chatbot and a learner dashboard). To prove the effect of the intervention, a measure of self-regulated learning was needed. In light of the application context and after an evaluation process, the researchers selected the Deep Learning Strategies Questionnaire (DLS-Q) (Panadero et al., 2021).

2.2 Participants

A total of 475 higher education students participated in this study, 78.53% of whom identified as woman, 20.21% as men, 1.05% as non-binary and 0.21% as another gender. All were students in different year groups (391 from first year, 57 from second year, 19 from third year, seven from fourth year and one from fifth year) from several faculties (99 students from the Faculty of Education, 321 from the Faculty of Pharmacy, 13 from the Faculty of Mathematics and Computer Science, 33 from the Faculty of Law, three from the Faculty of Biology and Geography and six from the Faculty of History) at the Universitat de Barcelona. This university is located in Catalonia, where the official languages are Catalan and Spanish. As a result, many academic activities and programmes are held in Catalan. All students were informed about the objectives of the project. The activities within the didactic sequence were mandatory, but the questionnaires used for research purposes were voluntary; therefore, the sample was clustered by convenience. The approval of the University Bioethics Commission was obtained (IRB 00003099).

2.3 Instruments

In this study, we used the Deep Learning Strategies Questionnaire (DLS-Q) (Panadero et al., 2021), which is designed to explore students’ thoughts while they perform academic tasks and is based on a four-dimensional model. DLS-Q is composed of 30 items, each of which presents statements that reflect students’ possible thoughts during academic tasks. Students are asked to express their level of agreement with each statement on a Likert-type scale with five possible responses, from 1 (strongly disagree) to 5 (strongly agree). The analysis carried out by Panadero et al. (2021) presents a four-factor structure for the 30 items. The proposed dimensions are: S1. Basic learning self-regulation strategies; S2. Visual elaboration and summarizing strategies; S3. Deep information processing strategies; and S4. Social learning self-regulation strategies. Table 1 below shows the distribution of the items across these dimensions. The original items in the Spanish questionnaire were translated into Catalan by the project’s principal investigator, who has bilingual proficiency in both languages (see Appendix 1). This translation was subsequently revised by two other team members. Efforts were made to ensure the translation remained as faithful as possible to the original content, which had already been validated. Most items were translated as literally as possible and minor adaptations were made only in cases where the meaning might have been lost due to the literal nature of the translation.

Additionally, four items were incorporated to explore students’ beliefs about the subjects before the beginning of the academic year. These items were drafted based on the items within the value task dimension of the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich, 1991). These items are designed to evaluate students’ perceptions of the importance, usefulness and relevance of a task (Pintrich, 1991). In the items developed for our study, students were asked to express their level of agreement with the following statements on a Likert-type scale with five possible responses, from 1 (strongly disagree) to 5 (strongly agree): i) I think that the topics of this subject  will be useful for me to learn; ii) I believe that I will be able to use what I learn in this subject in other subjects, iii) I think I will like the topics of this subject; and iv) Understanding the topics this subject  is very important to me.

2.4 Procedure

In all subjects involved in the project during the second semester of the 2021-2022 academic year, the intervention was carried out on the virtual campus. The instrument used in this study (initial questionnaire) was drafted in an online form approved by the university and was included in the virtual classroom for each participating subject to facilitate student access. On the first day of classes for each subject, a researcher presented the project and explained the research aims to the students. The students were invited to complete the initial questionnaire voluntarily (but not anonymously).

2.5 Data analysis

The data collected were anonymized, which involved eliminating any data that could identify the students, and a database without any personal information was created. Using this database, the negative DLS-Q items were recoded and exploratory and semi-confirmatory factor studies were performed using the Factor 12 software package (Ferrando et al., 2022; Lorenzo-Seva & Ferrando, 2013). These results were then used to conduct scale reliability, correlation and effect size analysis with the SPSS software package.

3 Results

To answer research question 1 (RQ1), we conducted an exploratory and semi-confirmatory factor study using the Factor 12 software package (Ferrando et al., 2022; Lorenzo-Seva & Ferrando, 2013). Based on this analysis, a four-factor structure emerged, across which the 30 items in the original scale are distributed (see Table 2).

Table 2

Rotated loading matrix of the Deep Learning Strategies Questionnaire (DLS-Q) for 30 items

Note. In Factor 12, loadings lower than absolute .300 were omitted. F1 = Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4= Visual elaboration and summarizing strategies. DSLQ 5_R, DSLQ 9_R, DSLQ 21_R, DSLQ 25_R: items recoded.

As shown in Table 3, the statistics corresponding to the fit of the model (semi-confirmatory factor analysis of the four dimensions for 30 items) indicated a good fit of the structure. The values of the goodness-of-fit indices were at values below .030 for RMSEA and above .90 for the rest of the indices.

Table 3

Semi-confirmatory factor analysis of the four dimensions for 30 items. Fit statistics

RMSEA = root mean square error of approximation; NNFI = Non-Normed Fit Index (Tucker & Lewis); CFI = comparative fit index; GFI = goodness of fit index; AGFI = adjusted g of fit index.

Based on these results, we decided to dismiss items 11, 19 and 23. The adequacy of the polychronic correlation was calculated; the Kaiser-Meyer-Olkin test was good (KMO = .869) and Bartlett’s statistic X2 (351) = 5239.2 (p <.001) indicated that enough equal variances could be assumed. Thus, a new exploratory and semi-confirmatory factor analysis was carried out. In this case, the 27 items were also distributed across four factors, as shown in Table 4 below.

Table 4

Rotated loading matrix of the Deep Learning Strategies Questionnaire (DLS-Q) for 27 items

Note. F1 = Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4 = Visual elaboration and summarizing strategies. DSLQ 5_R, DSLQ 9_R, DSLQ 21_R, DSLQ 25_R: items recoded.

The statistics corresponding to the fit of the model (semi-confirmatory factor analysis of the four dimensions for 27 items) confirmed the adequacy of this factorial structure, as shown in Table 5; the RMSEA statistics were less than .030 and the other statistics were greater than .90.

Table 5

Semi-confirmatory factor analysis of the four dimensions for 27 items. Fit statistics

RMSEA = root mean square error of approximation; NNFI = Non-Normed Fit Index (Tucker & Lewis); CFI = comparative fit index; GFI = goodness of fit index; AGFI = adjusted goodness of fit index.

Overall, a Cronbach’s alpha of .861 was obtained, with a confidence interval (95%) between .842 and .878. The statistics obtained showed acceptable reliability for each factor, with Cronbach’s alpha over .74 (Table 6).

Table 6

Scale reliability

Note. F1 = Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4 = Visual elaboration and summarizing strategies.

As shown in Table 4, there was correspondence between the distribution of items proposed in Panadero et al. (2021) and the findings of the analysis presented in this study. To that end, F1 would include items relating to basic learning self-regulation strategies, F2 items relating to social learning self-regulation strategies, F3 items relating to deep information processing strategies and F4 items concerning visual elaboration and summarizing strategies.

To answer RQ2, we conducted a correlational analysis to explore whether there was any relationship between students’ expectations or thoughts at the beginning of the academic year and their learning strategies.

We found significant positive correlations between all dimensions of deep-learning strategies and the different expectations or beliefs expressed by students at the beginning of the year. As shown in Table 7, Pearson’s correlation coefficient varied according to the dimension and expectation or belief posed. In addition to the fact that all correlations were significant, there was a clear difference between meanings. Factor 2 (social learning self-regulation strategies) presented low values for all expectations, while Factor 1 (basic learning self-regulation strategies) showed relatively high values for all expectations. In fact, the highest value corresponded to Pearson’s correlation coefficient between F1 (basic learning self-regulation strategies) and the statement “Understanding the topics of this subject is very important to me”; this value was closely followed by the correlation between Factor 1 and the statement “I believe that I will be able to use what I learn in this subject in other subjects”.

Table 7

Correlational analysis

Note. ** Correlations are significant at the .01 level (bilateral). Note. F1 = Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4 = Visual elaboration and summarizing strategies.

We used these results to conduct a mean difference analysis to identify any differences between students with high and low scores in the statement “I believe that I will be able to use what I learn in this subject in other subjects”. The categorization according to item scores was carried out based on percentile scores. In this case, we decided to create two categories to establish a more realistic comparison with the scores obtained on the five-point scale. Specifically, the cut-off was established at the 75th percentile (with a score of 5, with low scores below 5 and high scores above 5) (Table 8). To ensure that the groups created were comparable, Levene’s test was used in the comparison tests (T-test and ANOVA), in which the homogeneity of variances was verified.

Table 8

Frequencies for low and high groups in the ad hoc statements to explore students’ beliefs about the subjects before the beginning of the academic year

To obtain Cohen’s d and determine the effect size in SPSS, we saved the standardized values (z-scores) of the dependent variable as new variables. These z-scores represent the factor scores for each of the factors derived from the DLS_Q scale. Using these scores, we conducted the corresponding T-test. The results obtained are presented in Table 9 below.

Table 9

T-test for independent samples. High and low scores for the statement “I believe that I will be able to use what I learn in this subject in other subjects”

Note. F1= Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4 = Visual elaboration and summarizing strategies.

According to Cohen’s criteria, d values from .2 upwards are considered small effects, values from .5 upwards are medium effects, and values from .8 upwards are large effects (Cohen, 1988). In this regard, the score for the statement “I believe that I will be able to use what I learn in this subject in other subjects” showed statistically significant differences in the implementation of deep-learning strategies. In particular, the effect size (Cohen’s d) showed a large effect with the implementation of basic learning self-regulation strategies.

We also conducted the same analysis to identify any differences between students with high and low scores for the statement “Understanding the topics of this subject is very important to me”. Table 10 below shows the results obtained.

Table 10

T-test for independent samples. High and low scores for the statement “Understanding the topics of this subject is very important to me”

Note. F1 = Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4 = Visual elaboration and summarizing strategies.

In this case, we also found that the score for the statement “Understanding the topics of this subject is very important to me” presented statistically significant differences in the implementation of deep-learning strategies. In particular, the effect size (Cohen’s d) showed a medium effect with the implementation of basic learning self-regulation strategies (F1) and deep information processing strategies (F3).

Regarding RQ3, we conducted a T-test to detect any differences between the students who said they had previous experience of participating in peer-assessment processes and those who did not.

The sample was divided into group 1 (students with no reported experience) (N=140) and group 2 (students with reported experience) (N=335).

Table 11

T-test for independent samples

Note. F1 = Basic learning self-regulation strategies; F2 = Social learning self-regulation; F3 = Deep information processing strategies; F4 = Visual elaboration and summarizing strategies.

As shown in Table 11, no statistically significant differences were found between the groups (p > 0.05); in other words, previous experience of participating in peer-assessment processes made no difference in the use of deep-learning strategies.

4 Discussion

Based on the exploratory and semi-confirmatory factor analysis conducted, we provide evidence supporting the suitability of the four-dimensional structure proposed in the original version of DSL-Q. While the four-dimensional structure for the 30 items presented a good fit, we decided to eliminate the items identified as 11, 19 and 23 in the original version (Panadero et al., 2021; p. 9): (11) “I usually participate in class discussions, asking questions or making comments to the teacher”; (19) “If the teachers provide us with presentations, I take notes in them because it makes everything clearer” and (23) “If I do not do a good job on a task or an exam, I ask the teacher to give me more information about how to improve”. The low scores for these items could be attributed to teaching practices, such as the prevalence of lecture-based classes where students adopt a passive role, but they could also indicate a lack of student trust in lecturers as feedback providers. These factors should be explored through qualitative methods to gain a better understanding of students’ experiences.

Beyond these hypotheses, our results were consistent with the recent findings of Yaguarema et al. (2022), who also excluded item 19 (“If the teachers provide us with presentations, I take notes in them because it makes everything clearer”). In addition, the authors pointed out a lack of consistency in this item and argued that it could be included in “visual elaboration and summarizing strategies” instead of “social learning self-regulation strategies”.

In conclusion, the four-dimensional structure for the 27 items appears to be a good option in the context under study in light of the results obtained. In addition, we found that the distribution of the items across the dimensions was the same as those reported by the authors of the original version.

We also found significant positive correlations between all dimensions of learning strategies and students’ beliefs at the beginning of the year. In particular, the correlation between basic learning self-regulation strategies (F1) and the statement “Understanding the topics of this subject is very important to me” was consistent with the definition of learning strategy as any regulatory action that students implement to complete a task or comprehend content (García-Pérez et al., 2020).

With respect to the differences detected between students with low and high scores for the statements “I believe that I will be able to use what I learn in this subject in other subjects” and “Understanding the topics of this subject is very important to me”, this was consistent with the findings reported by Panadero et al. (2021) regarding the relationship between students’ regulation and depth of information processing and the use of deep-learning strategies. In turn, this is related to the theory underpinning the importance of the expectancy value (Eccles, 2005) when performing a learning task (Pintrich, 1991, 2000) and previous research in the field (Ghasemi & Dowlatabadi, 2018; Lawanto et al., 2014; Li & Zheng, 2018). Likewise, our results were in line with previous literature findings that have shown that learners with high values in task levels report higher self-regulation scores (Lee et al., 2020).

Despite extensive evidence pointing to the importance of peer assessment and peer feedback in developing self-regulated learning (Clayton Bernard & Kermarrec, 2022; Prompan & Piamsai, 2024; Stančić, 2021), our findings suggest that previous experience with peer-evaluation processes does not show statistically significant differences in the use of deep-learning strategies. Therefore, in this context, it may not contribute to enhancing self-regulated learning. However, it is important to note that students’ responses were obtained through self-reported yes-no questions. As a result, there is a lack of clarity on students’ understanding of the term “peer assessment” and a lack of information on the type of feedback involved in their experiences. As observed by Tooping (2021), the nature of students’ previous experiences with peer assessment is a key influencing factor. Consequently, our results in this area should be treated with caution, in light of the differences in effects depending on the type of feedback, as reported in previous studies (Hattie & Timperley, 2007; Lipnevich & Smith, 2009).

4.1 Limitations and future directions for study

This paper contributes to the body of literature on measuring self-regulated learning and its associated strategies. We provide evidence consistent with the findings presented by the authors of the original version of DLS-Q presented here. Moreover, we contribute to the field by translating and validating the items in a new language. This may encourage other researchers to translate the items into other languages and explore the validity and structure of the questionnaire in their own educational contexts.

Based on our findings and experience from this project, we recommend that interventions targeting self-regulated learning be viewed as long-term processes that develop gradually throughout students’ educational journeys. These interventions are mediated by contextual, intrapersonal and interpersonal variables. Therefore, it is advisable to gather initial information about students’ levels or domains and keep in mind that self-regulated learning should be linked to their learning objectives. It is also advisable to evaluate the effects of formative assessments on students’ academic performance and competences. Furthermore, regarding the link between formative assessment practices and self-regulation learning, our results should make educators and researchers aware of the importance of carefully designing and implementing such practices. The factors involved must be reflected on to provide suitable opportunities for enhancing self-regulation skills.

Despite these contributions, the study presents certain limitations. Firstly, there was insufficient representation of students from different year groups, which limited our ability to conduct analyses based on this variable. As mentioned, the information about students’ previous experiences with peer assessment had inherent limitations. Second, it should be kept in mind that the statements related to beliefs were specific to the subject, while DSL-Q focuses on strategies used by students while performing academic assignments in general.

In conclusion, this contribution should encourage other educational researchers to incorporate new instruments for measuring students’ strategies. This could provide valuable insights for the design and evaluation of educational interventions that could help students enhance their self-regulation and, ultimately, their lifelong learning competence.

References

Alonso-Tapia, J., Calderón, E. P., & Ruiz, M. A. D. (2014). Development and validity of the Emotion and Motivation Self-regulation Questionnaire (EMSR-Q). The Spanish Journal of Psychology, 17. https://doi.org/gt2bcd

Boekaerts, M., & Corno, L. (2005). Self‐regulation in the classroom: A perspective on assessment and intervention. Applied Psychology, 54(2), 199-231. https://doi.org/bjzv8p

Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245-281. https://doi.org/dmcw5x

Clayton Bernard, R., & Kermarrec, G. (2022). Peer assessment and video feedback for fostering self, co, and shared regulation of learning in a higher education language classroom. Frontiers in Education, 7. https://doi.org/gs8nzj

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd. ed.). Erlbaum (1st. ed., 1977, Academic Press).

Council Recommendation of 22 May 2018 on Key Competences for Lifelong Learning (2018/C 189/01). https://tinyurl.com/38xcynu6

Eccles, J. S. (2005). Subjective task value and the Eccles et al. model of achievement-related choices. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of competence and motivation (pp.105-121). Guilford Press.

Ferrando, P., Lorenzo-Seva, U., Hernández-Dorado, A., & Muñiz, J. (2022). Decálogo para el Análisis Factorial de los Ítems de un Test. Psicothema, 34(1), 7-17. https://doi.org/jjq8

Gajda, M., Małkowska-Szkutnik, A., & Rodzeń, W. (2022). Self-Regulation in Adolescents: Polish Adaptation and Validation of the Self-Regulation Scale. International Journal of Environmental Research and Public Health, 19(12), 7432. https://doi.org/nwrk

García-Pérez, D., Fraile, J., & Panadero, E. (2020). Learning strategies and self-regulation in context: how higher education students approach different courses, assessments, and challenges. European Journal of Psychology of education, 36, 533-550. (Advance online publication, 2021). https://doi.org/gt45kf

Ghasemi, A. A., & Dowlatabadi, H. R. (2018). Investigating the role of task value, surface/deep learning strategies, and higher order thinking in predicting self-regulation and language achievement. Journal of Asia TEFL, 15(3), 664. https://doi.org/nwrm

Guárdia, J., Fleixa Blanxart, M., & Peró Cebollero, M. (2008). Análisis de datos en psicología (2nd ed.). Delta Publicaciones.

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/bf4d36

Ibarra-Sáiz, M. S., Rodríguez-Gómez, G., & Boud, D. (2020). Developing student competence through peer assessment: the role of feedback, self-regulation and evaluative judgement. Higher Education, 80(1), 137-156. https://doi.org/nwrn

Lawanto, O., Santoso, H. B., Goodridge, W., & Lawanto, K. N. (2014). Task value, self-regulated learning, and performance in a web-intensive undergraduate engineering course: How are they related. Journal of Online Learning and Teaching, 10(1), 97. https://tinyurl.com/mrkz6fzh

Lee, D., Watson, S. L., & Watson, W. R. (2020). The relationships between self-efficacy, task value, and self-regulated learning strategies in massive open online courses. International Review of Research in Open and Distributed Learning, 21(1), 23-39. https://doi.org/gppgjt

Li, S., & Zheng, J. (2018). The relationship between self-efficacy and self-regulated learning in one-to-one computing environment: The mediated role of task values. The Asia-Pacific Education Researcher, 27(6), 455-463. https://doi.org/grpksg

Lipnevich, A. A., & Smith, J. K. (2009). Effects of differential feedback on students’ examination performance. Journal of Experimental Psychology: Applied, 15(4), 319–333. https://doi.org/d6n48h

Lorenzo-Seva, U., & Ferrando, P. J. (2013). Factor 9.2: A Comprehensive Program for Fitting Exploratory and Semiconfirmatory Factor Analysis and IRT Models. Applied Psychological Measurement, 37(6), 497-498. https://doi.org/jkth

Muñoz, J. L., Pons, L., Ion, G., Cano, E., Fuentes, M., Mercader, C., & Díaz, A. (2020). Guia de pràctiques d’avaluació per a l’autoregulació dels aprenentatges de l’estudiantat universitari. Edicions UAB. https://tinyurl.com/bdhwv46b

Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2), 199-218. https://doi.org/fh4qnb

Panadero, E. (2017). A Review of Self-regulated Learning: Six Models and Four Directions for Research. Frontiers in Psychology, 8, Art. 422. https://doi.org/gf4n8k

Panadero, E., Alonso-Tapia, J., García-Pérez, D., Fraile, J., Galán, J. M. S., & Pardo, R. (2021). Deep learning self-regulation strategies: Validation of a situational model and its questionnaire. Revista de Psicodidáctica (English ed.), 26(1), 10-19. https://doi.org/nwrp

Panadero, E., Andrade, H., & Brookhart, S. (2018). Fusing self-regulated learning and formative assessment: A roadmap of where we are, how we got here, and where we are going. Australian Educational Researcher, 45(1), 13-31. https://doi.org/gqr3jz

Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74-98. https://doi.org/gbvsn8

Panadero, E., Klug, J., & Järvelä, S. (2016). Third wave of measurement in the self-regulated learning field: when measurement and intervention come hand in hand. Scandinavian Journal of Educational Research, 60(6), 723–735. https://doi.org/ggf2x4

Pintrich, P. R. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). https://eric.ed.gov/?id=ED338122

Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In Handbook of self-regulation (pp. 451-502). Academic Press. https://doi.org/d5m37v

Prompan, J., & Piamsai, C. (2024). The effects of peer feedback and self-regulated learning on Thai EFL students’ writing ability and self-regulation. LEARN Journal: Language Education and Acquisition Research Network, 17(1), 100-132. https://tinyurl.com/5fsdm3fm

Schünemann, N., Spörer, N., Völlinger, V. A., & Brunstein, J. C. (2017). Peer feedback mediates the impact of self-regulation procedures on strategy use and reading comprehension in reciprocal teaching groups. Instructional Science, 45(4), 395-415. https://tinyurl.com/yyeyxf9k

Schunk, D. H., & Ertmer, P. A. (2000). Self-regulation and academic learning: Self-efficacy enhancing interventions. In Handbook of self-regulation (pp. 631-649). Academic Press. https://doi.org/ffnz4j

Stančić, M. (2021). Peer assessment as a learning and self-assessment tool: a look inside the black box. Assessment & Evaluation in Higher Education, 46(6), 852-864. https://doi.org/nwrq

Theobald, M., & Bellhäuser, H. (2022). How am I going and where to next? Elaborated online feedback improves university students’ self-regulated learning and performance. The Internet and Higher Education, 55, 100872. https://doi.org/nwrr

Topping, K. (2021). Peer Assessment: Channels of Operation. Education Sciences, 11(3), 91. https://doi.org/nwrs

Winne, P. H. (2018). Theorizing and researching levels of processing in self‐regulated learning. British Journal of Educational Psychology, 88(1), 9-20. https://doi.org/gd8cmv

Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In Handbook of self-regulation (pp. 531-566). Academic Press. https://doi.org/dpbrv7

Winstone, N., & Carless, D. (2019). Designing effective feedback processes in higher education: A learning-focused approach. Routledge. https://doi.org/nhd9

Wu, Y., & Schunn, C. D. (2020). From feedback to revisions: Effects of feedback features and perceptions. Contemporary Educational Psychology, 60, 101826. https://doi.org/gnr7sg

Yaguarema, M., Zambrano, R. J., & Salavarría, M. (2022). Analysis of the Deep Learning Strategies Questionnaire with Ecuadorian students. Frontiers in Education, 7, Art. 1004874. https://doi.org/nwrt

Zhang, Y., & Schunn, C. D. (2023). Self-regulation of peer feedback quality aspects through different dimensions of experience within prior peer feedback assignments. Contemporary Educational Psychology, 74, 102210. https://doi.org/nwrv

Zimmerman, B. J. (2000). Attaining self-regulation: a social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of Self-Regulation (pp. 13-40). Academic Press. https://doi.org/dhzpnw

Zimmerman, B. J. (2001). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated Learning and Academic Achievement: Theoretical Perspectives (pp. 1-37). Lawrence Erlbaum. https://doi.org/k3st

Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64-70. https://doi.org/cwgczh

Funding

This contribution was supported by the project “Analysis on the effects of feedback supported with monitoring digital technologies on generic competences” (grant PID2019-104285GB-I00 funded by MCIN/AEI/ 10.13039/501100011033).

This contribution is part of the first author’s PhD thesis for publication. It was written in the context of AYUDAS PARA CONTRATOS PREDOCTORALES PARA LA FORMACIÓN DE DOCTORES (PRE2020-095434) funded by MCIN/AEI.

Availability of data and materials

The datasets used and/or analysed for the current study are available from the corresponding author upon reasonable request.

Ethical approval and consent to participate

Informed consent was obtained from all participants before being included in the study. The project was approved by the Ethics Committee of the University of Barcelona (IRB 00003099).

Appendix 1

Deep Learning Strategies Questionnaire (DLS-Q) statements in Catalan language

1. 1.Analitzo en profunditat la tasca a realitzar perquè em quedi clar què he de fer. 

2. 2.Sovint elaboro esquemes o dibuixos per representar-me el que estudio o els problemes que he de fer.  

3. 3.Quan llegeixo o escolto una afirmació o conclusió a classe, penso en les alternatives possibles.  

4. 4.Quan he entès el que he de fer, procuro visualitzar de forma concreta el que he d'anar fent i aconseguint.  

5. 5.No acostumo a organitzar la informació en quadres o taules a l'estudiar perquè no serveix de molt per aprendre.  

6. 6.Relaciono el que estic aprenent a les classes amb idees pròpies.  

7. 7.Sovint comento amb els meus companys/es idees o aspectes del que he estat estudiant.  

8. 8.Mentre faig una tasca comprovo si els passos que vaig donant són els adequats. 

9. 9.Excepte que m'ho demani el/la professor/a, no acostumo a fer resums dels textos que estudio.  

10. 10.Quan estudio relaciono el material que llegeixo amb el que ja sé.  

11. 11.Normalment participo de manera activa a les classes, preguntant o fent comentaris al professor/a.  

12. 12.Si el/la professor/a em lliura alguna eina que em permeti avaluar si la manera de procedir en realitzar una tasca està bé, habitualment la utilitzo.  

13. 13.Quan estudio per a una avaluació, escric petits resums amb les idees i conceptes principals de les lectures.  

14. 14.Relaciono idees de la classe amb altres idees cada vegada que és possible fer-ho. 

15. 15.Demano l'opinió dels meus companys/es de classe sobre com estic fent un treball.  

16. 16.Quan estic fent una tasca m'aturo a comprovar si avanço segons el previst. 

17. 17.Acostumo a estudiar utilitzant estratègies diferents (memoritzar, fer esquemes, etc.) segons la matèria de què es tracti. 

18. 18.En estudiar, sovint relaciono mentalment els continguts que estic treballant amb els d'altres assignatures.  

19. 19.Quan els professors ens proporcionen les presentacions, prenc les notes sobre les mateixes perquè així em queda tot més clar.  

20. 20.En acabar una activitat de la universitat repasso el que he fet per veure si ho he entès i si està bé.  

21. 21.No acostumo a elaborar mapes conceptuals per relacionar els conceptes que estudio perquè són de poca utilitat.  

22. 22.En estudiar acostumo a buscar possibles relacions entre el que estudio i les situacions a les que podria aplicar-se. 

23. 23.Quan alguna cosa no m'ha anat molt bé en un treball o examen, demano al professor/a que em doni més informació sobre com millorar.  

24. 24.Abans de posar-me a realitzar una tasca, planifico acuradament el que he de fer. 

25. 25.No acostumo a fer gràfics o diagrames mentre estudio o resolc problemes perquè no m'ajuden a aprendre.  

26. 26.Cerco situacions a les quals aplicar els continguts del curs. 

27. 27.Intento, sempre que puc, comentar amb els meus companys/es idees o aspectes del que he estat estudiant per tal d'aprofundir-hi. 

28. 28.Llegeixo les instruccions dels exercicis i els exàmens les vegades necessàries per comprendre en profunditat què es demana. 

29. 29.Normalment, si és possible construeixo taules per organitzar la informació continguda en textos i problemes.  

30. 30.En general estudio tractant d'imaginar-me i "visualitzar" les situacions a què fa referència el contingut.