SABER 2.0 in STEM: Rewarded Correction and Subject Content – Active Learning Practical Matching Strategies

espanolINTRODUCCION. Se discute la implementacion 2.0 de la estrategia de clase practica para incrementar el rendimiento de los estudiantes universitarios, enfatizando en asignaturas del ambito CTIM (Ciencias, Tecnologias, Ingenierias y Matematicas). METODO. El punto de partida es un esquema basado en el concepto de clase invertida, flipped classroom (FC). No obstante, el trabajo comienza en la misma clase, definiendo una clase invertida sincrona. Las modificaciones posteriores se introducen para aumentar la capacidad de trabajo autonomo del estudiante. La metodologia practica se conoce como SABER (Supervision del Aprendizaje Basico con Ejercicios y autoReflexion). RESULTADOS. En este estudio se describe una version 2.0, que incorpora: a) correccion de errores recompensada, como parte importante para la consolidacion conceptual del trabajo realizado; y b) cambios substanciales en la forma en que la materia se introduce a los estudiantes. Asi, se utilizan experimentos comparativos y/o propiedades fisicas macroscopicas comparativas para introducir conceptos mas dificiles. DISCUSION. Esta aproximacion presenta los contenidos desde un punto de vista experimental que esta mucho mas cerca del conocimiento real de los estudiantes. En este sentido, el presente estudio tambien proporciona algunos ejemplos especificos y consejos practicos para poner de manifiesto la facilidad de implementar realmente esta metodologia. EnglishINTRODUCTION. The present paper addresses a 2.0 implementation of a practical classroom strategy to increase university students’ performance, with emphasis given to STEM subjects (Science, Technology, Engineering and Mathematics). METHOD. The starting point is a scheme based on the flipped classroom (FC) concept. The work, however, starts in the classroom, using a synchronous FC, and modifications are introduced to increase students’ ability to work autonomously. The practical methodology is known as SABER (after the Spanish Supervision del Aprendizaje Basico con Ejercicios y autoReflexion). RESULTS. The paper describes a 2.0 version that incorporates (a) rewarded mistake correction as a key part in students’ consolidation of concepts; and (b) substantial changes in how subject content is introduced to students. In the latter case, comparison experiments and compared macroscopic physical properties have been used to introduce difficult concepts. DISCUSSION. This approach presents content from an experimental perspective that is much closer to students’ existing knowledge. The paper also provides some specific examples and practical tips to demonstrate how easily the methodology can be implemented. catalaINTRODUCCIO. Es discuteix una implementacio 2.0 de l’estrategia de classe practica per incrementar el rendiment dels estudiants universitaris, amb emfasi en assignatures de l’ambit CTEM (ciencies, tecnologies, enginyeries i matematiques). METODE. El punt inicial es un esquema basat en el concepte de classe inversa (flipped classroom). El treball s’inicia a la classe mateix, definint una classe inversa sincrona. Les modificacions posteriors s’introdueixen per incrementar la capacitat de treball autonom de l’estudiant. La metodologia practica es coneix com SABER (Supervisio de l’Aprenentatge Basic amb Exercicis i autoReflexio). RESULTATS. En aquest estudi es descriu una versio 2.0 que incorpora: a) correccio d’errors recompensada, com a part important per a la consolidacio conceptual de la feina realitzada, i b) canvis substancials en la manera com s’introdueix la materia als estudiants. Sobre aquest segon punt, es fan servir, per exemple, experiments comparatius i propietats fisiques macroscopiques comparatives per introduir els conceptes mes dificils. DISCUSSIO. Aquesta aproximacio presenta els continguts des d’un punt de vista experimental, que esta molt mes a prop del coneixement real dels estudiants. En aquest sentit, l’estudi tambe proporciona alguns exemples especifics i consells practics per evidenciar la facilitat d’implantar realment aquesta metodologia.


Introduction
In June 1963, near the end of the preface to his celebrated "Feynman Lectures on Physics" (Feynman, Leighton and Sands, 1965, Preface), the renowned physicist Richard Feynman wrote: I think, however, that there isn't any solution to this problem of education other than to realize that the best teaching can be done only when there is a direct individual relationship between a student and a good teacher-a situation in which the student discusses the ideas, thinks about the things, and talks about the things.It's impossible to learn very much by simply sitting in a lecture, or even by simply doing problems that are assigned.But in our modern times we have so many students to teach that we have to try to find some substitute for the ideal.
The teaching approach used in Caltech classes was traditional, but Feynman's three-volume series became a seminal work, and for a seemingly not-so-relevant reason: the way in which physics content was introduced throughout his volumes.Instead of starting with powerful abstract statements, Feynman's approach was to start with a common-experience introduction to a concept, follow this with its relation to mathematical language and end with a development of the subject in full.
Thus, Feynman's introductory remarks (and, of course, his lectures) capture the complexity of the current movement in learning and teaching innovation, namely that the teaching methodology should be based on active work by students, even in the most theoretical classroom setting, and that subject content is not independent of specific choices about active learning methodologies, but instead is so tightly bound up with them as to become inextricable.
In this regard, many innovative active teaching methodologies have been developed in recent decades, including peer instruction (Crouch and Mazur, 2001), team-based learning (Johnson, Johnson and Stanne, 2000), flipped classroom (Ryan and Reid, 2015) and project-and problembased learning (Gijbels, Dochy, Bossche and Segers, 2005).These general, widely applicable methodologies are also complemented by a broad range of practical techniques, including frequent questioning (Kerezstes, Kaiser, Kovács and Racsmány, 2014), retrieval practice (Karpicke and Blunt, 2011) and rewarded mistake correction (Brown, Mason and Singh, 2016), and these techniques are widely applicable to any of the above active teaching strategies.However, Feynman's remarks and lectures go even further, implying that in such innovations, subject content must either be adapted to the techniques or substantially transformed.
Recent results on learning mechanisms offered by emerging sciences such as cognitive psychology and neurobiology certainly point in this direction.For instance, the role played by myelination in neural circuitry, which provides outstanding physiological evidence of learning, stresses the need for repeated, deliberate practice of error-free action patterns (see Holmes, Wieman and Bonn, 2016, on the importance of repeated practice in the development of critical thinking).In addition, recent advances in working and permanent memory management indicate that multiple connections with existing knowledge are crucial for proper, solid learning, and that one cannot exclude the role of forgetting in the fostering of permanent learning.Therefore, it is necessary to explore students' initial background, current knowledge and learning ability in order to design pedagogical strategies to introduce new concepts that increase learning efficiency (Storm, Bjork and Bjork, 2008).Over the past twenty years, the theoretical and practical advances noted above have led to a genuine implementation of innovative active learning methods in the classroom.This is the growing reality in primary and secondary school systems, and even in institutions of higher education.The seminal work by Freeman and collaborators (Freeman et al., 2014), a meta-study of 225 previous publications on STEM subjects, provides solid grounds for the real improvement produced by active methods in comparison to traditional teaching.More recent work has further confirmed Freeman's conclusions on the enhanced performance of students engaged in active learning methods (Weaver and Sturtevant, 2015).Because the superiority of active learning methods compared to the traditional lecture system has become increasingly evident, the focus needs to turn to further enhancing students' efficiency within the active learning context and to dealing with new difficulties that may arise as such pedagogical paradigms are extended across several subjects in the same course.
The last remark raises an important issue: any new pedagogical framework must be sustainable.This means that the implementation of active learning techniques needs to be general and universal, i.e., for all subjects and degrees.A major implication is, therefore, that students' efforts must be adequately balanced across disciplines, or put in other words, one cannot claim better results for a specific new pedagogical technique if it requires much greater effort from students than could be sustained if all disciplines in the same semester were taught using the same methodology.This is much more than reasonable thinking.It is actually a crucial need, because the important benefits of active learning have recently been shown to be short-lived unless active learning becomes the methodology of use in undergraduate teaching (van Vliet, Winnips and Brouwer, 2015).
The search for more efficient classroom settings, the pursuit of sustainability, and an attention to the important specificities of content in STEM subjects have prompted the development of a variant of the flipped classroom, called the synchronous flipped classroom (SFC), which is designed to assist students from the very first moment they tackle a theme (Medina, 2016).SFC brings in a repeating pattern of error-free practice involving reading, conceptual analysis and operations.It establishes a proper balance in student workloads and it focuses on the needs raised by teaching mathematically oriented subjects, which are known to pose major difficulties in the absence of instructor help (Stinson, Harkness, Meyer and Stallworth, 2009), underscoring the valuable realisation that minimal guidance in teaching does not work (Kirschner, Sweller and Clark, 2010).Since these significant conceptual difficulties arise right in the beginning of a theme, students can easily become stalled or even drop out of the learning dynamic.
The practical implementation of SFC has been named SABER (an acronym for Supervisión del Aprendizaje Básico mediante Ejercicios y autoReflexion, which may be loosely translated to English as Supervision of Basic Learning through Exercises and Self-Reflection).The first deployment of SABER helped to shift the traditional classroom setting toward one marked by active learning methods (Giménez, 2018).To improve the methods further, however, it becomes necessary to cope with the content problem, i.e., to adapt specific content difficulties to the particularities of the new pedagogical paradigm.Even though the methodology was developed for heavily mathematical subjects, it has been found to be of general validity as a practical classroom scheme.Content problems are well-known to present major issues in current university teaching.Students and teachers alike are overwhelmed by the "mile-wide, inch-deep" problem of content descriptors, which pose a common issue in any university subject and are particularly important in introductory undergraduate courses (Wood, 2009a).In short, there is too much content and the time to cover it is dwindling.Arguably, the need for content coverage raises an unsurmountable barrier to the adoption of active learning strategies; professionals opposing the active learning movement state that only traditional lecture techniques are sufficiently fast to meet the strict requirements for content coverage.
As is well-known, this fast information transmission system has a major drawback, since it leaves students far from any meaningful learning.It is clear, therefore, that learning must be freed from such restrictions on content coverage.Modern content availability for students is a help: the main content can be obtained at near zero cost, in zero time.This is an advantage that facilitates a disruptive change: content must be curated, instead of covered.In other words, the teacher acts as a content selector, with the specific items selected for the classroom being those that students cannot work out for themselves.
Several authors have proposed that the new paradigm must bring in content selection in terms of core ideas, crosscutting subjects and scientific practices (Cooper et al., 2015).The new content should also be delivered in a form that can be learnt efficiently by students, as has already been pointed out above.This means that the form of teaching is chosen so that (a) it is close enough to students' prior knowledge, and (b) skills and abilities are developed through sufficient error-free practice (Wood, 2009b).An example of such content delivery, in a specific undergraduate subject in science and technology, is provided below.
The remainder of the paper is organised into three sections.The second section describes the SABER methodology, which is based on the synchronous flipped classroom concept.A new version of the method, SABER 2.0, is also discussed, including test results from its preliminary implementation.For any methodology to become truly practical, however, several daily aspects need to be addressed to achieve smooth classroom dynamics.The third section, therefore, includes a short description of some practical tips that have actually been implemented in the SABER synchronous flipped classroom environment.Finally, the last section sets out our main conclusions.

Method
SABER is a practical implementation of the so-called synchronous flipped classroom (SFC), a kind of flipped classroom setting where students do not work on any material in advance, but rather start work in the classroom under direct teacher supervision.SABER has been designed to allow for error-free deliberate repetition patterns in students' activity, while also calling for a balanced workload.A primary reason for changing the basic flipped classroom scheme, however, is that subjects with increased mathematical content pose difficulties from the very first moment students begin work on a theme, with the consequence that medium-or low-achieving students simply drop any work to be done in advance.Their abandonment arises from an inability to make progress through the provided material on their own with a minimum of understanding.
One solution is to guide students from the outset.This need strongly suggests starting the work in the classroom under teacher supervision.Student questions might then be answered from the start and students will be able to proceed through the proposed activities.If one reasonably assumes that the rate of students' questioning diminishes as they proceed through the autonomous working units with Activity Sheets (hereafter, AS), then help from the teacher becomes less and less necessary toward the end.What actually happens is that a kind of "cruising speed" is achieved in students' autonomous work.
Following on from the above assumptions, the classroom work is organised according to the brief scheme set out below.Full details appear in another publication (Giménez, 2018): • Students are organised into groups of 2-5 individuals and asked to work together in a peer teaching scheme.Discussions among groups can be facilitated by the teacher, whenever necessary.
• A predetermined calendar of AS is introduced, consisting of a very detailed workout pathway for each specific subject.
• AS are worked out in class, under direct teacher supervision.
• The teacher does not lecture; she only helps students to solve questions.
• Questions are solved individually, within each group.However, if several groups raise the same question, the teacher may ask for the students' attention and explain the point using a traditional lecture format.
• Answers to questions from the student groups must be short and concise, pointing to a reworking of a portion of the readings that has not been understood or to exercises that show faulty procedures, assumptions or calculations.The aim is for students to find the solution on their own whenever possible as they correct conceptual and procedural mistakes.
• Reports containing answers to AS are submitted electronically.The teacher only corrects a sample of the reports, so that each student will have had a meaningful fraction of the report set corrected by the end of semester.This will keep teachers' correcting work to a reasonable amount, while providing a necessary input through the marking of students' work.
• A questionnaire is administered to students at the end of each section.There will be between 3 and 5 for the whole course, depending on the specific content features of the subject.
• An end-of-term examination can be administered.The examination has a traditional format.
The evaluation of SABER was carried out in Computer Programming, a first-year subject taught in the Bachelor's Degree in Chemistry at the University of Barcelona, Spain.The subject is taught directly in the computer room, making it necessary to divide the approximately 350 students enrolled in the course into 16 groups.The SABER group was the test group, while the remaining 15 were control groups, which were taught using traditional lecture sessions together with a substantial list of well-developed exercises.Student performance was assessed for all 16 groups using a traditional questionnaire that included some exercises.Figure 1 shows the percentage of passing students as a function of time.The results demonstrate the superiority of the SABER active learning methodology in line with the main conclusions of Freeman et al. (2014).The first version of SABER permitted its practical use in several STEM subjects, e.g., computer programming, general chemistry, physical chemistry of materials and environmental chemistry.However, it was clear from the outset that such a general methodology could be further developed to increase student engagement and efficiency, and also to better adapt subject content.
From the experience gained with the above courses and the tested proposals in the bibliography, we added two new features: i) rewarded mistake correction (RMC) combined with explicit analysis of error sources; and ii) content development to match active learning Laura Arévalo, Pablo Gamallo, Xavier Giménez.SABER 2.0 in STEM: Rewarded Correction and Subject Content -Active Learning… Universitat de Barcelona.Institut de Ciències de l'Educació strategies.A specific chemistry example is provided below for the sake of concreteness: the properties of solids based on their macroscopic aspect, texture and behaviour.

Rewarded Mistake Correction
It has recently been shown that results in traditional exams may improve substantially if rewarded mistake correction is incorporated as supervised formal work within the classroom.Brown, Mason and Singh (2016) tested the strategy in a course on quantum mechanics for undergraduate non-major physics students.By rewarding mistake correction, the authors found that students correcting midterm exam mistakes outperformed those who did not and that such correction also had a greater impact on the students who had performed lower on the first exam.
SABER 2.0 incorporates RMS in AS reports, but not in exams.The change has been introduced as a way to practice continuous mistake correction as a normal procedure, i.e., not tied to exams.The specific implementation of RMC is as follows: • A reasonable amount of time after students have submitted their report for a given AS, the teacher's solution to the AS questions is made available on the virtual campus.
• Students must then submit a new report for the same AS, with their solutions corrected.
• Students must also add a key part: an analysis of the reasons for the mistakes made in their first submission.
• The reward system works as follows: every properly analysed mistake counts as if there were no mistake.If all the causes of a mistake are properly addressed, then the student gets the maximum score, as if the first submitted report had been perfect.
RMC has very recently been tested for the first time, with students enrolled in Physical Chemistry of Materials, a second-year subject taught in the Bachelor's Degree in Materials Engineering at the University of Barcelona.By comparing AS reports from previous years with the report set for the present year's students, it is possible to extract some qualitative results.Specifically, students have shown: • a substantial increase in key concept awareness, • a developing ability to correct faulty answers, • an increased attention to details, including the value of careful reading, • an emerging sense that knowledge is built up gradually through continuous practice, checking and evaluation.
The full details of a quantitative comparison between previous and present results will be published elsewhere.

Content Development
As stated in the introduction, recent advances in the cognitive sciences indicate that knowledge is obtained as complex structures are built up in several brain areas.The building of structures is made possible by connecting to previous knowledge and using and checking new knowledge repeatedly.The ultimate proof of solid learning comes when learners are able to transfer learning, i.e., apply their knowledge in contexts substantially different from the original ones.
Traditional content teaching does not take this into account.Most science and technology textbooks have been written according to expert criteria.Powerful abstract constructs prevail over simple practical statements that connect to learners' experiences.Lessons are delivered in the classroom at a ridiculous speed, with little focus on difficult items.
Abstraction and speed combine to produce almost no solid learning.In response, several authors have proposed modifications to classic textbooks to accommodate, at least in part, the demands of cognitive science.In this regard, the inclusion of mathematical chapters in "Physical Chemistry, 11th Ed." (Atkins, de Paula and Keeler, 2017) is noteworthy.Similarly, a much greater effort was made by Eric Mazur, who wrote an entire first-year university Physics course following a scheme in which concepts were discussed first in terms of simple measurements and data analysis, and then in terms of mathematical derivations (Mazur, 2015).
To the best of our knowledge, no similar proposal exists for General Chemistry.A new proposal is therefore being put forward to correct the problem.New content is being designed to teach a specific chapter of the first-year general chemistry curriculum of the University of Barcelona's Bachelor's Degree in Chemistry from a new perspective: instead of focusing on the microscopic properties of atoms and their relation to macroscopic properties, the approach describes the relevant macroscopic properties first, followed by the differences in behaviour among different compounds, and only then offers a succinct discussion of the unifying microscopic view.
With this twist, the traditional reductionism of modern science is reversed for the sake of better pedagogy.It starts from properties and material behaviour that students may know readily from their common experience.Then it prompts relevant differences that indicate a possible connection between properties and composition, before finally providing an explanation.
A text was written and tested during the academic year 2016-2017.Two groups of upper secondary students completing studies in advanced chemistry on a student placement programme were selected (13 students from the school IES Moianès, and 38 students from the school IES Ramon Casas i Carbó, both schools in the province of Barcelona).The reason for the selection is that the subject is taught with nearly the same content and level and our experience could therefore be extended to upper secondary school courses.Students were organised into groups of 3-5 members and asked to work out the text content and activities in the classroom, with no previous work.They were also asked to read the text carefully and search the Internet for any additional material, including animations and videos.
The amount of material was designed to be covered in one week, the same length as the same lesson delivered with traditional methods.In order to test students' ability to cope with the subject exclusively in the classroom, no homework was assigned.The results appear in Figure 2.They correspond to a quiz taken by all students after the lesson was worked out in the classroom, including questions of the same difficulty as in previous years.Student scores are notably higher than the results for the same lesson in previous years, with the main reason being increased student engagement.Students and teachers alike were enthusiastic about the new methodology and subject content, according to a survey they completed after the active learning work in the classroom.
Our overall conclusion is that the combined synchronous flipped classroom plus adapted content methodology is a valid approach for the teaching of AP high school and first-year university Chemistry, producing superior results to traditional approaches.

Practical tips in real saber active learning classroom settings
The above methodology has now been implemented in one discipline, Physical Chemistry of Materials, a subject taught in the second year of the Bachelor's Degree in Materials Engineering at the University of Barcelona.Below are some practical aspects that must be considered to obtain proper results: • Activity Sheets (AS) must be done in the exact same order that text and questions are introduced.Even though it may seem obvious, there is a widespread practice among students to skip "theoretical" questions and go directly to "practical" (e.g., numerical) exercises.Instructor supervision should therefore ensure good, systematic reading practices, so that enough time is devoted to conceptual discussions.
• A similar situation happens with introductory texts, which may or may not include contextual presentations of a subject.Students regard these parts of AS as "superfluous" and a waste of time, mainly because they can easily cause students to run short of time.A solution to the problem is to break any text into pieces and insert questions.
• Students usually make serious mistakes, given their mathematical background.Since it is very important to detect such deficiencies, it has proven a good strategy for teachers to react to any basic mathematical question by starting from the lowest level that is reasonably possible.It is never redundant to repeat an explanation of correct mathematical procedures, even if students should already have the basis needed to perform them adequately.
• As stated, students work in groups to benefit from peer teaching, but they respond to exercises and deliver their answers individually because this is the core of the work they must do throughout the semester.It is very important, therefore, to identify groups where not all members are working actively.A practical approach is to force all group members to write down responses to AS questions individually and then start a discussion by comparing each member's answers.
• SABER's classroom dynamics involve continuous questions from student work groups.The analysis has already indicated how a teacher should respond to questions, but a very important feature is that students must provide the information source they have been using both in their questions and in their AS reports.In other words, all answers to AS questions should provide the complete list of bibliographical references used to build up the answers.The bottom line for this procedure is that answers come from a sufficiently thorough search of all the available information, and not from "student's free thinking", which is much easier to do and never the source of new learning.

Conclusions
The paper describes version 2.0 of the SABER methodology, using a synchronous flipped classroom.The original SABER method is a flipped classroom methodology where students do not perform any work in advance, but rather start each theme in the classroom under instructor supervision.The purpose is to provide assistance to students from their very first interaction with science-oriented, mathematics-grounded subjects, which are commonly regarded as particularly demanding because of the difficulties posed by their abstraction and language.
Version 2.0 incorporates two important new aspects.The first is rewarded mistake correction, i.e., an explicit revision of students' work, together with a comparison against solutions provided by the teacher.Students are asked to carry out an analysis of their error sources, which the teacher evaluates, and they are rewarded for analysing their error sources accurately.The procedure has proven to be a powerful way to build awareness of the continual need to understand and use concepts.
The second aspect is that version 2.0 has introduced and tested relevant modifications to how subject content is taught.The aim is for subject content to benefit from starting points that are much closer to students' existing knowledge, as well as a constructivist framework for the Laura Arévalo, Pablo Gamallo, Xavier Giménez.SABER 2.0 in STEM: Rewarded Correction and Subject Content -Active Learning… Universitat de Barcelona.Institut de Ciències de l'Educació Laura Arévalo, Pablo Gamallo, Xavier Giménez.SABER 2.0 in STEM: Rewarded Correction and Subject Content -Active Learning… Universitat de Barcelona.Institut de Ciències de l'Educació

Figure 1
Figure 1 Percentage of passing students, as a function of time, for the subject Computer Programming taught in the first year of the Bachelor's Degree in Chemistry at the University of Barcelona, Spain

Figure 2
Figure 2Score results for the quiz on the properties of solids.The scale is from 0 to 10