InContext: Learning Comparison of Mexican and Colombian Students Using a Mobile Application

This paper, co-written by Claudia Lerma, María Leticia Palacios, Tania Lucía Cobos and Genaro Rebolledo, was published in the journal Pixel-Bit No. 67 published by the University of Seville (Spain). The pdf version of the same, in Spanish and English, includes a bibliography, footnotes and tables.

InContext: Learning Comparison of Mexican and Colombian Students Using a Mobile Application

1. Introduction

Educating in a social environment where most communication is mediated by technology, old and new, requires attention to media education and the ability to handle all aspects of interacting with them (Mateus et al., 2019). It is essential to develop media competencies in this new environment to interact in the changing media ecosystem (Mateus et al., 2019) and the new scenarios left by the pandemic in the social environment. Multiple digital educational spaces and access to content through mobile devices facilitate the development of new learning in students accustomed to hyperconnected living from a very early age (Gil, 2019).

In this way, education acquires broad relevance because it must train to acquire these new competencies among the citizens integrating into the current and future society, appropriating an extraordinarily transcendent educational system (Díaz-García et al., 2020). As Sales et al. (2020) mentioned, technology can be a critical enabler, but critical thinking is essential, allowing students to advance from personal autonomy. From stages before higher education, the continued use of technology configures habits and technical skills and a predisposition to digital activities, but this is not accompanied by a solid base around the use, management, and production of information. Students come to the university with a deep need to improve their critical reflection.

In this context, learning must facilitate restructuring, mixing, and finding new connections; it is necessary to have competencies, which can be developed and built from the internal motivations of each individual and are defined as capacities that a person uses to face, dispose of, act on, or mobilize a set of cognitive resources, perception, and evaluation techniques to solve a complex situation (Arras-Vota et al., 2017). Thus, the student values the impact of information and communication technologies in developing learning strategies and facilitating academic tasks, which can become one of the keys to academic success (García-Valcárcel & Tejedor Tejedor, 2017).

This work has two objectives. The first is to validate an educational technology that has proven helpful in improving metacognitive thinking and developing learning strategies for students to exercise the competencies they need in the academic field and their professional future (Lerma-Noriega et al., 2020), i.e., the thinking that regulates the learning processes. Hence, the student is aware of what is necessary to do at all times (Panadero and Alonso- Tapia, 2014). The second objective of this work is to present how the InContext tool supported improving meta-cognitive thinking in a cohort of Latin American students different from the one originally used to validate it (Lerma-Noriega et al., 2020).

To understand how the innovation impacts learning, we must see the use of technological materials to support the pedagogy of higher education institutions began in 1992, defined as “interactive tools that support the learning of specific concepts by improving the cognitive processes of students” (Guerrero Posadas & García Orozco, 2016).

For García-Valcárcel and Tejedor Tejedor (2017), educational technologies promote autonomous learning, collaborative work, creative thinking, problem-solving strategies, interactive work, the ability to argue orally and written, and the creation of non-linguistic representations of what students learn.

For Urbina Nájera (2019), the learning objects must be matched with the competency-based model that aims to collect evidence on academic performance to ensure that the knowledge, skills, abilities, attitudes, and values that allow students to solve contextual problems in their daily lives and throughout life have been obtained. These tools are new determinants of academic performance since they affect students’ work at different levels and in different ways (García-Martín & Cantón-Mayo, 2019).

It can then be thought that technologies should allow students to enrich learning processes, achieving greater depth and significance in the knowledge acquired and better attitudes towards study. In addition, technologies can enhance greater autonomy and self-management of learning processes, developing cognitive and metacognitive and instrumental skills that allow them to continue learning permanently (García-Valcárcel & Tejedor Tejedor, 2017).

It seems clear that the use of technology within the classroom is an unquestionable reality, and its use (for teaching purposes) correctly and successfully depends on a set of variables that some authors have already begun to identify, such as the type of study, the characteristics of the student, or the teaching profile capable of implementing said technology (Carcelén et al., 2019).

Media literacy must necessarily address critical issues of our time, such as the phenomena that affect personal autonomy and decision-making capacity and promote the formation of a critical awareness of the new scenarios created by the media and a broad reflection on their characteristics (Mesquita-Romero et al., 2022).

In some areas incited by technology in academia, self-regulated learning is crucial. It is defined as the ability to act independently and proactively to learn, i.e., the ability that an individual has to know how to meet the learning objectives, what needs to be done, and how to act to achieve the goals (Dabbagh & Kitsantas, 2012 in Tur et al., 2021). The positive aspect is that this favors the learner taking control of the process by assuming responsibility and self-regulation of their learning (Tur et al., 2021).

Previous research has shown that students who receive training in self-regulated learning (SRL) strategies are more engaged in performing tasks and achieve better academic outcomes (Cerezo et al., 2019). This is based on the first SRL models in which different cognitive patterns were considered to be acquired through training and social interactions; thus, students mastered various tasks (Zimmerman, 2008 in Cerezo et al., 2019).

The use of mobile applications that strengthen competencies and skills in education has been observed in various studies at all academic levels. In primary education, Tavares et al. (2021) developed a mobile app to promote scientific competencies and self-regulation of learning. The authors point out that adopting resources such as games and animations to promote student engagement helps acquire scientific competencies.

In a study conducted with high school students, who used different apps to read texts, it was found that reading comprehension was higher when they used apps perceived as easier to use, more functional, and more enjoyable. While students could annotate directly in the apps to make reading easier, no significant association was found between the number of annotations and text comprehension (Léger et al., 2019).

Another study used Bingo and Socrative in a mobile application with undergraduate students and found that while Bingo has positive effects on learning motivation, Socrative allows knowledge sharing and critical thinking to develop (Wei-Lun Chang & Yu-chu Yeh, 2021). On the other hand, to review whether a mobile application can be used to improve communication and learning outside of class, Tan et al. (2020) developed research in which university students placed their academic questions, comments, and votes, achieving high participation with this tool in the exercise.

There are many mobile applications for educational purposes, so Caihua et al. (2021) presented a technique to evaluate apps using user feedback based on feelings expressed on digital social media and sites for apps, allowing the teacher to select the most suitable for academic purposes.

This growth of apps is also observed in physical and mental health (Torous et al. 2018), so it is vital that educators in these areas know the technological landscape and can evaluate existing digital resources for ethical and safe use. Similarly, there is a need to evaluate apps in art education (Gómez-Redondo et al., 2020).

In other areas of more procedural learning, such as dance, mobile technology has also been used to its advantage, as shown by Hsia and Hwang (2021). Conducted with 266 university students in two groups using mobile technology to learn choreography routines, the research produced evidence that technology improved the development of dance competency. The study by Long and Nie (2021), who used mobile applications with interactive software to improve students’ interest in learning and understanding of sports education, also significantly increased learning.

Finally, in the study by Lerma-Noriega et al. (2020), a first approach was made with the InContext app, applied to a population of Mexican students, and it was concluded that its use improved critical thinking, data search, cognitive self-regulation, and metacognitive thinking. These results are favorable because they suggest the application’s usefulness in the academic field. However, there is a need to extrapolate these results to other Latin American populations, which is the objective of this article.

According to Barrero (2007), metacognitive processes occur in people in a state of self-knowledge and self-regulation with the professional functions developed in their work environment. In learning situations, the teacher who wishes to improve the students’ meta-cognitive skills becomes a mediator with a catalytic function not located at the head of the classroom but next to the student, becoming a mirror that reflects their personal experiences with the learning objectives.

Pintrich and De Groot (2000) noted that students who believe they are capable engage in more metacognition, use more cognitive strategies, and are more likely to persist in a task than those who do not believe they can perform the task.

For their part, Pintrich et al. (1991) point out that cognitive skills related to the text writing process include:

1. Critical thinking: focuses on the degree to which the student uses their previous knowledge in new situations to make critical assessments, solve problems, or make decisions.

2. Elaboration of texts: allows evaluating elaboration strategies such as summarizing, paraphrasing, and creating analogies, among others.

3. Metacognitive self-regulation: comprises items related to the awareness, knowledge, and control that the student has over his cognition.

4. Effort regulation: highlights the ability to maintain effort and attention in the face of distractions or uninteresting or complex tasks.

5. Organization: refers to using information organization strategies, such as pointing out concepts in a text, structuring them in diagrams or concept maps, and extracting the main ideas.

6. Seeking help: measures the willingness to ask peers or teachers for help with a problem.

MSLQ (Pintrich et al. 1991) is a scale that allows measuring the skills that students possess with tasks that involve elaborating text. Different technologies and their contribution to learning in various areas were mentioned in the previous paragraphs. In addition, the importance of metacognition and the skills that are necessary to construct academic texts were mentioned.

This research analyzes explicitly the degree to which the InContext technology encourages the acquisition of structural text skills, including the degree of metacognition (Item 3). This article aims to extend the research of Lerma-Noriega et al. (2020) to find out how generalizable these findings are in a similar context but in another Spanish-speaking country, namely, Colombia. The research question that guides this work is:

PI 1. Does using the InContext mobile app improve the cognitive skills of university students in two Latin American countries?

In particular, the present research used the Arellano scale (2012) and started from the following hypotheses, testing them with the method indicated and how they manifested themselves in Mexican and Colombian students:

H1 Using the InContext app improves the ability to produce texts.

H2 The InContext app improves the organization of bibliographic material to construct texts.

H3 Using the InContext app improves students’ critical thinking.

H4 The use of the InContext app improves cognitive self-regulation in text construction.

H5 The InContext app promotes the regulation of effort and attention.

H6 Using the InContext app promotes the search for help.

2. Methodology

The need to use tools that facilitate the collection and creation of information led to the development of the InContext app (Lerma-Noriega et al., 2020), allowing students to have different formats at hand to organize their writing practices and collect information inside and outside the classroom.

The InContext application consists of specially designed software that adds several basic templates for news and opinion columns, research reports, and even basic editing. Students can write at each essential point of writing, add the required multimedia material, and send it directly to their email or the cloud.

By adding information to the InContext app, students continue to write text in their style without worrying about missing specific points. Also, they leave the material in the cloud so that it can be consulted when the teacher or they need it. The singular value of doing this is the indication that every professional should have a file for data comparison if necessary to facilitate the classification and cataloging of the material.

Users of the app choose from 16 news options, 4 research options, an opinion column option, and a basic writing option. They can attach linked photos, videos, audio, and text documents. InContext allows generating PDF reports that can be printed or sent electronically to teachers.

The tool explores new ways to submit content and facilitates more flexible learning, as students can work at their own pace and indicates, with asterisks, which fields are mandatory, such as the title or source. In turn, the teacher receives this information and can verify that all fields are covered and assign grades if they deem it appropriate.

To know if using the technology promoted the development of cognitive skills, we used the Motivated Strategies and Learning Questionnaire (MSLQ) scale (Pintrich et al., 1991) in its Spanish version (Ramírez-Dorantes, M., 2016) to measure the cognitive skills related to the six research hypotheses.

The questionnaire consisted of 81 learning-related items divided into two parts: the first focused on motivation and the second on learning strategies. This work used items directly related to measuring the impact of learning strategies per the established hypotheses.

The research was carried out in two stages. The pre-test sample had 30 students from the Mexican university and 30 from the Colombian university related to the students enrolled in the four courses. In the second stage post-test there were 14 Mexican students and 26 Colombians; there were less participants in the survey due to a lower assistance to class. The male and female students in a private Mexican university were enrolled in the courses related to journalism and research. In the private Colombian university, the young men and women studied in two courses related to journalism. The data for this research was chosen by the researchers due to them lecturing the previously mentioned courses and students from these courses who were invited to participate in the exercise and answer a questionnaire voluntarily between September and November 2021.

We applied both a pre-test and a post-test to the students at both universities so they could perform the relevant exercises in each of the courses and take the Motivated Strategies and Learning Questionnaire MSLQ scale (Pintrich et al., 1991) in its Spanish version. From this questionnaire, we used questions from section B of the MSLQ scale consisting of 50 items oriented to the Learning Strategy. Students responded to each item using a seven-option Likert scale, where “Nothing True in Me” corresponded to 1, and “Totally True in Me” corresponded to 7. The questionnaire was run using Qualtrics software.

The practice was carried out in two sessions. During the first, all students were asked to perform two activities: answer the scale questionnaire (pre-test) and then do an informational, opinion, research, or basic writing exercise from a case, using the corresponding InContext template. In the second stage, they were invited to answer the post-test scale questionnaire.

The exercise in the news journalism course included preparing a draft of text writing. The writing had to have an approximate length of 500 words and should contain two photos with captions, the title, and a link to a video made by the student.

In the opinion journalism course, students had to read five news items related to the topic selected for the opinion column and use the respective InContext form, which required the topic, summary, focus, arguments, and the news source URLs.

In another research exercise, students were asked to indicate a research question and write down two specific objectives, a justification, a title, and three keywords. They also used the InContext platform by selecting the Research option for their exercise.

3. Results

The following numerical data were analyzed to determine if the application helped promote learning strategies and respond to the questions and research hypotheses. A first statistical analysis contrasted the pre-test results applied in the cohorts of both universities (see Table 1).

Table 1. Pre-test descriptive statistics. See in the pdf.

Particularly in the Critical Thinking variable (4.2 for Mexico and 4.7 for Colombia), it is observed that Mexican students had a lower performance than Colombian students. This is a significant difference between the two country’s cohorts, and while the two populations are not entirely homogeneous, they are similar because there are no significant statistical differences between the other five variables (see Table 2). The populations, therefore, showed less diversity as a whole. These two student populations also performed primarily positive on the 1 to 7 scale.

Table 2. Test for homogeneity between the two populations. See in the pdf.

To analyze information and respond to the hypotheses, we conducted a statistical analysis with an intra-subject design to know if using the tool influenced students’ perceptions concerning the variables of interest. We used only data from students who answered both the pre-test and the post-test in both countries for this analysis. Because the measurements were made at different times of the semester, some students missed one of the two work sessions, and this condition contemplated only students who coincided at both times. It reduced the number of cases to only 40 students. Table 3 presents the descriptive statistics about the variables of interest.

Table 3. Descriptive Statistics for the hypotheses tests for both countries. See in the pdf.

Table 4 shows the statistical tests associated with the variables of our study. It can be observed that there is a significant difference between two variables: Elaboration (.016) and Self-Regulation of Knowledge (.001), which suggests that this difference is due to the use of the InContext tool.

Table 4. Non-parametric statistics for the hypotheses tests. See in the pdf.

4. Discussion

From the statistics analyzed, we can highlight that the application favors developing some of the learning strategies measured in this work: Elaboration of Texts and Self-regulation of Knowledge. The progress observed in the texting variables implies that the InContext app helps students’ work. In addition, the self-regulation of knowledge variable also improved, indicating that the tool is essential because it helps students know what their problems are and how to resolve them. From the educational point of view, the tool is very relevant because it may help develop the metacognitive process.

The study’s findings add to the existing literature on the use of technologies and their influence on academic performance. The study represents a first step in research on the academic effects of various technological tools on university students.

Among the metacognitive strategies that should be considered for the students, we suggest the following: planning (students direct and control their behavior), regulation (the student’s ability to follow the plan drawn up and check its effectiveness), and evaluation (verifying the results of the learning process) (García-Valcárcel & Tejedor Tejedor, 2017). In addition, as pointed out by Carcelén et al. (2019), the students with the best academic success recognize that using ICT significantly supports their learning process, especially in preparing for work, organizing academic activities, reviewing tasks, collaborating with colleagues, and searching for resources.

Regarding the research question (Does the use of the InContext mobile application improve the cognitive skills of university students?), it can be said that not all cognitive skills improved. Still, there was an improvement in the elaboration of texts and self-regulation, which are vital in developing competencies associated with metacognitive thinking. We also observed that using the InContext app favors text production because it helped students improve and be more expeditious. Finally, this research validated the results obtained in previous research (Lerma-Noriega, 2020), obtaining results in a new cohort of Mexican and Colombian students similar to those previously reported.

5. Conclusions

The importance of learning strategies is evident in the university environment, where a new student-focused conception has been imposed involving the activity that students develop in the learning process (Aizpurua, 2018). Hence, we see the relevance of conducting research where these educational innovation methods are visualized. When analyzing the comparison results between students and using the application, we observe its impact on metacognition.

These applications also support the teacher’s work as part of an educational innovation that strengthens the learning project of humanities courses, especially in fundamental aspects such as the elaboration of texts. In this study, self-regulation of knowledge showed a significant statistical difference; it is also a technique that strengthens the skills of the future professional.

The objective of creating this application was achieved, and the study results have the same tendency as the original research of Lerma-Noriega et al. (2020), which gave rise to this one. The sample populations in both countries responded in an almost homogeneous way, as perceived in Table 1.

Do not lose sight of the fact that searching for information on the internet involves selecting suitable sources and subsequently extracting, organizing, and integrating the information obtained, helping students acquire problem-solving skills. (García-Martín & Cantón-Mayo, 2019). Knowledge does not arrive by itself or appear spontaneously.

Educational institutions must strive to reduce the digital divide and bet on an interactive learning style, making students protagonists in the collective construction of knowledge, responding to their digital and participatory demands (Gil, 2019). The student competent in using information can determine the nature and level of the information they need (Arras- Vota et al., 2017).

Another highlight in this study is that the InContext app seems to facilitate and promote the self-regulation of knowledge and the elaboration of texts. Panadero and Alonso-Tapia (2014) defined self-regulation as “the control that the subject performs over their thoughts, actions, emotions, and motivation through personal strategies to achieve their objectives.” On the one hand, the definition includes “thought control,” that is, the cognitive component of self-regulation, also called metacognition, based on the strategic control of cognitive processes, something that has been sought since the original research that gave rise to this study (Lerma-Noriega et al. 2020).

Finally, it should be noted that this research can lead to developing investigations in larger samples and making comparisons with other Latin American populations to corroborate that strengthening the use of tools for educational innovations can lead to a promising future for word professionals.