Educational Research Without Numbers: A Debate
Here is what you need to know about the debate Educational Research Without Numbers:
Approach
Their approach is that neither Statistics nor Experimentation can answer key questions in education, such as:
- How do I discover the learning difficulties of my students?
- What do they owe?
- How does technology help learning?
- Why has a certain educational action been a success or a failure?
- So how do I investigate, how do I answer these questions? Do I renounce to Science, to the Scientific Method?
According to Nelson Medinilla, "it seems that any path of research with the scientific purpose of expanding our knowledge must have the objectivity of the experiment, the universality of mathematics. But thinking that way limits our ability to investigate issues where math and experiment are meaningless".
Process
It is a qualitative research that is carried out in nine of thirteen students that compose the class of the subject of operating systems, of the degree of Informatics of the Polytechnic University of Madrid. It aims to identify what are the intrinsic problems of the subject for it to be difficult (what are the difficulties). They work without initial hypotheses and without study variables. They are based on discovery and exploration.
Analysis Of Data
Through a program called Atlas.ti for the analysis of answers (collected in PDF) and by using Moodle, they follow the procedure of summarizing in short sentences each element of the course that the students detected as not properly understood. They then group these phrases into categories or concepts. In addition, they analyze the things that were well understood.
Results
Being open questions, the results that have been obtained are the groups of concepts that are more difficult to understand. One of them was the concept of "interruption". Being a term that outside the scope of the subject has a common meaning, it causes interference with the meaning that is given in operating systems.
Debate
It is proposed from the audience that there should be some objectives in the research (what do you want to know about the difficulties in learning this subject?). These difficulties would be directly asked to the students, in order to be able to launch initial hypotheses. Sonia Pamplona answers that this research could never have led to the result that has been obtained, because those are intrinsic to the subject. With the other procedure, only extrinsic causes would have been obtained. It is trying to investigate a problem that cannot be questioned directly to students or teachers. Through a series of indirect questions, the student has to infer things, they have to think and relate concepts, and it is then when they see whether they have understood it or not.
The objective of the subject at this level is not to learn to program, but to understand concepts. Therefore, the research has focused on the intrinsic causes of the subject, the contents. Extrinsic causes have been eliminated to focus research. Sonia Pamplona does not know the degree or weight in which one or other causes (intrinsic and extrinsic) influence the understanding of the subject.
The audience argues that if the objective is to improve the quality of the educational experience, so that people learn better, what should be sought in research is to maximize results. Sonia Pamplona answers that this investigation depends on the questions she put. She asks the audience the questions "What is the purpose of this investigation? Does it serve others?". The audience answers that if it serves to improve the subject, then it does. Another person replies that it surely will be useful, but then you have to ask yourself "Why? Does it serve to improve the didactic technique of content?". It is said that data serve according to what is done with them. Sonia asks if her results can be generalized and if she will find the same problem in other classrooms. The audience responds that she has now a new hypothesis to be validated with a larger sample. Nelson Medinilla believes that this is not necessarily the case.
It questions the methodology used to extract the concepts as a result of the students' answers. Sonia explains that she does this by highlighting words or phrases from these answers and assigning them a code. She then groups them by color. Color is a first grouping. The audience comments that what they find interesting about qualitative research is that it adds the interpretation layer, which adds value to the data. At the same time, it is a risk, but also allows to create a game environment from unstructured information to cover the concept. Personal interpretation generates a bias. Another person in the audience comments that, in his opinion, this is a poorly done qualitative analysis. Sonia argues that it relies on triangulation, because the results are not obtained from a single answer, but from many similar answers. It may come from a single student or from several students. A person of the audience comments that, for him, triangular is to apply different methods and to achieve the same result. Sonia responds that the triangulation she has applied is this: She has discussed the codes with the research team and has stated the investigator's biases, which increase because the researcher and the teacher are the same person. Nelson Medinilla asks the audience "What is the scientific method for you?". It is answered that for an investigation to be considered scientific, it must be replicable by other researchers, so that they obtain the same result, lasting in time (which does not depend on the moment of the investigation) and universal (which works equally in everywhere).
