Regardless of methodology, almost all researchers engage in a number of similar components in conducting their research. All of these components include, purpose and question of the research, research approach and methods, population and sample of subjects, data collection (e.g. tests or other measuring instruments, a description of procedures to be followed), a description of intended data analysis and interpretation, and conclusions. Those components are also considered as the key elements in doing critical analysis of research papers based on methodological perspective. In this text, I shall present my own critical analysis from methodological perspective of Lederman’s research paper titled Teachers’ understanding of the nature of science and classroom practice: factors that facilitate or impede the relationship.
Purpose and Question of the Research
As the researcher clearly stated in the abstract of his paper, there are two purposes of the research, that is:
- To investigate the relationship of teachers’ understanding of the nature of science and classroom practice
- To delineate factors that facilitate or impede a relationship.
The justification of what does it mean the nature of science is quite extensive and clear, though in a certain sense, it is more to students than to the teachers. In line with his research goals, the author also clearly stated his research questions in the end of the introduction. There are two research questions:
- Do teachers’ understandings of the nature of science influence classroom practice?
- What factors facilitate or impede the influence of teachers’ understandings on classroom practice?
It is quite acceptable that the research questions was formulated in a good sentence and can be understood by general readers.
Research Approach & Methods
It is clearly presented that the researcher employed multiple case study approach in conducting his research. This research approach was followed by several methods in order to collect the data. All of those methods include classroom observations, open-ended questionnaire, semi structured and structured interviews, and documents (instructional plans and materials). In my perspective, it is one of good sides of the research in which it employed many different methods during the investigation.
Population and Sample
According to the paper, the author forgets to mention the population of the research. In my point of view, the population of research seems a crucial thing to put in, since the scope of research is quite big based on its title. Writing the title in that way, without limitations (e.g. the area or region where the research has been conducted) seems to be less credible in relation to its goals.
Regarding to its sample, the researcher clearly stated that there were five high school biology teachers with different experience from 2 to 15 years from different areas (urban and rural locales) selected as a sample for the research. The method of selection is not clearly mentioned. The researcher only stated that there is initial reason for selection of those particular subjects for participating in the investigation. He said that the selection of those teachers was based on the researcher’s close working and personal relationship and the teachers’ possession of a view of the nature of science consistent with that advocated in the current reforms. Although, the sampling method is not clearly presented, I argue, based on those explanations that the researcher seems to use a convenience sampling method. It implies that for those reasons aforementioned, this research cannot be categorized as a good research. Furthermore, the close working relationship between the researcher and the subjects can also interfere with the validity of the research.
He also said that there were 10 students selected randomly in each teacher’s observed class as a source for additional data for creating teacher profiles. However, the researcher does not tell more about the kind of sampling method that he used in selecting those students randomly. The good thing is that the representativeness of this sample to its population was verified by the teachers (p. 921). Concerning the ecological generalizability of the research, the researcher seems to clearly state the nature of the environmental situations, the setting, under which a study takes place such as the background of the teachers and the students.
The researcher seems clearly mentioned almost all of the instruments that he used during the research such as open-ended questions (in open-ended questionnaire), interview schedules (semi-structured interviews at the beginning and at the end of the investigation) and observation forms (classroom observations).
He also clearly stated all the steps of data collection in his research, starting from the beginning (interviewing teachers 1 week before the beginning of the school year) until the end (interviewing a random sample of 10 students). By doing so, the readers can easily follow what the researcher has been done during the investigation. The other good side of the research is that the researcher provides a discussion of validity and reliability of the instruments which is usually common in qualitative research.
Data analysis and interpretation
Statistical analysis is not appropriate for this kind of study. As is usual in studies of this type, the results from various instruments are described. Generally, the researcher was analyzing the data of his research by using model of analytical induction. He also presents a subsequent step of analyzing the data, so that the readers can easily follow the chain of reasoning (trackability) from one step to the next step.
In relation to the interpretation, I find the results are quite impressive, particularly because he is often clear in stating “indicated” so that the readers can realize that this applies to many other statements as well.
Validity and reliability
Since the researcher employed a multiple case study research, I use the criteria of validity and reliability of qualitative research. In my point of view, the research much depends on the perspective of the researcher that implies a certain bias. The procedure for checking the validity and reliability of this research is as follows:
- Using a variety of instruments to collect the data
In my point of view, since the research is supported with number of different instruments, its validity is thereby enhanced. The questionnaire of the researcher, in particular, has been used and at least face validated elsewhere (p.920). Furthermore, the researcher also did a triangulation from many sources and types of data to construct a valid profile of the teacher’s beliefs and classroom practices.
- Conducting the data collections in a good order
The researcher clearly said that he did classroom observations and subsequent interviews without knowing the content of the completed questionnaire. Therefore, the researcher has a good reason to argue that there was no bias in the assessment of relationship between classroom practice and prior knowledge of teacher’s beliefs.
- Applying member checking
It is one of the good side of the research relating to its validity that the researcher did a member checking, asking the participants in the study (the teachers) to discuss their responses on the questionnaire, during the investigation. By doing so, the researcher can minimize misinterpretations when doing analysis of the research.
- Forgetting to apply the external audit or peer debriefing
As far as I know, based on the research paper, the researcher did the investigation and analysis by himself. Accordingly, he forgot to invite an individual outside of the study to review and evaluate the research findings such as the interview transcripts or the conclusions. At least, it is not explicitly mentioned in the paper.
- Observation is not clear
The researcher do not also stated about the use of both audio and video in conducting observation. In my point of view, it is very important to use both of them in doing observation, especially relating to the validity of the research.
- Observing the setting or situation of interest over a period of time
The other side of the research in relation to its reliability is that the length of an observation is very important in qualitative research. In my point of view, doing investigation for one full academic year is quite long and the researcher can derive a consistency over the time regarding to what he is seeing or hearing that can be a strong indication of reliability of the research.
This study illustrates both the richness of case study research and the difficulty of making firm conclusions. In many parts, the researcher put the citations from the participants in building the conclusions. He also derives remarks from combining several different sources of data (e.g. teacher’s interviews and questionnaires) and checked with the prior researches that have the same characteristics. Overall, it seems to me that the researcher is successful in answering his both research question in the conclusion.