(I am posting this reflection on a research article as a sample for students in my class. If this is useful or odd to others who may find it, so be it. Not like this blog gets a lot of traffic...)
This paper will provide a critical analysis of “The Land of Confusion? High School Students and Their Use of the World Wide Web for Research” by Michael Lorenzen which was published in Research Strategies 18 (2001), 151-163. As I am the author of the article being analyzed, I will use the first person to avoid awkwardness in writing about research.
In the Spring Semester of 2000, I took a doctoral level course (EAD 955B) on qualitative research methods at Michigan State University. As a requirement for the course, I conducted a study of local high school students to determine how they were using the Web for research. I received a 4.0 on the paper (and in the course as a whole) and after some rewriting I submitted it to the journal Research Strategies. The referees sent it back to me once for revision but it was published in 2001. I was very pleased with this result as Research Strategies was the leading journal for library instruction and information literacy research at that time.
Have I Described My Methodology and Theory Base?
Beginning a review of my study, it is evident that I was not clear about what exactly I was doing. I note that the study was the result of me having interviewed students. However, at no point did I describe that this is a case study. As such, I make no attempt to explain why a case study method was selected and why it is appropriate.
However, I did attempt to justify why I used a qualitative study. One of the peer reviewers voted to reject the paper as my sample size was too small and did not use any quantitative methods. The editor realized that the reviewer did not understand qualitative research and she asked me to add a paragraph explaining and justifying this approach. I wrote, “The methodology used for the study was qualitative. It differs from most research in the library science field, which tends to use a quantitative model. Many researchers have trouble accepting qualitative data as valid” (p. 156). I then detailed why this approach was valid and I think this helped to frame the study a bit for readers who are not used to qualitative research.
I believe the study is well grounded in a theory. I used Perry’s Scheme of Student Development to justify my methodology and to explain what I was expecting to find in the study. I liked the idea of dualism and multiplicity as rationales for student information seeking behavior. I believe I adequately explained Perry’s theory and referenced it properly throughout the entire article.
Did I Articulate a Set of Research Questions?
Actually, this study had only one research question. It was looking to see how high school students used the Web for research. This is clearly stated early and often in the study. That being noted, it probably would have been useful for me to have broken this one question down into some smaller research goals. I could have added several additional research questions to this which may have allowed me to probe the overarching question a little better. However, the question being examined is well stated, open ended, and clearly answerable with the method selected for the research.
Did I Identify a Research Context and Explain How Access Would be Attained?
I did not do a good job explaining this at all. I believe that the research was indeed doable and that the context selected was appropriate. However, I did not explain this. I did make it clear that the study was conducted at a high school but I make no attempt to justify why this is the best setting for the research. There are good reasons for using a high school to study high school students but I was not clear enough about this.
Further, I do not describe how I received permission to do this study at the high school. I knew the teacher who allowed me to interview her students and that is how I gained access. I also did not describe that the teacher had brought her class to the Michigan State University Library during the term and this may have influenced the results some. Conversely, I did an adequate job explaining how I gained consent from subjects and/or their parents.
Did I Describe Procedures for Selecting Participants?
I completely fail to mention at all how the participants were selected. I did not explain how the subjects were selected and invited. I do reference how many subjects participated but I do not indicate why I used the number I did and how much time was spent with them. I do note that some of the subjects had difficulty in answering some questions and gave brief responses but I do not really report how much the participants were involved. If I was to rewrite this study, this entire area would be described much better!
Did I Identify a Data Collection Method and Justify it?
This is another area that I did both right and wrong. I clearly indicated that I interviewed students. I provided the number of interviews conducted and provided a list of questions. These questions also relevant to the project and I think it would obvious to most readers that they would help to answer the research question. However, I make no real attempt to justify using interviews or any of the questions.
Did I Describe Data Analysis Procedures Used?
Incredibly, I did not do this! I looked over my findings section where I discovered that I just rattled off what I found with no explanation of how I got there. I am a little sheepish about this now. As such, there is no description of what I did with non-matching interviews that went against my findings. There is no way to determine if the data analysis method made sense as I did not list the data analysis method.
How Could I have Improved?
Looking back at this paper, I see many ways I could have improved it. To being with, I would have added a rationale for why I used a case study. I would have listed one or two more research questions to study as well beyond the one I pursued. I would have explained why I choose a high school as the research setting and I would have listed how I gained access. I would have explained how I selected the students who participated as subjects. I also would have noted my data analysis method.
This paper was my first attempt at qualitative research. I was very fortunate to have had it published. Had this been submitted to an education journal instead of a library journal, it probably would have been rejected. My methodology section is only three paragraphs long. However, I am happy with my use of theory to ground the research and I think my findings have held up well over the last five years. I am happy to see people citing this paper in their own research.