R7035-a methods & analysis of qualitative research

Assignment 2: Case Study

Read the following article:

Agostinho, S. (2004). Naturalistic inquiry in e-learning research. International Journal of Qualitative Methods, 4(1), Article 2. Retrieved Dec. 16, 2005, from
http://www.ualberta.ca/~iiqm/backissues/4_1/pdf/agostinho.pdf 

Read the article and respond to the following:

1. Critique the process of data analysis that was conducted in the original research, as described in the journal article.
2. In the original research upon which this article is based, the researcher indicated that dominant themes were identified, and the themes were clustered into categories. Critique the underlying inductive logic utilized to analyze the data.
3. Discuss the researcher’s rationale, as described in the journal article, of adopting a naturalistic inquiry paradigm with regard to the process of data collection and analysis.

All written assignments and responses should follow APA rules for attributing sources.

Assignment 2 Grading Criteria Maximum PointsCritique the process of data analysis.10Identified the dominant themes as  highlighted in the article.10Critique the underlying inductive logic used to analyze the data.10Discuss the researcher’s rationale of adopting a naturalistic inquiry.10Justified ideas and responses by using appropriate examples, material and references from texts, library, and online resource database (see LIBRARY link on left.).10Wrote in a clear, concise, and organized manner; demonstrated ethical scholarship in accurate representation and attribution of sources, displayed accurate spelling, grammar, and punctuation.15Total:65

*****M6A2 SAMPLE 

Case Study Critique

*********M6A2 SAMPLE 

Methods and Analysis of Qualitative Research

Introduction

            In 2004 Ms. Shirley Agostinho published an article which was titled “Naturalistic inquiry in e-learning research” In this paper the researcher Agostinho (2004) addresses how the naturalistic inquiry paradigm was used in an e-learning research study that looked at the use of the World Wide Web technology in higher education. The purpose of the paper is to review and critique the data analysis process, dominant themes presented, underlying logic and researcher’s rationale for adopting a naturalistic inquiry methodology.

Critiquing the data analysis process

            There are numerous approaches for analyzing qualitative data. According to Shilling (2006) content analysis is generally defined as a method of analyzing written, verbal or visual communication messages. Wildemuth and Zhang (2009) isolated eight steps which comprise the process of qualitative content analysis. The following table applies the eight step process to the data reflected in the researchers’ article.

Strengths

Weaknesses and limitations

1). Preparing the data

The researcher collected data through participant observations, interviews, questionnaire and reflexive journal.

Creswell (2009) identifies triangulation as the use of multiple sources and different methods for the express purpose of providing corroborating evidence in support of a theme or perspective.

Using the data three collection methods list above the credibility of the paper is increased through the triangulation of data.

The researcher has failed to include either a database or other analytic tool which would illustrate or support the data collected.

While not entirely fatal to the study, negative data, if any, should be disclosed and reflected in the analysis.

2). Defining the unit of analysis

The researcher clearly defined the context as “examining the interactions established among students and an instructor in an e-learning course delivered using World Wide Web and videoconferencing technologies”.

The strategy of inquiry was a collective case study. Two cases were selected for analysis. The cases consisted of two geographically separate classes of graduate students from the University of Wollongong

In the first case eight students met on campus, and six students met off campus (in Sydney, 80 km north of Wollongong). Both classes were held on the same evening for 3 hours over a semester of 14 weeks.

In the second case There were two geographically separate classes. Eleven students met on campus, and six students met off campus. The two classes were held on different evenings for 3 hours. During the 14-week semester, students attended eight class meetings and participated in asynchronous and synchronous online discussions during the non-meeting weeks.

Noting noted

3). Development of the categories and coding schemes

The researcher provided in tabular form a perspective of the process of inquiry.

The researcher stated citing Merriam (1988) that the data analysis technique used was abstractions into themes and categories. Additionally, the content technique was consistent with the work of Henri (1992).

Nothing noted

4). Testing coding scheme on a sample text

Citing the work of Denzin and Lincoln (2000) the researcher used “strategies of inquiry put paradigms of  interpretation into motion”.

Nothing noted

5). Coding text

As noted in step 3 above the researcher presented in tabular form a perspective of the process of inquiry (see Table 1). This table provides the underlying philosophy.

Citing Creswell (1998) the researcher noted “Qualitative researchers approach their studies with a certain paradigm or worldview, a basic set of beliefs or assumptions that guide their inquiries. These assumptions are related to the nature of reality (the ontology issue), the relationship of the researcher to that being researched (the epistemological issue), the role of values in a study (the axiological issue), and the process of research (the methodological issue)”

Nothing noted

6). Assessing coding consistency

The researcher noted that from the data collected and analyzed from the first case the role of the observer as well as other variables were modified.

Nothing noted

7). Drawing conclusions from coded data

As noted in step 2 the strategy of inquiry was a collective case study.

According to Creswell (2009) case studies are a strategy of inquiry in which the researcher explores in depth a program, event, activity or process within a bounded system.

The researcher used two cases which fits within the case study parameters. The researcher further drew conclusions from information obtained from the analysis of the case data.

Nothing noted

8). Reporting methods and findings

The researcher stated that “data analysis, which was conducted during and after data collection, involved the identification of dominant themes and the clustering of themes into categories (Merriam, 1988). Content analysis, based on the framework presented by Henri (1992), was an additional technique used to deconstruct what occurred in the online environment when synchronous and asynchronous discussions were held during both implementations of the course”.

There are two areas of concern.

The first is the lack of negative case analysis. While the researcher has claimed that it was applied it was omitted from the published study.

The second is the accessibility to an audit trail. In absence of a database or other description of the data, the researcher’s affirmation that the data was compiled, saved and stored in an electronic media form, there is an apparent lack of transparency.

            According to Bowen (2010) qualitative research involves the systematic collection, organization and analysis of largely textual material. The quality or validity of qualitative research is judged and evaluated using a set of generally accepted criteria. As a matter of data honesty Creswell (2009) noted that negative data should be used as a meaning of understanding or hypothesis refinement and not discounted by the process. The credibility of qualitative research lies within the successful validation of the study.  Qualitative researchers as well as users judge the credibility of qualitative research by how the research demonstrates the application of the criteria. Trustworthiness is a composite term which encompasses a variety of other terms which include but are not limited to credibility, dependability, transferability and believability. With the exception of the minor weaknesses noted the researcher has demonstrated academic integrity in the article.

Dominant themes in the article

            A detailed review of the study indicated that the themes were not obvious or stated. This article is centered on the research process and not any measurable outcomes. The conclusion is a justification for the research. The only theme that is apparent from the articles is based on observation that “technology, when used to its best advantage, helps reshape roles for teachers and learners and encourages new and different types of interactions in the classroom”. The study fails to conclude on the how’s and why’s. In order to ascertain the themes additional research would need to be conducted by the reviewer.

Critiquing the underlying logic used to analyze the data

            According to Zhang et al. (2009), “Qualitative content analysis involves a process designed to condense raw data into categories or themes based on valid inference and interpretation. This process uses inductive reasoning, by which themes and categories emerge from the data through the researcher’s careful examination and constant comparison”. As noted above this study was a collective case study where data analysis was conducted, during and after data collection for the purpose of identifying dominant themes and clustering of the themes into categories.

Creswell (2009) identifies triangulation as the use of multiple sources and different methods for the express purpose of providing corroborating evidence in support of a theme or perspective. As noted above the researcher collected data through participant observations, interviews, questionnaire and kept a reflexive journal. Considering the methodologies used and the data controls it is believed that the inductive logic used is appropriate.

Researchers’ rationale for adopting a naturalistic inquiry

            The researcher followed three quality criterions; appropriateness, rigor, and usefulness, in establishing the rationale of the research design. Citing Lincoln and Guba (1985), the researcher contends that there must be an agreement or logical fit between what is being studied and method of inquiry used in order for the study to produce meaningful results. The researcher evidenced the fit of which adhered the study to a naturalistic inquiry methodology.

In reviewing the rigor criteria the researcher discussed rigor as cited in Creswell (1998) and Merriam (1988) as trustworthiness and authenticity. Trustworthiness is a composite term which encompasses a variety of other terms which include but are not limited to credibility, dependability, transferability and believability. The researcher considered the verification procedures necessary to include prolonged engagement, persistent observation, triangulation, peer debriefing, negative case analysis, member checks thick description and accessibility and audit trail. The researcher addressed usefulness in terms of understandability, the need for the study, and how the results contribute to the existing body of knowledge in the area of study.

Conclusion

            The article was prepared as a presentation of a dissertation framework. As such the focus of the article was to illustrate how methodological issues pertaining to naturalistic inquiry were addressed and justified to represent a rigorous research approach rather than presenting the results of a research study (Agostinho, 2004). The study was thorough, well written and provided a platform for judging the quality of naturalistic inquiry research. The few noted weakness while not fatal to the article would have added to the completeness and added clarity to the study.

References

Agostinho, S., (2004). Naturalistic inquiry in e-learning research, International Journal of  Qualitative Methods, 4(1), Article 2, Retrieved Dec. 16, 2005, from

http://www.ualberta.ca/~iiqm/backissues/4_1/pdf/agostinho.pdf

Bowen, G., (2010). From qualitative dissertation to quality articles: Seven lessons learned, The     Qualitative Report, 15(4) 864-901

Creswell, J., (1998: 2007). Qualitative inquiry and research design: Choosing among five approaches (2nd). Thousand Oaks, CA: Sage

Creswell, J., (2009). Research design: qualitative, quantitative and mixed methods approaches

            (3rd), Thousand Oaks, CA: Sage

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions, and emerging   confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research           (2nd ed., pp. 163-188). Thousand Oaks, CA: Sage.

Henri, F., (1992). Computer conferencing and content analysis, In A. Kaye (Ed.), Collaborative   learning through computer conferencing: The Najaden papers (pp. 117-136). Berlin: Springer-Verlag

Merriam, S., (1988). Case study research in education: A qualitative approach. San Francisco:       Jossey-Bass.

Zhang, Y., & Wildemuth, B., (2009). Qualitative analysis of content:  retrieved from             http://ils.unc.edu/~yanz/Content_analysis.html