How are quantitative and qualitative analysis similar, and how might they reasonably differ?

Posted on Updated on

This piece compares and contrasts two case studies to discuss the ontological and epistemological differences between quantitative and qualitative analysis. The case studies are related to drug use and mental health. This discussion incidentally covers some issues that arise in philosophy of science, but its primary purpose is to evaluate the two different paradigms in social research. This essay is produced for a politics assignment and can be found here. The quantitative study can be found here, and the qualitative study can be found here.

plato's cave

Plato’s Cave – showing a simple relationship between what can be known and how we know it.

Comparative Evaluation of Quantitative and Qualitative Research: Two Case Studies

I           Introduction

The divide between quantitative and qualitative social research (“research”) is both institutional and methodological.[1] Historically, researchers have tended to define themselves within one school, and elected to use one approach over another.[2] At a methodological level, Bryman states that quantitative and qualitative research has divergent underlying assumptions regarding method, methodology and perspectives about knowledge.[3]

The division between quantitative and qualitative research stems from a fundamentally different conception about the ontological nature of knowledge.[4] Quantitative research views knowledge as being objective and separate from the individuals who perceive this world. On the contrary, qualitative research views knowledge as being inseparable from the subjects which perceive it, hence its subjective nature.[5] The ontological nature of knowledge thus informs the epistemological approach adopted by each paradigm.[6]

Despite this fundamental difference, the goal of both methods is to produce knowledge, to discover the ‘truth’ about this world.[7] This means for both methods, the steps in the research cycle need to be sound. Accordingly, this paper will compare and contrast the ways which quantitative and qualitative research both have to meet similar criteria of research plausibility, authority and validity.

II         Summary of Case Studies

The Quantitative Report:[8] (Appendix 1)

The report is titled Sexual orientation and drug use in a longitudinal cohort study of US adolescents by Corliss et al., conducted by Elsevier and published in Addictive Behaviours. The report was a longitudinal study using data collected from 1999 to 2005. The method of collection was self-administered questionnaires. The method of analysis used was multivariable analysis using Poisson regression to determine the relative level of risk. The research question focused on ‘why minority sexual orientation are at disproportionate risk for drug use’, focusing on the existing research-gap of drug use over time and identifying the most vulnerable demographics.

The Qualitative Report:[9] (Appendix 2)

The report is titled A qualitative study of illicit and non-prescribed drug use amongst people with psychotic disorders by Charles and Weaver, conducted by Informa Healthcare and published in the Journal of Mental Health. The report is a cross-sectional qualitative study of 14 patients with psychosis who misuse drugs, using analytical induction and the exploratory qualitative method through a qualitative interview. The sample was drawn from a random sample from the ‘Community Mental Health Team’ survey.

III        Comparative Analysis

Social research can be seen an attempt to organise and interpret a posteriori knowledge in order for researchers to make accurate and meaningful claims or conclusions about the world.[10] These claims nonetheless will always come with a degree of uncertainty (as discussed later). This section will begin by looking at general differences, then a closer evaluation of their similarities.

A         General Differences

Bryman sees quantitative research as being preoccupied with ‘measurement, causality, generalisation and replication’ and issues such as reliability and validity.[11] Quantitative social research is often described as using numerical data, with a narrow hypothesis and research that is focused and conclusive, and drawing conclusions using deductive reasoning.[12] Quantitative research makes use of statistical models. It is often conducted ‘late’ in research once a theory has already emerged. The data sample size tend to be large. The research questions have to be specific and the method of collection is often impersonal. Data analysis and presentation usually produce tables and graphs. Quantitative research can thus be categorised as positivist.[13]

In contrast, qualitative social research has been described as focusing on credibility, transferability, dependability and confirmability.[14] This paradigm normally uses verbal data, with a broad hypothesis and the research is more exploratory and focuses on the wider picture. Due to this, qualitative research is often deployed ‘early’ to canvass the issues and develop a theory whereas quantitative data analysis is usually used to confirm certain aspects of a theory. Accordingly, qualitative data analysis can be seen as reasoning inductively (‘moving out from the data’) where the hypothesis emerges.[15] In contrast, deductive reasoning places the hypothesis first and frames the rest of the research (‘moving in from the data’). Instead of using impersonal questionnaires and surveys, qualitative research normally focuses on a small sample size such as focus groups and structured, in-depth interviews. The data may take the form of interviews, videos, or even objects. The researcher is likely to be subjectively immersed in the research process. Qualitative research is thus influenced by constructivism and interpretivism.[16]

IV        Specific Comparison

This section will look at the similarities and differences of more specific issues: including (1) establishing research plausibility and (2) establishing authority. Application of the case studies will be interwoven.

A         Research Plausibility

At the heart of research plausibility is hypothesis formulation and grounding the research in academic literature.[17] Research plausibility is the exercise of ‘determining whether a case study is worth conducting; in other words, it might mean determining whether the theory derived from the case makes sense’.[18] Neither paradigm exists in a vacuum and in seeking to contribute knowledge about the social world, therefore both research looks at gaps in the existing knowledge.

However, within this there are some reasonable differences:

For quantitative research, the research question is normally highly specific and is posed at the beginning of the research cycle.[19] It is true that the researcher may need to return to the hypothesis if preliminary attempts at data collection and/or data analysis shows the data  is inadequate in answering the hypothesis, but in principle the hypothesis does  frame the rest of the research cycle.

The relationship with academic literature is one where the research question either looks to (a) confirm an aspect of the theory, or (b) narrowly focus on the effects of additional variables. This approach is known as hypothetico-deductive.[20]

For qualitative research, the research question can be posed at any stage in the research cycle.[21] The research question is also broader. Instead of identifying a theory and narrowing in, qualitative research tends to leave the hypothesis open as not to limit (a) the data collection process or (b) the theory that develops. In contrast to quantitative research, where the question presupposes the analysis, qualitative research arguably engages the collection and analysis at the same time, and allows the research question to emerge.[22] Therefore, the qualitative research question does not merely confirm a subset of a theory as in quantitative research, but instead seeks to expand the theory.

Case Studies

In the quantitative report, the researchers explicitly frame the research question early. The report cites previous reports which ‘have found that minority sexual orientation youth are more likely than heterosexuals to report use of… illicit drugs’.[23] From the academic literature, the researchers identify a knowledge gap (‘outstanding questions remain’[24]) by looking more closely at the variables age and gender. The research hypothesis then frames the rest of the research.

In the qualitative report, the researchers also grounds the research question in existing academic literature. A knowledge gap was also identified, stating that the ‘reasons for [the] high prevalence of drug use remain poorly understood.’[25] It is difficult to know whether the hypothesis was formed and fixed from the start, but we can see that during data collection and management, the researchers used a ‘flexible topic guide that was informed by a literature review and refined progressively during fieldwork’.[26] This shows the focus of the research often shifts during qualitative research as more data is collected.

B         Authority, Standards and Methods

Authority can be seen as the rules or justification of each paradigm.[27] In this section we will evaluate the use of standards and methods of each method in order to be (logically) persuasive. Each method needs to follow sound logic, so that the results are reproducible, but also meaningful. In this context, the definition of ‘meaningful’ is specific.[28] It does not necessarily mean that the data has some kind of normative importance (although it may), it means that the data is processed and analysed in a way which can be understood, both in terms of accessibility and its implications.

Authority is a broad concept, in this paper we will focus on (1) controlling error and making valid inferences,[29] (2) data reduction[30] and (3) standardisation and transparency of method.[31]

  • Controlling for Error

The need to control for error exist for both paradigms, and occurs at multiple stages in the research cycle[32]. There is both sampling error during the initial stages of research, and non-sampling error such as observational, classification and specification error during latter stages such as data processing and inference.[33] In order to make accurate claims about the world, both models can be seen as managing that error.[34]

The biggest similarity between the two approaches is in the way they address sampling error. Sampling error can be minimised by using random samples, which are sometimes adjusted later to reflect the larger population.[35] In research where the research sample is difficult to identify, sometimes randomness is abandoned and instead a referral process is adopted.[36] This is more common in research involving small samples in niche topics, but it is not exclusively employed in qualitative research.

In quantitative research, the methods used to minimise (non-sampling) error is normally statistical.[37] These include calculating the variance, confidence interval and margin of error. In order to identify the ‘best fit’, a regression model might be used.[38] The result also needs to be derived from the data, and any relevant data must also be included (not omitted). This is sometimes called operationalization.[39] The researcher also needs to justify classifications made, and whether to merge data, or how to weight variables.[40] This is similar to coding common themes in qualitative research, because it involves the subjective value-judgment of the researcher. This is when the simple ‘words versus numbers’ dichotomy breaks down.[41] An example is how to create a quantitative scale of a subjective concept such as ‘wellbeing/happiness’.

In qualitative research, the strength of the approach is our ability to gain insight into the motive, preferences and values of the person (the qualitative data).[42] Instead of addressing margins of error, instead the researcher focuses on the reliability of the source during the process of data collection.[43] There does appear to be a reliability ‘paradox’ when it comes to qualitative research, because the person is the data and at face value there appears to be a tension if  the researcher questions or interrogates the ‘uncertainty’ of the data that is supposed to be accepted as being the subjective but accurate experiences of the subject. This issue is complex, and is often where error arises. The researcher needs to maintain a level of ‘objectivity’ when selecting the sample, collecting the data, coding the data and interpreting the data.[44] Bryman compares this to the ‘goodness of fit’ in quantitative research – the researcher needs to ask questions such as ‘Do I believe your story’ and ‘Have you ignored other things that could change the picture’?[45] To some extent, qualitative researchers need to treat the data as if it is fallible, similar to quantitative research. The researcher also needs to classify the data in an objective way and not leave out data or inferences that is relevant. This is comparable to operationalization in quantitative research – the ‘empirical variables [need] to fit the theoretical concepts.’[46] Furthermore, the logic of inference for both approaches need to be sound.

Case Studies

Both reports used random samples to minimise sampling error. In the quantitative report, the researchers classified the data during data processing (variable-definition), such as grouping the subjects into social categories such as ‘completely’ or ‘mostly’ homosexual.[47] In the qualitative report, the researchers similarly needed to assign ‘thematic descriptive codes’ and ‘subcategories and conceptual codes’ to the transcripts.[48] These are domains where classification error could arise. In analysing the data, the researchers employed a ‘Poisson regression’ analysis to find the ‘best fit’.[49] In the same way, we can say that the researchers in the qualitative report also had to find trends by employing ‘analytic induction…. in the primary analysis.’[50] For both reports, a realist interpretation of the data was taken.

  • Data Reduction and Transparency

This section is related but distinct to the issue of error. Both is concerned with accuracy, but in this case data reduction focuses on producing meaningful data for interpretation.[51] Data reduction is the distilling of raw into a structured and organised form.[52] Transparency is concerned mostly with wilful bias, as opposed to error due to unintentional bias or noise. For both paradigms, there will (or should) be transparent standards and processes for the methods used, peer-review for journal publications, and disclosure of any conflicts of interest.[53] All of this is to ensure that the research maintains an adequate distance to the data. Lincoln and Guba terms this as authenticity such as whether the researcher has influenced the circumstances and actions of the person responding to the data.[54] Good quantitative and qualitative research both need to reduce their data and follow these transparency standards.

Case Studies

In the quantitative report, we can see that data reduction is important because the sample size is extremely large (n = 16,882).[55] Furthermore, not all the data is ‘useful’ because not all samples participated in all three longitudinal studies. Some data also needed to be removed because the sample size for the category is too small. The whole process is transparent, the data set is disclosed, and the process is clearly outlined, and the report is published in a reputable journal. Similarly, in the qualitative report, while the sample size is small (n = 14), the amount of interview data is also extremely large, which needs to be reduced using ‘frameworks’.[56] In both quantitative and qualitative analysis we can see the use of computer software for this process.

V         Conclusion

There is a systematic and congruent approach in the knowledge finding process.[57] Despite coming from historically and theoretically divergent backgrounds, both approaches have their own internal logic to research, but also have to address common epistemological issues such as error, bias and making valid inferences. There may be limitations of each approach, but ongoing division artificially limits the academic exchange amongst social scientists and the understanding that can be achieved in the social world.[58] With more scope it may be worthwhile to examine in more detail the inferential logic of each approach. Recognising the overlap also gives the opportunity for social scientists to employ the use of mixed methods.[59] As can be seen, mental health policy is a field which sees the intersection of the two extremes: the need for hard quantifiable data for treatment efficacy but also understanding its impact on the behaviour and subjective wellbeing of patients.[60] Greater exchange between the two paradigms will therefore be significantly beneficial to social research.

VI        References:

al., Bary Cooper et. Challenging the Qualitative-Quantitative Divide: Explorations in Case-Focused Causal Analysis. Bloomsbury Publishing, 2012.

al., Corliss et. “Sexual Orientation and Drug Use in a Longtudinal Cohort Study of U.S. Adolescents.” Addictive Behaviours 35 (2010): 517-21.

Brewer, John. “Meaning.” In The A-Z of Social Research, edited by Robert Miller, 191-92: SAGE Research Methods, 2003.

Bryman, Alan. Social Research Methods. 4th ed.: Oxford University Press, 2012.

Choy, Looi Theam. “The Strengths and Weaknesses of Research Methodology: Comparison and Complimentary between Qualitative and Quantitative Approaches”. IOSR Journal Of Humanities And Social Science 19, no. 4 (2014): 99-104.

Cruickshank, Justin. Realism and Sociology: Anti-Foundationalism, Ontology and Social Research. Vol. Canada: Routledge, 2003.

Emmel, Nick. Sampling and Choosing Cases in Qualitative Research: A Realist Approach. 4th ed.: SAGE, 2013.

Fulton, Carol Lynne. “Plausibility.” In Encyclopedia of Case Study Research, edited by Albert Mills, 683-85: SAGE Research Methods, 2010.

Guba, Egon. “Authenticity Criteria.” In The SAGE Encyclopedia of Social Science Research Methods, edited by Michael Lewis, 44-46: SAGE Research Methods, 2004.

Gupta, Arjun. Theory of Sample Surveys. 4th ed.: World Scientific, 2011.

Hardy, Alan Bryman and Melissa. Handbook of Data Analysis. SAGE Publications Ltd, 2009.

Hollway, Wendy. “The Importance of Relational Thinking in the Practice of Psycho-Social Research: Ontology, Epistemology, Methodology and Ethics.” In Exploring Psyco-Social Studies. London: Karnac, 2008.

Mason, Jennifer. “Working Paper: Six Strategies for Mixing Methods and Linking Data in Social Science Research ” In ESRC National Centre for Research Methods. NCRM Working Paper Series: University of Manchester 2006.

Morgan, David. “Practical Strategies for Combining Qualitative and Quantitative Methods: Applications to Health Research.” Qualitative Health Research 8, no. 3 (1998): 362-76.

Niglas, Katrin. “Quantitative and Qualitative Inquiry in Educational Research:  Is There a Paradigmatic Difference between Them?” In European Conference on Educational Research. Lahti, Finland: Tallinn Pedagogical University, 1999.

“Quantitative and Qualitative Research.” Explorable: Think Outside the Box, https://explorable.com/quantitative-and-qualitative-research.

Romm, Norma. Accountability in Social Research. Springer Science & Business Media, 2001.

Schwandt, Thomas. “Authority.” In The SAGE Dictionary of Qualitative Inquiry, 16-17: SAGE Research Methods, 2007.

Tashakkori, Abbas. Sage Handbook of Mixed Methods in Social & Behavioral Research. 4th ed.: SAGE Publications, 2010.

Velez, Angela. “Evaluating Research Methods: Assumptions, Strengths, and Weaknesses of Three Educational Research Paradigms.” Davenport University, http://www.unco.edu/AE-Extra/2008/9/velez.html.

Weaver, Vikki Charles and Tim. “A Qualitative Study of Illicit and Non-Prescribed Drug Use Amongst People with Psychotic Disorders.” Journal of Mental Health 19, no. 1 (2010): 99-106.

Footnotes on page 2

Pages: 1 2

One thought on “How are quantitative and qualitative analysis similar, and how might they reasonably differ?

    […] How are quantitative and qualitative analysis similar, and how might they reasonably differ? […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s