Research methodology, from philosophy to data analysis

1. The research onion

The research onion organises methodological decisions into six layers that are peeled from the outside in (Saunders et al., 2019): research philosophy, approach to theory development, methodological choice, research strategy, time horizon, and techniques and procedures for data collection and analysis. Each outer layer constrains the ones inside it. A positivist philosophy, for example, usually leads to a deductive approach, a quantitative choice, a survey or experiment, and statistical analysis.

2. Research philosophy

A research philosophy is a set of assumptions about the nature of reality (ontology), what counts as valid knowledge (epistemology) and the role of values in research (axiology) (Saunders et al., 2019). Stating your philosophy explains why your methods suit your question.

Positivism

Positivism treats reality as external and measurable. Researchers aim for objective, generalisable, law-like findings, usually by testing hypotheses on numerical data. It underpins most survey and experimental research.

Critical realism

Critical realism accepts a reality independent of our thinking but argues we only see part of it. Research looks for the underlying mechanisms that explain observed events, and can use both numbers and words.

Interpretivism

Interpretivism holds that people create meanings, so social reality is understood through the perspectives of those who live it. Research is usually qualitative, in-depth and context-bound (Bryman, 2016).

Postmodernism

Postmodernism questions dominant ways of thinking and the power relations behind them, giving voice to perspectives that have been marginalised.

Pragmatism

Pragmatism judges ideas by their practical consequences and lets the research question drive the choice of methods. It is the paradigm most often linked to mixed methods research (Johnson & Onwuegbuzie, 2004) (Morgan, 2007).

3. Approach to theory development

Deduction starts from existing theory, derives hypotheses and tests them with data. Induction starts from data and builds theory from patterns. Abduction moves back and forth: a surprising observation prompts a plausible explanation, which is then tested (Saunders et al., 2019). Quantitative studies are usually deductive; qualitative studies are usually inductive or abductive.

4. Methodological choice

A mono-method study uses a single quantitative or qualitative technique. A multi-method study uses more than one technique of the same type, for example interviews and focus groups. A mixed methods study combines quantitative and qualitative data and, crucially, integrates them (Creswell & Plano Clark, 2018).

5. Research strategies

The strategy is the overall plan for answering the question (Saunders et al., 2019). Common strategies:

• Experiment: manipulates a variable and compares groups to test cause and effect.
• Survey: collects standardised data from a sample, usually by questionnaire, to describe a population or test relationships.
• Archival and documentary research: analyses existing records and documents.
• Case study: investigates a case in depth within its real-life context (Yin, 2018).
• Ethnography: studies the culture of a group through extended immersion.
• Action research: works with participants in cycles of planning, acting and reflecting to bring about change.
• Grounded theory: builds theory systematically from data (Glaser & Strauss, 1967) (Charmaz, 2014).
• Narrative inquiry: studies the stories people tell about their experiences.

6. Time horizons

A cross-sectional study collects data at one point in time, which suits most dissertations. A longitudinal study follows the same people or organisations over time and can show change and development (Saunders et al., 2019).

7. Sampling and sample size

Probability and non-probability sampling

Probability sampling (simple random, systematic, stratified, cluster and multi-stage) gives every member of the population a known chance of selection and supports statistical generalisation. Non-probability sampling (purposive, snowball, quota, convenience and self-selection) is used when no sampling frame exists or when depth matters more than representativeness (Saunders et al., 2019).

Quantitative sample size

For estimating proportions in a known population, the table by (Krejcie & Morgan, 1970) is widely used; for example, a population of 1,000 needs about 278 respondents. For multiple regression, (Green, 1991) suggests N ≥ 50 + 8k to test the overall model and N ≥ 104 + k to test individual predictors, where k is the number of predictors. A power analysis is better still: choose the expected effect size, alpha (usually .05) and power (usually .80) and calculate N, for example in G*Power (Faul et al., 2007). Cohen's conventions for small, medium and large effects are r = .10, .30 and .50; d = 0.2, 0.5 and 0.8; and f² = .02, .15 and .35 (Cohen, 1988).

Qualitative sample size

Qualitative samples are judged by saturation, the point at which new data add little that is new. (Guest et al., 2006) found most themes appeared within 12 interviews. A systematic review found saturation typically between 9 and 17 interviews, or 4 to 8 focus groups (Hennink & Kaiser, 2022).

8. Data collection

Questionnaires measure constructs with several items each, commonly on 5- or 7-point Likert scales; using established, validated scales strengthens validity. Interviews range from structured to unstructured; semi-structured interviews use a guide of open questions and probes. Focus groups capture interaction between participants. Observation records behaviour in its setting, and secondary data reuse existing datasets or documents (Creswell & Creswell, 2018). Always pilot your instrument before full data collection.

9. Quantitative data analysis

Data screening and descriptive statistics

Check for missing values, outliers and coding errors, then report means, standard deviations, skewness and kurtosis for each variable. Many statistical tests assume roughly normal distributions (Field, 2018).

Reliability

Internal consistency is usually reported with Cronbach's alpha (Cronbach, 1951). Values of .70 or above are generally accepted (Nunnally, 1978); very high values (above about .90) can signal redundant items (Tavakol & Dennick, 2011).

Validity

Construct validity is examined with factor analysis. Convergent validity is supported when the average variance extracted (AVE) is at least .50, and discriminant validity when each construct's square root of AVE exceeds its correlations with other constructs (Fornell & Larcker, 1981). The heterotrait–monotrait ratio (HTMT) is a stricter test of discriminant validity (Henseler et al., 2015).

Correlation and group comparisons

Pearson's r describes linear relationships between continuous variables. Independent and paired t-tests compare two means, one-way ANOVA compares three or more, and chi-square tests associations between categorical variables. Non-parametric alternatives (Mann–Whitney, Wilcoxon, Kruskal–Wallis) are used when assumptions are not met (Field, 2018).

Multiple regression

Regression estimates how well a set of predictors explains an outcome. Report R², the F-test and each coefficient with its t and p values. Check linearity, independence of errors, homoscedasticity, normality of residuals and multicollinearity; variance inflation factors (VIF) well above 5 to 10 are a concern (Hair et al., 2019).

Mediation and moderation

A mediator explains how a predictor affects an outcome; a moderator changes the strength or direction of that effect (Baron & Kenny, 1986). Modern practice tests indirect effects with bootstrap confidence intervals rather than the older causal-steps approach (Preacher & Hayes, 2008). The PROCESS macro implements these models in SPSS, SAS and R: Model 1 for moderation, Model 4 for mediation, and Models 7 and 14 for moderated mediation in the first and second stage (Hayes, 2022). The indirect effect is supported when its 95% bootstrap confidence interval excludes zero.

Structural equation modelling

SEM tests a measurement model (how items load on constructs) and a structural model (how constructs relate) together. Covariance-based SEM (for example AMOS or lavaan) emphasises model fit; partial least squares SEM emphasises prediction and suits smaller samples and complex models (Hair et al., 2019) (Kline, 2016).

Effect sizes and reporting

Report effect sizes alongside p-values so readers can judge practical importance (Cohen, 1988), and follow APA style for tables and statistics.

10. Qualitative data analysis

Thematic analysis

Thematic analysis identifies patterns of meaning across a dataset. The six phases are familiarisation, generating initial codes, searching for (constructing) themes, reviewing themes, defining and naming themes, and writing up (Braun & Clarke, 2006). Reflexive thematic analysis stresses that themes are actively built by the researcher rather than “emerging”, and that researcher subjectivity is a resource (Braun & Clarke, 2019) (Braun & Clarke, 2021).

Interpretative phenomenological analysis (IPA)

IPA explores how individuals make sense of significant lived experiences, working case by case before looking across cases. Samples are small and homogeneous (Smith et al., 2022).

Grounded theory analysis

Grounded theory builds explanations through constant comparison and coding that moves from initial to focused or theoretical codes, with memo-writing and theoretical sampling until categories are saturated (Glaser & Strauss, 1967) (Charmaz, 2014).

Content analysis and the framework method

Qualitative content analysis can be conventional (codes come from the data), directed (codes come from theory) or summative (counting and interpreting key words) (Hsieh & Shannon, 2005). The framework method charts coded data into a matrix of cases by codes, which suits team-based and applied research (Gale et al., 2013).

Trustworthiness

Qualitative quality is judged by credibility, transferability, dependability and confirmability (Lincoln & Guba, 1985). Strategies include member checking, thick description, an audit trail of coding decisions, peer debriefing and reflexive journals.

11. Mixed methods and integration

Mixed methods research combines both strands to answer questions neither could answer alone (Johnson & Onwuegbuzie, 2004). The three core designs are convergent (both strands at the same time, then merged), explanatory sequential (quantitative first, then qualitative to explain the results) and exploratory sequential (qualitative first, then a quantitative phase that builds on it) (Creswell & Plano Clark, 2018).

Integration can happen at the design, methods, and interpretation and reporting levels (Fetters et al., 2013). A joint display sets quantitative results beside qualitative findings and states whether they confirm, expand or diverge, leading to meta-inferences (Guetterman et al., 2015).

12. Ethics

Every study needs informed consent, voluntary participation with the right to withdraw, confidentiality or anonymity, secure data storage and approval from the institution's ethics committee (Bryman, 2016). In Kenya, research also generally requires a research licence from the National Commission for Science, Technology and Innovation (NACOSTI), and personal data must be handled in line with the Data Protection Act, 2019.

13. Writing the methodology chapter

A clear methodology chapter usually covers, in order:

1. Research philosophy and approach, with justification.
2. Research design and strategy.
3. Population, sampling technique and sample size calculation.
4. Data collection instruments, piloting, reliability and validity.
5. Data collection procedure.
6. Data analysis techniques, linked to each research question or hypothesis.
7. Ethical considerations.
8. Limitations of the methodology.

Justify each choice with reference to your research questions and the literature, not only by describing what you did (Creswell & Creswell, 2018).

References

1. Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037/0022-3514.51.6.1173
2. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
3. Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. https://doi.org/10.1080/2159676X.2019.1628806
4. Braun, V., & Clarke, V. (2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238
5. Bryman, A. (2016). Social research methods (5th ed.). Oxford University Press.
6. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Sage.
7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
8. Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage.
9. Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.
10. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
11. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
12. Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—Principles and practices. Health Services Research, 48(6 Pt 2), 2134–2156. https://doi.org/10.1111/1475-6773.12117
13. Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th ed.). Sage.
14. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
15. Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13, Article 117. https://doi.org/10.1186/1471-2288-13-117
16. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Aldine.
17. Green, S. B. (1991). How many subjects does it take to do a regression analysis? Multivariate Behavioral Research, 26(3), 499–510. https://doi.org/10.1207/s15327906mbr2603_7
18. Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82. https://doi.org/10.1177/1525822X05279903
19. Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Annals of Family Medicine, 13(6), 554–561. https://doi.org/10.1370/afm.1865
20. Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage.
21. Hayes, A. F. (2022). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (3rd ed.). Guilford Press.
22. Hennink, M., & Kaiser, B. N. (2022). Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Social Science & Medicine, 292, Article 114523. https://doi.org/10.1016/j.socscimed.2021.114523
23. Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8
24. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
25. Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. https://doi.org/10.3102/0013189X033007014
26. Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.
27. Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607–610. https://doi.org/10.1177/001316447003000308
28. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage.
29. Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48–76. https://doi.org/10.1177/2345678906292462
30. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
31. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. https://doi.org/10.3758/BRM.40.3.879
32. Saunders, M. N. K., Lewis, P., & Thornhill, A. (2019). Research methods for business students (8th ed.). Pearson.
33. Smith, J. A., Flowers, P., & Larkin, M. (2022). Interpretative phenomenological analysis: Theory, method and research (2nd ed.). Sage.
34. Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
35. Yin, R. K. (2018). Case study research and applications: Design and methods (6th ed.). Sage.