Abstract:
One of the frequent objections to qualitative approaches and the use of qualitative
methods for research is the subjectivity of the evaluation – it is entirely up to the analyst
to choose which sections to analyse, what codes to use, what themes to generate
as significant and pass them on for further analysis or interpretation. This study, which
is part of the broader DAQUAo project (psycholinguistic analysis of therapeutic diaries),
is concerned with content analysis as a fundamental and usually the first step in most
qualitative approaches. The aim of this study is to verify the agreement of assessors
by statistically comparing codes obtained by the content analysis method in the open
coding phase, performed by two independent analysts onidentical texts. Stimulus material:
therapy journals, N=179 unique text entries, total 22,046 words. Results: Analytical
K identified 32 codes, analyst L identified 70 codes. Codes that were close in content
were referred to assessor agreement analysis (paired t-test). There was an agreement for
two codes (movement-travel, emotion-good mood). Most of the differences were due
to different levels of generality – there were mostly hyper- and hyponymy relationships
between the analyst’s codes K and L (the superordinate category K corresponded
to several specific categories of L) or adjacent categories (two specific instances of one
superordinate category). Approximately one-third of the codes were unique and could
not be easily compared. Conclusion: coding text through content analysis requires careful
training, a clear definition of the level of generality required in the first stage of coding,
and last but not least, a formulated research question. These three conditions can ensure
higher inter-assessor agreement and de facto better reliability of the method as such.
