Just a quick post to share this article by Ron Chenail about writing up...it covers so many of the things we have been talking about recently but really puts the turbo on the creativity and number of options availalbe to us in writing up
HereThe Qualitative Report, Volume 2, Number 3, December, 1995(http://www.nova.edu/ssss/QR/QR2-3/presenting.html)
Some quotes to reinforce the points:
“I write in order to learn something that I didn’t know before I wrote it…not to write until I knew what I wanted to say, until my points were organized and outlined… this static writing model coheres with mechanistic scientism and quantitative research… It ignores the role of writing as a dynamic, creative process….(Richardson L. 1990)
“Writing-up qualitative research inevitably results in the emergence of new ideas and ways of viewing the data and hence plays a crucial role in the analysis process.”(Pitchforth et al. 2005)
We had a very productive first ShutUp and Write meeting this Friday with 13 people joining for 7 hours of writing....many people writing up results, a few writing research proposals, discussions and other things...we had lots of interesting chats during the breaks, comparing analysis methods, working on adding some literature to results sections but generally I think the real benefit was simply having an opportunity to write without the distractions of being at home, without access to the internet but in a collegial and supportive setting
I found it pretty amazing how much I got done, writing about half a paper, which is really saying something..plus...Toblerone, Brownies, quiche, slice, fruit and many more goodies...
We planned to more meetings;
1. October19 9am-4pm in the Conference Room in Psychology Clinic, Mackie Building
2. Novermber 30th 9am-4pm OTC 405 as before
Bring your own lap top for the October meeting, computers provided for November.....please email me to express an interest...p.rhodes@sydney.edu.au
The more I read the more concerned I become about how we are lagging behind in clinical psychology in Australia when it comes to qualitative research in psychology..been reading this INTRODUCTION TO QUALITATIVE METHODS IN PSYCHOLOGY Dennis Howitt Loughborough University where he provides a history of qual in psychology, marking the 1980's as when qual became more accepted..
That is certainly not the case with clinical psychology here where we lag way behind...even in 1940 it was understood that science can include qualitative..
If we rejoice, for example, that present-day psychology is . . . increasingly empirical, mechanistic, quantitative, nomothetic, analytic, and operational, we should also beware of demanding slavish subservience to these presuppositions. Why not allow psychology as a science – for science is a broad and beneļ¬cent term – to be also rational, teleological, qualitative, idiographic, synoptic, and even non-operational? I mention these antitheses of virtue with deliberation, for the simple reason that great insights of psychology in the past – for example, those of Aristotle, Locke, Fechner, James, Freud– have stemmed from one or more of these unfashionable presuppositions. (Allport, 1940, p. 25)
Thank god for Health Psychology and Critical Psychology who still wave the flag...
Its a pretty interesting reflection on the conservatism of clinical psychology that we have never had a study conducted in our program using discourse analysis...never.....pretty astounding when you think of all the potential it holds for deconstruction of our practices and all the power plays requiring such......it demonstrates how easy it is to set up a research silo, a product partly of our poor engagement with other disciplines.......
Here is a really great article on it, particularly Foulcault influenced discourse analysis, by Linda Graham from Queensland University of Technology...she quotes Stephen Ball (1995: 267) t “the point about theory is not
that it is simply critical” ..its purpose is “to engage in struggle, to reveal and undermine what is most invisible and insidious in prevailing practices.”
Her work relates to ADHD and includes critque of the DSM...here.......pretty impressed by her work!!!
Graham, Linda (2006) The Politics of ADHD. In Australian Association for Research in Education (AARE) Annual Conference, 26th-30th November, Adelaide.
This essay offers a critical review of the problem we call “ADHD‿. In the first part of the discussion, the author presents an analysis of the literature surrounding Attention Deficit Hyperactivity Disorder. Adopting a lens informed by the work of Foucault, she teases out the medical and psychological models to show the interdependency between these otherwise competing knowledge-domains. She argues that as it currently stands the construct serves political ends and questions whether a diagnosis of ADHD is helpful - and if so, for whom? In the second part, the author considers what role schooling practices might play in the pathologisation of children and interrogates the ADHD phenomenon as a symptom of the pathologies of schooling. Graham concludes by suggesting how we might arrest the rising rate of diagnosis by thinking
Today we held a great collaborative coding session where a project was presented with focus groups held at three time points..we split the group in three, allocated transcripts from each time point and then came together after discussion in each group to compare notes as across the whole project...
This got me thinking about the best procedure to run such team reflections and how it can draw directly from Post-Milan Systemic Family Therapy reflecting teams
A 5 step reflecting process for collaborative coding....
1. Step 1...each group reads transcripts 2. Each team discusses meaning 3. Each team shares their analysis with each other while the researcher remains silent -This discussion should start with affirmations directed to the researcher before constructive comments -This discussion should always be tentative "Iwas wondering....." 4. The researcher is then asked what interested her the most about the process 5. The group is asked how this relates to their own projects
The allocation of discussion groups ca be made in many ways: analyse using different theories/look at transcripts from those with specific attributes/compare data from different data collection methods...ie: supports triangulation...
Im learning the hard way at the moment about how to write results after a thrashing by a journal for a study titled Exploring How Trainees Manage Their Own Distress In-Session: An Interpersonal Process Recall Study
The review was very constructive, pushing me to write better, particularly suggesting that I integrate theory from the literature in my results rather than saving it all for the discussion..I confess to finding this quite difficult but after about 8 hours of mental struggle Ive found my results have really come to life in a much better way that before.before they were descriptive...now they kick arse? well..they are better but it's still a work in progress...here they are, completely unproofed and informatted..they are obviously out of context without the intro and method but I thought some of you writing at the moment might find it useful..
Results
The
Discrepancy Between Planned Content and In-Session Process
Watzlawick, Bavelas and Jackson (1967) differentiate
between content and process in therapy. The former is seen as the digital or
rational where messages are unequivocal and can be analysed easily. The latter
is analogue or emotional and messages as more subtle and hard to interpret. In
this study we saw the overwhelming majority of trainees become distressed when
faced with unexpected processes in the therapy room.
They described feeling as “lost”, “confused”, “unsure”, “floundering”, having a “mental blank” or unsettled.
Trainees recalled preparing well for
their sessions, including ‘reading books
and notes’, ‘revisiting the formulation’ and ‘making a number of session plans’ but felt that this preparation
did not necessarily equip them for what was occurring that moment in the room.
In-Session Interpersonal Difficulties
Many of the unexpected processes
involved interpersonal difficulties between trainee and client, including
clients who are seen as resistant, disengageing, attempting to assert control
or seeking direct advice.
Here a clients is seen as being
resistant to cognitive restructuring.
What
was I meant to say to that? In all the cognitive therapy books the person says,
“Wow! Now that you put it like that, I completely agree.”
Here a trainee becomes confused when a
client disengaged, despite a previously strong therapeutic relationship.
When
I said something that wasn’t an answer to her mind, you could tell she immediately disengaged. She stopped looking
at me and started playing with tissues. Whereas every other time she was
completely looking at me, very, very engaged.
One trainee described feeling “frozen” and “backing down” when a client was seen as overtly counteracting
explicit guidelines for homework tasks. In this case a couple were asked to
write on the same sheet, to facilitate a comparison of perceptions concerning
events. The trainee recalls the husband disagreeing.
“Can we have separate ones of these?” I went
“Oh! Okay! Yep”, agreeing to it because I was just so thrown by something that
I thought we’d really explicitly talked about.
Another trainee described feelings of
powerlessness in the face of a client who frequently requested direct advice.
It’s
an expectation of me to provide advice, it didn’t sit comfortably. I felt like
I was being trapped in a corner. She was putting me on the spot – trying to
nail me down kind of thing. I didn’t feel very comfortable [slight nervous
laugh]. I notice that I wave my hands around a lot more when I’m feeling scared.
Each of these examples suggest that
trainees had an expectation that client’s would follow therapy as described in
text books or in their own pre-session planning, as if what Bennet-Levy &
Thwaites (2007) calls declarative knowledge would be translated directly to
procedural (skills-in-action) without reflection on the interpersonal dynamics
at play in the room.
Complex or incongruous presentation
While trainees found interpersonal
interactions difficult they also had difficulty making sense of the complexity
of the client’s story.
What
I had sitting on that chair were three different session plans. What I did not
expect was for her to have all three as a presenting issue. “Ahh! Which one?!”
I got frazzled, there was a sense of urgency.
These stories were seen to “lack of
coherence and stability,”and trainess were unsure of how they related to the
presenting problem or how to integrate these experiences into their
formulations or treatment
Here a trainee is poised to intervene
with a client who has a history of social anxiety, only to discover that he is
telling the story of breaking up with his girlfriend.
I
was lost. I thought he was going to go and have coffee with this girl and
explore his feelings, so I thought we were testing predictions like, “It’ll be
awkward” and “It won’t be fun.” It became apparent that actually he was going
there to tell her he wasn’t interested. I was like like, “Riiiigggght. Now I’m
going to leave.”
Here a trainee finds it difficult to
make sense of seemingly incongruous reactions to a death in the clients life.
I’m
thinking, “What’s going on with you? This guy had died and you’re . . .
reacting with humour.’”He does impressions. He does impressions when asked
emotional questions.
The emotional reactions described by
students regarding both interpersonal processes and complex client
presentations reflect discomfort with what family therapists Anderson, H. &
Goolishian, H. (1992) call a ‘not-knowing’ position, an essential prerequisite
to curiosity (Palazzoli et al. 1980) that leads one to ask open-ended reflexive
questions of the client, rather than concern oneself with how to best
intervene. It would seem in both of these types of situations trainees are
momentarily “frozen’ or ‘baffled. Many also described feelings
of inadequacy at not knowing what to do next. ’
I
didn’t like that feeling. It felt like I was ineffective, like I’d lost
purpose, um, and I don’t see how that’s useful to the client.
One trainee described how her supervisor
helped her see that her negative judgments were related to her idea that
therapy had to be formal and technical, rather than natural and conversational.
I
was thinking “That part wasn’t special – it’s conversation” but my supervisor
today was going, “That – that’s therapy. That bit where you casually dismiss
that you re-framed that helost his job?
That’s therapy, What about the bits where I couldn’t fill in the sheets and do
the ABC and he’s like, “Yeaaahhh.”
In this example the trainee seem to have
misconstrued the collaborative nature of therapy, where meaning is contracted
through dialogue. (Overholser, 2011)
Activation
of Rudimentary Reflections
Bennett and Thwaites (xxxx) provide some
conceptual confirmation for the findings in this study, asserting that the
reflective system is activated when there is a mismatch between expectations
and reality. In their approach trainees are taught to focuss attention on the
problem at hand, develop a mental representation of it and engage in an active
cognitive proves of problem solving. However this occurs in supervision, as a form
retrospective of reflection on-action rather than in-session reflection
in-action (Schon)
While they were initially ‘taken aback’ Trainees describe how, in “what seems like a fleeting moment” or a ‘rush’
they are able to acknowledge the emotion they are feeling then struggle to engage
in a process of reflection (in my head I
was like “What the hell?”).
Three different strategies were then
described, each of which has been described here as rudimentary.
Referencing the expert
A significant number spoke about
mentally referencing a supervisor or perceived superior to help them decide how
to proceed. For one the supervisor operated as a “voice in your head going, you should do that”. For another it
served as a reminder of “what is the
right thing to do in terms of an intervention point-of-view”?
In some cases this strategy was seen as
helpful, particularly when their supervisor was trusted or open about their own
mistakes. For others comparisons with an “expert”
only exacerbated feelings of inadequacy.
He’d
have something meaningful to say, with his level of insight. Whereas I don’t
know what to do with her.
Attempts at transference
Despite the range of distressing
emotions experienced by trainees the majority described an ability to
acknowledge these emotions while they were happening in sessions. In some cases
trainees discussed how their own emotional response communicated to them the
emotional experience of their client.
“I was feeling lost, and I guess that’s
how she was feeling as well.”
“He’s probably feeling really upset and
really disappointed, too”
These reflections, while important, do
not yet mirror the complexities of transference dynamics, in terms of
recognising that interpersonal difficulties encountered with clients are
actually a potential tool for exploring complex unconscious processes (Gabbard,
2001).
Check
the therapeutic relationship
Trainees also engaged in momentary
reviews of the therapeutic relationship, either checking whether the
relationship was strong enough to handle the difficult moment, or reviewing it
so as to make a choice between trying to re-establish rapport or continue with
the skills-based intervention. Relationships reviewed as strong made it “easier to say a lot of things” and
caused some easing of distress because “I
don’t really mind making mistakes in front of her.”
Engage in Self-talk
Self-talk was also a common strategy
used to try and get the session back on track after distressing events.
Trainees described talking themselves through fleeting reviews of their
formulation and treatment plans.
Just
trying to quickly re-formulate in my head how I could quickly change th direction of therapy to be still effective.
I’m
sort of not wanting to shut that possibility down but at the same time I do
sort of have a therapy plan which I went back to. I’m trying to make sure that
we take each session somewhere and that I use the time well.
This differs significantly from the kind
of complex inner conversations described by Rober et al. (2008) in his analysis
of work by more experienced therapists. Here the therapist can make mental room
for a conversation between multiple positions, including the interaction in the
room, client’s story and planning for action. This conversation is held
internally but at a distance, a process of constructive hypothesising about how
to respond in the moment (Rober, 2002).
Retreat
to Safety of Non-Directive Counselling
For many trainees, especially those who
lacked confidence in treatment techniques the next step after reflection was to
retreat to the safety of non-directive counselling.
I’m
supposed to be using some sort of technique and I don’t know what I’m doing.
I don’t have any strategy for her, CBT
or DBT or schema stuff– I have nothing to structure her thinking.
Skills used included listening,
affirming, summarising, reflecting or asking questions. For the majority of
trainees these basic counselling skills came more “naturally”
than skills that were seen as more interventive.
I
feel like being warm and being empathic towards a client are not things I have
to fake, and not things I have to remind myself to do. I suppose things that
I’m still less comfortable with, and still getting used to, are you know,
particular questioning techniques, or, you know, more sophisticated cognitive challenging
and things like that.
Using basic counselling skills in the
moment also served as a “coping
mechanism”, “buying time,” supporting a “facade of control” and the maintenance of a “professional stance.” For many therapists “surviving” the distressing moment was the goal.
Risk
Taking
Other trainees, however, decided to push
themselves ‘outside of their comfort
zones’and focus on intervention. Some were motivated to “take these risks” by “professional growth.’ Intervention did
not necessarily flow from a reconceptualization of the case or the formulation
based on the distressing experience but was instead described as an attempt to
‘push ahead’, be more ‘directed’
See! It
works! You’ve just got to talk. Now you
can get back to what you’re doing.” I’m pleased I did something. That it
actually went okay – that when I interrupted they both seemed to settle fairly
quickly – was a good reminder to me that when I do this it doesn’t actually
damage the relationship too much.
Arguably, despite the limitations, this
type of risk taking is critical to learning (Spellman & Harper, 1996).
Trainees who took risks, however, were careful to review the therapeutic
relationship first to see if it was secure. . Those trainees who felt it was less
seemed more tentative in their decisions and less likely to respond in a way
that might “challenge” the client.
Im thinking heavily about running an Unconference or QualCamp at Sydney Uni next year..a what? I hear you say...
Here is how it works..first the things not done
No pompous plenaries or panels
No prepared talks
No $780 rego fees
No overheads
No horrible meals
No domination by people who want to show off
Unconferences are a new trend, about 5 years or so in Australia, (see News-com.au article ) where the emphasis is placed on collaborative learning rather than pontificating....basically everyone is a particpant, they all turn up and through skilled facilitation develop the agenda on the first day.....a number of streams might be developed, cented around actual problem solving, story telling and also teaching technical skills...but there are no dyadic talks..maybe 7mins max of talk time given to many people, then discussion...
If we held one at Uni, for example, 100 people might meet in the quadrangle then split up based on interests and find spaces to run sessions.. uni cafes, parks, Glebe Point Road cafes, lecture theatres, lecture rooms wherever..
Instead of panels there are Unpanels done in a fishbowl discussion style with speakers in the middle, rotating with participants..
Spectrograms can be used to place people along a continuum of agreement/disagreement with a controversial topic..people are then interviewed along the line
Community-Mapping can be done to visually picture how everyones research projects are related to each other....
..also Lightening Talks, SpeedGeeking and more...
There are many other creative ideas..check this blog and others.......want to be involved?
Development of Qualitative Research Education Unit in School of
Psychology
Dr Paul Rhodes, Clinical Psychology Unit
Background
An informal research support group was developed 18 months
ago, Qualitative Research in Psychology (QRIP), to support qualitative
researchers in the School of Psychology. Fortnightly meetings have been held
since this time focussing on educating students on the full gamut of
qualitative research knowledge, from epistemology to design, methods, data
collection techniques, data analysis methodologies, rigour, coding, writing up,
publishing and more.
Methods explored have included thematic analysis, grounded theory,
discourse analysis, conversational analysis, interpersonal process recall,
interpretative phenomenological analysis, narrative inquiry, community-based
participatory action and more.
This material is presented in an informal setting, involving
discussion and development of current research and serves to augment lectures
given in the school. Many of the group meetings have involved interview role
playing, collaborative research design, collaborative coding and many other
activities that are most suitable for a group setting.
Achievements to Date
Since QRIP’s inception 87 students and academics have
joined, from the School of Psychology, but also from Health Sciences, the
Medical School, Social Work, Occupational Therapy and other affiliated
disciplines. Fifteen to twenty students and academics attend meetings,
depending on content, and a majority are Honours, DCP and PhD students from the
School of Psychology. Many of these students have teamed up to work together in
between meetings, particularly for cross coding purposes. Formal feedback concerning student
satisfaction and feedback regarding these groups will be sought at the end of
2012 and annually from then.
The remaining members
are supported by our listserve, our Usyd E-Community site, our blog and
occasional one-to-one meetings conducted by myself. The blog has proved a
particularly innovative and effective resource, well suited to the way in which
students source information and interaction. The blog receives approximately
500 hits per week by those seeking advice on qualitative research, including
our own students. http://qual-rip.blogspot.com.au/.
It appears as the second international Google search item under search terms
‘qualitative research psychology.’ It has also been recently recommended by the
British Psychology Societies Readers Digest to U.K Psychologists. Recognition and encouragement has also come
from Prof. Lyn Richards (Founder of QSR International), "Love your blog – and your openness and
practical de-mystifying approach. Qualitative research needs so much more of
such input – and an ability to laugh at itself".
This group has also been recognised by the Institute of
Teaching and Learning, University of Sydney, including an article in their
regular Newsletter and presentation at ITL Seminar. I am also presenting
related material at The Australian
Conference on Science and Mathematics Education (The 18th UniServe
Science Conference). Rhodes, P. What standards should be set for
qualitative research conducted in a science faculty: Psychology, rigour and the
politics of evidence. A
close relationship with The Centre for Values, Ethics and Law in Medicine,
University of Sydney has also developed, including research supervision of our
students being supported by their academics and exchanges in teaching
responsibilities.
Given these developments I am proposing that QRIP be
developed into the Qualitative Research Education Unit, hosted by the School of
Psychology.
Objectives
1. To provide technical support to research students in the
School of Psychology conducting qualitative research
2. To promote methodological diversity in qualitative
research in the School.
3. To promote high standards of rigour in qualitative
research in the School, including the use of methods to ensure dependability
and credibility of findings.
4. To foster collaboration with other academic qualitative
researchers in the University of Sydney and other Universities, who can further
support research students in the School of Psychology, providing specific
technical expertise when required, including serving as Associate Supervisors.
5. To actively advocate for the further recognition of
qualitative research in the School of Psychology and Psychology as a field.
Functions
Many of the functions of the Unit would be consistent with
QRIP as it now stands.
1. Provision of regular education and research support group
meetings
2. Provision of Digital resources: Listserve, Usyd
E-Community and Blog resource
3. Provision of one-to-one consultations for research
students in the School of Psychology
Additional functions would include:
4. Development of a group of Academic Associates to support
the endeavours of the Unit and research students in the School of Psychology
5. Hosting regular small seminars with guest speakers to
support research students in the School of Psychology
6. Develop and advocate for clear standards of excellence in
the design and conduct of qualitative research in psychology
7. Support a mentoring system between students at different
levels of research, providing opportunities for PhD students to gain experience
in supporting research students.
Proposed Outcomes
1. Increased number of research students including
qualitative methods in their research, where appropriate.
2. Increased standard and rigour of qualitative research
conducted by students in the School of Psychology.
3. Greater number of students asking and answering questions
that are suitable for qualitative research, including therapeutic process
research, research on reflective practice, community-based action initiatives,
consumer-driven research.
4. Enhanced pool of Associate Supervisors and technical
expertise available to students.
5. Further recognition of the School of Psychology as
supporting innovation and diversity in research.
6. Enhanced capacity to manage larger numbers of students
requiring supervison.
Structure
To date QRIP has been essentially coordinated informally, with feedback from students. I am seeking to change this by:
1. Convening a committee to meet once every four months to
discuss directions and organisation of events, made up of both academic and
student representatives.
2. Committee would also conduct annual reviews of both
student satisfaction and outcomes (including number and quality of
publications)
3. Report to Head of School each year on achievements and
future goals.
4. Development of a central web page, linked from School of
Psychology from which the details regarding meetings, events, blog could be
accessed.
The best Phd blog in the universe is obviously the Thesis Whisperer who has developed fantastic idea for research students, Shut Up and Write meetings held in RMIT Cafes or thereabouts....the idea started in San Francisco where writers met together to chat, drink coffee, then 'time-box', keeping quiet for 45 minutes until the next break....I notice the library at UTS has started doing it and that Melbourne Uni has also done Thesis Boot Camps....
Seems a perfect idea to mediate against the isolation of writing AND procrastination...
Im flagging the idea for QRIP too, but with some of the breaks for consultation about the writing of qual research, with academics but also between students....in situ group problem solving...Ive sent an email out to all members, lets see what happens
I have a talk coming up titled What standards should be set for qualitative research conducted in a science faculty: Psychology, rigour and the politics of evidence which i will present at the Australian Conference on Science and Mathematics Education 2012, Sydney, Australia..I havent written it yet but have been reading a great paper, below and a few others
They propose the following standards to judge good qual research:
Clarification and justification;
Procedural rigour;
Representativeness;
Interpretative rigour;
Reflexivity and evaluative rigour; and
Transferability.
Clarification and justification
As in all forms of research, clarity of research question reflected in the aims of the study is essential for evaluating results and their interpretation. The demonstration of theoretical rigour (referring to the soundness of fit of the research question, aims and the choice of methods appropriate to the research problem11) is extremely important.
There is a wide variety of named qualitative approaches that are underpinned by particular theoretical perspectives. In addition, the researcher may use basic field research (question, investigation, interpretation). Regardless of the theoretical approach used, the choice requires justification in reference to the research question of the study.
Procedural rigour
Procedural, or methodological, rigour concerns the transparency or “explicitness” of the description of the way the research was conducted. It involves detailing issues of accessing subjects; development of rapport and trust; how data are collected, recorded, coded and analysed; and accounts of the manner in which errors or subject refusals are dealt with.4,11,22 In this regard, readers and reviewers may ask the following questions while examining descriptions of qualitative methods: How were participants/settings accessed? Who was interviewed/observed? How often? For how long? What interview questions were asked? What was the purpose of any observation? Which policy documents/case notes were accessed? How were they assessed? How was collected data managed?
Representativeness
There are a number of commonly available, non-probability sampling approaches. Maximum variation sampling seeks representativeness of all aspects of the topic in terms of participants. Homogenous sampling consists of the selection of a group fitting specified criteria. Snowball samplinginvolves networking from one difficult-to-access type of participant to a wider range of participants. Finally, convenience sampling involves studying easily accessed individuals or groups. This last technique obviously presents its own ethical dilemmas of the “insider” type and is possibly the weakest form of sampling in terms of allowing conceptual generalisability.4,15,16,22,23Maximum variation is the ideal when a holistic overview of the phenomenon is sought; for instance, the question of how a particular hospital department operates may involve sampling in the wider organisation as well as within the individual department and among recipients of services.
Simply mentioning the sampling strategy in the methods section of a qualitative research paper is not sufficient. The key findings of the research need to be evaluated in reference to the diverse characteristics of the research subjects. Through constantly comparing the experiences and responses of the participants against each other, subtle but significant differences can be uncovered that can generate profound insights into the phenomena under study.19
Interpretative rigour
Interpretative rigour relates to as full as possible a demonstration of the data/evidence. In qualitative research, a commonly used concept is inter-rater reliability. This refers to using a type of researcher triangulation by which multiple researchers are involved in the analytical process. This is an attempt to increase the validity and reliability of the study19 through the provision of a more complex and nuanced understanding of the possible interpretations of the objects of the research.11 In contrast to the quantitative research paradigm, what is important in this process is not the level of consensus, but the opportunity for discussion among analysts to provide opportunities for developing further coding.19
A related technique is that of respondent validation, or member checking. This entails offering subjects interviewed the opportunity to view and amend their transcripts as a type of validity.12 However, this approach does have limitations due to the evolution over time of the positions and purposes of the researchers and participants, thereby potentially affecting interpretations and accounts. Respondent validation should be thought of as part of a process of reducing error, which involves the generation of further original data, which then requires interpretation.8
Other techniques that enhance interpretative rigour are the differing forms of triangulation: data (multiple evidentiary sources; ie, documents, interviews, survey data, observation), methods (multiple methods), and theory (multiple theoretical and conceptual frames applied to the research to enhance insights into phenomena). Using these forms of triangulation allows the development of a comprehensive understanding of the phenomena and can ameliorate the potential bias of simply using one method.4,5,8,11,16,22
In the interpretive process, accounts of “negative” or “deviant” cases are especially important. These are explanations pertaining to data or evidence that contradicts the researchers overall explanatory account of the phenomena.5
In sum, a clear description of what forms of analysis were used, the process and what were the major outcomes of the analytical process in terms of findings is needed to ensure quality for the author, and to enable an assessment to be made in terms of the analytical quality of the research by the reader.
Reflexivity and evaluative rigour
Reflexivity is where researchers openly acknowledge and address the influence that the relationship among the researchers, the research topic and subjects may have on the results.4,11,13 Fundamentally, reflexivity requires a demonstration by the researchers that they are aware of the sociocultural position they inhabit and how their value systems might affect the selection of the research problem, research design, collection and analysis of data.15 It also refers to an awareness by the researchers of the social setting of the research and of the wider social context in which it is placed.4
Evaluative rigour refers to ensuring that the ethical and political aspects of research are addressed. Typically, this refers to proper ethics approval from appropriate committees covering confidentiality, informed consent and steps to avoid possible adverse effects on the subjects. Importantly, where appropriate, relevant community leaders should be consulted in the design and conduct of the research.11 Researchers should revisit their actions and interactions within the research process to ensure as “accurate” as possible portrayal of the production of their findings.
Transferability
Conceptual generalisability and transferability refer to how well the study’s findings inform health care contexts that differ from that in which the original study was undertaken.4 For example, a review of data from qualitative studies was conducted on a wide variety of doctor–patient interactions about medication compliance.24 The authors examined barriers to patients taking prescribed medication as directed by their doctors and found that patients were often inclined to resist taking medicines, not because of problems with the patients, doctors or systems, but because patients were concerned about the medicines. This type of study allows for the construction and transfer of general policy on medicine-taking (through, for example, less emphasis on patient behaviour modification and more emphasis on production of safer medicines) and practice (suggesting, for instance, that doctors should assist lay evaluations through provision of more information, support, feedback and safe prescribing practices
Here is an approach that perhaps has more consideration for the many different types of qual research
m.hammersley@open.ac.uk MartynHammersley 0000002007 330Taylor & Francis 2007 Original Article 1743-727X (print)/1743-7288 (online) International Journal of Research & Method in Education 10.1080/17437270701614782 CWSE_A_261329.sgm Taylor and Francis
This article addresses the perennial issue of the criteria by which qualitative research should be evaluated. At the present time, there is a sharp conflict between demands for explicit criteria, for example in order to serve systematic reviewing and evidence-based practice, and arguments on the part
of some qualitative researchers that such criteria are neither necessary nor desirable. At issue here,
in part, is what the term ‘criterion’ means, and what role criteria could play in the context of qualitative enquiry. Equally important, though, is the question of whether a single set of criteria is possible across qualitative research, given the fundamental areas of disagreement within it. These reflect
divergent paradigms framed by value assumptions about what is and is not worth investigation. In
addition, there are differences in methodological orientation: over what counts as rigorous enquiry,
realism versus constructionism, and whether the goal of research is to produce knowledge or to serve
I was involved in a really useful meeting today looking at a mixed method study that involved the sequential use of community-based participatory action research as a hypothesis generation tool, followed by a more quantitative testing of emergent themes and theory from that study....
Mixed methods studies have so many advantages and can be used in many different ways.
1. Quant data can support qual analysis, triangulating, or adding trustworthiness to findings
2. Qual studies can be nested in bigger quant studies like randomised control trials
3. Quant and qual studies can be done simultaneously with both sets of findings analysed together (the inside and outside views)
4. Qual studies can generate hypotheses that are then tested by quant studies
There are many more designs of course...if you want to know more I highly reccommend the following book chapter, just email me p.rhodes@sydney.edu.au if you are a Usyd student and want access to it..
Research design : qualitative, quantitative, and mixed methods approaches / John W. Creswell.
I have recently been discussing the issue of making meaning from your data not just listing lots of themes with a couple of my students...trawling through your transcripts and coding similar phenomenon might be time consuming and important but in the end this type of coding is only a tool to help you think about the depth and complexity of what is really going on in the data.....the aim is to move from CATEGORISING to developing CONEPTS, to let higher order meaning emerge from your analysis.
Here are 10 tips I would recommend to help:
1. Go for a walk, untangle yourself from your data and just see what comes to you
2. Try telling someone what it all means in five minutes....you may already know how it hangs together
3. Draw lots of diagrams and pictures until one comes out that seems to fit
4. Talk to colleagues to get it out of your head and bounce it off someone else
5. Cross reference categories with demographics.
6. Have a look at Negative Cases or Outliers...they will tell you alot about the rest
7. Go to theories in the literature..do they kick start your thinking?
8. Summarise each of your participants findings into one paragraph, lay all of them out on the floor and then compare and contrast these, not the codes.
9. Struggle
10.and keep struggling!
REMEMBER..build an argument or a narrative...not a list!
Here is an incredibly good paper by the master, Pat Bazeley Research Support P/L and Australian Catholic University
Ive been having an interesting experience writing up the results of a PAR study this week, particularly in regard to issues of confidentiality.
Participatory action research prides itself on dealing with issues of power is research, positioning the partners as equals to the researcher who together co-construct meaning and develop something that can lead to valuable local change.
The possible dilemmas that can arise, however, can mediate against this effort...in my own case, for example, the participants became concerned about identifying as authors on the paper as it would mean people might guess who they were...some wanted authorship and others voted no, leaving us in a quandry..should we change the results so they are less revealing so authorship is possible? should we say no authorship but then they miss out? Its not as void of power as we might think..
Also what to do if the participants are concerned about some of their direct quotes in retrospect...there perspective on the issues may have changed since the research was conducted...should the paper be changed? Is the paper editing meeting another 'interview' ? Should it be coded and integrated into the results?
Triangulation is one of the core concepts in qualitative research, supporting the production of rigourous, trustworthy studies...there are many ways to do it..it's all about multiple perspectives
-use multiple theories to analyse data to mediate against using only your pet concepts
-use multiple coders, chosen because of different views to add richness to your analysis
-use multiple sources of data to look at a phenomenon from different angles
This week i have helped students design 2 new studies,each taking advantage of these methods
One involves looking at ethical dilemnas clinicians face when having to use the DSM in everyday practice. We are looking at how different professions deal with this, including family therapists, clinical psychologists and psychiatrists. We will have 3 data sources; a survey of a 100 people, in depth interviews with 20 and 3 detailed case studies of work with clients. We will also form a coding committee of representatives from these professions to meet 4 times during the coding process.
The other involved a study of the experience of families admitted to a kids hospital with a child with anorexia, an admission for those families struggling with standard outpatient family therapy. Our info sources are family interviews, audio-diaries to catch significant moments and videos of family meal sessions, all at three time point before, during and after admissions
These two designs take advantage of multiple sources of data and multiple perspectives and mean that our conclusions will be more robust....
HERE is a great paper, an old one, on use of team work in qualitative research that supports our collaborative coding parties in QRIP!
"Multidisciplinary teams, in particular, result in a broadening of possibilities in the research: “A multi-faceted investigation can yield more information and be more exciting than one which is restricted to a single mode of knowing” (Riesman & Watson, 1964, p. 286). Multidisciplinary teams bring together people whose training calls on highly diverse assumptions and different knowledge bases (Opie, 1997). A team can draw on all the “fore-understandings” of individual members about the area being researched, giving a wider base from which to explore the data."
I had a great research design meeting with a student and a colleague yesterday where the term ethnography was discussed in reference to a study looking at the cultural appropriateness of Australian parenting training in Africa.
We suggested a participatory and ethnographic framework but what the hell were we talking about when it comes to 'ethnography'?
Ethnography is essentially work done in the actual field, a case study of a social group and has it's origins in anthropology
Data collection is done through multiple sources.... interviews, questionairres but also participant observation (ie: taking your own notes when actually observing the phenomenon under investigation/close up personal experience not distant observation).. particular attention is also taken to interview knowledgeable community members.....
Believe it or not we have a student in our Department wanting to study Reallifesuperheroes...emersing yourself in this community might be ideal for the production of an ethnography, even if it is to some degree an on-line community
Richardson (2000)provides 5 criteria to evaluate ethnographies
Substantive Contribution: "Does the piece contribute to our understanding of social-life?"
Aesthetic Merit: "Does this piece succeed aesthetically?"
Reflexivity: "How did the author come to write this text…Is
there adequate self-awareness and self-exposure for the reader to make
judgments about the point of view?"
Impact: "Does this affect me? Emotionally? Intellectually?" Does it move me?
Expresses a Reality: "Does it seem 'true'—a credible account of a cultural, social, individual, or communal sense of the 'real'?
some examples from the recent Issue of the Journal of Contemporary Ethnography
Boredom and Action— Experiences from Youth Confinement
Few
studies have examined boredom as a central experience of everyday life.
This article adds to the boredom-related literature by examining the
role of boredom and boredom-aversion in the everyday life of young
people confined in secure care for young offenders. Data are primarily
drawn from an ethnographic study in a Danish secure care unit and
include both participant observation and interviews with unit
residents. Drawing on theories of boredom and young people’s creation
of action through risk-taking edgework, the article demonstrates how
boredom is a key experience in daily life in secure care. Waiting is a
defining aspect of the experienced boredom, and the young people spend
much time “doing nothing,” finding it difficult to relate to the unit’s
daily routines. Analyses show that the young people deal with the
experience of boredom through the generation of risk-taking action.
Ringing the Chord: Sentimentality and Nostalgia Among Male Singers
Singing
in barbershop choruses and quartets evokes sentimentality and nostalgia
as a means of interpreting the present and establishing an identity as
a barbershop singer. In choruses devoted to the preservation of a
highly stylized form of singing, a gendered social context shapes the
acquisition of identities in the development of relationships. An
analysis of the barbershop experience reveals that men, particularly
those with common backgrounds, are routinely involved in expressing
deep emotions through ritualized means, allowing them to maintain a
version of traditional masculine identity.
