Writing Up...

Just a quick post to share this article by Ron Chenail about writing up...it covers so many of the things we have been talking about recently but really puts the turbo on the creativity and number of options availalbe to us in writing up

Here The Qualitative Report, Volume 2, Number 3, December, 1995(http://www.nova.edu/ssss/QR/QR2-3/presenting.html)

Some quotes to reinforce the points:
“I write in order to learn something that I didn’t know before I wrote it…not to write until I knew what I wanted to say, until my points were organized and outlined… this static writing model coheres with mechanistic scientism and quantitative research… It ignores the role of writing as a dynamic, creative process….(Richardson L. 1990)
“Writing-up qualitative research inevitably results in the emergence of new ideas and ways of viewing the data and hence plays a crucial role in the analysis process.”(Pitchforth et al. 2005)

Shut Up and Write continues

We had a very productive first ShutUp and Write meeting this Friday with 13 people joining for 7 hours of writing....many people writing up results, a few writing research proposals, discussions and other things...we had lots of interesting chats during the breaks, comparing analysis methods, working on adding some literature to results sections but generally I think the real benefit was simply having an opportunity to write without the distractions of being at home, without access to the internet but in a collegial and supportive setting

I found it pretty amazing how much I got done, writing about half a paper, which is really saying something..plus...Toblerone, Brownies, quiche, slice, fruit and many more goodies...

We planned to more meetings;

1. October19 9am-4pm in the Conference Room in Psychology Clinic, Mackie Building

2. Novermber 30th 9am-4pm OTC 405 as before

Bring your own lap top for the October meeting, computers provided for November.....please email me to express an interest...p.rhodes@sydney.edu.au

Behind the Times

The more I read the more concerned I become about how we are lagging behind in clinical psychology in Australia when it comes to qualitative research in psychology..been reading this INTRODUCTION TO QUALITATIVE METHODS IN PSYCHOLOGY Dennis Howitt Loughborough University where he provides a history of qual in psychology, marking the 1980's as when qual became more accepted..

That is certainly not the case with clinical psychology here where we lag way behind...even in 1940 it was understood that science can include qualitative..

If we rejoice, for example, that present-day psychology is . . . increasingly
empirical, mechanistic, quantitative, nomothetic, analytic, and operational,
we should also beware of demanding slavish subservience to these presuppositions. Why not allow psychology as a science – for science is a broad and beneficent term – to be also rational, teleological, qualitative, idiographic, synoptic, and even  non-operational? I mention these antitheses of virtue with deliberation, for the simple reason that great insights of psychology 
in the past – for example, those of Aristotle, Locke, Fechner, James, Freud– have stemmed from one or more of these unfashionable presuppositions. (Allport, 1940, p. 25)

Thank god for Health Psychology and Critical Psychology who still wave the flag...

Discourse Analysis??

Its a pretty interesting reflection on the conservatism of clinical psychology that we have never had a study conducted in our program using discourse analysis...never.....pretty astounding when you think of all the potential it holds for deconstruction of our practices and all the power plays requiring such......it demonstrates how easy it is to set up a research silo, a product partly of our poor engagement with other disciplines.......

Here is a really great article on it, particularly Foulcault influenced discourse analysis, by Linda Graham from Queensland University of Technology...she quotes Stephen Ball (1995: 267) t “the point about theory is not
that it is simply critical” ..its purpose is  “to engage in struggle, to reveal and undermine what is most invisible and insidious in prevailing practices.”

Her work relates to ADHD and includes critque of the DSM...here.......pretty impressed by her work!!!

Graham, Linda (2006) The Politics of ADHD. In Australian Association for Research in Education (AARE) Annual Conference, 26th-30th November, Adelaide.

This essay offers a critical review of the problem we call “ADHD‿. In the first part of the discussion, the author presents an analysis of the literature surrounding Attention Deficit Hyperactivity Disorder. Adopting a lens informed by the work of Foucault, she teases out the medical and psychological models to show the interdependency between these otherwise competing knowledge-domains. She argues that as it currently stands the construct serves political ends and questions whether a diagnosis of ADHD is helpful - and if so, for whom? In the second part, the author considers what role schooling practices might play in the pathologisation of children and interrogates the ADHD phenomenon as a symptom of the pathologies of schooling. Graham concludes by suggesting how we might arrest the rising rate of diagnosis by thinking 

A 5 step reflecting process for collaborative coding?

Today we held a great collaborative coding session where a project was presented with focus groups held at three time points..we split the group in three, allocated transcripts from each time point and then came together after discussion in each group to compare notes as across the whole project...

This got me thinking about the best procedure to run such team reflections and how it can draw directly from Post-Milan Systemic Family Therapy reflecting teams

A 5 step reflecting process for collaborative coding....

1. Step 1...each group reads transcripts
2. Each team discusses meaning
3. Each team shares their analysis with each other while the researcher remains silent
-This discussion should start with affirmations directed to the researcher before constructive comments
-This discussion should always be tentative "Iwas wondering....."
4. The researcher is then asked what interested her the most about the process
5. The group is asked how this relates to their own projects

The allocation of discussion groups ca be made in many ways: analyse using different theories/look at transcripts from those with specific attributes/compare data from different data collection methods...ie: supports triangulation...

Writing Results With Refs

Im learning the hard way at the moment about how to write results after a thrashing by a journal for a study titled   Exploring How Trainees Manage Their Own Distress In-Session: An Interpersonal Process Recall Study

The review was very constructive, pushing me to write better, particularly suggesting that I integrate theory from the literature in my results rather than saving it all for the discussion..I confess to finding this quite difficult but after about 8 hours of mental struggle Ive found my results have really come to life in a much better way that before.before they were descriptive...now they kick arse? well..they are better but it's still a work in progress...here they are, completely unproofed and informatted..they are obviously out of context without the intro and method but I thought some of you writing at the moment might find it useful..

Results

The Discrepancy Between Planned Content and In-Session Process

Watzlawick, Bavelas and Jackson (1967) differentiate between content and process in therapy. The former is seen as the digital or rational where messages are unequivocal and can be analysed easily. The latter is analogue or emotional and messages as more subtle and hard to interpret. In this study we saw the overwhelming majority of trainees become distressed when faced with unexpected processes in the therapy room.

They described feeling as “lost”, “confused”, “unsure”, “floundering”, having a “mental blank” or unsettled.

Trainees recalled preparing well for their sessions, including ‘reading books and notes’, ‘revisiting the formulation’ and ‘making a number of session plans’ but felt that this preparation did not necessarily equip them for what was occurring that moment in the room.

 In-Session Interpersonal Difficulties

Many of the unexpected processes involved interpersonal difficulties between trainee and client, including clients who are seen as resistant, disengageing, attempting to assert control or seeking direct advice.

Here a clients is seen as being resistant to cognitive restructuring.

What was I meant to say to that? In all the cognitive therapy books the person says, “Wow! Now that you put it like that, I completely agree.”

Here a trainee becomes confused when a client disengaged, despite a previously strong therapeutic relationship.

When I said something that wasn’t an answer to her mind, you could tell she  immediately disengaged. She stopped looking at me and started playing with tissues. Whereas every other time she was completely looking at me, very, very engaged.

One trainee described feeling “frozen” and “backing down” when a client was seen as overtly counteracting explicit guidelines for homework tasks. In this case a couple were asked to write on the same sheet, to facilitate a comparison of perceptions concerning events. The trainee recalls the husband disagreeing.

 “Can we have separate ones of these?” I went “Oh! Okay! Yep”, agreeing to it because I was just so thrown by something that I thought we’d really explicitly talked about.

Another trainee described feelings of powerlessness in the face of a client who frequently requested direct advice.

It’s an expectation of me to provide advice, it didn’t sit comfortably. I felt like I was being trapped in a corner. She was putting me on the spot – trying to nail me down kind of thing. I didn’t feel very comfortable [slight nervous laugh]. I notice that I wave my hands around a lot more when I’m feeling scared.

Each of these examples suggest that trainees had an expectation that client’s would follow therapy as described in text books or in their own pre-session planning, as if what Bennet-Levy & Thwaites (2007) calls declarative knowledge would be translated directly to procedural (skills-in-action) without reflection on the interpersonal dynamics at play in the room.

 Complex or incongruous presentation

While trainees found interpersonal interactions difficult they also had difficulty making sense of the complexity of the client’s story.

What I had sitting on that chair were three different session plans. What I did not expect was for her to have all three as a presenting issue. “Ahh! Which one?!” I got frazzled, there was a sense of urgency.

These stories were seen to  “lack of coherence and stability,” and trainess were unsure of how they related to the presenting problem or how to integrate these experiences into their formulations or treatment

Here a trainee is poised to intervene with a client who has a history of social anxiety, only to discover that he is telling the story of breaking up with his girlfriend.

I was lost. I thought he was going to go and have coffee with this girl and explore his feelings, so I thought we were testing predictions like, “It’ll be awkward” and “It won’t be fun.” It became apparent that actually he was going there to tell her he wasn’t interested. I was like like, “Riiiigggght. Now I’m going to leave.”

Here a trainee finds it difficult to make sense of seemingly incongruous reactions to a death in the clients life.

I’m thinking, “What’s going on with you? This guy had died and you’re . . . reacting with humour.’”He does impressions. He does impressions when asked emotional questions.

The emotional reactions described by students regarding both interpersonal processes and complex client presentations reflect discomfort with what family therapists Anderson, H. & Goolishian, H. (1992) call a ‘not-knowing’ position, an essential prerequisite to curiosity (Palazzoli et al. 1980) that leads one to ask open-ended reflexive questions of the client, rather than concern oneself with how to best intervene. It would seem in both of these types of situations trainees are momentarily “frozen’ or ‘baffled. Many also described feelings of inadequacy at not knowing what to do next. ’

I didn’t like that feeling. It felt like I was ineffective, like I’d lost purpose, um, and I don’t see how that’s useful to the client.

One trainee described how her supervisor helped her see that her negative judgments were related to her idea that therapy had to be formal and technical, rather than natural and conversational.

I was thinking “That part wasn’t special – it’s conversation” but my supervisor today was going, “That – that’s therapy. That bit where you casually dismiss that you re-framed that he lost his job? That’s therapy, What about the bits where I couldn’t fill in the sheets and do the ABC  and he’s like, “Yeaaahhh.”

In this example the trainee seem to have misconstrued the collaborative nature of therapy, where meaning is contracted through dialogue.  (Overholser, 2011)

Activation of Rudimentary Reflections

Bennett and Thwaites (xxxx) provide some conceptual confirmation for the findings in this study, asserting that the reflective system is activated when there is a mismatch between expectations and reality. In their approach trainees are taught to focuss attention on the problem at hand, develop a mental representation of it and engage in an active cognitive proves of problem solving. However this occurs in supervision, as a form retrospective of reflection on-action rather than in-session reflection in-action (Schon)

While they were initially ‘taken aback’   Trainees describe how, in “what seems like a fleeting moment”  or a ‘rush’ they are able to acknowledge the emotion they are feeling then struggle to engage in a process of reflection (in my head I was like “What the hell?”).

Three different strategies were then described, each of which has been described here as rudimentary.

Referencing the expert

A significant number spoke about mentally referencing a supervisor or perceived superior to help them decide how to proceed. For one the supervisor operated as a “voice in your head going, you should do that”. For another it served as a reminder of “what is the right thing to do in terms of an intervention point-of-view”?

In some cases this strategy was seen as helpful, particularly when their supervisor was trusted or open about their own mistakes. For others comparisons with an “expert” only exacerbated feelings of inadequacy.

He’d have something meaningful to say, with his level of insight. Whereas I don’t know what to do with her.

Attempts at transference

Despite the range of distressing emotions experienced by trainees the majority described an ability to acknowledge these emotions while they were happening in sessions. In some cases trainees discussed how their own emotional response communicated to them the emotional experience of their client.

“I was feeling lost, and I guess that’s how she was feeling as well.”

“He’s probably feeling really upset and really disappointed, too”

These reflections, while important, do not yet mirror the complexities of transference dynamics, in terms of recognising that interpersonal difficulties encountered with clients are actually a potential tool for exploring complex unconscious processes (Gabbard, 2001).

Check the therapeutic relationship

Trainees also engaged in momentary reviews of the therapeutic relationship, either checking whether the relationship was strong enough to handle the difficult moment, or reviewing it so as to make a choice between trying to re-establish rapport or continue with the skills-based intervention. Relationships reviewed as strong made it “easier to say a lot of things” and caused some easing of distress because “I don’t really mind making mistakes in front of her.”

Engage in Self-talk

Self-talk was also a common strategy used to try and get the session back on track after distressing events. Trainees described talking themselves through fleeting reviews of their formulation and treatment plans.

Just trying to quickly re-formulate in my head how I could quickly change th  direction of therapy to be still effective. 

I’m sort of not wanting to shut that possibility down but at the same time I do sort of have a therapy plan which I went back to. I’m trying to make sure that we take each session somewhere and that I use the time well.

This differs significantly from the kind of complex inner conversations described by Rober et al. (2008) in his analysis of work by more experienced therapists. Here the therapist can make mental room for a conversation between multiple positions, including the interaction in the room, client’s story and planning for action. This conversation is held internally but at a distance, a process of constructive hypothesising about how to respond in the moment (Rober, 2002).

Retreat to Safety of Non-Directive Counselling

For many trainees, especially those who lacked confidence in treatment techniques the next step after reflection was to retreat to the safety of non-directive counselling.

I’m supposed to be using some sort of technique and I don’t know what I’m doing. I  don’t have any strategy for her, CBT or DBT or schema stuff– I have nothing to structure her thinking.

Skills used included listening, affirming, summarising, reflecting or asking questions. For the majority of trainees these basic counselling skills came more  “naturally” than skills that were seen as more interventive.

I feel like being warm and being empathic towards a client are not things I have to fake, and not things I have to remind myself to do. I suppose things that I’m still less comfortable with, and still getting used to, are you know, particular questioning techniques, or, you know, more sophisticated cognitive challenging and things like that.

Using basic counselling skills in the moment also served as a “coping mechanism”, “buying time,” supporting a “facade of control” and the maintenance of a “professional stance.” For many therapists “surviving” the distressing moment was the goal.

Risk Taking

Other trainees, however, decided to push themselves ‘outside of their comfort zones’and focus on intervention. Some were motivated to “take these risks” by “professional growth.’ Intervention did not necessarily flow from a reconceptualization of the case or the formulation based on the distressing experience but was instead described as an attempt to ‘push ahead’, be more ‘directed’

See! It works!  You’ve just got to talk. Now you can get back to what you’re doing.” I’m pleased I did something. That it actually went okay – that when I interrupted they both seemed to settle fairly quickly – was a good reminder to me that when I do this it doesn’t actually damage the relationship too much.

Arguably, despite the limitations, this type of risk taking is critical to learning (Spellman & Harper, 1996). Trainees who took risks, however, were careful to review the therapeutic relationship first to see if it was secure. . Those trainees who felt it was less seemed more tentative in their decisions and less likely to respond in a way that might “challenge” the client. 

Qual Unconference 2013 Anyone?

Im thinking heavily about running an Unconference or QualCamp at Sydney Uni next year..a what? I hear you say...

Here is how it works..first the things not done

No pompous plenaries or panels

No prepared talks
No $780 rego fees
No overheads
No horrible meals
No domination by people who want to show off

Unconferences are a new trend, about 5 years or so in Australia, (see News-com.au article ) where the emphasis is placed on collaborative learning rather than pontificating....basically everyone is a particpant, they all turn up and through skilled facilitation develop the agenda on the first day.....a number of streams might be developed, cented around actual problem solving, story telling and also teaching technical skills...but there are no dyadic talks..maybe 7mins max of talk time given to many people, then discussion...

If we held one at Uni, for example, 100 people might meet in the quadrangle then split up based on interests and find spaces to run sessions.. uni cafes, parks, Glebe Point Road cafes, lecture theatres, lecture rooms wherever..

Instead of panels there are Unpanels done in a fishbowl discussion style with speakers in the middle, rotating with participants..



Spectrograms can be used to place people along a continuum of agreement/disagreement with a controversial topic..people are then interviewed along the line

Community-Mapping can be done to visually picture how everyones research projects are related to each other....

..also Lightening Talks, SpeedGeeking and more...

There are many other creative ideas..check this blog and others.......want to be involved?

Get in Quick!

Onwards and Upwards? Lets See

Proposal to Head of School

Development of Qualitative Research Education Unit in School of Psychology

Dr Paul Rhodes, Clinical Psychology Unit

Background

An informal research support group was developed 18 months ago, Qualitative Research in Psychology (QRIP), to support qualitative researchers in the School of Psychology. Fortnightly meetings have been held since this time focussing on educating students on the full gamut of qualitative research knowledge, from epistemology to design, methods, data collection techniques, data analysis methodologies, rigour, coding, writing up, publishing and more.

Methods explored have included thematic analysis, grounded theory, discourse analysis, conversational analysis, interpersonal process recall, interpretative phenomenological analysis, narrative inquiry, community-based participatory action and more.

This material is presented in an informal setting, involving discussion and development of current research and serves to augment lectures given in the school. Many of the group meetings have involved interview role playing, collaborative research design, collaborative coding and many other activities that are most suitable for a group setting.

Achievements to Date

Since QRIP’s inception 87 students and academics have joined, from the School of Psychology, but also from Health Sciences, the Medical School, Social Work, Occupational Therapy and other affiliated disciplines. Fifteen to twenty students and academics attend meetings, depending on content, and a majority are Honours, DCP and PhD students from the School of Psychology. Many of these students have teamed up to work together in between meetings, particularly for cross coding purposes.  Formal feedback concerning student satisfaction and feedback regarding these groups will be sought at the end of 2012 and annually from then.

 The remaining members are supported by our listserve, our Usyd E-Community site, our blog and occasional one-to-one meetings conducted by myself. The blog has proved a particularly innovative and effective resource, well suited to the way in which students source information and interaction. The blog receives approximately 500 hits per week by those seeking advice on qualitative research, including our own students. http://qual-rip.blogspot.com.au/. It appears as the second international Google search item under search terms ‘qualitative research psychology.’ It has also been recently recommended by the British Psychology Societies Readers Digest to U.K Psychologists.  Recognition and encouragement has also come from Prof. Lyn Richards (Founder of QSR International), "Love your blog – and your openness and practical de-mystifying approach. Qualitative research needs so much more of such input – and an ability to laugh at itself".

This group has also been recognised by the Institute of Teaching and Learning, University of Sydney, including an article in their regular Newsletter and presentation at ITL Seminar. I am also presenting related material at The Australian Conference on Science and Mathematics Education (The 18th UniServe Science Conference). Rhodes, P. What standards should be set for qualitative research conducted in a science faculty: Psychology, rigour and the politics of evidence. A close relationship with The Centre for Values, Ethics and Law in Medicine, University of Sydney has also developed, including research supervision of our students being supported by their academics and exchanges in teaching responsibilities.

Given these developments I am proposing that QRIP be developed into the Qualitative Research Education Unit, hosted by the School of Psychology.

Objectives

1. To provide technical support to research students in the School of Psychology conducting qualitative research

2. To promote methodological diversity in qualitative research in the School.

3. To promote high standards of rigour in qualitative research in the School, including the use of methods to ensure dependability and credibility of findings.

4. To foster collaboration with other academic qualitative researchers in the University of Sydney and other Universities, who can further support research students in the School of Psychology, providing specific technical expertise when required, including serving as Associate Supervisors.

5. To actively advocate for the further recognition of qualitative research in the School of Psychology and Psychology as a field.

Functions

Many of the functions of the Unit would be consistent with QRIP as it now stands.

1. Provision of regular education and research support group meetings

2. Provision of Digital resources: Listserve, Usyd E-Community and Blog resource

3. Provision of one-to-one consultations for research students in the School of Psychology

Additional functions would include:

4. Development of a group of Academic Associates to support the endeavours of the Unit and research students in the School of Psychology

5. Hosting regular small seminars with guest speakers to support research students in the School of Psychology

6. Develop and advocate for clear standards of excellence in the design and conduct of qualitative research in psychology

7. Support a mentoring system between students at different levels of research, providing opportunities for PhD students to gain experience in supporting research students.

Proposed Outcomes

1. Increased number of research students including qualitative methods in their research, where appropriate.

2. Increased standard and rigour of qualitative research conducted by students in the School of Psychology.

3. Greater number of students asking and answering questions that are suitable for qualitative research, including therapeutic process research, research on reflective practice, community-based action initiatives, consumer-driven research.

4. Enhanced pool of Associate Supervisors and technical expertise available to students.

5. Further recognition of the School of Psychology as supporting innovation and diversity in research.

6. Enhanced capacity to manage larger numbers of students requiring supervison.

Structure

To date QRIP has been essentially coordinated  informally, with feedback from students. I am seeking to change this by:

1. Convening a committee to meet once every four months to discuss directions and organisation of events, made up of both academic and student representatives.

2. Committee would also conduct annual reviews of both student satisfaction and outcomes (including number and quality of publications)

3. Report to Head of School each year on achievements and future goals.

4. Development of a central web page, linked from School of Psychology from which the details regarding meetings, events, blog could be accessed.

Dr Paul Rhodes

Senior Lecturer

Clinical Psychology Unit.

Thesis Boot Camp Anyone ? Shut Up and Write

The best Phd blog in the universe is obviously the Thesis Whisperer who has developed fantastic idea  for research students, Shut Up and Write meetings held in RMIT Cafes or thereabouts....the idea started in San Francisco where writers met together to chat, drink coffee, then 'time-box', keeping quiet for 45 minutes until the next break....I notice the library at UTS has started doing it and that Melbourne Uni has also done Thesis Boot Camps....

Seems a perfect idea to mediate against the isolation of writing AND procrastination...

Im flagging the idea for QRIP too, but with some of the breaks for consultation about the writing of qual research, with academics but also between students....in situ group problem solving...Ive sent an email out to all members, lets see what happens

Standards for Quality ?

I have a talk coming up titled What standards should be set for qualitative research conducted in a science faculty: Psychology, rigour and the politics of evidence which i will present at the Australian Conference on Science and Mathematics Education 2012, Sydney, Australia..I havent written it yet but have been reading a great paper, below and a few others

Quality in qualitative research

Simon C Kitto, Janice Chesters and Carol Grbich

Med J Aust 2008; 188 (4): 243-246.

They propose the following standards to judge good qual research:

• Clarification and justification;
• Procedural rigour;
• Representativeness;
• Interpretative rigour;
• Reflexivity and evaluative rigour; and
• Transferability.

Clarification and justification

As in all forms of research, clarity of research question reflected in the aims of the study is essential for evaluating results and their interpretation. The demonstration of theoretical rigour (referring to the soundness of fit of the research question, aims and the choice of methods appropriate to the research problem11) is extremely important.

There is a wide variety of named qualitative approaches that are underpinned by particular theoretical perspectives. In addition, the researcher may use basic field research (question, investigation, interpretation). Regardless of the theoretical approach used, the choice requires justification in reference to the research question of the study.

Procedural rigour

Procedural, or methodological, rigour concerns the transparency or “explicitness” of the description of the way the research was conducted. It involves detailing issues of accessing subjects; development of rapport and trust; how data are collected, recorded, coded and analysed; and accounts of the manner in which errors or subject refusals are dealt with.4,11,22 In this regard, readers and reviewers may ask the following questions while examining descriptions of qualitative methods: How were participants/settings accessed? Who was interviewed/observed? How often? For how long? What interview questions were asked? What was the purpose of any observation? Which policy documents/case notes were accessed? How were they assessed? How was collected data managed?

Representativeness

There are a number of commonly available, non-probability sampling approaches. Maximum variation sampling seeks representativeness of all aspects of the topic in terms of participants. Homogenous sampling consists of the selection of a group fitting specified criteria. Snowball samplinginvolves networking from one difficult-to-access type of participant to a wider range of participants. Finally, convenience sampling involves studying easily accessed individuals or groups. This last technique obviously presents its own ethical dilemmas of the “insider” type and is possibly the weakest form of sampling in terms of allowing conceptual generalisability.4,15,16,22,23Maximum variation is the ideal when a holistic overview of the phenomenon is sought; for instance, the question of how a particular hospital department operates may involve sampling in the wider organisation as well as within the individual department and among recipients of services.

Simply mentioning the sampling strategy in the methods section of a qualitative research paper is not sufficient. The key findings of the research need to be evaluated in reference to the diverse characteristics of the research subjects. Through constantly comparing the experiences and responses of the participants against each other, subtle but significant differences can be uncovered that can generate profound insights into the phenomena under study.19

Interpretative rigour

Interpretative rigour relates to as full as possible a demonstration of the data/evidence. In qualitative research, a commonly used concept is inter-rater reliability. This refers to using a type of researcher triangulation by which multiple researchers are involved in the analytical process. This is an attempt to increase the validity and reliability of the study19 through the provision of a more complex and nuanced understanding of the possible interpretations of the objects of the research.11 In contrast to the quantitative research paradigm, what is important in this process is not the level of consensus, but the opportunity for discussion among analysts to provide opportunities for developing further coding.19

A related technique is that of respondent validation, or member checking. This entails offering subjects interviewed the opportunity to view and amend their transcripts as a type of validity.12 However, this approach does have limitations due to the evolution over time of the positions and purposes of the researchers and participants, thereby potentially affecting interpretations and accounts. Respondent validation should be thought of as part of a process of reducing error, which involves the generation of further original data, which then requires interpretation.8

Other techniques that enhance interpretative rigour are the differing forms of triangulation: data (multiple evidentiary sources; ie, documents, interviews, survey data, observation), methods (multiple methods), and theory (multiple theoretical and conceptual frames applied to the research to enhance insights into phenomena). Using these forms of triangulation allows the development of a comprehensive understanding of the phenomena and can ameliorate the potential bias of simply using one method.4,5,8,11,16,22

In the interpretive process, accounts of “negative” or “deviant” cases are especially important. These are explanations pertaining to data or evidence that contradicts the researchers overall explanatory account of the phenomena.5

In sum, a clear description of what forms of analysis were used, the process and what were the major outcomes of the analytical process in terms of findings is needed to ensure quality for the author, and to enable an assessment to be made in terms of the analytical quality of the research by the reader.

Reflexivity and evaluative rigour

Reflexivity is where researchers openly acknowledge and address the influence that the relationship among the researchers, the research topic and subjects may have on the results.4,11,13 Fundamentally, reflexivity requires a demonstration by the researchers that they are aware of the sociocultural position they inhabit and how their value systems might affect the selection of the research problem, research design, collection and analysis of data.15 It also refers to an awareness by the researchers of the social setting of the research and of the wider social context in which it is placed.4

Evaluative rigour refers to ensuring that the ethical and political aspects of research are addressed. Typically, this refers to proper ethics approval from appropriate committees covering confidentiality, informed consent and steps to avoid possible adverse effects on the subjects. Importantly, where appropriate, relevant community leaders should be consulted in the design and conduct of the research.11 Researchers should revisit their actions and interactions within the research process to ensure as “accurate” as possible portrayal of the production of their findings.

Transferability

Conceptual generalisability and transferability refer to how well the study’s findings inform health care contexts that differ from that in which the original study was undertaken.4 For example, a review of data from qualitative studies was conducted on a wide variety of doctor–patient interactions about medication compliance.24 The authors examined barriers to patients taking prescribed medication as directed by their doctors and found that patients were often inclined to resist taking medicines, not because of problems with the patients, doctors or systems, but because patients were concerned about the medicines. This type of study allows for the construction and transfer of general policy on medicine-taking (through, for example, less emphasis on patient behaviour modification and more emphasis on production of safer medicines) and practice (suggesting, for instance, that doctors should assist lay evaluations through provision of more information, support, feedback and safe prescribing practices

Here is an approach that perhaps has more consideration for the many different types of qual research

International Journal of Research & Method in Education

Vol. 30, No. 3, November 2007, pp. 287–305

The issue of quality in qualitative 

research

Martyn Hammersley*

The Open University, UK

m.hammersley@open.ac.uk MartynHammersley 0000002007 330Taylor & Francis 2007 Original Article 1743-727X (print)/1743-7288 (online) International Journal of Research & Method in Education 10.1080/17437270701614782 CWSE_A_261329.sgm Taylor and Francis

This article addresses the perennial issue of the criteria by which qualitative research should be evaluated. At the present time, there is a sharp conflict between demands for explicit criteria, for example in order to serve systematic reviewing and evidence-based practice, and arguments on the part 

of some qualitative researchers that such criteria are neither necessary nor desirable. At issue here,

in part, is what the term ‘criterion’ means, and what role criteria could play in the context of qualitative enquiry. Equally important, though, is the question of whether a single set of criteria is possible across qualitative research, given the fundamental areas of disagreement within it. These reflect 

divergent paradigms framed by value assumptions about what is and is not worth investigation. In 

addition, there are differences in methodological orientation: over what counts as rigorous enquiry, 

realism versus constructionism, and whether the goal of research is to produce knowledge or to serve 

other goals.

Designing MIxed Methods Research

I was involved in a really useful meeting today looking at a mixed method study that involved the sequential use of community-based participatory action research as a hypothesis generation tool, followed by a more quantitative testing of emergent themes and theory from that study....

Mixed methods studies have so many advantages and can be used in many different ways.

1. Quant data can support qual analysis, triangulating, or adding trustworthiness to findings

2. Qual studies can be nested in bigger quant studies like randomised control trials

3. Quant and qual studies can be done simultaneously with both sets of findings analysed together (the inside and outside views)

4. Qual studies can generate hypotheses that are then tested by quant studies

There are many more designs of course...if you want to know more I highly reccommend the following book chapter, just email me p.rhodes@sydney.edu.au if you are a Usyd student and want access to it..

Research design : qualitative, quantitative, and mixed methods approaches / John W. Creswell. 

Edition 2nd ed. Publisher Thousand Oaks, Calif. : Sage Publications, c2003. Chapter 11: Mixed methods procedures

Mix it up!!!!

How to Derive Meaning (Not Just Codes) From our Data: Ten Tips

I have recently been discussing the issue of making meaning from your data not just listing lots of themes with a couple of my students...trawling through your transcripts and coding similar phenomenon might be time consuming and important but in the end this type of coding is only a tool to help you think about the depth and complexity of what is really going on in the data.....the aim is to move from CATEGORISING to developing CONEPTS, to let higher order meaning emerge from your analysis.

Here are 10 tips I would recommend to help:

1. Go for a walk, untangle yourself from your data and just see what comes to you
2. Try telling someone what it all means in five minutes....you may already know how it hangs together
3. Draw lots of diagrams and pictures until one comes out that seems to fit
4. Talk to colleagues to get it out of your head and bounce it off someone else
5. Cross reference categories with demographics.
6. Have a look at Negative Cases or Outliers...they will tell you alot about the rest
7. Go to theories in the literature..do they kick start your thinking?
8. Summarise each of your participants findings into one paragraph, lay all of them out on the floor and then compare and contrast these, not the codes.
9. Struggle
10.and keep struggling!

REMEMBER..build an argument or a narrative...not a list!

Here is an incredibly good paper by the master, Pat Bazeley Research Support P/L and Australian Catholic University

Qual-Memes?

Confidentiality in Participatory Action Research

Ive been having an interesting experience writing up the results of a PAR study this week, particularly in regard to issues of confidentiality.

Participatory action research prides itself on dealing with issues of power is research, positioning the partners as equals to the researcher who together co-construct meaning and develop something that can lead to valuable local change.

The possible dilemmas that can arise, however, can mediate against this effort...in my own case, for example, the participants became concerned about identifying as authors on the paper as it would mean people might guess who they were...some wanted authorship and others voted no, leaving us in a quandry..should we change the results so they are less revealing so authorship is possible? should we say no authorship but then they miss out? Its not as void of power as we might think..

Also what to do if the participants are concerned about some of their direct quotes in retrospect...there perspective on the issues may have changed since the research was conducted...should the paper be changed? Is the paper editing meeting another 'interview' ? Should it be coded and integrated into the results?

Going Multiple

Triangulation is one of the core concepts in qualitative research, supporting the production of rigourous, trustworthy studies...there are many ways to do it..it's all about multiple perspectives

-use multiple theories to analyse data to mediate against using only your pet concepts
-use multiple coders, chosen because of different views to add richness to your analysis
-use multiple sources of data to look at a phenomenon from different angles

This week i have helped students design 2 new studies,each taking advantage of these methods

One involves looking at ethical dilemnas clinicians face when having to use the DSM in everyday practice. We are looking at how different professions deal with this, including family therapists, clinical psychologists and psychiatrists. We will have 3 data sources; a survey of a 100 people, in depth interviews with 20 and 3 detailed case studies of work with clients. We will also form a coding committee of representatives from these professions to meet 4 times during the coding process.

The other involved a study of the experience of families admitted to a kids  hospital with a child with anorexia, an admission for those families struggling with standard outpatient family therapy. Our info sources are family interviews, audio-diaries to catch significant moments and videos of family meal sessions, all at three time point before, during and after admissions

These two designs take advantage of multiple sources of data and multiple perspectives and mean that our conclusions will be more robust....

Paper Alert:Barry (1999) Using Reflexivity to Optimize Teamwork in Qualitative Research

HERE is a great paper, an old one, on use of team work in qualitative research that supports our collaborative coding parties in QRIP!

"Multidisciplinary teams, in particular, result in a broadening of possibilities in
the research: “A multi-faceted investigation can yield more information and be
more exciting than one which is restricted to a single mode of knowing” (Riesman &
Watson, 1964, p. 286). Multidisciplinary teams bring together people whose training calls on highly diverse assumptions and different knowledge bases (Opie,
1997). A team can draw on all the “fore-understandings” of individual members
about the area being researched, giving a wider base from which to explore the data."

What is Ethnography?

I had a great research design meeting with a student and a colleague yesterday where the term ethnography was discussed in reference to a study looking at the cultural appropriateness of Australian parenting training in Africa.

We suggested a participatory and ethnographic framework but what the hell were we talking about when it comes to 'ethnography'?

Ethnography is essentially work done in the actual field, a case study of a social group and has it's origins in anthropology

Data collection is done through multiple sources.... interviews, questionairres but also participant observation (ie: taking your own notes when actually observing the phenomenon under investigation/close up personal experience not distant observation).. particular attention is also taken to interview knowledgeable community members.....

Believe it or not we have a student in our Department wanting to study Reallifesuperheroes...emersing yourself in this community might be ideal for the production of an ethnography, even if it is to some degree an on-line community


 Richardson (2000) provides 5 criteria to evaluate ethnographies

1. Substantive Contribution: "Does the piece contribute to our understanding of social-life?"
2. Aesthetic Merit: "Does this piece succeed aesthetically?"
3. Reflexivity: "How did the author come to write this text…Is there adequate self-awareness and self-exposure for the reader to make judgments about the point of view?"
4. Impact: "Does this affect me? Emotionally? Intellectually?" Does it move me?
5. Expresses a Reality: "Does it seem 'true'—a credible account of a cultural, social, individual, or communal sense of the 'real'?

some examples from the recent Issue of the Journal of Contemporary Ethnography

Boredom and Action— Experiences from Youth Confinement

1. Tea Torbenfeldt Bengtsson ttb@sfi.dk

Abstract

Few studies have examined boredom as a central experience of everyday life. This article adds to the boredom-related literature by examining the role of boredom and boredom-aversion in the everyday life of young people confined in secure care for young offenders. Data are primarily drawn from an ethnographic study in a Danish secure care unit and include both participant observation and interviews with unit residents. Drawing on theories of boredom and young people’s creation of action through risk-taking edgework, the article demonstrates how boredom is a key experience in daily life in secure care. Waiting is a defining aspect of the experienced boredom, and the young people spend much time “doing nothing,” finding it difficult to relate to the unit’s daily routines. Analyses show that the young people deal with the experience of boredom through the generation of risk-taking action. 

Ringing the Chord: Sentimentality and Nostalgia Among Male Singers

1. Jeffrey Eugene Nash jenash@ualr.edu
   1. University of Arkansas, Little Rock, AR, USA

Abstract

Singing in barbershop choruses and quartets evokes sentimentality and nostalgia as a means of interpreting the present and establishing an identity as a barbershop singer. In choruses devoted to the preservation of a highly stylized form of singing, a gendered social context shapes the acquisition of identities in the development of relationships. An analysis of the barbershop experience reveals that men, particularly those with common backgrounds, are routinely involved in expressing deep emotions through ritualized means, allowing them to maintain a version of traditional masculine identity.