Social Research Update 45: The Use of Piloting �

Social Research Update is published quarterly by the Department of Sociology, University of Surrey, Guildford GU2 7XH, England. Subscriptions for the hardcopy version are free to researchers with addresses in the UK. Apply by email to sru@soc.surrey.ac.uk.

Dr Elizabeth McDermott is currently postdoctoral research assistant on this AHRB funded project; previously she was archivist in the Ceramic Archive, UW, Aberystwyth. Her main research interest is in women’s career development.

Dr Jill Venus is a Senior Lecturer in the Department of Management and IT at the UW, Lampeter. Her research interests include women and business and, rural business and development.

Ms Moira Vincentelli is Senior Lecturer in Art History and Curator of Ceramics at the University of Wales (UW), Aberystwyth. Her major publications are Women and Ceramics, Gendered Vessels (Manchester UP, 2000) and Women Potters, Transforming Traditions (A&C Black, 2003).

This article addresses the dearth of reports of pilot studies (van Teijlingen and Hundley, 1998) by describing one used to strengthen a funding application for a collaborative project between the Lampeter and Aberystwyth colleges of the University of Wales. Piloting helped us to re-formulate and gain funding from the then Arts and Humanities Research Board (AHRB) for the full project, which aims to explore the experience of women working in ceramics and determine their motivations for becoming and remaining self-employed ceramic artists and craft potters. In the first half of this article we describe the issues identified in the literature and in the second half we report on the decisions made during the 21 day pilot.

Those issues concern the size and nature of the sample and the most appropriate mode of interviewing, especially in relation to the possibility of bias, bearing in mind that value for money is uppermost in any funding application.

Sampling

Our original intention for the main project was to interview 60 women ceramic artists and craft potters working on a self-employed basis in Wales, using a small sample of men in similar situations as a control. A representative sample would include the different types of pottery and pottery businesses, from craft and studio pottery made in a potter’s studio to small scale mass produced pottery made on larger premises, and from the sole maker to the small entrepreneur employing several people. Participants could be either working with partners or alone. In either case they should have sufficient control over the process and product to be able to give us information about creativity and economic considerations. We also needed to investigate women working in the different areas and contexts provided in Wales in order to determine the effect of different environments on their lives and work. We knew from experience that there are potteries in all the furthest reaches of Wales and, as the researchers have all lived and worked in Wales for many years, we also knew that travelling from one end of the country to the other could be both time consuming and costly. We therefore needed to consider using telephone interviews in order to maximise value for money.

Interviewing

Several factors needed to be considered during the pilot: the length of the interview, the importance of observation to data collection and to interaction between interviewer and interviewee, the possibility of bias, ethical considerations, and the cost. The literature and previous experience suggested that face-to-face interviews are more costly than telephone interviews, because, by eliminating the cost and time of travel, more interviews can be fitted into a day (Wilson, 1998).

Vaus (1991) suggests that telephone interviews are unsuitable for complex subjects such as ours and, certainly more than an hour on the telephone could be tiring for both parties, affecting the quality of the data. We designed Interview Guidelines for an interview that lasted approximately one hour, which we intended to pilot in both telephone and face-to-face situations to test the length of interview that can be sustained over the telephone and the suitability of the Interview Guidelines.

The discourse surrounding the representation of participants’ perspectives in feminist research and the importance of not misrepresenting them was considered during the pilot (Harding, 1991; Stanley, 1990). Tang (2002) discovered during a pilot study that there was a noticeable difference between interviewing women academics in a formal situation and in their homes. She describes the latter as more “relaxed and intimate” (Tang, 2002) and tried to arrange all the interviews in the main investigation in women’s homes. We are aware that, for people working in the creative arts, it is important that due recognition is given to their work and the researcher needs to know or have seen it. It therefore seemed important to visit them in order to observe their products and the context of their daily lives, rather than interviewing and reporting on them from a distance.

On the other hand, some of the literature suggests that face-to-face interviews are particularly prone to influence by the interviewer’s body language and other physical signals, whereas telephone interviews are not. It is argued that telephone interviews have a distancing effect that eliminates some of the elements leading to bias in face-to-face interviews. Furthermore, previous experience in telephone interviewing indicated that a rapport can be quickly established between women on the telephone. For these reasons we were reluctant to abandon the idea of telephone interviews and we decided to carry out at least one, to help us decide whether or not it was worth pursuing the idea of telephone interviews for the main project.

Bias is present in research for many reasons such as memory distortion, self-serving bias and social desirability bias (Finch, 1984; Ribbens, 1989; Oakley, 1981; Stanley, 1990; Stanley, 1983; Bourdieu, 1996). In qualitative research, bias is often a matter of the actual or perceived power differences in the social relationships between researchers and researched. It is possible for participants to be or to feel that they are situated “as the passive givers of information, with the researcher acting as a sponge soaking up the details provided” (Maynard, 1994). In our case the Ceramic Archive and the School of Art, Aberystwyth, embody a certain amount of status amongst the ceramic community. For example, potters’ curriculum vitae often include a list of the galleries that have bought their work and Moira Vincentelli (a member of the research team) is frequently the person with the power to buy on behalf of the Ceramic Collection, Aberystwyth.

As Finch (1984) asserts, while it is easy for women to get women to talk, this rapport can also increase the potential for exploitation. She describes a process where women relax to the point where they forget they are being interviewed and, despite having been fully informed and signed a permissions sheet for the use of the data, reveal very private things that they later regret (Kirsch, 1999).

Findings and Conclusions

Sampling

The pilot gave us the time to consider our initial ideas about the sample and we revised our intention to include men as a control, especially as we were not intending to perform a truly comparative investigation. Moreover, our study is essentially exploratory inasmuch as there is little academic writing on women and ceramics in Wales. The stereotypical perception of ‘the potter’ is that of a bearded man in sandals (Vincentelli, 2000) and we wished to help redress the imbalance by performing a study of women only.

During the pilot sampling process we located eight potters to give a reasonable spread of types of pottery and working situations in Wales. It became evident that identifying a population from which to sample would be more difficult than we first thought. There are several disparate information sources in Wales from which gendered and geographical data may be extracted, for example: Crefft Forwm Cymru, Crafts Council Wales, the South Wales Potters Association, and the North Wales Potters Association. Many studio potters market their wares actively on the internet, in craft exhibitions, galleries, in brochures and leaflets, and advertisements in journals such as Ceramic Review and Crefft while other, usually small scale potteries seem to rely on word of mouth and a sign at the nearest junction to a trunk road -- sometimes five or more miles along a single track road -- and have to be actively sought out. The pilot indicated that more time than we had originally allotted would be needed for identification of the research population. A stage of visiting craft fairs, galleries and exhibitions was introduced as a method of identifying those with a lower profile.

Interviewing

Most of the women we interviewed considered themselves to be fairly isolated and did not feel part of the community in which they lived. Some had only limited contact with the world of ceramics. All were glad to be interviewed, perhaps because of their isolation, as well as the buying power of the Ceramic Collection and the status attributed to potters whose interviews are held in the Ceramic Archive. Thus the pilot revealed that recruiting interviewees, once they had been located, was not a problem.

The fact that we were only able to carry out one telephone interview restricted the conclusions we were able to draw, but the data gathered was of such good quality that we were assured that the use of telephone interviews should not be discounted. Despite the fact that Aberystwyth is situated fairly centrally in Wales, mileage and time used during the pilot were significant and we were anxious to use our funding efficiently. We discovered that the furthest we would have to travel for interviewing to ensure that our research covered all of Wales would be a round trip of about 240 miles. Clearly only one face-to-face interview per day could be performed, rather than several telephone interviews, confirming that telephone interviews are both time and cost effective (Frey, 1995).

Both face-to-face and telephone interviews allowed the interviewee to be in their own workshop/home whilst being interviewed and there seemed to be little difference in terms of rapport between the two modes. The interviews took between 50 minutes and one hour and twenty minutes. The single telephone interviewee showed no signs of interview fatigue and we were able to use the same Interview Guidelines for both.

The pilot confirmed our expectation that women’s creative work is very important to them and they clearly needed to know that the interviewer was at least aware of it. All those who were interviewed in their workshop/studio were keen to show the interviewer around. They seemed to be more comfortable knowing that they were talking to someone who at least knew and was interested in their work and the making process. One came to the interviewer’s home and brought, unbidden, some of her work with her. The woman who took part in the telephone interview asked the interviewer, when describing her work, “do you know what I mean?” and was reassured by being reminded that, although we were talking on the phone, her shop had been visited and her work seen.

Having decided that we cannot discount the cost and time advantages of telephone interviews we decided to use the visits to craft fairs, galleries and exhibitions, mentioned earlier as a means of identifying a research population, in order to accommodate more cost effective observation of the work. In a few cases the work will not be available to observe in this way and a face-to-face interview will have to be arranged in order to observe it. Either way, the participant will be aware that the interviewer is familiar with her work and telephone interviews can still be used in some cases. Furthermore, we came to realise that interview technique and the skill of the interviewer is more important than the mode of interview, as long as careful planning accommodates the above considerations.

The interviewer carefully monitored any indication of differences in the power dimension of the social relationship. It was clear that a few of the participants were quite nervous at the beginning of the interview. However, the interviewer felt that her role as a working mother, dog owner, and her knowledge of ceramics and business practices gave them enough commonality to ameliorate some of the differences. The pilot confirmed that we will have to employ strategies to redress the power balance. Although “it is not always so easy to reduce the power dynamics that are likely to be present in research and it is unlikely that they can ever be eradicated completely” (Maynard, 1994, 16), they can be monitored.

We learnt a great deal from the piloting of this important project and recommend the process for future research. We were successful with our application for funding and we will have a much improved research process.

References

BOURDIEU, P. T. B. F. (1996) Understanding. Theory, culture & society: explorations in critical social science, 13, 17-37.

FINCH, J. (1984) It’s great to have someone to talk to: the ethics and politics of interviewing women. In C. BELL, A. H. R. (Ed.) Social Researching: Politics, Problems, Practice. London, Routledge and Kegan Paul.

FREY, J. H. (1995) How to conduct interviews by telephone and in person, Thousand Oaks; London, Sage.

HARDING, S. (1991) Whose Science? Whose Knowledge? Thinking From Women’s Lives, Milton Keynes, Open University Press.

KIRSCHE, G. E. (1999) Ethical dilemmas in feminist research, State University Of New York Press.

MAYNARD, M. (1994) Methods, practice and epistemology: The debate about Feminism and Research. Researching Women’s Lives from a Feminist Perspective. London, Taylor and Francis. OAKLEY, A. (1981) Interviewing Women: A contradiction in terms? In ROBERTS, H. (Ed.) Doing Feminist Research. London, Routledge.

RIBBENS, J. (1989) Interviewing - an “unnatural situation”? Women’s Studies International Forum, 12, 579- 92.

STANLEY, L. W. S. (1983) Breaking Out: Feminist consciousness and Feminist Research, London, Routledge & Kegan Paul.

STANLEY, L. W. S. (1990) Method, methodology and epistemology in feminist research processes. In STANLEY, L. (Ed.) Feminist Praxis: Research, Theory and Epistemology in Feminist Sociology. London & New York, Routledge.

TANG, N. (2002) Interviewer and Interviewee Relationships between Women. Sociology, 36, 703-721.

van TEIJLINGEN, E. and V. Hundley (2001) The importance of pilot studies. Social Research Update 35, University of Surrey.

VINCENTELLI, M. (2000) Introduction. Women and Ceramics: Gendered Vessels. Manchester & New York, Manchester University Press.

WILSON, K. A. R., B. (1998(Dec.)) Telephone or Face-to-Face Interviews? A Decision Made on the Basis of a Pilot Study. International Journal of Nursing Studies, 35, 314-321.

Permission is granted to reproduce this issue of Social Research Update provided that no charge is made other than for the cost of reproduction and this panel acknowledging copyright is included with all copies.



Issue 45	Spring 2005