The Effects of Interviewer Mood Manipulation on Telephone Consent Rates

Richard S. Wilson, Sam G. Everingham, Mark L. Day.

| Method | Results | Discussion | Conclusion |

ABSTRACT

This paper argues that one of the most powerful determinants of response is in fact not respondent incentives, but rather interviewer incentives. Specifically, it examines the effects of various mood re-focusing techniques in raising telephone interviewer performance levels. In addition, the effect of such techniques on raising transient Positive Affect scores is tested. Results showed that for 'interactive' interventions (food & drink, compliments and criticism only) there were significant positive correlations of Positive Affect with consent rates. Across all interviewers, a significant positive correlation was noted between response rates and scores on Positive Affect. Discussion focuses on the implications of the findings for maximising interviewee consent rates.

BACKGROUND

The main difficulty with mail surveys is obtaining adequate response rates. In terms of volume of questions and numbers of respondents (40,000), Wilson MLI's Integrated Data Source is arguably the largest single source survey in the world. It is based on telephone recruitment and mail returns. To provide representative data to a diverse client base, research firms need to ensure their random sample selection is not overly influenced by poor response rates.

MLI has two levels at which response rates come under the microscope. The first is at the recruitment phase — how many people contacted can we induce to assist us in our endeavours?

Secondly, having obtained a high degree of consent, how many questionnaires can we get back? Over the last two years we have focused more on questionnaire return rates. In 1995, with the move to a CATI-based recruitment system, we have refocused on survey recruitment rates.

Australia is a country where some 95% of people are contactable by telephone, making this medium a viable method of interviewing. Its value is greatly enhanced when response rates (traditionally low in Australia) are raised.

Wilson MLI has successfully used a range of techniques over the past two years to improve response rates to its mailed omnibus survey. Varying combinations of reminder phone calls and mailed thank you/reminder cards have been effectively trialled to arrive at an optimum mix to maximise response rates. 'Call-back specialists' and introductory letters are also used to provide expert 'encouragement' to potential respondents.

In early 1995 we turned our attention to the initial recruitment process. We conducted a 'refusals' recontact survey to examine reasons respondents refused at the initial telephone recruitment point. These reasons varied across age groups depending particularly upon respondents’ lack of knowledge of market research concepts and lack of appropriate incentives to participate. As a result, we have developed customised telephone introductions for males and females of different age groups in order to easily redress these failings.

We also implemented a team structure to the interviewer pool, assigning weekly performance-based ratings to interviewers. Interviewers are placed in one of four teams, tiered in terms of hourly pay rates and extra benefits. Such a strategy might, according to Process theory, be expected to increase interviewer motivation and hence work performance. In short, a 'gold standard' has been applied to the interviewers themselves.

Common sense would suggest that obtaining good response rates with telephone interviews will depend on two main factors: locating the respondent and getting them to agree to take part in the interview (we make up to ten attempts to reach each number dialled). However, we also argue that to improve consent rates, job satisfaction and/or mood state of the interviewer plays a major role. This paper will concentrate on examining these variables as they interrelate and their subsequent impact on recruitment rates.

Types of Experiments

Various strategies have been tried in the past to improve consent rates. Peacock(1995) conducted an experiment designed to increase survey return rates by reducing the waiting period between recruitment phone call and receipt of the survey in the mail. It was found that instead of effecting response rates, the agreement to participate (consent) rate increased significantly. He theorised that recruitment for a closer, more immediate task may be an easier sell, as respondents may feel more confident in committing to a task closer-at-hand.

Peacock's experiment also showed that increasing unanswered number redial attempts from eleven to thirteen increased telephone consent rates by 1.5 percentage points. This study also found a two percent improvement in response rates among ethnic and lower socio-economic groups by turning the mailer into a presentation pack – almost a present.

Howard (1990) looked at the influence of verbal interviewee responses to common greetings on compliance behaviour. This research found that a combination of asking subjects how they are feeling, allowing them to respond and acknowledging their response has a significant affect on increasing compliance rates with a charitable action. The implications of this experiment are limited however by the minor commitment required of subjects, the small sample sizes used and the mores of the cultural group surveyed (Americans from Texas).

Cook (1995) describes a process by which potential subjects for television metering are invited to be members of a research panel (rather than asking them for permission to install meters), thus potentially giving them implied 'membership' benefits. This strategy was found to increase consent rates significantly to 68.5%.

However, those contacted who refused to have meters installed were recontacted in person six months or more after initial refusal. At this stage they were asked again to become members. This strategy only increased consent rates marginally (Cook, 1995).

Measures of Personality and Effect

Mabey & Bridges (1972) were able to collect personality data on 200 female interviewers working for the British Market Research Bureau throughout the United Kingdom using Cattell's 16 Personality Factor inventory. The traits that emerged most strongly amongst interviewers were social boldness, confidence, assertiveness and emotional stability. In terms of work performance, it was found that the poorer performers were usually more extreme in these personality characteristics. Although statistically significant differences were not given, Mabey & Bridges (1972) concluded that interviewer performance is negatively related to extremity in these traits.

Other researchers (Macfarlane Smith, 1972) have similarly found that the extroverted, sociable person is less capable of standing back and viewing others impartially and thus not likely to produce the most valid data. Moser (1979) also states that interviewers should be neither over-aggressive nor over-sociable.

Consequent on these observations, Wilson MLI undertook additional once-only measures of these four traits –­­ confidence, assertiveness, social boldness and emotional stability in their interviewers, in order to test whether such traits have an effect on either transient mood or interviewer performance. Extreme scores on these traits were hypothesised to correlate with poor consent rates.

Previous research by Watson & Clark (1984) examined negative affectivity, finding it a fairly stable trait related to individual differences in self-esteem and self-satisfaction. Brandstatter (1983, cited in Clark & Watson, 1988) found subjects' mood to be better when they were interacting with other people than when they were alone. This finding itself may suggest telephone interviewers are likely to have higher positive affect (PA) scores due in part to the nature of their work and arguably the layout of their workspace (Wilson MLI does not use enclosed booths for its interviewer team).

Clark & Watson (1988) in a Japanese study of 18 upper-class psychology students found that high scores on Positive Affect were significantly associated with various indices of subjects' social behaviour.

Positive and Negative Affect were also examined by Watson & Clark (1991) in terms of the validity of self- versus peer-ratings of Positive and Negative Affect. Results showed significant self-peer agreement for such scales as well as significant discriminant validity. Self-rated affect was thus concluded to be a most valid mood measure.

De Vaus (1991) reports extensive investigations of self-reported transient mood have repeatedly found two independent factors – positive and negative affect – to be the most important dimensions of subjective emotional experience (Watson, Clark & Tellegen, 1988). High positive affect can be defined by words like active, excited, alert, enthusiastic and strong whereas low positive affect is characterised by sadness and lethargy.

Levin & Stokes (1989) have examined this trait in relation to its possible effect on job satisfaction. They conducted a laboratory study as well as a correlational field study of professional workers. In both studies they found the relation of job satisfaction to task/job characteristics to be much stronger than its relation to worker negative affectivity.

George (1991) in a study of the effect of positive mood on pro-social work behaviours found that individuals who experienced positive moods at work were more likely to engage in helping behaviours (i.e. customer service).

Previous research has also shown that in group settings, negative affect is far more transferable than positive affect. The present paper examines whether it is possible to intervene to improve transient interviewer mood in such group settings, and whether such improvement can increase work performance.

Sample

A sample of 96 casually employed telephone interviewers working for a private field research firm between August and September 1995 in Sydney, Australia were used for this study.

Design

A repeated measures approach was adopted for this experiment, in which 853 observations (shifts) were recorded across the sample size. Mean number of work shifts for which questionnaires were completed by the sample was 8.9 (S.D. = 5.36) over the eight-week period.

We argue that if we can shut down the preceding mood state and allow interviewers to refocus positively this will be passed through to respondents, resulting in higher response rates.

Measurement Scales

The study used self-reported interviewer current mood state as its dependent variable. Watson, Clark & Tellegen's (1988) Positive and Negative Affect Scale (PANAS) items were used to measure mood state. These ten-item scales have been found to be internally consistent, showing excellent convergent and discriminant correlations with lengthier measures of the underlying mood factors. When used with short-term instructions (e.g. right now), Watson et al. (1988) report these items to be sensitive to fluctuations in mood, whereas they exhibit trait-like stability when longer-term instructions are used.

Interviewer affect was recorded on a self-rated computer-generated scale at the start of interviewer shifts each evening, though subsequent to mood re-focusing techniques.

Interviewer performance data was obtained via computer generated consent rates. This gave a fractional measure of successful recruits per interviewer shift divided by total number of eligible persons who were called. Computer-generated number of successful recruits per shift was used as an additional indicator of performance.

Overall daily and weekly interviewer performance were also measured by averaging all consent rates for that day or week. In this way the effect of day- or week-long interviewer mood interventions could be analysed.

Procedures

Over the two-month study, a half-hour period each night immediately preceding interviewer shifts was devoted to refocusing interviewer mood. This refocusing was varied in each of the eight test weeks, using a distinct theme. Interviewers are routinely asked to arrive 15 to 30 minutes prior to the commencement of their shift. This period is used for affect intervention. Weeks one to four involved positive mood themes – these were Contemporary Pop Music (1); Ambient Music (2); Compliment Your Neighbour (3) and Aromatherapy (4). The first two days of week five were purposely neutral with no mood manipulation and the balance of this week's intervention was positive – Food and Drink provided (5). The first two days of week six were designed to produce negative mood responses (6) by the use of criticism and beratement. Weeks seven and eight involved the screening of popular half-hour television comedies, Absolutely Fabulous (7) and The Simpsons (8) respectively.

Data analysis was performed using SPSS for Windows, Version 6.0.

Sex Differences

T-tests of Positive Affect scores by sex showed that males demonstrated significantly higher transient Positive Affect compared with females (t = 4.55, p<.001). No significant sex differences were found in regard to Negative Affect scores, number of recruits or consent rates per work shift.

Age Differences

Significant differences in transient interviewer mood were found when interviewers under 25 years were compared with those over 25 years. The younger group reported significantly higher mean scores on Negative Affect (14.5 vs. 13.0, t = 4.89, p<.001) and significantly lower scores on Positive Affect (23.9 vs. 27.6, t = –5.95, p<.001).

No significant age group differences were found in respect to numbers recruited per shift.

Table 1 below summarises the affect score, consent rate and job satisfaction differences by demographic and interviewer performance variables.

Positive & Negative Affect Scores

Cumulative scores on these two sets of questions were found to be weakly negatively correlated (r = –0.27, p<.001). Scores on a Likert scale measure of job satisfaction showed a strong positive correlation with scores on Positive Affect (r = 0.6, p<.00l) and a weak negative correlation with Negative Affect (r = –0.18, p<.001). Similar agreement was found between Affect scores and a Likert scale measure of life satisfaction.

Table 1: Demographic & Performance measures vs. Key Dependent Variables

Figure 1: Positive Affect Score by Week

However mean Positive Affect scores did vary significantly over the period as Figure 1 shows. Analysis of variance (ANOVA) tests showed significant variation between weekly means (F ratio = 5.55, p<.001). The Least Significant Difference test showed Positive Affect scores during the 'Pop Music' and 'Simpsons' weeks to be significantly higher than mean scores for the 'Ambient Music' (week 2), ‘Compliments’ (week 3) or 'Aromatherapy (week 4) periods (p<.05).

Surprisingly however, mean Positive Affect scores for the negative intervention week were also significantly higher than for the afore-mentioned 'positive' weeks two to four.

Positive Affect scores for the 'Absolutely Fabulous' (week 7) interventions were found to be significantly higher than mean scores for the 'Ambient music' or 'Compliments' weeks.

On the basis of this preliminary exploration, the eight weeks of 'mood refocusing' were regrouped into three categories of affect; Foreground (pop music and TV comedy weeks); Background (ambient music, aromatherapy weeks) and Interactive (criticism, food/drink & compliments weeks). PANAS mean scores were then compared across these three affect types via paired t-tests.

During 'foreground' intervention weeks, Negative Affect mean scores were shown to be significantly higher than in 'background' or 'interactive' intervention weeks (15.8 vs. 14.9, t = -3.08. p<.05). There were no significant differences in Positive Affect scores across the three intervention types.

Consent Rates

Mean number of respondents recruited per interviewer was 7.26 (S.D. = 3.3). When the data concerning mean weekly consent rates was explored, variances were found to be homogenous. Consent rates were found to be lowest for the Absolutely Fabulous TV comedy week (7) as shown in Figure 2 above. The Food and Drink provision week (6) was found to produce highest consent rates (significantly higher than for all other weeks).

Analysis of variance with a Least Significant Difference test of the means showed all these differences to be significant (F ratio = 7.6, p<.001).

Figure 2: Weekly Consent Rates

Figure 3 below shows that mean number of consents did not improve significantly over the eight one-week periods. Neither did mean number of consents vary significantly from first day of intervention compared to weekdays two to five for any of the eight weeks.

Figure 3: Time Series Analysis of Contents

A significant, though small negative correlation was also found between a rating of interviewer appearance (presentation) and consent rates (r = –0.19, p<.05). No correlations were found between once-only scores on a number of personality trait measures and consent rates. A strongly negative correlation was found between interviewers scoring extremely high on assertiveness, boldness, stability and confidence, and consent rates (r = –0.48, p<.08).

Relationships between Transient Interviewer Mood and Consent Rates

In support of the main hypothesis, a significant positive correlation was found between consent rates and Positive Affect (r = 0.15, p<.005). When positive item correlations with PA were compared, items measuring 'how strong' (r = 0.18), 'how proud' (r = 0.18), 'how inspired' (r = 0.21) and 'how attentive' (r = 0.13) were independently significant in their correlation with consent rates.

When mood intervention weeks were again separated into three types, (foreground, background and interactive), only for the interactive intervention weeks (food & drink, compliments and criticism) were there significant positive correlations between consent rates and Positive Affect (r = 0.19. p<.05).

No correlation was found between consent rates and Negative Affect measures.

The Effect of Job Satisfaction

The PANAS scale measure of Positive Affect showed high positive correlations with interviewer scores on job satisfaction, but only for fully accredited interviewers (r = 0.65, p<.001). Trainee interviewers showed no such correlation.

No correlations were found between interviewer experience and consent rates or number of successful recruits. Neither did self-reported job satisfaction vary significantly over the eight interventions.

Pre-study, during and post-study means of consent rates were taken in order to show the effect of Wilson MLI's transient mood interventions. The mean consent rate for the three months preceding the intervention was 45.4%, compared to 52% for the three months during which refocusing of interviewer affect was carried out. In the one month following the study, when interventions were deliberately withheld, consent rates were shown to have dropped to 46% again, clearly illustrating the effectiveness of refocusing interviewer transient mood state when other factors are held constant.

Figure 4: Consent Rate Comparisons

The results of the mood interventions showed that the more subtle positive interventions were unsuccessful in elevating Positive Affect, perhaps due to less attention being paid to these 'background' type effects (ambient music, aromatherapy). Significantly more positive scores on interviewer transient mood were found using 'foreground' or 'interactive' techniques. It may be that the more attention-grabbing interventions (TV comedies, loud pop music) are more successful in raising Positive Affect scores.

However the most success in raising response rates came from the 'Interactive' mood refocusing weeks, such as criticism and food & drink interventions. Interactive interventions may in fact work on a group-level, as outlined by George (1990) who illustrates that group socialization techniques such as the 'interactive' techniques used at Wilson can be responsible for consistency of pro-social behaviours.

The high negative correlation found between extreme scores on extrovert traits and consent rates is in agreement with Macfarlane Smith's (1972) and de Vaus' recommendations that the performance of interviewers extreme on such traits is likely to suffer. Although not significant due to the low numbers of interviewers scoring at the extreme end of these traits, this deficiency is of benefit to Wilson MLI's consent rates.

Non-homogeneity of variances between the eight weeks of intervention were caused by the large variation in sample sizes over the eight time periods. This may explain the lack of detectable variation in Negative Affect scores over the eight interventions. Another factor restricting variation may be that telephone interviewers as a group show more stability of transient Negative Affect.

A further limitation of the study lay in the inability to compare interviewer mood scores following interventions with scores before interventions took place. Although such effect data (“how have you been feeling 'today' and 'lately'?") was collected during the first month of the study, it proved too time-consuming to continue with.

The experiments are continuing and although the results are preliminary, they indicate that significant increases in interviewer transient mood over a four-hour shift can be achieved by interactive mood re-focusing techniques. This paper also gives credence to the view that increasing interviewer Positive Affect is correlated with increases in interviewee consent rates.

Negative Affect has been found in previous research to be significantly more transferable than Positive Affect. It is in the market researcher's interest then to minimise Negative Affect. The current study has shown that higher Positive Affect was correlated with low Negative Affect. As a result, we at Wilson MLI are committed to continuing to use pro­active measures that increase Positive Affect in order to maximise consent rates. 'Interactive' mood refocusing techniques are now an important component of our strategy to improve quality of data through higher response rates.

REFERENCES

| Method | Results | Discussion | Conclusion |

Back to top