SHIFTING
THE FRAME OF REFERENCE IN PRINT MEDIA RESEARCH:
QUALITY vs QUANTITY vs RELATIONSHIP
Richard Wilson and Geoff Isaac
of Wilson MLI, Sydney, Australia
With statistical analysis by:
Michael Edwardson of the University of NSW
| Background
| The Wilson Project | The Experiment
| Analysis | The Wilson Involvement
Model |
| Implications for Syndicated Readership Studies
| Further Work | Conclusion
|
Introduction
The debate
for many in the print industry today is no longer about mass readership
politics. It's about rather what their readers and advertisers are getting
out of their particular publications.
Today individuals
are faced with an ever-increasing range of local and imported publications
catering for increasingly defined niche markets. In addition magazines as
a medium are facing new competition for reading time, from cable television
and cyber-based media options, which have the potential to reduce the time
(and money) of individuals (and advertisers) who are prepared to invest
in magazines.
Developing
the relationship between reader and magazine is now the primary goal of
many publishers, a goal that could have significant advertising ramifications.
There have
been several attempts to define and develop the concept of reader involvement.
Some evidence suggests involved readers are more likely to recall advertising
in publications and as a consequence, it is assumed, more likely to respond
to those adverts.
The concept
of involvement is difficult to define. It also sidesteps the issue Wilson
MLI hypothesises to be central to the relationship between readers and advertising
response, namely trust. If a reader "trusts,' a magazine, it can be
argued he/she may be more likely to believe the magazine would not include
"questionable advertising" and therefore, more likely to believe
the advertising he reads in that "high trust" publication and,
even buy certain reader relevant products advertised.
Although many studies exist on involvement
indicators or quality of reading, few have directly related high involvement
with an above average opportunity for advertising contact, let alone explored
the concept of trust or demonstrated the link between involvement and increased
response to advertising.
In 1968, an attempt at defining a Reader-Magazine
Relationship (RMR) measurement was made by the US Home Testing Institute.
The MAG Q tests identified 14 questions as the most valuable indicators
of RMR.
1. This
magazine is one of my favourite magazines out of all I read
2. Reading
frequency
3. Place
of reading
4. How last
issue obtained
5. Times
picked up
6. Time
spent reading
7. Concentration
whilst reading
8. Evaluation
of topical articles
9. Evaluation
of fictional stories
10. Evaluation
of pictorial articles and illustrations
11. Evaluation
as a source of entertainment
12. Evaluation
as a source of information and education
13. Agreement
with personal views
14. Evaluation
of credibility of ads
No syndicated readership study we would
argue could measure all 14 attributes for all titles measured.
In 1972 Dieter Mueller completed a project
for the German Readers Digest aimed at reducing the number of indicators
required to achieve a RMR score. Summary results were presented in a brochure
called "Love Story".
That study concluded regular frequency
alone is not a good indicator of RMR. Magazines that achieved a high RMR
in fact tended to have a relatively low proportion of regular (3 or more
out of 4 issues) readers.
It was argued that 8 elements should be
included (grouped under 3 factors) to measure RMR.
However, this eight-item model still proved
too long to be included in syndicated research studies and was subsequently
reduced to five elements, known as the "Love Story Model" as follows:
Factor 1 — Qualitative Evaluation
Agreement with personal views
Evaluation as a source of information
and education
Factor 2 — Handling/Utilisation
Reading frequency
This magazine is one of my favourite magazines out of all I read
The study then went on to examine the influence
of RMR on advertising effectiveness.
In his paper delivered to the
ESOMAR conference in Budapest (1976) D. Mueller
concluded, RMR readers utilise information in their preferred magazines
more intensively, and made 30% above more advertising contacts than
all readers in general.
To qualify as a RMR reader a respondent
had to give a very positive answer to at least one of the questions in Factor
1 or 2 above.
The majority of work completed since this
time has focused on identifying involvement indicators suitable for addition
to syndicated readership studies.
Trust, as alluded to in Factor 1 above,
was given little consideration.
In Barcelona (1988) Scott
McDonald presented a paper on "Quality of Reader Involvement: Report
on Research in Progress". Referring to the findings of Ursula Wangard
and RoIf Speetzen (Salzburg 1986) concluded that of all the attitudinal
statements tested by their work only two were worthy of further consideration:
-
"I usually read all the
articles in this periodical" — a measure of thoroughness of
reading, and
-
"If necessary, I would go
to great trouble to obtain even issue of this magazine" — a
measure of enthusiasm, habit, or loyalty!
George Gramse and Peter Beike presented
work (financed by Die Zeit (Barcelona 1988) and discrediting Wangard and
Speetzen's findings) demonstrating a demographic classification to have
the same predictive value for advertising recognition as the former classification
of contact quality.
Work completed for Time Magazine (McDonald,
Barcelona) and Cade's work for the RSL (presented in
San Francisco, 1993) found estimates based on units of time to be unreliable
— failing to correlate with estimates of the amount of an issue read.
Similarly, thoroughness of reading claims did not correlate well with article-based
estimates.
Baxter (Barcelona
1988) reported on a thorough experiment that proved a solid case for the
value of reading occasions as a measure of involvement. Increase in reading
days correlated to a higher recall of advertised brands.
Jochen Hansen (Barcelona 1988) spoke on
the use of new qualitative criteria in media planning. In addition to emphasising
the importance of amount read, Hansen went on to produce a measure of involvement
by asking readers how closely they identified with each publication.
However, the influence of reader involvement
on purchasing was not considered. Hansen instead used the Allensbach Single
Source Survey to correlate interest in a given type of media and involvement
in the respective market. He showed that if a respondent read more than
one title from a magazine genre he was more likely to agree with attitudinal
statements relevant to products associated with that genre.
For example, people who read more than
one car magazine were found to be more likely to agree with statements such
as, "Decide alone on car purchase".
Common to all of this work, though useful,
was lack of follow-through to any purchasing outcomes.
San Francisco (1993) produced three papers
on qualitative reading measures. Hilary Cade from RSL described qualitative
research that had led to a quantitative test covering the following involvement
indicators:
-
-
Source of copy
-
Place of reading
-
Proportion of copy read —
which respondents had difficulty with
-
Time spent reading — which
respondents had difficulty with
-
Way in which publication read —
which respondents had difficulty with
-
"Favourite" publication
-
Reaction if publication were no
longer available — which was found to be better than the favourite
question as it was less emotive
-
Mood — which the researchers
had difficulty with due to the wide variety of answers
This study proved particularly useful in
assisting with questionnaire design; eliminating questions which this research
demonstrated respondents had trouble answering.
Perry and Philport
(San Francisco 1993) reported on work done in the US for Simmons and MRI.
A battery of qualitative variables has been included in both studies and
the aim of that project was to use factor analysis to reduce the number
of variables to a core list.
The original 13 Simmons variables were
reduced to 8 factors and the original MRI list of 14 variables was reduced
to 9 factors. Cluster analysis was used to allocate publications covered
by the survey into 10 or 15 groups respectively. Individual results were
not presented but the value of this research in promoting magazine genres
was demonstrated.
Importantly, different measures of involvement
were found to be relevant for different categories of magazines.
But, this was as far as the research went.
No attempt to relate the findings to advertising effectiveness was made.
Chris Minter from
the Readers Digest alluded to the "trust" relationship in his
paper (San Francisco 1993). However, many of the variables measured were
specifically relevant to the Readers Digest and, in any event, the relevance
of involved readers to advertisers was not elucidated.
Although several researchers have been
playing in the reader involvement sand pit, no one has yet built the castle.
Current involvement indicators
Wilson Single Source Survey is conducted
over 11 months, with over 3,700 interviews per month, or over 40,000 per
annum. Involvement indicators (frequency of reading, primary purchase frequency
and number of times picked up) are included in the Wilson MLI self-completion
based syndicated readership study.
Average issue readership is calculated
from the frequency question, asked after the recency question. The recency
question is only used to focus the respondents' attention on publications
that have been read or looked into within the time scale covered by the
frequency question.
A numeric response is requested (the respondent
indicates the number of issues read/looked into out of the last four). This
methodology is consistent with the findings of the British RSL study presented
in San Francisco; a numeric scale produces more accurate data compared with
a verbal scale.
In a separate question, in a different
section of the survey booklet, respondents are asked to indicate the magazines
they purchase:
An estimate of primary readership is calculated
from these results (again a numeric scale is used).
In a separate question (to avoid the possibility of affecting readership
claims due to respondent fatigue) respondents are asked to indicate the
number of reading occasions for the most recent issue of each publication
read:
|
|
Past 7 days/
4 weeks
|
Longer ago
|
| Have you read or looked into any of the magazines below in the last
month/6 months? |
|
|
| When did you last read or look into any copy of the following magazines? |
|
|
Mark the oval which best describes
the number of different occasions you looked into the most recent issue
you read (radio buttons for illustration purposes
only):
In addition several relevant attitudinal
statements are asked:
-
I use the coupons I find in magazines
or on packets
-
I often enter competitions run
by newspapers or magazines
-
I don't read the ads in newspapers
and magazines
-
These statements record general attitudes
to magazines — not to specific titles.
In having the largest Single Source Survey
database in the world Wilson MLI was ideally placed to test involvement
as an indicator of higher advertising exposure. Is involvement a better
predictor of advertising effectiveness (as measured by recall)?
We hypothesised that highly involved readers
should demonstrate higher levels of advertising recall compared with readers
attributed with lower levels of involvement. In addition, it was decided
to take the research a step further and attempt to identify the relationship
of involvement with increased purchase of brands advertised. Thus the effectiveness
of measuring recall as a surrogate for involvement could also be tested.
Question Content
There is no universally accepted definition
of involvement or quality of reading. From the review of international work
Wilson MLI has identified those variable most commonly identified as good
indicators of involvement.
To quality for inclusion we sought variables
that had been tested successfully. Notably, we eliminated attributes which
respondents found difficult to answer. The work done by RSL (as reported
by Hilary Cade in San Francisco) was particularly useful here. Most notably
the RSL study (among others) demonstrated readers have great difficulty
in estimating the time spent reading a publication (failing to correlate
with observation or proportion of copy read). Therefore, we excluded this
measure from our experiment.
In general any measurement requiring a
calculation by the respondents would be likely to encounter a high degree
of error, thus the scale used to estimate the proportion of copy read was,
minimised to reduce errors.
No attempt was been made to measure mood.
The RSL study also demonstrated that meaningful results are difficult to
obtain due to the diversity of response, which must be incorporated into
a coding frame. More work is needed on this and the entire mood issue respondent
and interviewer is the subject of other work currently being done by Wilson
MLI.
Rationale
In formulating our approach we attempted
to include only measures that could be applied meaningfully to all publications.
Attributes likely to illustrate the relative strengths of specific groups
of titles have been therefore eliminated. For example, statements such as
in this publication are useful for my work relevant for only business and
special interest publications were not included.
The existing Wilson MLI database (of respondents
who previously participated in the syndicated study) was used to identify
readers of publications included in our experiment. Sampling stratification
took into account both primary and pass-along readers at different levels
of frequency of reading claims. Respondents who qualified for selection
were selected at random form the pool of all qualifying respondents.
Three significant "Women's" titles
were included in the experiment. Each was represented by 200 readers as
follows:
|
Frequency of Reading
|
Family Circle
(Monthly)
|
House and Garden (Monthly)
|
Who Weekly (Monthly)
|
|
3 or 4/4 issues
|
100
|
100
|
100
|
|
1 or 2 /4 issues
|
100
|
100
|
100
|
|
Total
|
200
|
200
|
200
|
Although frequency of reading alone has
been widely discredited as an indicator of involvement, it still proved
a useful basis on which to construct reader stratification.
Respondents were screened to include only
those who had read the most recent edition of the given publication and
had access to a copy of the publication to refer to during interview. In
order to maintain representation from less frequent readers (and those who
had read someone else's copy) it was found necessary to mail copies of the
publication to some respondents in order to complete the through the book
the experiment.
In the case of the two monthly titles interviewing
commenced in the third week of the month when readership had reached a sufficient
level for targets to be met.
Interviewing was completed over the telephone
using a CATI based system to reduce the margin for interviewer error.
Major demographics were checked for changes
since the major interview was conducted and current reading habits obtained.
Only experienced interviewers selected
from those who have achieved "Gold Standard" within the organisation
(from working on our syndicated study) were used for this experiment.
All interviewers were fully briefed by
Wilson MLI's National Field Manager who has been running the main syndicated
project, together with all ad hoc projects conducted by the company, since
1989.
Interviewers introduced themselves from
Wilson MLI and explained they were not representing a publisher but conducting
some general research on magazines.
Ad recall — unprompted
Recall was conducted in the first instance
as George Gramse and Peter Beike found (in their work for Die Zeit in West
Germany) that "when the contact-qualifying questions are posed after
the copy test, the variables change to a certain extent".
A general question on whether the respondent
remembered any advertising in the most recent issue was followed by a prompt
for product categories and brands. A measure of claimed and validated unaided
recall was therefore available.
Purchase involvement
A list of product categories relevant to
advertisements included in the pages of the magazine being tested was then
read. The respondent was instructed to stop the interviewer if they had
been involved with purchase activity within that product category within
the past month (week, in the case of Who Weekly).
For all categories selected respondents
were asked to nominate the brand purchased. Results were recorded as either
the advertised brand or another brand from within the same product category.
Ad recall — prompted
Respondents were asked to find the magazine.
If the respondent no longer had possession of the magazine, arrangements
were made to forward a copy by mail and an appointment made to call back.
Respondents were asked to review the first
15 pages of the magazine and mention any articles or advertisements they
remembered seeing, however briefly. They were encouraged to mention all
pages looked at, even if only for a second, leaving out only those they
only turned the corner on in search of a page number.
Respondents then skipped to the back of
the magazine and work backwards answering the same questions (for 15 pages).
Identifying copy
Respondents were then asked to take a further
look at the articles and advertisements they mentioned and identify those
they considered interesting for whatever reason. Answers were recorded using
a five-point scale:
Level of appeal was then measured using
the same technique.
Respondents were then asked to respond
to a number of attitudinal statements concerning the magazine.
Number of times picked up
Number of times a specific issue is read
or looked into was expected to correlate with Opportunities To See and therefore
with recall of advertising. Respondents were, therefore, asked:
How many times have you read or looked into the copy of ** you are
currently holding?
Respondents were informed that the rest of the
questionnaire did not refer to a specific issue of the magazine but to
their general attitudes toward the publication.
Favourite Magazines
Although others have discredited (e.g.
Reference RSL) was not impressed with this measure (males in particular
appeared reluctant to use this emotive expression) the preference given
to it by other researchers (most notably the early work done by Mueller
(1976) warrants its inclusion in the Australian study, that is:
Would you say that ** is one of your favourite magazines?
Thoroughness of reading
To eliminate the need for respondents to
quantify reading time or make an estimate of the proportion of magazine
read, the following attitudinal statement was used:
1) I usually read all the articles in **.
Loyalty
2) If necessary, I would go to great trouble to obtain every issue
of this magazine.
Identity
The Hansen (Barcelona 1988) question (with
a reduced scale) was used here:
3) You sometimes say of a person
that you are very close to him or not so close. This can also be applied
to magazines. Would you please tell me for whether in general you identify
very closely with this magazine or not so closely?
The following statements were used by the
researchers:
4) I like looking at the adverts
5) I find the advertising helpful
6) I find the advertising a useful
source of product information
Editorial
Attitudes to editorial were measured using
the following statements:
7) I have complete confidence in the editorial
8) I believe what I read in it
-
Bought it personally from shop
or newsagent
-
Someone else in household bought
it
-
Home-delivered by newsagent
-
Paid subscription delivered to
home
-
Free copy delivered to home
-
Office/work copy
-
Friend's/relative's copy
-
Seen at a library, waiting room
or other public place
-
None of these
Reaction if no longer available
The work presented by Cade
in San Francisco in particular prompted us to include this measure in our
experiment.
How would you feel if ** were no longer
available?
Analysis
Back
to top 
Due to the limited time available for completion
of this project preliminary results reported are restricted to the two monthly
titles included in our survey.
Stage 1
Cross-tabulations were produced to examine
recall and purchase behaviour across all the involvement indicators used
in the study.
Virtually all the variables measured in
our survey could have (and often have) been argued to be indicators of involvement.
From an advertisers perspective the most important outcome of involvement
must be a greater response to advertising. For involvement to be a useful
selling tool, advertisers must satisfy themselves that involved readers
are more likely to respond to advertising within a selected publication.
Four measures of advertising effectiveness
were recorded by the survey:
-
Unaided recall summary
(where respondents were simply asked
to state whether or not they remembered any advertising in the issue)
-
Unaided recall (where
respondents were asked to nominated specific adverts they remembered seeing
in the issue)
-
Prompted recall (where
respondents were asked to look through selected portions of the magazine
and nominate pages they remembered seeing before)
-
Purchase behaviour (where
respondents were asked about their recent purchase activity within selected
product categories)
Early analysis (based on the simple unaided
recall summary) suggested the most obvious correlation with an increase
in the number of respondents who recall an ad was with the number of times
respondents opened the magazine.
FAMILY CIRCLE
Percent of respondent who claimed to recall
any advertising
Graph 1
HOME BEAUTIFUL
Percent of respondent who claimed to recall
any advertising
Graph 2
The most significant difference appears
to be between those respondents who had only read or looked into the magazine
three times or less and those who had looked into the magazine at least
four times.
Also of significance was frequency of reading
and source of copy.
FREQUENCY OF READING
Percent of respondent who claimed to recall
any advertising
Graph 3
SOURCE OF COPY
Percent of respondent who claimed to recall
any advertising
Graph 4
Higher recall scores were associated with
respondents who said: they identify closely with the publication, like looking
at the adverts, find the adverts helpful, find the adverts a useful source
of product information, have 100% confidence in the editorial, believe what
they read and would be disappointed if the publication was no longer available.
The replies to all three of the advertising related questions (4 to 6 above)
appeared to hold significant relationships with purchase behaviour while
only two of the three editorial variables (7 to 9 above) showed signs of
a relationship. (Whether a respondent agreed they learnt from the publication
or not appeared to have no effect on advertising recall).
The above analyses were based on the results
to the question, "Can you recall any advertising in the issue?"
For the purposes of more detailed analysis recall scores were calculated
as a percentage of the total number of advertisements in the tested sections
of the magazine. In the case of unaided recall respondents were encouraged
to nominate advertisements they remembered seeing. Whereas aided recall
was measured by respondents looking through the selected sections of the
magazine being tested and nominating adverts they remembered seeing before.
Multivariate analysis
Multivariate factor analysis was used in
an attempt to examine the nine general attitudinal variables (labelled 1-9
above) from this survey (following the work presented in San Francisco by
Cade from RSL and Perry & Philport
for Y&R).
A varimax rotation resulted in three factors
with eigenvalues greater than 1 and factor loadings greater than 0.3 with
no cross loadings greater than 0.3.
Based on the meanings of the individual
questions these factors have been labelled:
Ad relevance
5) I find the advertising helpful
6) I find the advertising a useful source of product information
4) I like looking at the adverts
8) I believe what I read in it
9) I learn a lot from it
7) I have complete confidence in the editorial
2) If necessary, I would go to great trouble
to obtain every issue of this magazine
1) I usually read all the articles in it
3) You sometimes say of a person that you are very close to him or not
so close. This can also be applied to magazines. Would you please tell
me for whether in general you identify very closely with this magazine
or not so closely?
|
Rotated Factor
Matrix
|
Factor 1
|
Factor 2
|
Factor 3
|
|
Variable
|
Ad relevance
|
Trust
|
Commitment
|
|
5) ads help
|
.84793
|
.21993
|
.14990
|
|
6) ads useful
|
.80967
|
.23427
|
.08474
|
|
4) like ads
|
.79622
|
.03854
|
.20259
|
|
8) believe
|
.13938
|
.80580
|
.03158
|
|
9) learn lots
|
.18141
|
.73938
|
.15957
|
|
7) editorial confidence
|
.12404
|
.68474
|
.28805
|
|
2) go to trouble to
obtain
|
.08378
|
.24236
|
.79847
|
|
1) read all articles
|
.20301
|
.02957
|
.77537
|
|
3) identify closely
|
.13159
|
.27956
|
.66002
|
Responses for all nine attitudinal questions
were measured using ratings on five point Likert scales. Eight of the variables
were anchored by 1 = Strongly Agree to 5 = Strongly Disagree. Question 3
was scaled using 1 = Very Close and 5 = Not At All.
Factor scores were computed as averaged
linear composites, so as to preserve the original scaling information. For
the structural equation analysis, factor scores were reverse scored, i.e.
Strongly Agree = 5.
When these factors were correlated with
outcomes, such as purchase behaviour and ad recall, the following results
were achieved.
The factor "Ad Relevance" was
found to have the strongest relationship with purchase behaviour (with a
correlation coefficient of .2416).
The three variables which explain the factor
"Trust" were also found to have a significant relationship with
purchase behaviour (.1699). The factor " Reader Commitment" appeared
to have a weaker direct relationship with actual purchase behaviour (.1198)
although still significant.
"Reader commitment" however,
had a strong relationship with "Aided Recall" (.2719) but a significantly
weaker relationship with unaided recall (.1345). Similarly, "Ad Relevance"
though having a strong relationship with "Aided Recall" (-.2251),
had no significant relationship with "Unaided Recall".
Path analysis
In order to examine the complex interrelationships
between the variables of interest, and to develop an exploratory causal
model of these relationships, the structural equation modelling technique,
LISREL 7, was used (Joreskog and Sorbom, 1988).
As the researchers were primarily interested in overall effects, data from
the two monthly titles were combined.
The nine variables included in this analysis
were:
1) Brand purchased
2) Unaided ad recall
3) Aided ad recall
4) Ad relevance *
5) Trust *
6) Disappointment if magazine no longer available
7) Times an issue read
8) Number of issues bought
9) Reader commitment *
* = Factors summarising three core variables
each as described above
The input for the LISREL analysis was the
correlation coefficient matrix:
|
Variable
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
8
|
9
|
|
1
|
1.00
|
|
|
|
|
|
|
|
|
|
2
|
0.16*
|
1.00
|
|
|
|
|
|
|
|
|
3
|
0.14*
|
0.13*
|
1.00
|
|
|
|
|
|
|
|
4
|
0.24*
|
0.11
|
0.23*
|
1.00
|
|
|
|
|
|
|
5
|
0.17*
|
0.07
|
0.02
|
0.39*
|
1.00
|
|
|
|
|
|
6
|
0.06
|
0.05
|
0.16*
|
0.24*
|
0.39*
|
1.00
|
|
|
|
|
7
|
0.10
|
0.12*
|
0.06
|
0.06
|
0.09
|
0.11
|
1.00
|
|
|
|
8
|
0.00
|
0.02
|
0.09
|
0.02
|
0.11
|
0.11
|
0.03
|
1.00
|
|
|
9
|
0.12*
|
0.14*
|
0.27*
|
0.37*
|
0.41*
|
0.49*
|
0.16*
|
0.20*
|
1.00
|
* = significant relationship
It was hypothesised that Trust would be
the central exogenous variable and that through various causal paths, would
influence the other variables. The aim was to explore this theoretical hypothesis
using the empirical data relationships found in the survey.
LISREL is usually used for confirmatory
model testing, however it is also used for exploratory model development
(Hayduk, 1957). Recent editions of journals such as
The Journal of Marketing, Journal of Advertising Research, Journal of Consumer
Research testify to the popularity of LISREL analysis for exploratory analysis
(see Crosby, Evans and Cowles, 1990; and Morgan
and Hunt, 1994 for examples).
Caution should be observed when generalising
models that result from data-driven modifications of models (MacCullum,
Roznowski & Necowitz, 1992). Nonetheless use of this exploratory
technique is useful for theory building followed with subsequent confirmatory
testing.
In using the exploratory procedure to develop
the structural equation model, successive paths were specified, building
each time on the assessment of individual estimated parameter fit for each
previously specified model (Byrne, 1989). A series
of models were subsequently specified and estimated until one was obtained
which exhibited not only an acceptable statistical fit but also indicated
a substantively meaningful representation of the observed data.
The final LISREL model (see chart) is a
good fit, Chi-Square (with 21 degrees of freedom) = 20.89 (p = .47). The
adjusted goodness-of-fit index (GFI) is 0.99 and root mean square residual
(RMR) is 0.03. All parameters are significant at p<0.5. Significant causal
paths between variables are shown in the model below. Where no arrows exist
between variables, no significant direct relationship exists between them.
Path coefficients are maximum likelihood parameter estimates.
Wilson Involvement
Model
| Analysis for both Monthlys combined
(N=414) |