Creative Testing

The Hawai’i Visitors and Convention Bureau (HVCB) turned to SMARInsights for concept testing of the campaign that would restart tourism after a year of suspended advertising. The high stakes advertising had a complex set of goals. Most critically, it would jump start the economy. However, the target audience wasn’t just any traveler – the campaign had to target and motivate people who value sustainability and local culture.

One challenge of this study was its ambitious timetable. Creative production was underway when HVCB decided to evaluate consumer reactions to the message at the foundation of the video spots. Topline data was provided during data collection – just 21 days from the kick-off meeting, and the report was presented in less than six weeks from our first point-of-contact with HVCB.

We worked with HVCB research and marketing teams at the inception of the study to understand the contextual organizational concerns and how the findings would be applied. This collaborative approach informed the survey design and the selection of survey tools that were implemented; and, ultimately helped us deliver actionable insights in the presentation of findings.

Another challenge was the rigorous screening criteria across multiple attributes. To address this challenge, multiple panel vendors were coordinated, and an incidence test was conducted to ensure that we could deliver the target audience on time and within budget. During early data collection and preliminary analysis, we identified an underrepresentation of high potential millennial travelers that could not pass the income screener and we recommended an oversample of this population. The final sample included 1,300 completed surveys divided between western domestic markets and the rest of the United States.

Survey participants rated the appeal of Hawai’i and five competing destinations along with their likelihood to visit. They were exposed to the campaign messages, in the form of written statements, and they rated how well the statement fit with a variety of destination image attributes, and how it impacted likelihood to visit. Within the survey instrument, they used a highlighter tool to select words and phrases they considered positive or negative.

The survey also asked participants to rate the importance of various attributes in choosing a vacation destination and then rated destination performance for each attribute. These responses were used to segment visitors and compare how well the statements resonated with core audiences and which were potentially alienating.

Creative testing does not produce a right or wrong answer. It seeks to understand how the creative resonates with various segments of potential visitors and whether the creative motivates intended actions. It is important to discover which aspects of the creative are working and not working, and for which target audiences. Our findings identified the destination attributes that could strengthen the sustainability message and broaden the impact of the message with consumers, to achieve alignment between cultural values and economic objectives.