Survey Administration Best Practices: First Steps
Since 2000, Ithaka S+R has run the US Faculty Survey, which tracks the evolution of faculty members’ research and teaching practices against the backdrop of increasing digital resources and other systemic changes in higher education. Starting in 2012, Ithaka S+R has offered colleges and universities the opportunity to field the faculty survey, and a newly added student survey, at their individual institutions to gain better insight into the perceptions of their faculty members and students. More than 70 local faculty and student surveys have been fielded thus far and have enhanced Ithaka S+R’s expertise in higher education survey administration.
Over the next few months, I will be writing a series of blog posts about the survey administration process and ways to ultimately increase survey completion rates. Ithaka S+R is often queried about our survey administration practices, and this series of blog posts will explore both our experience in fielding surveys and current research from email marketers.
The first steps in convincing someone to complete your survey involve making sure that the email invitation you have sent is both received and opened. Considering the overwhelming amount of email that so many of us receive on a daily basis, doing so is not an easy task, but crafting an enticing subject line and selecting an appropriate signatory can mean the difference between garnering valuable survey responses and your invitation getting deleted.
To increase the response rate to your survey, you need to maximize the number of emails that are received and opened by your participants. The number of participants who click on the survey link, start the survey, and complete the survey, are all dependent upon these first two steps.
Above all else, the most important factor in crafting an effective subject line is understanding your audience. Recommendations regarding survey administration vary across organizations and industries and thus should always be considered in the context of both the sender and the recipient.
The signatory matters
Choosing the right person from whom the survey email invitation comes is crucial to ensuring that the recipient will open the email. The signatory should be someone that the recipient will recognize immediately.
Most survey platforms will allow you to customize the signatory when distributing survey invitations, enabling you to make the message appear to come from whomever is most appropriate.
For the Ithaka S+R local surveys, in some cases, the chief academic officer, library director, faculty senate chair, or other campus-level representative can effectively reach the broadest possible group. In other cases, a more personal note from a dean or department chair is more effective. The key is to make sure that whomever you pick as the signatory is someone that the recipients know and trust.
Ithaka S+R has found that varying the signatory across the email invitation and subsequent reminders is also an effective way to increase email open rates.
Choose the right words
Certain words in email subject lines have been shown to increase email open rates.
In 2013, MailChimp, an email marketing service provider, analyzed approximately 24 billion email subject lines composed of approximately 22,000 distinct words to determine which words were most effective in boosting email open rates.
The findings showed that words that imply time sensitivity, including “important”, “urgent”, and “alert” resulted in higher open rates, as did using the words “announcement” or “invitation.” Personalizing the subject line also was shown to increase open rates, especially when the both the recipient’s first and last name were included in the subject line.
Ithaka S+R advocates using differing subject line text across invitations and reminders, as different language entices different individuals to open an email; varying the subject line text across different communications enables us to reach a wide range of faculty members and students.
Avoid being marked as spam
Certain words like “free”, “click here”, and “opportunity” may trigger a spam filter, causing your email to land in a recipient’s junk folder. Using excessive capitalization and exclamation points can also lead to your email being categorized as spam.
Other words may not trigger a spam filter, but may cause recipients to ignore your email. Some words common in survey email invitations like “reminder” or using promotional, sales-related phrasing may lead to a decreased open rate.
Additionally, prior to sending email invitations and reminders, Ithaka S+R asks each institution to whitelist our IP addresses. This allows our emails to bypass all spam filter rules and arrive in participants’ inboxes.
Subject line length may not matter
Much research has been performed to determine whether there is a relationship between the number of characters in an email subject and the frequency with which the recipient opens the message. Ithaka S+R has not noted any relationship between the two and research has been mixed on this topic.
Mailer Mailer, an email marketing and newsletter service provider, advocates for using 4-15 characters and found a slight correlation between the subject length and open rate, while Adestra, a digital marketing technology solution provider, found that using longer subject lines, especially in business to consumer messaging, leads to higher open rates.
In February 2015, Return Path, a global data solutions provider, analyzed over 9 million emails from more than 3,000 senders to determine how the length of the subject line affects the average open rate. While they found that subject lines of 61 to 70 characters had the highest average read rate at 17%, these findings were not statistically significant and the differences were small and random at the aggregate level.
Bottom line: When deciding on a subject line, you should start with your audience in mind and consider what will catch their attention and entice them to open your message. Know what language does and does not speak to them. Focus on conveying the value of your message to the recipient and encourage the recipient to take action.
Increased response rates often lead to more diverse, representative perspectives from respondents and lend validity to survey results. Before a respondent even starts taking the survey, he/she needs to have received the email invitation and opened it; these first communications with a respondent are of upmost importance to the final response rate.
In my next blog post, I’ll focus on how Ithaka S+R and its partner institutions have crafted email invitations and reminders to encourage respondents to click on local survey links and how these strategies can be broadly applied to survey administration.