UX Mini-lesson: Diary Studies

For today’s post I a present a “UX mini-lesson” on how to conduct diary studies. We will cover the value of longitudinal data sets, and strategies to maximize the quality and number of diary entries from your participants.

What are Diary Studies

Usability.gov defines a diary study as…

“Research method that involves providing participants with the materials and structure to record daily events, tasks and perceptions around a given subject in order to gain insight into their behavior and needs over time.”

The main benefit of diary studies is that they provide longitudinal data. That is, they can help you answer questions about user behavior and feelings over-time. This means, diary studies can help you answer questions like:

Do users stay engaged with [product] several weeks after first use?
What are typical weekly patterns of [product] usage?
How do people feel emotionally after they [have some experience] with the [product] ?

The other benefit to diary studies is that they don’t require a researcher to present to gather the data directly. There are other research methods, like contextual inquiries and ethnographies that can also help you gather longitudinal data, but they require the researcher to be present. For this reason, many people find diary studies to be less invasive than other methods. You can use diary studies to gather data about more private or intimate settings where a researcher’s presence would be awkward (ie. the home, bathroom, bedroom, etc.)

How to conduct a diary study

Diary studies can run anywhere from a weekend to several months long. And once a diary study is underway, there is very little opportunity to make adjustments. So you will want to take a lot of time to construct the study, prepare materials, and make sure your participants are ready. Here are the basic steps of conducting a diary study:

Fully articulate the goals of the study. This will help you determine exactly what kind of data you’ll need to gather.
Carefully construct your diary prompts and materials. This can be in the form of daily SMS messages, surveys, or paper forms.
Run a pilot study on a few friends. Make sure they answer your diary prompts in the way you intended.
Carefully recruit participants. Make sure they understand exactly what is expected of them and how to respond to the diary prompts.
Check in with your participants regularly. You’ll also want to schedule a wrap-up interview at the end of the study to get their thoughts and impressions.
Analyze the data. Diary studies can produce a lot of data, so you’ll want to leave plenty of time for this.

As with any type of research method, there are some potential risks. Here are some caveats to consider:

Attrition will be high. Because a diary study runs for a long stretch of time, and requires a fair amount of work, there is a higher likelihood of participants dropping out of the study along the way. You’ll want to recruit more participants than you actually need, and check in with them regularly to mitigate this.
You should pay your participants well. Like I said in point #1, participating in a diary study is a lot of work. You should reward your participants for that work (and thereby reduce attrition even more).
Stay on top of it. There is nothing worse than getting to the end of a diary study and realizing you didn’t get the information that you need. You’ll want to keep track of the data as it’s coming in, so you can intervene or even halt the study if need be.

Tools

A diary study tool can be as cheap as an online survey that you share with your participants. Or it can be as expensive as a full-on research suite that handles recruitment, reminders, and data analysis. Here are a few of the diary study tools I have found to be effective:

eSurv (Free) – eSurv is funded by educational research institutions and provides a completely free survey tool. With eSurv you could create your diary study survey and collect an unlimited number of results. The interface for it might not be as swanky as some other survey tools on the market – but it is secure, unlimited, and free.
Dovetail (Free, while in beta) – Dovetail is a new research tool designed specifically to facilitate diary studies. It allows you to create your diary entry form, and send email or SMS notifications to participants. It seems like they will be adding more features soon.
SurveySignal ($) – You create a registry of participants, and then set-up automatic email or SMS reminders to fill out their diary. Cost is 10 cents per “signal” (aka. SMS) you send out. It might not be free, but it would make your life a whole lot easier.
PollEverywhere ($) – While I don’t think diary studies is exactly what this product is designed for, you could certainly use it that way. You would ask your participants to download the app (or participate via SMS), and then send them poll questions when you like.
ESMCapture.com ($$ to $$$) – ESMCapture is an all-in-one diary study tool. You can set-up your diary survey, sign-up participants, and set a reminder schedule all with either a web or iOS based interface. (Side note: ESM stands for “Experience Sampling Method,” which is another term for diary study.)
dscout ($$$) – dscout is another all-in-one research tool that allows you to create beautiful looking diary studies. It also appears to allow you collect all types of data, including user submitted videos.
LifeData ($$$$) – LifeData allows you to easily build a diary study tool that will live as an app on your participants’ smart phone. This is advantageous because it means you can utilize app functionality to notify and keep track of your participants, as well as enable them to fill out their diaries “on the go.” LifeData also provides a data dashboard for analyzing and tracking all your results. But a license can run you ~$2500, so I’d really only choose this option if diary studies are going to be the core of your user research strategy.

What are Diary Studies

How to conduct a diary study

Tools

More Reading

Want more? Get more!