Participant selection
Include a knockout question in the screener that asks if they found it difficult to remember to take short-term medication. Use this information to knockout those who said no.
Formality
I often used the term “I” when communicating with the participants because I alone was running the experiment. Similar to the Hawthorne effect, I believe that this might have affected the participants behaviour. There was at least one comment that they were “doing this for me” because I was observing them and they all to some degree knew me, so may have modified their behaviour to please me. In further testing, I would communicate with the participants more formally in the hope that this would reduce the number who would try to please me with their performance.
Placebo
Posted all participant Tic-Tc’s instead of allowing them to buy their own so that I knew the pill count was accurate. This would require that I also gather the participants address when setting up the study.
When setting this up, I would post only the Tic-Tac’s that they needed for that stage of the study.
Reminder participants to treat the Tic-Tac’s like real medication
I included a question asking if the participants had eaten any Tic-Tac’s outside of the test scenario in the post-study survey. Two participants said that they had.
Getting people started
Set up a schedule to get people ready for the study. This would include reminders 3 days before, 1 day before and the day the study begins so that fewer participants would drop out/not be prepared to start the experiment.
Make sure that all participants received the placebo package at least 5 days before the commencement of the study. The 5-day window would allow enough time to get replacement packages out in case of postage issues. When posting, I would make sure that the package can fit through a letter box reducing the hassle of starting the study, in case they would need to pick up the package from the Post Office if it were delivered when they were not at home.
Instructions for non-intervention group
Make sure that participants realise that they should be using their usual method of remembering to take their medication
Second stage of study
Create shorter instruction message (if it has not been a long time since the first stage). Telling them – you’re sick again, repeat the study but with one change;
- You’ll receive reminders
- Use your usual methods to remember to take your medication
The second message could be rephrased to either tell the participants;
- what we want to learn this time
- that I want to compare how they did under each condition
Post study
Automate the adherence chart yes/no then interview to find out the reason. I found that it was sometimes difficult to get hold of the participants for the interview straight after the study and it was important that I asked about their adherence soon after the experiment finished so that there was less chance of reconstruction bias.
Consider letting people know that they were being ‘tracked’ in two different ways (pill count and self-reporting). Participants being aware could be another discussion topic for the interview e.g. “did you do as well as you thought?”
Survey
Try to reduce the number of questions in the survey, so that I only collect information that is directly relevant at this stage. e.g. removing the questions “I could use the text message reminder without further instructions” and “Both occasional and regular users would like the text message reminder.”
Move the question – “The text message reminder works the way I want it to work” to the interview so that I can explain what I mean by this question in case there is any confusion.