This post is part of a series, describing the assessments used to develop the Institute for Research Design in Librarianship (IRDL).
One month after the Summer Research Workshop was complete, we emailed them an anonymous link to a Qualtrics survey that took about ten minutes to complete, to ask about their workshop experience. LMU’s IRB reviewed the protocol ahead of sending it to the group.
The survey covered for areas of concern: the curriculum, instructors, their time at the workshop, and their cohort. The questions about the curriculum were to gather information about the amount of pre-workshop activities, how challenging they found the curriculum, how appropriate the curriculum was for their current needs, and the time set aside for hands-on activities and exercises.
We asked the Scholars to rate the instructors on clarity, helpfulness, and approachability.
We asked about the time commitment needed to participate in the workshop, and if they had sufficient support from their home institutions to focus exclusively on the workshop during the two-week period.
We were interested to learn if the workshop design impacted how each Scholar felt about belonging in a cohort. We asked if they felt part of the group, if the felt comfortable contributing to discussions, and how likely it was that they would stay in touch with at least one person in their cohort.
To end, we asked two open-ended questions for them to suggest any improvements for future workshops and if they had any other feedback about their workshop. Linked here is the version of the survey in use from 2022-2024.
My reflection on the use of this tool for assessing the program
I expected that this survey would give the Scholars the opportunity to tell us anonymously some critical things that they might not have told us in person (or via email) about their experience while the workshop was underway. I wanted to give the Scholars a bit of time after the workshop to mentally process their experience before responding, to allow them time to pause and reflect before we sent the survey.
We did seek their more immediate feedback during the workshop, even setting up an anonymous form that I checked every day, with this as the question: “Got something to say but would rather do it anonymously? We want to hear about it. We will check this once a day during the IRDL workshop. This form will collect a timestamp and the feedback you provide, that’s it.” From that form we learned that they felt as if they needed more one-on-one consultation time with experts, a suggestion for crowdsourced notes, rather than taking them individually, and that they loved the prompts for the morning sessions that were Scholars-only spaces to chat about non-research topics.
I expected from this post-workshop survey that we would learn that the workshop was too long, at two full weeks, but we heard quite the opposite (!). Over the years, the general feedback received was that the Scholars wanted more time for their expert consultations and wanted more down-time with their cohorts to process what they were learning. Time together, even with two weeks carved out specifically to think about research, proves to be too little for those seeking to begin their research endeavors.
I honestly did not expect the impassioned compliments we received in the open-text responses, one being, “Overall, this was a super impactful experience. I would HIGHLY recommend this program to anyone who wants to do their own research. It did exactly what I hoped it would do…I feel prepared to complete my research project!” I’m glad we provided a mechanism to receive this kind of feedback, too.
The cost of this assessment tool
Because LMU has an institutional subscription to Qualtrics, no grant funds needed to be used for the assessment. No specialized software was needed for the analysis of data.
Earlier posts in this series:
Introduction post, Confidence scale, Research networks of the Scholars, External review
