Organization Monkey

Using Albert Bandura’s theory of self-efficacy (1977) as a theoretical base, we designed the Institute for Research Design in Librarianship (IRDL) to address the four main sources of influence on self-efficacy: mastery experiences, social persuasion, modeling, and emotional arousal. We interpreted the component of “mastery experiences” as successfully learning new tasks, providing the participants with the ability to persevere in the face of obstacles. We interpreted “social persuasion” to structured situations in which the participants receive encouragement and experience success in working through challenges. We viewed “emotional arousal” as how people were feeling when they were in the research process. We interpreted “modeling” to allow for viewing the research process of the other Scholars in their cohort, as well as having access to a formal mentor to provide accurate guidance on the research process. Over the course of IRDL we designed multiple points of measurement to assess the efficacy of those components.

To assist us in developing sound assessment mechanisms, in the first year of IRDL (2014) the two program co-investigators (me and Kristine Brancolini) collaborated with two staff members on campus who focused on assessment metrics as part of their regular work. Leading the IRDL assessment effort was the LMU Director of Assessment, Laura Massa, and the LMU Manager of Surveys and Evaluation, Christine Chavez. The four of us mapped the desired outcomes of the program and designed appropriate assessment tools to measure how successful the program model was in meeting the outcomes.

Since we based the design on the components of the Bandura theory, our initial focus of assessment was on them. To determine the possible gains program participants had related to mastery experiences, we used a confidence scale we designed around the components of the research process. To measure possible changes related to social persuasion, we used a personal network series of surveys over the course of the year, to observe the nature of the research-related relationships the participants developed. To determine how the Scholars were self-regulating, we checked in with them via a survey, at the mid-point in the year-long experience. To determine how a mentor-mentee relationship may have affected the participants sense of confidence, we created a survey for the mentors and a separate survey for the participants (the mentees), that was delivered at the end of the program.

With each year of the IRDL program, we had nine separate formalized assessments in place, to help us evaluate if we were meeting our programmatic goals: 1) the confidence scale noted above; 2) a series of research network surveys, noted above; 3) a post-workshop survey; 4) an external review; 5) a pre-/post-workshop research proposal evaluation; 6) a mid-point check-in survey; 7) a summative survey; 8) a mentor feedback survey, noted above; and 9) a scholar feedback survey on the mentor program, noted above.

In a series of blog posts I will focus on each assessment, when it was delivered, what mechanism of data collection was used and its analysis, how much the assessment cost, and a reflection on its effectiveness in measuring what we intended.

I include here the annual schedule, showing the year according to the grant terms, that I used to coordinate the assessments.

I want to put “ask me about my Gantt chart” on a t-shirt someday.

[Edited 8/15/2025 to include the costs of the snacks]

Part of a series of reflections on some of the affective components of the Institute for Research Design in Librarianship (IRDL) program, things I put in place to make the Scholars feel welcomed and delighted in the learning environment.

Candy and snacks of the in-person workshop

As we were planning the first in-person Summer Research Workshop of IRDL we talked about how we would set up the environment for comfort and maximize easy physical interactions for group work and networking. We decided on round tables in the center of the room and two small tables in the back for me and Kris to observe. We decided not to have a lectern or formal panel table at the front of the room; the instructors would have wireless microphones (we had both lapel and hand-held) and could teach throughout the room.

In the back of the room were two couches and a big table between them. I asked the library’s Events Manager, who was partnered with us to make the event as smooth as possible, to load that table up with snacks. There were already going to be planned meals and snacks throughout the days, but in those between times, when someone was maybe feeling like they needed a momentary distraction, I wanted to have snacks available.

The Events Manager came through for me in a big way. She put so many snacks on the table that it was like a party. She bought clear glass candy jars with silver metal lids, candy scoops, and filled them with individually wrapped snacks (Nerds, gummy bears, trail mix) and bulk snacks (peanut M&Ms, etc.). She even brought fresh whole pieces of fruit to mix in.

Throughout the days of the workshop we’d say, “Meet me at the snacks,” if we wanted to chat about something during a break. During the tougher cognitive parts of the workshop, we’d notice more frequent trips of the Scholars to the back of the room to fuel themselves.

Clear glass candy jar with a metal lid, filled with peanut M&Ms and a candy scoop

Starburst, fruit gels, Haribo

I estimate that snacks for the in-person workshop cost around $750 per year. The first year would have cost more, to include the purchase of the candy jars and scoopers. Grant funds were used until a change in allowable costs for this particular grant, restricting being able to pay for food. At that point the library absorbed the cost.

Candy and snacks of the online workshop

During the Zoom orientation for the online Summer Research Workshop we hinted that there would be information coming to them soon about snacks, but that is all we told them. Then, later, when they received an email alerting them that they had been gifted a build-your-own box of snacks at SnackMagic, they were delighted. The gift had a set amount budgeted for each person, and they could choose whatever snacks they wanted within that amount. Then a box would appear on their doorstep ahead of the Workshop, so they were stocked with snacks for the whole ten days.

The first year we sent these boxes, the Scholars told us to tell future cohorts to mail themselves their snacks at their work addresses rather than at home, so their kids wouldn’t get to the boxes ahead of them!

We chose SnackMagic after a comparison of a few companies like this because the price point was within our expectation, the snack options were diverse (they even have vegan snacks), and the offer drinks, too. One of the Scholars one year chose nothing but drinks, to try new things.

Every day of the Workshop, during a break, we’d compare notes about which snacks we’d selected, which ones we preferred, and which one we were snacking on at that moment.

We ordered a box ahead of time for quality control. I was excited to receive my box.

The cost of snacks for 25 participants is $1,000 ($40 per box, for 25 people). No grant funds were used to pay for these snacks; the library Dean paid for them.

[Edited 8/15/2025 to include the costs of the awards and prizes]

Part of a series of reflections on some of the affective components of the Institute for Research Design in Librarianship (IRDL) program, things I put in place to make the Scholars feel welcomed and delighted in the learning environment.

Awards and prizes in the in-person workshop

To encourage participation during the lecture parts of the summer research workshop, the instructors embedded questions, quizzes, and group activities into their presentations. While the workshop was in person, the instructors would award small prizes for the participants brave enough to answer a question, jump in to share an opinion, or offer a reflection. I bought a bunch of prizes at the Dollar Store ahead of the workshop and put them in a basket so the instructors could easily see all of them. The instructors could choose to award a big sticker, a sheet of stickers, a puzzle, a frisbee, a travel game, etc., depending on how “big” the question was that the participant answered. Once the participants saw what the prizes were in the basket, they’d answer a question in order to make a request for a specific prize.

The prizes met the program design goal we were after, to make participating fun and silly, with a low threat. The instructors even awarded prizes to wrong answers if they were especially good or funny.

Here are a few of the prizes:

A screenshot from an old tweet from a participant that won a puzzle.

Crossword puzzle book prize

A tattoo sleeve prize

In addition, there were group prizes, of which the favorite was for the sampling visualization challenge, where a group of participants needed to work together to use available items in the room to demonstrate an assigned sampling technique. Here’s a photo of one of the group awards:

A photograph of the group that won a group prize of stick-on mustaches, for their efforts in visualizing the simple random sampling technique.

The awards and prizes for the in-person workshop were purchased from the Dollar Store and Amazon, for a cost of around $100.

Awards and prizes in the online workshop

A consideration when we moved the 10-day workshop to an online environment was the amount of time spent synchronously on Zoom. Even with plenty of scheduled and spontaneous screen breaks, asking the Scholars to maintain focus on lectures and exercises seemed like a big request.

A small reward for the Scholars to keep their eyes on the PowerPoint slides was a challenge I designed, named Hidden Iggys. This was the premise:

LA is very close to Disneyland and here at LMU we are familiar with all things Disney. One thing you can do at Disneyland is look for Mickey-Mouse shaped objects throughout the theme park, called “Hidden Mickeys” (https://en.wikipedia.org/wiki/Hidden_Mickey). They’re just for fun, to see how close you’re paying attention as you go through the park.

IRDL has taken that idea and has hidden some images of the LMU mascot lion, named Iggy, in the slide decks. If you spot an Iggy in the slide deck during the lecture, then fill out the Google form telling us where you saw it and get entered into a raffle to win an LMU t-shirt. The Iggys can be large, they can be small, they can be distinct, they can be blurred, and they may be different colors.

There are 2 Iggys each day with lectures, for a total of 18 Iggys. Your name will be entered into the raffle for each Iggy you find. Get them all!

FILL OUT THE FORM FOR EACH IGGY YOU SPOT

All the Iggys will have this same basic shape:

The Google form requested the title of the slide deck where they spotted the Iggy, the slide number, and the Scholar’s name. The Scholars had access to the slides, so they could look for the Iggys there if they didn’t notice them during the lecture session.

After the last lecture section, I used a random number generator to choose one of the entries from the form. The winner got to select from LMU’s online store a branded item to be sent to them in the mail.

Here’s a screenshot of one of the slides with an Iggy “hidden” on it:

A hidden Iggy on a lecture slide

The Hidden Iggy prize cost around $25. The group prize for collaborative demonstration of a sampling technique cost $25 ($5 gift card each, for 5 Scholars).

[Edited 8/15/2025 to include the costs of the snacks]

Part of a series of reflections on some of the affective components of the IRDL program, things I put in place to make the Scholars feel welcomed and delighted in the learning environment.

Badges (in person)

IRDL Advisory Board member Jennifer Masunaga (a librarian employed at LMU at the time) suggested to me that she could design and make some buttons for me to give out during the workshop, as motivators for the attendees. The library had just purchased a button machine, and we were finding a lot of creative inspiration with the tool. I welcomed the idea and created a list of workshop achievements that could be matched with a button.

I broke the list into three categories: completion; competition; for fun. I made sure to consider how many people could win each badge, so that everyone left the workshop with at least a few.

During the first year the badge design included an icon to represent the achievement and in subsequent years we added the four-digit year underneath the icon to distinguish them as belonging to that year’s cohort. These were the badges Jenn created for us in the first three years of IRDL:

A photograph of the badges used during the in-person IRDL workshops

Completion badges

1. Human subjects training is complete: Everyone was awarded this badge, as completing training at their institution ahead of the workshop was a requirement.
2. Research question is complete: Everyone could win this badge when they determined that their research question was in its final form.
3. Methods section is complete: Everyone could win this badge when they determined that their data collection strategy was solid.
4. Focus group expert: Everyone was awarded this badge, at the end of the focus group hands-on exercise in which the entire cohort practiced writing a focus group interview guide, leading a focus group, participating in a focus group, and notetaking in a focus group.
5. Completed IRDL: Everyone was awarded this badge on the last day of the workshop.

Competitive badges

1. Best sampling demonstration: Each person in the group that won the sampling competition was awarded this badge, as well as a group prize.
2. Most number of words in application proposal: One person was awarded this badge after putting into a shared Google document the total number of words in their IRDL research proposal, the person using the most words as winner.
3. Largest number of tweets: We encouraged social media participation on the most robust platform at the time, Twitter, using the hashtag #IRDL. The two people who posted the most tweets during the workshop won this badge.
4. Best oral presentation: On the last day of the workshop, each Scholar had to present to the rest of the group for seven minutes about the current state of their research project. The two best presenters (as judged by their peers) were awarded this badge.

For-fun badges

1. Ate at In-n-Out (available only on Wednesday): Everyone could win this badge if they went to the California-famous chain restaurant, In-n-Out, and brought back a receipt or a photograph. We normally would provide a dinner at the campus dining facility for each Scholar, but Wednesday afternoon was a day off to explore Los Angeles and try dining on their own.
2. Went in the ocean: Everyone could win this badge if the dipped at least one toe into the Pacific Ocean, providing photographic proof.
3. Saw a celebrity: Everyone could win this badge if they reported a sighting of a famous person during their time in Los Angeles.
4. Traveled the farthest to be at IRDL: Using Google Maps to determine the number of miles from the Scholar’s house to the address from LMU, if driving, the one person who “traveled” the farthest to attend the workshop was awarded this badge.
5. In 2019, the west side of Los Angeles, where LMU is situated, was especially impacted by the weather phenomenon called June Gloom, during which the mornings were heavily fogged, with a break for sun in the afternoon. To commemorate this, I made a “Survived June Gloom” button that was awarded to everyone.

Screen capture of a tweet from a Scholar, proof of going to In-n-Out, in order to win a badge

Screen capture of a tweet from a Scholar, proof of dipping at least a toe in the Pacific Ocean, in order to win a badge

The Scholars developed clever ways to display the badges they had won, affixing them to a cardigan or their backpack/computer bag. We noticed during our monthly video check-in sessions with the Scholars after the workshop that some had made sashes to display their badges on or put them on a corkboard behind their computer workstations.

The library already had both the button machine and bulk materials so the cost for making the badges was minimal; no grant funds were spent on this.

Badges (online)

When we moved to an online environment it was clear that I would not be able to award physical badges during the workshop and so created virtual badges instead. To facilitate the awarding process, the library’s student designer in the Outreach and Engagement department created a custom Zoom background that the Scholars could use during the workshop.

The Scholars started the workshop with this custom – but empty- Zoom background. As they won badges, I updated the empty background by adding an image of the badge. The Scholar would then use their updated background. To facilitate this during the workshop, I created an image file named for each Scholar and in Photoshop added each new badge as a layer, flattening the resulting image to send to the Scholar.

Personalized Zoom background, with badges won by that Scholar

It was clear that some of the previous in-person badges would not apply and offered an opportunity to design some new ones. I removed these three since they were LA-situated: ate at In-n-Out; went in the ocean; saw a celebrity.

I changed the traveled the farthest to be at IRDL to virtually travelled the farthest and the person who from their house address to LMU’s address (using Google Maps, driving route) had the greatest number of miles won the badge (one badge awarded).

I introduced four new badges to account for the new video software environment delivery of the workshop, via Zoom. Each Scholar was assigned to meet with the program team several times during the workshop, for personalized consultations about their research protocol. When the Scholar had met at least once with all five members of the team, they were awarded the met with all research experts badge.

The Zoom Crash badge

During the in-person workshop the Scholars would often share stories and pictures of their loved ones back home, tagging posts of photos of them on twitter with the hashtag #IRDLkids and #IRDLpets. The Zoom environment allowed us to meet those loved ones in a live setting. To encourage the Scholars to bring their kids and pets into the frame of the video screen with them, I created a Zoom-crashed! badge, that was awarded to anyone who introduced us to their pet, child, roommate, or coworker. A creative Scholar who wanted to earn the badge but did not have a pet drew one and shared the drawing over the screen.

I wanted to keep a tie to LMU even though the Scholars were not on our campus for the workshop. I created an Iggy badge, for any Scholar whose Zoom background image included LMU’s mascot, Iggy, for at least one day.

To keep another tie to Los Angeles, instead of asking the Scholars to go in the ocean, we changed the entire theme of the first Friday of the workshop to “Beach Day.” In the online instruction portal, we pointed to several live beach cameras in the Los Angeles area, so they could experience the Pacific Ocean from wherever they were situated. To further the fun for that first Friday, when we thought the Scholars might be weary after five days of instruction via Zoom, we encouraged them to wear beach-themed outfits. The fidgets we sent them for that day were drink umbrellas and plastic sunglasses. Scholars were awarded the Beach badge if they changed their Zoom background on that day to a photograph of any beach. Their beach photographs provided a lot of conversation about desired vacation spots and stories about growing up or visiting those beaches.

To keep things fun for the instruction team and Zoom assistant, I created customized Zoom backgrounds for them to use on that day, with drawings of palm trees on the sides.

Beach Day Zoom background for IRDL instructors

These are the badges used in the online workshop, from 2022 to 2024:

Completion badges

1. Human subjects training is complete
2. Research question is complete
3. Methods section is complete
4. Focus group expert
5. Consulted with all 5 research experts
6. Completed IRDL

Competitive badges

1. Best sampling demonstration (group award)
2. Most number of words in application proposal
3. Most tweets (used from 2022-2023)
4. Best oral presentation (as judged by your peers)

For-fun badges

1. (Virtually) traveled the farthest to be at IRDL
2. Zoom background photo includes LMU’s mascot, Iggy, for at least 1 day
3. Best beach-themed Zoom background
4. Zoom-crashed! (interrupted by a pet, kid, roommate, coworker)

The cost for producing the online badges was in personnel time, for the student designer to create the Zoom background and for me to produce the digitized versions of each badge. No grant funds were used for the badges.

The first in a series of reflections on some of the affective components of the IRDL program, things I put in place to make the Scholars feel welcomed and comfortable in the learning environment.

Fidgets (in person)

When the summer workshop was hosted in the LMU library (Los Angeles, CA) as an in-person event (2014-2019), I put a bunch of “fidgets” on each of the round tables in the training room, so that people sitting at each table could choose their own toys throughout the days to occupy their fingers during the learning sessions. I placed objects like chenille pipe cleaners in a variety of colors, squishy balls, toy cars, LEGO building blocks, crayons, coloring book pages, and small plastic animals on the tables.

At the end of each day of the workshop, I cleared the tables and reset them for the next day. As I picked up the used fidgets, I noticed that the people at the tables had cooperated in building scenes for the items found at the tables, like building a pipe-cleaner house for a plastic animal or a drawn road on several pieces of paper for the toy cars to drive on. We replaced used toys with fresh ones each day over the course of the nine-day workshop.

As we wrapped up the last day of the workshop and prepared to leave the training room for the last time, a Scholar asked if they could take their favorite fidget home with them. Clearing the tables on that last day was a breeze because they took everything. We heard from the Scholars after the workshop that the toys occupied a prized place in their work offices, as a fun memory of the workshop.

The fidgets and hands-on materials for exercises for the in-person workshop were purchased from the Dollar Store and Amazon, for a cost of around $300.

Fidgets (online)

As we moved to hosting the summer workshop online via Zoom instead of in person (2022-2024), I thought through some options for fidgets, that might create a similar fun atmosphere in the new setting. I remembered back to when I was young and travelled during the summer in the car with my mom from Texas to visit relatives in Ohio, she would pack for me a little prize to open every day of the car trip. Those packages were delightful to me, and I couldn’t wait to see the prize each day (a paddle game, a can of Play-Doh, a jump rope). Keeping that delight in mind, I decided to pack a fidget for each Scholar, for each day of the workshop.

I put each fidget in a brown paper bag and folded it closed, with a sticker on the outside for the day on which the Scholar was to open the bag. I put ten bags into a box and sent it to each Scholar via UPS, with instructions not to open any bag until the first day of the workshop.

Here’s what I packed in each bag:

Open on Day 1: Four printed pages of advice from previous Scholars/Mentors, a Slinky, some pipe cleaners
Open on Day 2: building blocks, toy cars
Open on Day 3: 25 google eyes, giant foam dice (these fidgets could be used during the hands-on practice with sampling design)
Open on Day 4: 2 pre-stamped IRDL postcards (one to send to a friend, one to send to a colleague they thought should apply to IRDL), LMU library sticky notes, an LMU library magnet
Open on Day 5: paper drink umbrellas, plastic sunglasses (for the “beach” theme day)
Open on Day 6: single item large toy (Nerf football, dinosaur, Spirograph)
Open on Day 7: sheets of stickers
Open on Day 8: 3 small plastic animals
Open on Day 9: 3 glow bracelets, Rubik’s cube
Open on Day 10: IRDL completion badge

Office full of treat bags, ready to be packed and sent

The fidgets and hands-on materials for exercises for the online workshop were purchased from the Dollar Store and Amazon, for a cost of around $200. Boxes and tape to send the materials cost around $60. The library covered the cost of the mailing of the materials via UPS.