Social engagement in Dino 101

Happy new year folks! There are no pictures in this post, SORRY NOT SORRY.

A group of researchers at the University of Alberta recently published a study on learner engagement in Dino 101, and I thought I'd summarize it briefly here and share a few thoughts about it. You can read the original article online for free via Google Books: "Emotional and social engagement in a massive open online course: an examination of Dino 101". You might also want to check out another summary of their data at the University of Alberta's site.

Daniels et al. lay out four components for describing the otherwise somewhat nebulous term 'engagement':

• Cognitive engagement (investment into thinking about tasks and mastering content; could also be thought of as motivation for learning)
• Behavioural engagement (things like attendance, paying attention, and participation)
• Emotional engagement (things like anxiety, boredom, interest, etc.)
• Social engagement (willingness to socialize with others, to make connections)

They mainly investigated emotional and social engagement, since cognitive and behavioural engagement are largely measured by things like course completion rates and grade averages. Daniels et al. sent a survey to people enrolled in Dino101 towards the end of its initial offering, as well as the students enrolled for credit in Paleo200/201 at the UofA, and also interviewed a total of 30 students representing all three versions of the course. Because all of the lessons and study material for Dino101 was released at the beginning of the semester (in contrast to many MOOCs which release new lessons on a weekly basis, probably in line with the in-person lectures at the university), I wonder if conducting the survey at the end of the semester, in December, influenced the results. We heard from many students that they completed Dino101 very very quickly – some in as little as a day or two, and many within 2-6 weeks. From managing the forums, I can also attest that discussion board participation dropped pretty dramatically after about six weeks. Therefore, students that completed the course are not necessarily students that stuck around until December, and I wonder how well this slice of the long statistical tail might represent the majority of the people who engaged with Dino101.

Overall, the results showed that we did a pretty good job of emotionally engaging Dino101 students: many said they were inspired by the material, didn't get bored, and felt attentive during the lectures. We fared less well in terms of social engagement: many students were neutral about whether or not they had a sense of belonging in Dino101, but only 16% had expected student-student interactions to contribute to their own learning. (In other words, most students expected to learn primarily from the videos and course notes, and not from discussions with other students.) I'm not sure if this is typical for a MOOC or unique to Dino 101, so I'd be curious to see if there are similar studies for other courses that show similar results.

In the data from the student interviews, Daniels et al. highlighted both positive and negative aspects to the discussion forums – some students were really interested in the forums even if they didn't participate, some didn't look at them at all, and some found them overwhelming because of too many email notices. I think there are a couple of takeaways just from this one portion of their paper:

•  If you're doing a MOOC, make sure the button for the discussion forums is prominently displayed towards the top of the course page near the lecture videos. I think ours was located kind of far down the menu and some students might just overlook it.
• Give students some guidelines for using the forums, both technically (i.e. how to turn off email notifications for new posts, because that would have driven me COMPLETELY MAD if I hadn't known how to turn it off for most posts), and by giving suggestions for how to participate in the forums. Do this right at the beginning of the course.

Another thing that popped up in this paper is that while social engagement online wasn't as high as it could be, social engagement in meatspace was unexpectedly common. There were lots of people taking the course with their kids or other family members at home, and they would sit and do the course together. So, here is another takeaway suggestion: find ways to increase social engagement by giving people things to do at home together, in the physical world. Maybe we could provide suggestions for local museum trips or fossil sites based on where people in the course are based, or give some suggestions for hands-on activities people can do with stuff you find in your house.

Finally, there were mixed feelings about engagement with the instructors of the course (here limited to Phil Currie and Betsy Kruk, who presented the material, but excluding myself and the other teaching assistants who did most of the online interactions in the forums). Some students felt like they had a good connection with Phil and Betsy despite the scripted presentations, and others found they didn't like the scripted format at all. Strangely, the paper doesn't include much discussion about engagement with the instructors (and here I'm including the TAs) in the discussion forums, the only part of the course where that was really possible given the pre-recorded nature of the videos.

My final comment here is that if engagement with an instructor is important for social engagement in MOOCs (and I think engagement with the instructor is important in education generally, so it should probably be important in MOOCs), I don't know what we're going to do. One of the comments from the interviews that's highlighted in the paper is that one student didn't have his question answered in the forums – and with my reading between the lines, that probably means we failed to socially engage this student, which sucks. I feel badly that a student felt ignored. But the reality is also that we had 23 000 people enrolled in Dino101, and had to manage hundreds of forum posts on very limited time budgets, in addition to managing the for-credit version of the course at the university. There is no way to scale up personal interactions between students and instructors in a learning environment without scaling up your teaching staff – either you need more instructors (in the form of profs, TAs, whatever), or you need fewer students, or you're not going to be able to interact with every student that wants interaction. And we shouldn't be asking instructors in a university to educate for free, so somebody needs to be paying for those extra people. So that's one more important takeaway here: social engagement requires a lot of time investment from the instructors to encourage discussion and set up an environment that encourages social interaction, so if you want to run a MOOC with high social engagement, you need to budget money for lots of instructor/TA time.

Anyway, that's probably one of the last posts I'm going to have about Dino101/MOOCs for a while, but I thought I would mention one other piece of news: before I moved down to Raleigh in 2014, I had started work on some new palaeontology mini-MOOCs in my role as the Science Digital Learning Manager at the UofA. After much hard work from the palaeo crew at the UofA, these courses are now just about finished and will be launching on Coursera over the next few weeks – if you liked Dino101, you might want to take a look at Theropods and the Origin of Birds (starting later in January), Ancient Marine Reptiles (starting in February), and Early Vertebrate Evolution (starting in March, not yet available for registration at Coursera but keep an eye on the UofA's page there).