Happy new year folks! There are no pictures in this post, SORRY NOT SORRY.
A group of researchers at the University of Alberta recently
published a study on learner engagement in Dino 101, and I thought I'd
summarize it briefly here and share a few thoughts about it. You can read the
original article online for free via Google Books: "Emotional and social engagement in a massive open online course: an examination of Dino 101".
You might also want to check out another summary of their data at the
University of Alberta's site.
Daniels et al. lay out four components for describing the
otherwise somewhat nebulous term 'engagement':
- Cognitive engagement (investment into thinking about tasks and mastering content; could also be thought of as motivation for learning)
- Behavioural engagement (things like attendance, paying attention, and participation)
- Emotional engagement (things like anxiety, boredom, interest, etc.)
- Social engagement (willingness to socialize with others, to make connections)
They
mainly investigated emotional and social engagement, since cognitive and
behavioural engagement are largely measured by things like course completion
rates and grade averages. Daniels et al. sent a survey to people enrolled in
Dino101 towards the end of its initial offering, as well as the students
enrolled for credit in Paleo200/201 at the UofA, and also interviewed a total
of 30 students representing all three versions of the course. Because all
of the lessons and study material for Dino101 was released at the beginning of
the semester (in contrast to many MOOCs which release new lessons on a weekly
basis, probably in line with the in-person lectures at the university), I
wonder if conducting the survey at the end of the semester, in December,
influenced the results. We heard from many students that they completed Dino101
very very quickly – some in as little as a day or two, and many within 2-6
weeks. From managing the forums, I can also attest that discussion board
participation dropped pretty dramatically after about six weeks. Therefore,
students that completed the course are not necessarily students that stuck
around until December, and I wonder how well this slice of the long statistical
tail might represent the majority of the people who engaged with Dino101.
Overall,
the results showed that we did a pretty good job of emotionally engaging
Dino101 students: many said they were inspired by the material, didn't get
bored, and felt attentive during the lectures. We fared less well in terms of
social engagement: many students were neutral about whether or not they had a
sense of belonging in Dino101, but only 16% had expected student-student
interactions to contribute to their own learning. (In other words, most
students expected to learn primarily from the videos and course notes, and not
from discussions with other students.) I'm not sure if this is typical for a MOOC or unique to Dino 101, so I'd be curious to see if there are similar studies for other courses that show similar results.
In
the data from the student interviews, Daniels et al. highlighted both positive
and negative aspects to the discussion forums – some students were really
interested in the forums even if they didn't participate, some didn't look at
them at all, and some found them overwhelming because of too many email notices.
I think there are a couple of takeaways just from this one portion of their
paper:
- If you're doing a MOOC, make sure the button for the discussion forums is prominently displayed towards the top of the course page near the lecture videos. I think ours was located kind of far down the menu and some students might just overlook it.
- Give students some guidelines for using the forums, both technically (i.e. how to turn off email notifications for new posts, because that would have driven me COMPLETELY MAD if I hadn't known how to turn it off for most posts), and by giving suggestions for how to participate in the forums. Do this right at the beginning of the course.
Another thing that popped up in this paper is that while social engagement online wasn't
as high as it could be, social engagement in meatspace was unexpectedly common.
There were lots of people taking the course with their kids or other family
members at home, and they would sit and do the course together. So, here is
another takeaway suggestion: find ways to increase social engagement by giving
people things to do at home together, in the physical world. Maybe we could
provide suggestions for local museum trips or fossil sites based on where
people in the course are based, or give some suggestions for hands-on
activities people can do with stuff you find in your house.
Finally, there were mixed feelings about engagement with the
instructors of the course (here limited to Phil Currie and Betsy Kruk, who
presented the material, but excluding myself and the other teaching assistants who
did most of the online interactions in the forums). Some students felt like
they had a good connection with Phil and Betsy despite the scripted presentations,
and others found they didn't like the scripted format at all. Strangely, the
paper doesn't include much discussion about engagement with the instructors (and
here I'm including the TAs) in the discussion forums, the only part of the
course where that was really possible given the pre-recorded nature of the
videos.
My final comment here is that if engagement with an
instructor is important for social engagement in MOOCs (and I think engagement
with the instructor is important in education generally, so it should probably be
important in MOOCs), I don't know what we're going to do. One of the comments
from the interviews that's highlighted in the paper is that one student didn't
have his question answered in the forums – and with my reading between the
lines, that probably means we failed to socially engage this student, which
sucks. I feel badly that a student felt ignored. But the reality is also that
we had 23 000 people enrolled in Dino101, and had to manage hundreds of forum
posts on very limited time budgets, in addition to managing the for-credit
version of the course at the university. There is no way to scale up personal
interactions between students and instructors in a learning environment without scaling up your teaching staff – either you need more instructors (in
the form of profs, TAs, whatever), or you need fewer students, or you're not
going to be able to interact with every student that wants interaction. And we
shouldn't be asking instructors in a university to educate for free, so
somebody needs to be paying for those extra people. So that's one more
important takeaway here: social engagement requires a lot of time investment
from the instructors to encourage discussion and set up an environment that
encourages social interaction, so if you want to run a MOOC with high social
engagement, you need to budget money for lots of instructor/TA time.
Anyway, that's probably one of the last posts I'm going to
have about Dino101/MOOCs for a while, but I thought I would mention one other
piece of news: before I moved down to Raleigh in 2014, I had started work on some
new palaeontology mini-MOOCs in my role as the Science Digital Learning Manager
at the UofA. After much hard work from the palaeo crew at the UofA, these
courses are now just about finished and will be launching on Coursera over the
next few weeks – if you liked Dino101, you might want to take a look at
Theropods and the Origin of Birds (starting later in January), Ancient Marine Reptiles (starting in February), and Early Vertebrate Evolution (starting in
March, not yet available for registration at Coursera but keep an eye on the
UofA's page there).
