Collaborative auto-ethnography – an antidote to big data in MOOCs?

Firstly, thanks to @helencrump for the title of this post. The alternative title was ‘#oldsmooc – the MOOC that keeps giving”. But I think Helen’s one sounds much more impressive 🙂

I’m not quite sure where I’m going with this post but there was a flurry of activity on my twitter stream last night/this morning and I just wanted to try and capture some of the ideas that have been floated.

Regular readers of this blog will know that a group of #oldsmooc-ers recently presented a paper at the EMOOCs Conference, called “Signals of success and self directed learningwhere we took a collaborative auto-ethnographical approach to create a range of narratives which described our different measures of success in participating in the mooc.  We felt that this personal reflection would help to address some of the gaps in understanding what actually motivates learners in MOOCs and probably more importantly for us as a group to explore the extent “connection, self-efficacy and self-directed strategies facilitate learning in a MOOC”.   As a self selecting group we were very conscious that we could not ensure our narratives were typical of other leaners on the MOOC, but we did try and collect some more data from other participants.  I have tried to document the story of our collaboration, which has been a key  signal of success for us all.

Getting the paper accepted was a great moment for us all, as was the conference presentation and tweets it generated.  We were still unsure if our approach and methodology really had any traction.  But last night Paige altered us all to this bit of activity on the current #rhizo14 MOOC.  Again through connections and network (aka some of our group taking part in another mooc) it seems our ideas and approach are being explored by other learners.

I think this is really important.  We know that existing accepted educational metrics don’t really apply in the MOOC context, particularly for retention. Despite the promise of MOOCs and big data being able to give us insights into how people learn, I, like others, am still not so sure about some of the methods being used and in turn the patterns that are emerging. As well as the quantitative data, we need to get much more qualitative data exploring as many different narratives as possible from learners. It’s only by doing that that can we really start to help develop our understanding of how people define success in MOOCs. And in turn, we can ask more challenging questions of/from the quantitative data.

Being a bear of very little brain, I like seeing the pretty patterns and swirly diagrams, but find it confusing when they don’t seem to relate to my own experiences.  Mind you,  if this article in the Sunday Observer is to be believed we won’t need to grapple with big data -v – little data -v- educational theory for much longer as soon the google robots will have worked it out all out for us and will have “fixed” education.

My experience of learning on MOOCs has been very different from my traditional educational experiences. I know I didn’t (and still don’t really) enjoy formal education, and I am much happier (and hopefully more creative) in connected, loosely structured learning experiences than read a bit, do the test, read a bit more ones.

Anyway below is a collation of the tweets from last night this morning, which range from us being all “check us out with starting an auto-ethnographic revolution” to more serious questions about the nature of open collaborative spaces, self disclosure and the importance of failure. On the last point, Pat Parslow referenced the “confessional” booth at the PELeCon Conference, which again got me thinking about the use of the language of guilt around what are perceived to be non traditional ways of doing things. But that’s probably a post for another day.

Cloudworks, purple dots, tea and biscuits (Warning this post contains references to MOOCs) #emoocs2014

How do you measure success on a MOOC? It’s a question that has been causing a lot of consternation as our traditional measures of success in education don’t seem to apply. Large drop out rates, challenges of assessment at scale, I don’t really need to go into all the details, it’s documented by others far more eloquently than me.

However, despite my unsuccessful attempts in many MOOCs I have managed to complete a few (4 now!).  They all have been successful learning experiences for me. I got a certificate for one of them, but nice as they spacemen are on it, it isn’t really my biggest signal of success. As I blogged about before, it’s been after the MOOC  that the really collaboration and reflection on success has begun.

This week at the EMOOCs 2014 Conference my colleagues Paige Cuff and Helen Crump are presenting our paper Signals of Success. Now having a paper accepted for a conference might not be your measure of success for a MOOC, but for me, Paige, Helen, Penny, Briar, Iwona and Yishay, actually pulling of this international collaboration has been a real triumph.

You can see a pre-print of our paper and some more information here.

And if you are wondering where the title of this post came from, this twitter conversation will give you a clue.

After the mooc has gone – the real collaboration and connectivisism begins

In my invited speaker slot at ALT-C last month I mentioned that MOOCs despite what you may think of them have given the sector a rush of some kind of energy.  Measuring success on MOOCs tho is still fraught. Many much wiser souls than me have shared various theories as to why this may be, including the drop out rates, the “you take what you need”, “it’s more like a conference” etc.  I’ve blogged a lot about  my adventures with MOOCs over the past year. I’ve analysed and reflected on my own disengagement (and engagement) with them.  By traditional measures I’ve only actually completed three MOOCs but from each of them I still have sense of community and connections that have endured beyond the set time of the actual MOOC.  

#oldsmooc ( the learning design MOOC run by the OU) has been probably the most challenging MOOC for me but in many ways also the most rewarding as it was actually the first time that I felt that I had true learner autonomy.  It was a challenging for many reasons but towards the end there was a real spirit of community, trust and support – and that wasn’t just because there were only a few of us left standing 🙂  

Through the wonders of social media (particularly twitter) I’ve managed to stay in contact with a number of my felllow #oldsmooc-ers., and over the last couple of weeks I think we’ve done something quite extraordinary. Six of us have collaboratively written a paper. Like so many things these days, it started with a tweet.

Image

And so the power of connectivism sprang into life (see this storify for more)

Coordinating suitable times between  Australia, Canada, England, Ireland and Scotland  was quite a challenge, but thanks to some people being willing to get up at 6am, we did it. Using google hang-outs we started to make our plans.

After much discussion we decided that it would be good share our contrasting experiences of the #oldsmooc and what we have individually and collectively classified as our own measures of success.  Several months have passed since the end of the course, and I think it is only now that we all are able to begin to articulate and properly reflect on what we have all experienced and what impact it has had on us.

For me it was having the confidence to be able to in a way do what I wanted in terms of thinking about the context of learning and in particular the user interface of Cloudworks which  believed could be far more user friendly and useful through visualising connections. For others it was about developing and extending their own personal learning networks, using some of the actual course materials and methodologies in their own practice and collaboratively developing new learning designs. There is a really rich stream of data from our conversations and notes – enough for several papers.  

Fingers crossed our paper will be accepted but even if it isn’t, I’ve really enjoyed this extended learning experience and working with my peers from #oldsmooc.  Another measure of a successful learning experience imho.

Badges? Certificates? What counts as succeeding in MOOCs?

Oops, I did it again. I’ve now managed to complete another MOOC. Bringing my completion rate of to a grand total of 3 (the non completion number is quite a bit higher but more on that later). And I now have 6 badges from #oldsmooc and a certificate (or “statement of accomplishment”) from Coursera.

My #oldsmooc badges

My #oldsmooc badges

Screenshot of Coursera record of achievement

Screenshot of Coursera record of achievement


But what do they actually mean? How, if ever, will/can I use these newly gained “achievements”?

Success and how it is measured continues to be one of the “known unknowns” for MOOCs. Debate (hype) on success is heightened by the now recognised and recorded high drop out rates. If “only” 3,000 registered users complete a MOOC then it must be failing, mustn’t it? If you don’t get the certificate/badge/whatever then you have failed. Well in one sense that might be true – if you take completion to equate with success. For a movement that is supposed to be revolutionising the (HE) system, the initial metrics some of the big xMOOCs are measuring and being measured by are pretty traditional. Some of the best known success of recent years have been college “drop outs’, so why not embrace that difference and the flexibility that MOOCs offer learners?

Well possibly because doing really new things and introducing new educational metrics is hard and even harder to sell to venture capitalists, who don’t really understand what is “broken” with education. Even for those who supposedly do understand education e.g. governments find any change to educational metrics (and in particular assessments) really hard to implement. In the UK we have recent examples of this with Michael Gove’s proposed changes to GSCEs and in Scotland the introduction of the Curriculum for Excellence has been a pretty fraught affair over the last five years.

At the recent #unitemooc seminar at Newcastle, Suzanne Hardy told us how “empowered” she felt by not submitting a final digital artefact for assessment. I suspect she was not alone. Suzanne is confident enough in her own ability not to need a certificate to validate her experience of participating in the course. Again I suspect she is not alone. From my own experience I have found it incredibly liberating to be able to sign up for courses at no risk (cost) and then equally have no guilt about dropping out. It would mark a significant sea change if there was widespread recognition that not completing a course didn’t automatically equate with failure.

I’ve spoken to a number of people in recent weeks about their experiences of #oldsmooc and #edcmooc and many of them have in their own words “given up”. But as discussion has gone on it is apparent that they have all gained something from even cursory participation either in terms of their own thinking about possible involvement in running a MOOC like course, or about realising that although MOOCs are free there is still the same time commitment required as with a paid course.

Of course I am very fortunate that I work and mix with a pretty well educated bunch of people, who are in the main part really interested in education, and are all well educated with all the recognised achievements of a traditional education. They are also digital literate and confident enough to navigate through the massive online social element of MOOCs, and they probably don’t need any more validation of their educational worth.

But what about everyone else? How do you start to make sense of the badges, certificates you may or may not collect? How can you control the way that you show these to potential employers/Universities as part of any application? Will they mean anything to those not familiar with MOOCs – which is actually the vast majority of the population. I know there are some developments in California in terms of trying to get some MOOCs accredited into the formal education system – but it’s very early stages.

Again based on my own experience, I was quite strategic in terms of the #edcmooc, I wrote a reflective blog post for each week which I was then able to incorporate into my final artefact. But actually the blog posts were of much more value to me than the final submission or indeed the certificate (tho I do like the spacemen). I have seem an upward trend in my readership, and more importantly I have had lots of comments, and ping backs. I’ve been able to combine the experience with my own practice.

Again I’m very fortunate in being able to do this. In so many ways my blog is my portfolio. Which brings me a very convoluted way to my point in this post. All this MOOC-ery has really started me thinking about e-portfolios. I don’t want to use the default Coursera profile page (partly because it does show the course I have taken and “not received a certificate” for) but more importantly it doesn’t allow me to incorporate other non Coursera courses, or my newly acquired badges. I want to control how I present myself. This relates quite a lot to some of the thoughts I’ve had about using Cloudworks and my own educational data. Ultimately I think what I’ve been alluding to there is also the development of a user controlled e-portfolio.

So I’m off to think a bit more about that for the #lak13 MOOC. Then Lorna Campbell is going to start my MOOC de-programming schedule. I hope to be MOOC free by Christmas.

Prototyping my Cloudworks profile page

Week 5 in #oldsmooc has been all about prototyping. Now I’ve not quite got to the stage of having a design to prototype so I’ve gone back to some of my earlier thoughts around the potential for Cloudworks to be more useful to learners and show alternative views of community, content and activities. I really think that Cloudworks has potential as a kind of portfolio/personal working space particularly for MOOCs.

As I’ve already said, Cloudworks doesn’t have a hierarchical structure, it’s been designed to be more social and flexible so its navigation is somewhat tricky, particularly if you are using it over a longer time frame than say a one or two day workshop. It relies on you as a user to tag and favourite clouds and cloudscapes, but even then when you’re involved in something like a mooc that doesn’t really help you navigate your way around the site. However cloudworks does have an open API and as I’ve demonstrated you can relatively easily produce a mind map view of your clouds which makes it a bit easier to see your “stuff”. And Tony Hirst has shown how using the API you can start to use visualisation techniques to show network veiws of various kinds.

In a previous post I created a very rough sketch of how some of Tony’s ideas could be incorporated in to a user’s profile page.

Potential Cloudworks Profile page

Potential Cloudworks Profile page

As part of the prototyping activity I decide to think a bit more about this and use Balsamiq (one of the tools recommended to us this week) to rough out some ideas in a bit more detail.

The main ideas I had were around redesigning the profile page so it was a bit more useful. Notifications would be really useful so you could clearly see if anything had been added to any of your clouds or clouds you follow – a bit like Facebook. Also one thing that does annoy me is the order of the list of my clouds and cloudscapes – it’s alphabetical. But what I really want at the top of the list is either my most recently created or most active cloud.

In the screenshot below you can see I have an extra click and scroll to get to my most recent cloud via the clouds list. What I tend to do is a bit of circumnavigation via my oldsmooc cloudscape and hope I have add my clouds it it.

Screen shot of my cloud and cloudscape lists

Screen shot of my cloud and cloudscape lists

I think the profile page could be redesigned to make use of the space a bit more (perhaps lose the cloud stream, because I’m not sure if that is really useful or not as it stands), and have some more useful/useble views of my activity. The three main areas I thought we could start grouping are clouds, cloudscapes (and they are already included) and add a community dimension so you can start to see who you are connecting with.

My first attempt:

screen shot of my first Cloudworks mock up

screen shot of my first Cloudworks mock up

Now but on reflection – tabs not a great idea and to be honest they were in the tutorial so I that’s probably why I used them 🙂

But then I had another go and came up something slightly different. Here is a video where I explain my thinking a bit more.

cloudworks profile page prototype take 2 from Sheila MacNeill on Vimeo.

Some initial comments from fellow #oldsmooc-ers included:

and you can see more comments in my cloud for the week as well as take 1 of the video.

This all needs a bit more thought – particularly around what is actually feasible in terms of performance and creating “live” visualisations, and indeed about what would actually be most useful. And I’ve already been in conversation with Juliette Culver the original developer of Cloudworks about some of the more straight forward potential changes like the re-ordering of cloud lists. I do think that with a bit more development along these lines Cloudworks could become a very important part of a personal learning environment/portfolio.

Learning from our MOOC-stakes and sharing learning designs

It had to happen at some time, and not sure if it was karmic retribution or chaos theory, or plain old sod’s law that this week the first high profile MOOC collapse occurred with the pulling of Georgia Tech’s Fundamentals of Online EducationCoursera MOOC.

As many have already commented the route of the problem was the actual course design and implementation. From what I have seen on the twitter and blog-o-spheres, some very fundamental issues such as trying to promote group work without a clear reason as to why it was necessary coupled with technical problems with the chosen technology to facilitate the work general lack of guidance and support, all ask question of the underlying course design and quality assurance processes of (in this instance) Coursera MOOCs. But there are more fundamental questions to be asked about the actual design processes used by the staff involved.

As readers of this blog will know, I’m documenting my own “adventures in mooc-land” at the moment, and I’m in week 4 of #oldsmooc, which is all about learning design. This week is very much focused on the practicalities and planning stages of a design – be that a whole course or an individual activity. The week is led by Professor Diana Laurillard and Dr Nial Winters of the London Knowledge Lab with Dr and Steve Warburton from the University of London.

The week started with a webinar where Diana introduced the PPC (Pedagogical Patterns Collector). Designing for MOOCs were inevitably part of the discussion, and Diana raised some very pertinent points about the feasibility of MOOCS.

which led to these questions

Well it would seem that the design used by the Georgia tech course is one that shouldn’t be shared – or is that case? Elements of what they were suggested can (and have worked even in MOOCs). So can we actually turn this round and use this in a positive way?

I always get a slightly uneasy feeling when people talk about quality of learning materials, as I’m not convinced there are universal quality controls. What on the surface can look like a badly, designed artefact, can actually be used as part of a very successful (and high quality) learning experience -even if only to show people what not to do. Perhaps this is what Coursera need to do now is turn this thing around and be open so the whole community can learn from this experience. Already many, many experienced teachers have shared their views on what they would have done differently. How about using a tool like the PPC to share the original design and then let others re-design and share it? As George Siemens said so eloquently

“the gift of our participation is a valuable as the gift of an open course.”

The community can help you Coursera if you let it.

#oldsmooc week 2 – Context and personal learning spaces

Well I have survived week 1 of #oldsmooc and collected my first online badge for doing so -#awesome. My last post ended with a few musings about networks and visualisation.

I’m also now wondering if a network diagram of cloudscape (showing the interconnectedness between clouds, cloudscapes and people) would be helpful ? Both in terms of not only visualising and conceptualising networks but also in starting to make more explicit links between people, activities and networks. Maybe the mindmap view is too linear? Think I need to speak to @psychemedia and @mhawskey . . .

I’ve been really pleased that Tony Hirst has taken up my musings and has been creating some wonderful visualisations of clouds, cloudscapes and followers. So I should prefix prefix this post by saying that this has somewhat distracted me from the main course activities over the past few days. However I want to use this post to share some of my thoughts re these experiments in relation to the context of my learning journey and the potential for Cloudworks to help me (and others) contextualise their learning, activities, networks, and become a powerful personal learning space/ environment.

Cloudworks seems to be a bit like marmite – you either love or hate it. I have to admit I have a bit of a soft spot for it mainly because I have had a professional interest in its development.(I also prefer vegemite but am partial to marmite now and again). I’ve also used it before this course and have seen how it can be useful. In someways it kind of like twitter, you have to use it to see the point of using it. I’ve also fully encouraged the development of its API and its open source version Cloud Engine.

A short bit of context might be useful here too. Cloudworks was originally envisaged as a kind of “flickr for learning designs”, a social repository if you like. However as it developed and was used, it actually evolved more into an aggregation space for ideas, meetings, conferences. The social element has always been central. Of course making something social, with tagging, favouring etc, does mean that navigation isn’t traditional and is more “exploratory” for the user. This is the first time (that I know of anyway) it has actually been used as part of a “formal” course.

As part of #oldsmooc, we (the leaners) are being encouraged to use Cloudworks for sharing our learning and activities. As I’m doing a bit more on the course, I’m creating clouds, adding them to my own #oldsmooc and other cloudscapes, increasingly favouriting and following other’s clouds/cloudscapes. I’m starting to find that concept of having one place where my activity is logged and I am able to link to other spaces where I create content (such as this blog) is becoming increasingly attractive. I can see how it could really help me get a sense of my learning journey as I process through the course, and the things that are useful/of interest to me. In other words, it’s showing potential to be my personal aggregation point, and a very useful (if not key) part of my personal learning environment. But the UI as it stands is still a bit clunky. Which is where the whole visualisation thing started.

Now Tony has illustrated how it possible to visualise the connections between people, content, activities, what think would be really useful would be an incorporation of these visualisations into a newly designed profile page. Nick Frear has already done an alpha test to show these can be embedded into Cloudworks.

A move from this:

My Cloudworks profile page

My Cloudworks profile page

To something kind of like this:

Potential Cloudworks Profile page

Potential Cloudworks Profile page

Excuse the very crude graphic cut and paste but I hope you get the idea. There’s lots of space there to move things around and make it much more user friendly and useful.

Ideally when I (or any other user) logged into our profile page, our favourite spaces and people could easily been seen, and we could have various options to see and explore other network views of people/and our content and activities. Could these network views start to give learners a sense of Dave Cormier’s rhizomatic learning; and potentially a great level of control and confidence in exploring the chaotic space which any MOOC creates?

The social “stuff” and connections is all there in Cloudworks, it just needs a bit of re-jigging. If the UI could be redesigned to incorporate these ideas , then I for one would be very tempted to use cloud works for any other (c)MOOC I signed up for. I also need to think a lot more about how to articulate this more clearly and succinctly, but I’d be really interested in other views.