This is now Week 4 of the Learning Challenge (LC) and I've been a little naughty in that I have allowed the "Learner Analysis" assignment to interfere with the LC. The "Learner Analysis" assignment is due to be submitted on Sunday 26th October and I have probably spent more than is actually necessary on it.

I am back on the "I Will Knot" website to refamiliarise myself with the knots that I was practicing before the assignment got in the way. It's amazing how much you quickly forget because you haven't been practicing to the point where it "sticks" or "clicks" into place. At the moment, I am practicing once with the knot with the aid of the step-by-step video and then a further three times without the aid of the video - the trick is to try and repeat this tomorrow without the aid of the video at all!

On the subject of the "I Will Knot" website, I mentioned to a colleague of mine who teaches Chemistry at my University and has just been appointed a Learning and Teaching Co-ordinator for my Faculty that I was doing this learning challenge as part of my MSc and showed her the website I was using. She was most impressed by it - NOT because it showed you how to tie knots in easy step-by-step videos BUT how the idea behind the easy step-by-step videos could be used to show a particular Science experiment / practical to a group of students.

This visual aspect to learning is indeed a powerful motivator, and if done properly it could instruct students better than just reading it out of a textbook.

It's Week 1 of the Learning Challenge (LC) and I have set aside 30 minutes to practice the art of knotcraft. The first website I went to was "Knots: How to Tie Knots"; I was not impressed with the rather short but pedestrian instructions with the naff looking illustrations - if I was having trouble understanding how to tie easy knots, what was it going to be like with the more complex ones?

The next website, "Ropers Knot Page", has slightly longer, but equally, pedestrian instructions with rather confusing looking illustrations that don't do anything to enlighten me as to what I am suppose to be doing. I am beginning to wonder if I have chosen an appropriate learning challenge - certainly the free resources I am using are not explaining themselves very well.

My next resource, "I Will Knot", is something of a revelation to me. It uses a mixture of short, sharp videos and short, but still, pedestrian instructions. But it is the videos that work for me. When it comes to something that involves a motor skill, I am someone who likes to be shown so that I can mimic and keep practicing that skill. In that sense I could be considered as a kinesthetic learner.

This also suggests to me that if you want someone to learn something online, the materials / resources you create will be dependent upon the task / skills / knowledge / information that you wish to convey to your students.

So, do you use drawings, photographs, videos, audio recordings or text to put across something that needs to be learnt? Should this resource be passive or interactive? Can the learner work in isolation or does this require a collaborative / community dimension?

Interesting questions indeed!

Here we are again, a new term and a term module, this time it's "Understanding Learning in the Online Environment" led by the incomparable Hamish Macleod. This is my third module to date and its looking good. My only concern is that 10% of the course assessment is based upon me writing something worthwhile on the discussion board on a reasonably regular basis.

Interestingly, I have just read in Section 2 (B5) of the QAA (2004) "Code of practice for the assurance of academic quality and standards in higher education" that students should have:

"where appropriate, regular opportunities for inter-learner discussions about the programme, both to facilitate collaborative learning and to provide a basis for facilitating their participation in the quality assurance of the programme"

One of the other assessed pieces of work is the "Learning Challenge" which contributes about 20% of the overall mark. Some of the examples of a learning challenge included juggling three balls; performing a conjuring trick; origami; writing a computer program; or tying a complex knot. Given that my sense of balance and eye / hand co-ordination is shot to pieces, any notion of doing juggling or riding a unicycle was quickly dismissed as a bad idea. Having programmed in a variety of different computer languages over the past 20 years or so didn't fill me with any great sense of desire.

The combination of not being in the scouts and being a fan of "The Worst-Case Scenario Survival Handbook" led to the appealing idea of trying to attempt to perform a range of knots that could have practical applications should I find myself inexplicably castaway upon an exotic island, a bit like the cast from "Lost".

Based upon Damien DeBarra's initial idea of using social bookmarking for the course, Hamish Macleod suggested that we could give Diigo a try. Unlike Delicious (which I use a lot), Diigo allows users to create public / private groups for people to collaboratively work in - sharing resources and research material. Diigo, also, has the ability to highlight and comment on pieces of text. So I created a list of bookmarks on Diigo to support by learning challenge with knots - the list goes by the unimaginative title of "Get Knotted".

So, I've got the guides and tutorials that I need to perform the difficult knot exercises; but I was lacking that one vital piece of apparatus - the rope! I dutifully went off to C and H Fabrics where I purchased myself about a metre's length of soft cord. This wasn't without incident either; the shop assistant gaved me such a funny look over my purchasing of this piece of cord. Heaven knows what went through her mind as she was serving me.

I now have everything I need for my 10 (more like 8 to 9) week learning challenge. I shall be using the blog to record my thoughts about the actual cognitive processes involved in tying a knot and the strategies that I have employed to try and master the exercises.

References

QAA, (2004). Code of practice for the assurance of academic quality and standards in higher education. QAA [online]. Available at: http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/ [Accessed 23 September 2008]

It was going to be my intention to keep a semi-regular-ish posts on my engagement with the "Effective Course Design for e-Learning" module. As you can see - this didn't happen!

The first week introduced us to some of the current theories around styles of course design (Toohey, 1999) and how some of them might have been adopted based upon personal preferences or, even, encouraged by external political pressures and agencies.

Weeks 2 to 4 explored the different approaches, that were:

• Traditional or discipline-based approach
• Performance or systems-based approach
• Cognitive approach
• Experiential or personal relevance approach
• Socially critical approach

Using a combination of different readings and a wiki to collect and collate thoughts about the readings, looking for real-world examples of these different approaches in action and suggesting additional resources to help build up a coherent bank of knowledge - I found myself quite enamoured with the socially critical approach that attempted to look at a particular issue that needed to be debated and discussed with a view to making significant changes to how that issue was currently operating.

Whilst I felt that this module would be enormously valuable to me as a learning technologist who is advising and developing staff to use the University's learning systems like Blackboard to the best of their abilities - I also felt hampered that I didn't have enough traditional teaching experience to actually get to the nitty-gritty of some of the concepts and ideas that were presented. Something that would have quite a profound effect upon my assignment.

In weeks 5 to 7, my peers were put into groups (and named after fruit) to discuss, devise and develop a miniature "learning event" around a topic or theme that was of interest to us and using one or more of the approaches that we had been looking at for the past 4 weeks. The other members of the group would then take part in the "learning event" and feedback upon it. I wanted to do something that involved the socially critical approach and was rather inspired by the work done by Turnley (2005). I wanted my participants to look at the developments within the so-called "Web 2.0" phenomena and how that would impact upon and enhance their research practices - I called this concept "Research 2.0", being a pun upon how people have used the notion of versioning to try and attempt to describe something that was different (and in some cases better!).

I used the Holyrood Park Elgg site to deliver the event and asked my participants to write a little critique - whilst they said that they enjoyed it; it was debateable as to whether any actual "learning" occured. These experiences would then form the basis of the reflective report - the feedback from that report suggested to me that I was being overly ambitious with what I wanted to achieve, especially with my lack of teaching experience - so I had probably chosen an approach that was best adopted by someone with considerably more teaching experience than myself.

Week 8 looked at assessment and how that was partly defined by well constructed aims and learning outcomes. Weeks 9 to 10 covered course evaluation and course usability; again my peers could have chosen which topic to spent 2 weeks exploring in some depth.

Finally, in weeks 11 to 12, we spent that time working on our assignments which involved writing a course outline; a course rationale that explained our thinking and some semblance of a course that was constructed within some kind of learning environment. Despite the rather good mark for this assignment; I personally felt that I didn't spend enough time to do the course any justice - illness, project meetings across the country and a much needed holiday got in the way of that.

The big thing that I learnt from this module is that online courses don't start with the technology - it begins using pen, paper, a whole lot of thinking and several cups of coffee later as to what you want to try and achieve with the course and what you expect people to get out of it, in terms of what is learnt and what you want them to experience and how you challenge their thinking in the process. 

References

Moon, J., (2002). The module and programme development handbook. London: KoganPage 

Toohey, S., (1999). Designing Courses for Higher Education. Buckingham: Open University Press.

Turnley, M., (2005). Contextualized design: Teaching critical approaches to web authoring through redesign projects. Computers and Composition. 22(2), pp. 131-148.

Whilst almost all official mention of these tests has disappeared from the Internet, they remain in my thoughts*. The tests only reached the pilot phase of development, and I was lucky enough to be placed in one of the schools that had been ear-marked for testing. They were aimed at year 9 pupils, or those at the end of their compulsory Key Stage 3 ICT programmes.

This test was a first for many schools as it delivered the exam in the form of an on-screen assessment. The software was a mock up of a traditional GUI interface such as Windows XP or Mac OS X. Pupils received their test questions ‘via email’ (time released by the software) in the built in email client. The aim was to put to use all the skills they should have learnt during their 3 years of ICT classes.

As recommended by Bull and McKenna (2003), the students were exposed to the new software environment up to 7 weeks before the tests for around an hour per week (more if they wanted) in order to familiarise themselves with the layout of the applications and the available tools. Many (senior) teachers criticised the tests as they thought it criminal not to test students in the environment that they had learnt in – in my school this was Windows XP. They assume that will be the only environment that the students will use outside of school in the workplace, therefore question the need to ever learn a new environment. This simplistic view of ICT from senior management (and curricula level) is one of the reasons I have moved away from teaching it this year. If anything, the use of a new software environment helped us (as teachers) to identify those students who had been learning surface level routines in XP, rather than a deep understanding of what they were actually doing.

From the reading I have already conducted around this topic I can tell that this environment was an innovation in CAA. Firstly, it shied away from the traditional MCQs and Boolean questions that would be expected, in favour of contextualised tasks. This was made possible by the sophistication of the assessment system, which was also to check the file structures and contents at the end of the exam and keep a record of how students performed specific tasks. One downside was that there was no instant feedback for the teacher or student as the marks had to be independently moderated for anomalies.

Just before the final pilot went ahead in May 2007, it was announced that the tests was not be introduced compulsorily as expected for summative exams, instead they would be made available for formative assessment as and when teachers and students were ready. Probably a wise decision as there would be no real benefit for this kind of summative assessment for the pupils’ learning or the teachers’ teaching. The primary role of the summative test would be to provide accountability and fuel more league table competitiveness.

In summary – this is a good and useful assessment technology which was partially introduced with less than useful intentions, but held its own in pilot testing on a fairly wide scale. This type of software will be a useful assessment tool for early stages of ICT education.

 

* One of the remaining official documents can be found on a secondary school server here

On my quest to discover ‘what is online assessment’ I have read the first two chapters of Bull and McKenna’s seminal work on Computer Assisted Assessment, as they define it. At first glimpse this title, and the content of their book does not appear too relevant to my study as the word ‘assisted’ implies a weak influence. In fact, on further reflection, the majority of online assessment is ‘assisted’ as humans still maintain control over various factors, such as question content, assessment times and formats.

They note that some of the most common forms of online assessment are MCQs and Boolean options. Whilst I have experienced these methods both as a teacher and a student, I have also submitted assignments that I have produced on my computer through either email or a VLE for assessment by a teacher – surely this constitutes a form of online assessment? I have also participated in one of the QCA KS3 ICT pilot tests as a teacher. These tests provided new software environments for students to perform tasks based on what they had learnt in their classes. The tasks were recorded by the software using mouse click tracking and file scanning at the end, but were also submitted to human moderators for ‘authentication’. Hopefully I will find time to post more about this experience as part of this 2-week block.

They offer a concise list of reasons for using CAA:

1.    To increase frequency of assessment thereby motivating students to learn and encouraging students to practise skills.
2.    To broaden range of knowledge assessed.
3.    To increase feedback to students and teachers
4.    To extend the range of assessment methods
5.    To increase objectivity and consistency
6.    To decrease marking loads
7.    To aid administrative efficiency.

This list is fairly logical, although with every positive there is a potential negative (as with most things!). Increased frequency of assessment could lead to pupil agitation and subsequent disengagement. A broader range of assessment could mean subjects don’t get assessed as in depth as previously. Increased feedback could overload and confuse students (a bit far-fetched I suppose). An increased range of assessment tools could definitely confuse students who are used to alternative assessment methods and it could even put a different skew on results as would be expected when assessed using different methods. If questions are tailored towards more objective subjects then some topics could be overlooked. If teachers are further removed from the marking process they run the risk of misinterpreting the results. I’m struggling to think of the negative side of increased administrative efficiency – but I’m fairly sure that’s not too connected to the learning that is taking place.

Bull and McKenna make some sensible observations about how to go about using CAA. Is it actually appropriate? More on this with some other readings hopefully (e.g. Brosnan, M. (1999). Computer anxiety in students: should computer-based assessment be used at all?). A couple of other sensible points: “CAA objective tests should only be used as one of a number of assessment methods….The implementation of a learning technology should be integrated with the structure and delivery of a course.” It is also important to guard against testing IT skills rather than subject content.

In the second chapter the authors touch on a number of issues raised by the e-assessment conundrum. A big concern here is the interface between the software development industry and education. Could CAA technology influence (or even determine) pedagogic practise by including certain question formats and enabling specific feedback formats? Furthermore, the cost of various CAA softwares might determine which products are used and therefore determines the types of test formats that are available.

“CAA enables collection of detailed data on formative activities – but this should be balanced against surveillance concerns raised by Land and Bayne (2002).”

Electronic literacy: CAA is more than assessment of subject expertise, also understanding how online environments mediate and even construct knowledge. Rather than traditional linear texts, students are exposed to “visual literacy” (Kress, 1998): the logic of simultaneous presence of a number of elements and their spatial relation to each other – a core issue that is somewhat addressed by the Prensky (2001) and Monereo (2004).

In this section of the module I will look at how assessments have moved online and what the impacts of this transition have been. It is important to distinguish between online assessments of a face-to-face based course and the online assessment of an online course. I think that it is the case that there are currently more online assessments than online courses and hopefully throughout this topic we will begin to understand the reasons and motivations for this. I will also hopefully get time to briefly review a couple of the technologies that make online assessment possible.

Initial readings for this topic:

Bull, J and McKenna, C (2003) Blueprint for Computer Assisted Assessment (chapter 1-2)

John A. Ross & Maura Ross (2005) Student Assessment in Online Courses: Research and Practice, 1993–2004. Canadian Journal of University Continuing Education
Vol. 31, No. 2.

Scalise, K. et al. (2006) Assessment for e-Learning: Case studies of an emerging field. 13th International Objective Measurement Workshop.

Ridgway, J. (2004) Literature Review of E-assessment. FutureLab. Available at: http://www.futurelab.org.uk/resources/documents/lit_reviews/Assessm

Further suggestions about what to read or what assessment tools to review are always welcome.

I liked this paper's no-nonsense approach (and it was written in a style I could access easily). Several good bits of content:-

From the paper:-

 The essence of the challenge for all educators in the 21st century is to get the learners to:-

• read more widely
• see more clearly
• think more clearly
• (why am I thinking of the song "Day By Day"?)
• challenge authority on every occasion
• more importantly get learners to challenge themselves

 The aim is to promote the free-flow of information and ideas in the interest of all and to promote a thriving culture, economy and democracy.

 Information Literacy is the ability to deal with complexities of the current information environment - it must

• subsume all the skill-based literacies but not be restricted by them
• not be restricted to any one technology / technology group
• centre around understanding, meaning and context

 So much e-learning remains as e-teaching (the provision of lecture material online) - is this due to poor information literacies amonst the tutors?

 The "information literate" are those who know when they need information and are able to identify, locate, evaluate, organise and effectively use the information to address and resolve problems

 The Australian Information Literacy Standards

An information literate individual has learned how to learn and is able to:-

1. recognise a need for information
2. determine the extent of the information needed
3. access the needed information efficiently
4. evaluate the information and its sources
5. incorporate selected information into their knowledge base
6. use information effectively to accomplish a purpose
7. understand the economic, legal, social and cultural issues around the use of information
8. access and use information ethically and legally
9. classify / store / maipulate the information generated
10. recognise information literacy as a pre-requisite for lifelong learning

I sorry but I found this paper rather dull although it did contain some little gems of information I could use.

Firstly, Barrett attests that most graduates did not have a clear sense of their research aims at the start of the process - they fumbled about and were guided by colleagues, tutors and supervisors. This is so good to hear as I am usually in the same boat. The important thing here is that this is probably when most of the searching of libraries and whatnot occurs - so that searching can at bet be unfocussed and at worst be blind fishing. Without good IL skills the period of fuzziness is probably an awful lot longer than it needs to be.

The second little gem was that most students lack personal collections and substantial subject expertise. Again, I thought I was alone but so many people I have spoken to lack a personal collection or just have haphazard piles of documents in cupboards or piled on desks. From this knowledge I feel I can make best use of the web-based personal catalogues offered by del.icio.us, Connotea, Furl It, Zimbio and the like. All the tools are there - we just don't use them. I shall catalogue all my piles of paper.

Hellfire!

What a paper to start us off on! It was like pulling teeth but I got there in the end I think. A good (content) opener for the course as it provided so much food for thought.

Intertextuality has to exist otherwise we would have to write everything de novo each time - scientific advances would be limited to the lifespan of any one scientist.

Newspapers of ten translate the "official" laguage of politicians and the like into the vocabulary of the the day-to-day spoken word (or rather the newspaper's interpretation of the spoken word). Why do they have to do this? Is it that "official" language is not digestable by the masses or are we losing the ability to understand "proper" vocabulary? I fear I do not know the answer to this!

From the paper:-

 Many non-commodity institutions are being drawn more and more into the commodity model and the matrix of consumerism - they are under pressure to "package" their "commodities" and "sell" them to "consumers".

 Presuppositions (based on prior texts of the text-producers or by other texts) can be manipulative as well as sincere - they are a good way of manipulating people as they are very difficult to challenge.

 A genre is not only a particular text type but a particular process of producing, distributing and consuming that text

 A discourse is a particular way of constructing a subject matter. E.g. Medicine is an area of knowledge constructed from a technological and scientific perspective unlike that of "alternative medicine"