Oct 19 : On the use of Moodle quiz “adaptive mode” on summative assessment tasks

On the use of Moodle quiz “adaptive mode” on summative assessment tasks:

– the idealistic side of me responds immediately: it’s never too late to learn!   Adaptive mode will help students grasp the conceptual associations even as a parting kiss, a last chance!

– the realistic side of me notes that summative assessment is usually limited in time, and this will affect any decision to use adaptive mode.  While an experienced teacher can estimate quite accurately how long it will take for a group of students to do 100 MC questions with single-answer / no looking at the result responses, it is much more difficult to estimate how long it will take students to do 100 MC questions with adaptive mode / keep answering until you find the right response.   Especially when 100 MC questions are paired with an essay or two in a 1.5 or 2 hour summative assessment exam, I would say that adaptive mode will penalize the lower-performing students, who will take much longer to explore all the wrong answers, and have much less time remaining to work on the essays!

Oct 9 : On the use of the essay assignment or problem type to elicit higher-order thinking: Susan vs Robert

On the use of the essay to elicit higher-order thinking: Susan vs Robert

Perhaps one way of looking at the two views of the essay problem type would be to think of two approaches to learning: Susan creates an original essay answer that is built on her deep understanding, while Robert merely lists or repeats key phrases he copied down in his notes, based on his surface approach to learning.

Both students were given the same essay problem to solve, but one used deep understanding to produce an answer, while the other only used surface understanding.

Resource: http://exchange.ac.uk/learning-and-teaching-theory-guide/deep-and-surface-approaches-learning.html

Sep 28 : Which comes first, upgrading a course or writing new Learning Outcomes?

Which comes first, upgrading a course or writing new Learning Outcomes?

I mean that when a teacher wants to upgrade a course, it’s better to start by writing better LOs:

– more active verbs

– raise the expectations for the students : can they reach Relational level?

– can the LOs integrate new technology available?

– can we expect greater depth of understanding, or more collaboration?

Once the LOs have been redeveloped, THEN the teacher can implement a better course!

Just like the pony goes before the cart, the LO goes before the course.

Sep 24 : On why Biggs says that the SOLO Extended Abstract level is “outside of the learning cycle”

On why Biggs says that the SOLO Extended Abstract level is “outside of the learning cycle”:

I agree that in sanda (散打; sparring) you learn to relate the moves you were taught in taolu (套路; forms), and you can reach a deeper, more abstract understanding of the art of Wushu (武术).   You might watch a video of the fight and analyze what you did, and relate your moves to what you were taught.

When a student has the engagement+effort+talent to reach the Extended Abstract level, perhaps they may create new combinations that are original, not something they were taught.    They may even create an entirely new “style” of fighting, such as Changquan (長拳 or Long Fist), or Nanquan (南拳 or Southern Fist).

How can teachers require students to go beyond what they have been taught?

As Biggs said: extended abstract thinking is easily recognized, but difficult to specify.

This is why Extended Abstract is considered to be “outside of the learning cycle”.

Sep 24 : On the issue of participation vs criterion-referenced assessment:

On the issue of participation vs criterion-referenced assessment:

I require participation in the discussion forums because I want everyone to “dance” (for example).

If you don’t try it yourself, it’s very hard to learn to dance.

Participation isn’t criterion-referenced: it’s just black or white = you try, or don’t try.

Once I get everyone dancing, I can give feedback on what makes for good dancing: rhythm, expression, etc.

We also watch recordings of experts dancing, and I try to demonstrate some good dancing myself!

Group-based discussions can generate lots of peer feedback on individual dancing, far more than I can myself.

At the end of the course, the summative assessment should be criterion-referenced, as opposed to norm-referencing.

That is when the teacher evaluates the work based on a set of criteria, and at what level each student has achieved on each criterion.   SOLO levels are useful for applying to each criterion.

Criterion for summative assessment of a dance might be:

  • rhythm
  • expression
  • social interaction
  • effort
  • symbology

At the end of THIS course, I will evaluate each student’s performance on the summative assessments: a Learning Portfolio, and their eLearning Project.   Each will be assessed on a range of criteria.

Participation helps me get everyone dancing, and at the end of the course I evaluate each student on how well they can dance.   Some try harder than others.    Some have more talent in dance than others.   Some are more engaged with the class than others.

Part of my job is to try and engage as many students as possible, so more students can become good dancers.

Sep 24 : On the problem with unknown terms in syllabus Learning Outcomes

On the problem with unknown terms in syllabus Learning Outcomes

A course syllabus is often not very enlightening as to what the learning experience will be like in the course.   Often the “word of mouth” or gossip about a course is much more insightful to help a student find out the truth.

For example, in the syllabus of the 6311 eLearning course, the Course Intended Learning Outcomes (CILOs) refer to an “eLearning environment”, but what does this mean?   Not until the students take the course do they find out this could be an LMS like Moodle, a web conference such as Adobe Connect, or an immersive virtual environment such as Second Life!

One way to relieve some of the confusion is also supply Unit-level Aims and ILOs, so at least the context is reduced to just one unit, instead of the entire course!

But even in a unit, you will see an ILO such as:

  • list typical VLE functions and relate them to PBL

What is a VLE?   What is PBL?    At least the Key Terms Glossary will help, and the meaning of these terms becomes clear during the unit Teaching & Learning Activities (TLAs)!    (I hope)

Sep 20 – On the differences between F2F learning and online learning

On the differences between F2F learning and online learning:

One big difference between the online environment and F2F is that everything we discuss is archived and can be read later by everyone in the class, but especially your groupmates.

In F2F, most of what happens flies into our ears, and only perhaps 1% is recorded via accessible declarative memories or written down.

Do you remember copying down everything the lecturer wrote on the chalkboard?    I do.

It’s a pretty slow way to learn, and where is the problem solving, the student-centered inquiry-based learning?

Sep 20 – On Biggs’ idea of “backwash from assessment”:

On Biggs’ idea of “backwash from assessment”:

Typically students are not very aware of LOs, but they tend to be very focused on Assessment Tasks (ATs, which determine if they will pass the course).

Check out this article about “backwash from assessment”:
http://link.springer.com/article/10.1007%2Fs11251-005-3002-8#page-1

Students DO change their behaviors to help them achieve the ATs, so if the ATs are aligned with the LOs, then students will automatically guide themselves toward achievement of the LOs (which is what the teacher hopes)!

Sep 20 – An analogy to illustrate the SOLO levels of understanding

An analogy to illustrate the SOLO levels of understanding

Imagine a course where the teacher wants to train the students how to use construction tools, like a saw, hammer & nails, a drill, and screwdriver & screws.

The context is to build a small bridge, and the assessment objective criteria is that it must be able to support a weight of 50 kg, and perhaps some subjective assessment of “joint build quality”.

Each student can design their bridge in any way they want.

Unit 1: the teacher shows them various bridge designs, and after this unit perhaps they can IDENTIFY selected bridge designs.

Unit 2: they review and discuss various ways to join wood together and they practice making structural joints using a technique the instructor demonstrates.   After this unit the students should be able to LIST joint types and DESCRIBE how wood can be joined together, COMBINE techiques to design a joint, and they are assessed on their ability to PERFORM SERIAL SKILLS (or follow a procedure) to create joints using their tools and the wood supplied.

Unit 3: the students CREATE their first bridge designs, ANALYZE each other’s designs, COMPARE AND CONTRAST them with their groupmates in CRITICAL feedback, and are assessed on their ability to RELATE the use of various joints to the overall design of each bridge.

At this point most of the students can only create bridge designs which are very similar to what they saw in the slideshow or textbook.   A FEW students may already be capable of original creative work, and they design bridges using their own unique designs.   These students are starting to GENERALIZE THEIR LEARNING TO A NEW DOMAIN, so they have reached the Extended Abstract level.   They have GONE BEYOND the domain of bridge designs they were shown, and are applying their understanding to create new designs they have never seen before.

Unit 4: the teacher asks the students to REFLECT on their prior learning, and HYPOTHESIZE a new unique bridge design based on the concepts, materials, and principles supplied.   The students are assessed on their ability to CREATE actual wooden bridges by ANALYZING their prior designs and APPLYING THE SERIAL SKILLS to build their new bridge designs.

The students are assessed on whether their bridge can hold up 50 kg, and the teacher evaluates the quality of their joint construction.

Typically almost all of the students can pass the course at Multi-structural and Relational levels, and some of the students can now create original designs.   A few might even apply their understanding to create designs which are even outside of the domain of bridge design, such as designing a SHOPPING MALL which doubles as a bridge, or a BOAT which can motor along the river and then transform itself into a bridge!

Should Relational level be assigned a “B” grade or an “A”?    (is this an introductory or advanced bridge design class?)

Note that the top students can create designs which makes summative assessment difficult: how do you reward innovation and still require the basics?   What if it’s a great shopping mall and only a mediocre bridge?

Can the teacher expect ALL of the students to create original designs, beyond what they have been shown or taught?    (Isn’t this the requirement for PhD level work: “an original contribution to the field”?)

This is why typically the Extended Abstract level is considered to be “outside of the learning cycle”, as this level of performance demonstrates understanding BEYOND what the teacher has taught.