Thursday, May 10, 2007

CIPP Evaluation Model

Well, here I was thinking about how all I ever do was the formal/summative evaluation in my teaching, focusing on the assessment of student learning, when it all hit me (right between the eyes it seemed!) that I do complete much of the evaluation outlined in the readings.

The CIPP model, as I have said recently, now not only seems to be the most relevent, but actually is the most accurate description of what I do! Up until now, I was thinking that it was the model that most represented what I should be doing.

This unit of evaluation seems to have taken the most brain energy to get through, despite having the least number of readings. The limited number has maybe encouraged me to think deeper rather than just take in as much as I can.

In this process, the evaluation of my learning in this course is improving! Very exciting!

Signing off, ready to hand in assignment Discussion 3!

Tuesday, May 8, 2007

Evaluation of Flexible Programs - Module 4

Having now read through the entire module 4 and through some of the checklists, it is now clear to me the whole process of evaluation and the full extent of the different processes and the procedures that it can inform.

Whilst reading through Eseryel as shown in the previous post, Shufflebeam's CIPP (Conext, Input, Process, Product) model came out as the most appropriate to my context in my mind. It seemed the most appopriate to my situation in catering for online/flexible delivery and the face to face classroom.

But as read through the notes of module 4, the depth of the evaluation becomes more apparent. The 4 areas of evaluation are:
  1. Context
  2. Input
  3. Process and
  4. Product

However, as I now know, these are not just the different types of evaluation that can take place, but these indicate 4 completely different areas, and line up well (it seems to me) to the ADDIE method of instructional design as each area focuses and infroms the different areas of ID using the ADDIE method.

It certainly seems exhaustive, so will keep reading and thinking about it's implementation in my context.

Monday, May 7, 2007

Approaches to Evaluation of Training

In this article, (found at http://www.ifets.info/journals/5_2/eseryel.html) it outline show evaluation is a complex task, and often not undertaken due to the complexity or lack of experience/skills.
The 4 purposes:
Evaluation of student learning
evaluation of instructional materials
transfer of training
return on investment
Were interesting as it seems that only the first 2 were relevent for the situation that I have chosen.
When looking at the systems approaches, CIPP (1987) is the most useful one in my situation:
Context: obtaining information about the situation to decide on educational needs and to establish program objectives
Input: identifying educational strategies most likely to achieve the desired result
Process: assessing the implementation of the educational program
Product: gathering information regarding the results of the educational intervention to interpret its worth and merit
However, as he suggests, they do not address the collaborative process of evaluation.
Eseryel also suggests that systematic and planned evaluation was generally not found in practice, nor was there a distinction between formative evaluation and summative evaluation. The most common type of evaluation is of the student performance, in the form of assessment (which is a heavy focus in our school and educational setting) and not enough on the reviewing the design of the instruction based on the results of the assessment. Much of the focus is on the learner, not the course design.
In my experience and in discussing this issue with a number of staff, it seems that some staff are aware of inadequacies of the instructional design, but are also equally aware of the weakness of the students, and only use assessment to fromally "prove" that their judgements are correct.

It has also recently arisen the plan to modify the assessment to suit the final outcome, as Eseryel suggests - the bias for internal evaluators (and in this situation - the assessers) the bias may have a very positive effect on course uptake - even at the school level.

One staff member has said to another: "Mark this task easily so that we don't turn off the students from choosing this subject!"

Similarly, evaluation tools are limited, thereaction sheets are adding to the failure of evluation in training scenarios.

My thoughts about the parts of this article worth mentioning.

Friday, May 4, 2007

Models of Evaluation

I have started Module 4 about Evaluation, and I am a little confused...the 3 models suggested are not clear to me.

Part of the problem I see here, is that I am planning a unit that includes online components, but is not totally online. So there are some distinctions that I need to make, and maybe the simplest is to focus the work of my project/unit of work on the components that will be solely online.

The naturalistic evaluation seems like it would be more difficult to method to utilise in the online environment, whereas it is one of the simplest when working face to face with students.

I will need to think hard about these models and my context that I am applying this course to.

Where am I at now???

Well I have not read for a little while as I have been stewing over the whole instructional design concept and thinking through all of the articles that I have read so far. I keep coming back to this blog and am appreciating the work that I put in early. (The concept of the blog works well for me, I am keen to continue it!)

I am still struggling with the difference between the design and the development of the course. The assignments have been pushing design, and not Development, but the comments from Shirley (facilitator) suggest that I am blurring the lines here. I need to refocus on design.

My thoughts are that there are 2 key areas for this course:
1. Choosing an instructional design model
2. Choosing a model for sequencing.

I am not confident this is the case, but for my task this seems to suit the situation. My application of this is:
1. Follow the ADDIE model of instructional design (Intulogy)
2. Utilise the Simplifying Conditions Method of sequencing (Reigeluth)

I particularly like in the ADDIE method, the flowchart outline and the detail provided in the clear, but numerous steps. It forces me to think about all of the concepts, and address the relevent ones, with the flowchart keeping the big picture in focus.

The SCM of sequencing, gives a title and structure for a method of sequencing that I have been using regularly in class for certain topics - that has provedn very effective.

My weakness is the assessment and evaluation. I am now looking forward to Module 4 and evaluation.

Tuesday, April 24, 2007

The Elaboration Theory: Guidance for Scope and Sequence Decisions" Charles M. Reigeluth

This article has enabled a formalisation of my initial model of instructional design...my model did outline a learning centred model which had little or no formal basis, except through experience in the classroom and success in a variety of situations. Not all of the courses I am involved with are structured in this manner, however, as I progress through the paper by Reigeluth, I find a stronger and stronger link to my experience as "... the elaboration theory was developed to provide ... a holistic approach to sequencing"

Types of Sequencing Strategies - Relationships.
I found it useful to have some direction to the topic of sequence and the models offered, summarised below:
Historical: chronological relationship - E.g. Timelines in history
Procedural: order of performance - E.g. Software development cycle
Hierarchical: the various skills and subskills that comprise a task - E.g. Authoring and Multimedia topics
Simplifying conditions: based upon the degree of complexity of different versions of a complex task - E.g. IPT Intro to Information Systems topic and IST - Digital Media (Video editing) unit

I think the spiral sequencing links well to the Simplifying conditions method and this relates well to my initial model of ID.

Comments from Siemens "Learning Development Cycle: Bridging learning Design and Modern Knowledge needs

Here are some quotes and my thoughts on the Siemens Article:

Quote 1:
"Learning design is primarily about creating guideposts, not describing how to walk on a particular path. The best that a well designed course, workshop, or work-integrated learning
resource can offer is the climate in which a learner can choose to learn."

I agree, you cannot force a learner to learn. However, you can create a positive learning environment (positive in the sense that the students will want to learn through intrinsic motivation to learn the content) and you can create a negative learning environement where stduents will learn because they are required to. The latter is a questionable about whether students learn, or just form an ability to reproduce or know or "learn enough to pass the assessment".

Quote 2:
"Designing courses requires set steps and guidelines for instructors and learners to follow. Learning design, in contrast, is concerned with more than simply creating courses. Instead, the intent is to create the constructs within which learning will occur - networks and ecology."

Again, this comment highlights the differences in learning strategies, through identifying the differences between setting courses and designing instruction.

Quote 3:
"Bridging prior learning with academic standards requires...that learners verify stated learning through a variety of sources and means."

Again, linking this back to my initial definition of Instructional design, this fits as I try to verify students learning through multiple real life scenarios and problems.

Quote 4:
" Learner-centred design focuses on giving the learner the ability to decide what he/she feels is important and relevant."

But how can I do this? How do I cater for the student who is not motivated to learn, or not motivated to learn in this particular course?

Quotes 5:
"Research (neuroscience) is beginning to indicate that the primary learning component of our
brains is pattern recognition, not information processing. ...Replacing the causal model of learning (need highlighted, instructional intervention planned, measurement enacted) with “network phenomenon”:" and

"...distributed representation has a profound implication for pedagogy, as it suggests that learning (and teaching, such as it is) is not a process of communication, but rather, a process of immersion.”

This begs the question of assessment, how does one tie this all in? Immersion is fine, but I struggle with assessment of the immersed topic...have I come at it from the wrong (or different) perspective? Is problem based learning a solution to this? In which case, again, my initial definition is still intact!

The following quote again from this article suggests that this is the case:
"...learning can simply be defined as actuated or actionable knowledge. This definition has two components - knowledge: understanding of an implicit or explicit nature, and actuation: doing
something appropriate (defined as contextually aware) with knowledge."

Quote 6:
Not so much a quote, but a response to Siemens' "Learning Domains" of accretion, transmission, aquisition and emergence...

If I have the concept right, one method of implementing this is to set a project or task or set the outcomes, the students can decide how they are going to demonstrate achievement of this outcome and the teacher provides resources for the students to use. Assessment can be through rubrics. Or is this the domain of accretion only? Should we have a broad spectrum of types of instruction to cover all 4 domains?

I love the idea of the learning network/environment/ecology as an ideal, but this is not always possible...the basics must be taught here...(I see a strong link here to Reigeluth's "Elbaoration Theory")

How do I create a learning ecology for the Robotics unit I am designing? Must think more...

Quote 7:
Conclusion:

"Taking a panoramic view of learning, and accounting for unique facets and domains, equips a designer with numerous approaches and methods. Instead of only transmitting learning, educators begin to create structures and networks that will foster a lifetime of learning and learning skills....The monochromatic world of course design is replaced with a vibrant environment where learning occurs in an integrated ecosystem. Learning is a continuous stream, rather than a dammed up reservoir."

Reigeluth (1993) ID definition:

In the Siemens article, he quotes Reigeluth's definition of ID as: " A discipline that is concerned with understanding and improving one aspect of education: The process of instruction". And as Siemens goes on to suggest, this is too narrow a field for my liking, despite my appreciation for much of what Reigeluth suggests in other docs.
That said, there can be a major effect on the learning process of the accurate and effective design of the instruction.

Thursday, March 29, 2007

Principles of Online Design

This article from Florida Coast University was very helpful in outlining the process of analysis. Although focused on Online media, the concepts are almost identically linked to a blended environment. I found the principles outlined in this model to be clear and helpful in the design of the unit that I would ike to design.

Principles from the website are listed below:

Instructional Design
1.1 Instructional & Audience Analysis
Principles:
1.1.1 Formal or informal instructional and audience analyses should be conducted prior to the onset of a course design.
1.1.2 Instructional analysis should determine the suitability of a course for online delivery with present technology capability.
Principle:
1.1.3 Instructional analysis should determine areas of knowledge and skills involved in achieving instructional goals.
Principle:
1.1.4 Audience analysis should determine the learner's personal characteristics, intellectual skills, subject knowledge level, and the purpose of taking the course.
Principle:
1.1.5 Audience analysis should also include the learner's technology skills and previous experiences with online courses.
1.2 Goals & Objectives
Principle:
1.2.1 Course learning goals and objectives should be stated in a manner that is clear and measurable.
Principle:
1.2.2 Course learning goals and objectives should be publicly available and explicitly communicated to the learner.
1.3 Instructional Activities
Principle:
1.3.1 Specific instructional activities should be directed toward providing learners with the necessary skills, knowledge, and experience required to meet the goals and objectives of the course.
Principle:
1.3.2 Content should be sequenced and structured in a manner that enables learners to achieve the stated goals.
Principle:
1.3.3 Instructional and learning activities should encourage frequent and meaningful interactions among learners and between learners and instructors.
Principle:
1.3.4 While selecting appropriate instructional materials, the instructor should be aware of the online instruction copyright issues and carefully observe all applicable laws.
1.4. Evaluation
Principle:
1.4.1 Methods and procedures for formative and summative course evaluation should be carefully planned in the course design process.
Principle:
1.4.2 Methods and procedures for evaluating student learning must be well articulated and directly linked to the stated learner objective
Principle:
1.4.3 The content of course evaluation should closely link to the course objectives for the purpose of course improvement.
1.5 Teaching Strategies
Principle:
1.5.1 Teaching strategies should reflect personal teaching philosophy. They should be congruent with that philosophy and capitalize on the strengths of the instructor. Effective strategies assist learners in achieving learning goals and objectives.

Big Dog ISD Page - I like it!

<> That has to be one of the best titles for a web site - BIG DOG!

From Clarke's above titled page, some comments and forming part of my analysis of my chosen project - which is the Robotics Unit for Yr 10 Students.

Analysis: The building block of a training program. The:
Who - must be training
What - must be trained
When - will training occur
Where - the training will occur

Must take into account the wants and needs of the clients (Students and school/NSW Board of Studies). As Director of Curriculum and teacher, I know what the requirements are in terms of School and BOS, but what about the students???? I need to get some feedback from them. Will plan a forum in class to review what we did last year and what they want out of the next unit.

I am one of the SMEs….getting projects started, keeping projects ruinning, but starting from the right spot is not always my strength…hence this course will give me the skills to do this properly. The first unit on Robotics was exactly this…we have the gear, get into it, keep the excitement happening…but lost some focus on the learning path – and came out the end not knowing if the unit was successful or not – why? Because I didn’t know what would have made it successful! I am ready to change this!

Monday, March 26, 2007

Project Proposal #1

Have been thinking more about what to do for the project...
I teach a Robotics unit to Year 9 and 10 students. Last year I received the ordered equipment (Lego Mindstorms) and had little time to prepare for it...poor excuse really...

But it is an area that I would like to push, not only in the computing subject area, but as an area of extension and challenge and problem solving...then maybe even into getting the school onto the competition map!

So what I need is a clear introductory unit, or a follow up unit (which is where my Yr 10 class is at as they got the poor intro unit last year!) which allows a greater depth to be reached...

Still initial thoughts, will solidify soon and start the analysis...

Chris :-)
From the conclusion of Module 1:

David Merrill makes an excellent point when he states:

"Design is a very general type of activity that takes in many fields. Often mistakes are attributed to people when the design of the artifact involved is the real cause of failure. We often make design decisions that cause students to fail. This failure is often attributed to student effort, ability, or motivation when in fact it may be the instructional materials that are poorly designed."

How often have I heard this, that the course failed from a lack of motivation, the students jst were not into it...how much of this was because of poor instructional design?? Maybe even weak ID...I think a bigger factor than my previous thoughts...

Chris :-)

Where did start at??

In the discussion: "Performing an analysis of FET5601 - theory into practice... " initiated by course facilitator Reushle, Shirley under the heading:

USER/LEARNER/LEARNING CONTEXT ANALYSIS - Who are/were the learners?

She wrote:

Student evaluations also revealed that two main groups of learners were emerging:
  • Those at the postgraduate (construction – dialogue) level who are looking for ways to assimilate instructional design principles that underpin their approach to instructional design, a chance to debate, consider alternative approaches
  • Those at the conceptualization level who are looking for templates, recipes, definitive examples of how to “do” instructional design (McKendree et al. 1997; USQ Student Evaluations data 2000).
  • Those who are between both stages.

Where did I start is my question? Initially floored by the topic, I thought I started at the conceptualisation level (or even before that!) . However, now I am not so sure, I have been at the first stage for a while, looking for ways to assimilate ID principles, but the formal side was missing...so I suppose that puts me in the middle, or out to the side...

Chris :-)

Saturday, March 24, 2007

Objectivism and Constructivism

In this table outlined in the Module 1 notes, I read through them struggling to see which one was better or worse in each situation. I grappled with which was the stronger for a range of situations and came to the conclusion that I am both an objectivist and a constructivist...and then I read the following paragraph!

"As you move through this course, you will find that we have taken an “eclectic” approach to our design, attempting to address a number of different approaches and perspectives. Although it may appear that there are many tensions between the various schools of thought, if you do adopt a more eclectic approach, you will find you spend a lot less time trying to determine which approach is the “best” and more time considering which is the most appropriate mix for your particular context."

And this is what I was doing...I try and relate the articles to me situation and I found myself a little disjointed in my thinking...but I like the eclectic approach...that is more my style...however, I am starting to think more about trying out certain units of work following a very constructivist approach....this might be the start of my project...I am thinking of a unit I have planned little for and am not happy with the result - mainly because I didn;t know what I wanted the students to learn - this topic is robotics....we have only just received the equipment necessary for this topic late last year...and I implemented the topic in a very ad-hoc manner. I would like to formalise this more....will keep thinking.

Chris :-???

IBSTPI, Instructional design competencies...a comment

In the article, IBSTPI, Instructional design competencies, it suggests that: "Reflect upon the elements of a situation before finalizing design solutions and strategies. " is essential in the Design and Development compentency...

It would seem to me to be more appropriate for this to be placed in the "Implementation and management" section.

Otherwise, the comments about essential and Advanced were helpful.

Does this list appear in chronological order? It is unclear as to the sequence of this list.

Chris :-?

First Principles of Instruction

Maybe I am a lazy learner, but I like the idea of what Merril has completed in this article. Drawing together all of the common elements. However, one needs to be careful to not simplify things too much...

Key points for me:
"...an attempt to identify prescriptive principles that are common to the various theories....
  1. Learning is promoted when learners are engaged in solving real-world problems
  2. Learning is promoted when existing knowledge is activated as a foundation for new knowledge.
  3. Learning is promoted when new knowledge is demonstrated to the learner.
  4. Learning is promoted when new knowledge is applied by the learner.
  5. Learning is promoted when new knowledge is integrated into the learner’s world.

...It is concluded that, although they use a wide variety of terms, these theories and models do include fundamentally similar principles."

Chris :-)

ISD Framework


Just contemplating this model...one of my weaknesses is formal evaluation techniques...I like to do it, I often evaluate with my classes...but rarely with peers and rarely formally. I have a concern that this will detract from my teaching if I spend too much time/effort on evaluation...however, must do if you want to improve...these are just my initial thoughts about this one.
Chris :-)

Amateur to Professional

Whilst completing my under grad studies I was regularly confronted by a number of people asking why? When I could have walked into a successful business and landscaping career? I suppose I took the attitude that I wanted to be a teacher – with plenty of encouraging remarks from some to do so. This attitude then came with me through Uni where I was told by some teachers that the degree I was doing had little to do with teaching, but it would be my ticket to teaching – so keep at it. I would say I had this attitude to studying the models of teaching…and after prompting from family (in-laws) and through leaders in my school, I have started to study again (hence this course).

Why this comment? Because I link this to the comment in Module 1 below:

“Most teachers use some sort of model when they develop a course, whether they realise it or not. Why bother looking at instructional design models and learning theories then, if you are an experienced teacher? To be thorough, whether you are a teacher or instructional designer assisting teachers, it is very useful to become familiar with models that have been developed and trialled by educational experts. As a result of trialling and research, many of these models have become theories of learning and instruction.”

So here I see my reason for longing to have a basis for my understanding. I want to move from what I see as an amateur teacher to a professional teacher who is able to take into account the knowledge of experts and apply it to my situation. This is not saying that I am not a good teacher – I believe I am – but it has been through trial and error, a desire to learn from those who teach well (and I have had plenty of opportunity to learn what not to do from a variety of poor teachers!). Now I gaining an understanding of my profession! I am working towards being a professional teacher…I am excited!

Chris :-))

Friday, March 23, 2007

Comments on the Mergers article

As suggested in the Module 1 notes, the article by Brenda Mergel was great in clarifying not just the 3 “isms” in education (Behaviourism, Cognitivism and Constructivism) but it also clarified the basis for a number of policies that we have in my school where I work (Secondary School in Sydney). Some examples include:
Behaviourism:
· Skinner’s Operant conditioning Mechanisms is very clearly the basis for our discipline policy overall.
· A substantial part of our discipline policy outlines a highly structured positive reinforcement system which is based on the Behavioural Shaping, and uses the reinforcement schedules very appropriately.
· Our sports award program also uses the reinforcement schedules to shape the bahviours we want to encourage in the school.
· There are aspects of my classroom discipline that I resort to should other models fail. Some of these are based on parts of the behaviourism model.
Cognitivism
· A highlight here was the realisation that I base a few some of my instructional design on cognitivism. I try to make allow the theory aspects of courses come out of the experience I put the students through, and try to stop the question: “When are we ever going to use this?” which is typical of students that I have taught over time.
Consturctivism
· Again, another highlight here was how I use the this model to develop the higher order thinking that I seek from my students. However, the problem I see is that at times I try to introduce the higher order thinking, or constructuvist approach, where they do not have the basis for the making the step to generalised problem solving.

Overall, an excellent article, that has cemented in my opinion a need for at least a basic understanding of the different learning theories. I have been under pressure have students think at a higher level, but have fought for a return to the basics, this article has given me a basis for justifying those decisions.

Siemens - Instructional Design in Elearning

Have been out of action for a week with a Year 8 camp and now getting back into it again...

Looking at Module 1 again to ensure I have the right perspective and notivced that I didn't mention the Siemens article....this was most helpful in the Module 1 Actvities as it outlined and clarified a whole range of loose ends that I have had since the course started.

Particular thoughts:
  • Instructional Design is the systematic process of translating general principles of learning and instruction into plans for instructional materials and learning
  • similar to lesson planning, but more elaborate and more detailed
  • Instructional design provides a framework for the creative process of design, and ensures the learners' needs are met
  • Instructional Design is the art and science of creating an instructional environment and materials that will bring the learner from the state of not being able to accomplish certain tasks to the state of being able to accomplish those tasks. Instructional Design is based on theoretical and practical research in the areas of cognition, educational psychology, and problem solving

In reading through the models, I started instantly applying the different models to some of the situations that I need to teach in....eg

  • ADDIE - for developing the teaching skills of the whole school staff over a long period
  • Algo-Heuristic -for teaching programming
  • Minimalism - as suggested is good for computer training which I do with staff and students.
  • Epathic Instructional Design - this model seems relevant for a particular course I teach in information processes.

Overall:

  • ID is a quality process. It seeks to ensure that critical concepts are explored through content presentation and learning activities.
  • ID is to serve the learning needs and success of students through effective presentation of content and fostering of interaction
Benefits:
Out of all the benefits listed later in the article, I find that the most relevant for me is the way that ID can give structure and reason to what I do. I crave structure in my teaching, learning and life. Yet I have been wading through teaching with ideas, dreams and ad-hoc methods of instructional design...I am looking forward to developing my skills in ID and more importantly, gaining a base for the plans that I have. In the words of the late Big Kev - "I'm excited!"

Sunday, March 18, 2007

The ADDIE System of ID

Went through the website recently from Intulogy (http://www.intulogy.com/addie/) that works through how they view the ADDIE system of ID. They have a very neat summary of their system (http://www.intulogy.com/addie/ADDIE-Flowchart-by-Intulogy.pdf) - which certainly helps to understand it all.

This one for me is incredibly detailed and gives great strcture when dealing with large groups of people, or a large number of stakeholders - which Intulogy seems like they deal with all the time. However, many of the concepts are useful for indivs as well.

I reflect on my experience, and I have had little formal training in instructional design, despite being a teacher for 15 or so years! I thknk now - "What have I been doing? How have I been able to scrape through with the results I have been getting? Is it all luck? I have a strong work ethic, and at times I have the resolve of: If at first I don't succeed, then work harder! Get it right next time!"

I know it is better to "Work smarter, not harder", but then, I often try the harder before realising I need to be smarter.

I have had little or no formal training, except through the role models that I have had time to work with. I love learning from other people and lately (last 5 years or so) I am enjoying refining my questioning techniques to encourage and also learn from other people in areas that I have links to. I suppose in that sense, I constantly am thinking about how to improve, but now is the time to start getting the smarter working harder!

This ADDIE method then, sounds great, but for larger projects - I am working on a larger project at the moment - which deals with teaching ID of all things!!! Maybe I need to consider applying the ADDIE method to this situation....but it sounds like too much hard work...and little time to do this, but then, how will I get it right? Or even more importnantly, how will I know IF I get it right? Time to think hard about ADDIE....

Instructional Design - Chris Woldhuis


Thoughts and the development of my thoughts on Instructional Design (ID):
The mind map (sort of) on the left is my initial picture of the mothodology I have for creating my instruction. This being before I start looking further at what ID is really about.