INTRODUCTION

Authentic Assessment

By Joyce Arnold

Introduction
Welcome to this unit within the virtual seminar of Online Pedagogies.

In this space we are using an asynchronous approach so that you can discuss and learn at your own pace, across different time-zones and geographical boundaries.

This unit will be available from 13/5/2007 until 1/6/2007.
We are using a social constructionist model so that all participants of the seminar can include their thoughts, prior knowledge and experience. A series of authentic or simulated dilemmas have been proposed with some background readings, which I have kept brief and easy to read. Your comments and discussions are encouraged to enhance connectivity and construction of meaning for all participants.
A reflective journal towards critical thinking will assist you in this unit and help with the assessment at the end.
I hope you enjoy the materials provided and discussion. I will moderate the discussions on a daily basis during the seminar so please contact me if you are having any problems.
Joyce Arnold
drjoycearnold@gmail.com

Aims and Objectives
This unit aims to provide an understanding of the central role of assessment towards learning. Using the online pedagogies, in particular, assessment is an integral part of scaffolding learners. Whether you are using individual learning programmes or collaborative, social constructivism, connectivity or socio-cultural learning pedagogies, assessment provides both behavioural intervention and cognitive learning.

There are 7 dilemmas for discussion in this unit

Dilemma 1
How can I make authentic assessment a central part of my teaching?

Dilemma 2
How can I judge when to give authentic feedback, scaffolding and assistance to individuals and collaborative groups ?

Dilemma 3
What are the best, summative or formative assessments?

Dilemma 4
How can you deal with plagiarism?

Dilemma 5
What is an online portfolio?

Dilemma 6 How can an authentic project be assessment?

Dilemma 7 What rubric to use?

Discussion

Assessment needs to be authentic or simulated for authenticity, to provide relevance towards learning. The use of dilemmas, generated by the facilitator or student, provides authentic learning in this unit.

Assessments are seen as summative or formative and this unit aims to provide a good understanding of the differences and uses of these approaches to learning.

Assessment can be provided by feedback by facilitator (teacher), peers or self- assessment. It is aimed that an understanding of the process, pitfalls, problems and outcomes of this process will be understood from this unit.

The issue of plagiarism is next addressed as it is a common dilemma with the advent of technology, the Internet and online education.

This unit aims to explore the portfolio, online portfolio and project approach to authentic learning and assessment.
There are many approaches to rubrics for assessment which are an integral part of authentic assessment, self-assessment and formative assessments. There is a large area of expertise with rubrics, much prior learning from unit participants so that the discussion should be useful to all . The self-assessment of this unit is an holistic, authentic rubric for your experience and my feedback.
The reflective journal can be part of summative and formative critical learning and is also explored in this unit. Your reflections in the asynchronous learning environment will provide authentic learning as an example of the process.Above all, this unit aims to provide a fun learning environment. As all the participants already know each other from their online learning community, I hope that this seminar topic will provide a pleasant place for learning and networking

Assessment
This unit is to be assessed using a self-assessment tool as described below.
Using your reflective journal will help you to make this assessment. This will also inform the facilitator through your feedback and I thank you for your participation.

Holistic rubric
Understanding of authentic assessment within the aims and objectives of this unit- self

5.

You have developed an excellent understanding of the aims and objectives

4.
You have a full understanding of all the aims and objectives

3.
You understand most of the concepts and practical uses

2.
You understand some of the aims and objectives.

1.
You have little understanding of this unit.

Please post your self-assessment at the end of the unit, in the space marked Assessment, thank you.

Dilemma 1

Dilemma 1
How can I make authentic assessment a central part of my teaching?

Authentic
When a person has a sense of real meaning, hands on learning, they are more likely to understand the processes, learn from mistakes and integrate their new learning with prior learning. This is a naturalistic learning process, occurs every day of our lives in a life-long learning fashion. It is easier to understand when it comes to a physical activity like playing golf, driving a car or making a cake. We cannot always provide an authentic experience so we can substitute a simulated or hyper-real proposition. Simulated flying instruction, trial projects or pilot studies are good examples of these propositions.

Assessment
A central part of the natural cognitive learning process is to Do, Assess, Modify, and Learn. If assessment is central to learning, then serial modifications and learning can be constructed. This is the basis of social constructivism or connectivity towards learning. These pedagogies have been found to help a student develop skills towards life-long learning. Assessment can be summative or formative. Feedback assessment can be provided by the facilitator, peers or be a self- assessment.
Assessment is also performed as feedback about the course, curriculum programmes and facilitators or administration towards better teaching.

If you place aims and objectives, assessment requirements, assessment rubrics at the start of a learning activity, you assist the student and the learning group to become outcome focused towards authentic learning.


Discussion within the learning triad ( Student- Teacher- Peers) provides authentic feedback. Pedagogies use assessment and feedback for learning.

Authentic assessment differs from Traditional assessment in a number of fundamental ways.


Traditional Assessment
Cognitive and theoretical
Theoretical, contrived task
Memorising or recall
Teacher centred
Indirect measurement
Norm based
One assessment
Teacher assessment

Authentic Assessment
Behavioural, task oriented
Authentic or simulated real life task
Constructivism and deep learning
Student centred
Assessment is of direct output
Criteria Based
Multiple assessments and practices
Self-assessment, reflection

Instead of the curriculum driving the assessment, the assessment drives the curriculum.

Perhaps you can add some more differences.

Is there any reason why we can’t have both traditional and authentic assessment?

Different types of authentic assessments:
Self-testing
Pre-labs
Pre-tests
Portfolio
Past examinations
Worked solutions which models expectations
Multiple-choice questions,
Projects
Dilemmas for discussion,
Research,
Reflective journals.
Performance assessment
Short investigations
Open response questions
Self -assessment
Rubrics
Online, asynchronous learning
Wikis and Blogs,
Pod-casts and web-quests,
Games

Some extra readings:
Mueller, J (2006) What is Authentic Assessment (Authentic assessment toolbox) retrieved 10/5/2007
http://jonathan.mueller.faculty.noctrl.edu/toolbox/whatisit.htm


Sandova, P & Wigle, S.E. (2006) Building a unit assessment system: creating quality evaluation of candidate performance. Education 00131172 . 126 (4)
retrieved Academic Search Primer 19/4/2007


Pearson Education Development Group (2007) Authentic Assessment Overview retrieved 10/5/2007
http://www.teachervision.fen.com/teaching-methods-and-management/educational-testing/4911.html

Dilemma 2

Dilemma 2
How can I judge when to give authentic feedback, scaffolding and assistance to individuals and collaborative groups ?

How can you make the decision when to interject, assist and help a student or group? Too soon and the student doesn’t use their anxiety created by their dilemma for learning and becomes dependent upon others to provide the answers. Too soon and you can stop peer interaction. Too late and you risk abandonment by the student who finds it all too hard to cope with.

Too much in the way of learning activities and authentic assessments and students are too busy coping with the workload to be able to interact.
Too little work to cover and people will lose interest.

Asynchronous feedback can miss the moment but can also allow time to prepare feedback and research answers.
Synchronous discussion allows for instant feedback but little time to plan it.

Some students join an asynchronous discussion almost daily and others once a week or so. It becomes an individual programme for each person and their needs. If you respond in a timely way to one student you may not be timely for the next.

Forcing people to interact with a set number of posts can create an overload and anxiety. Formal assessment of online discussions can stop people freely expressing themselves.
What about the problem of assessing ‘lurkers’?
How can you control the feedback that is given by peers?

Peer feedback in asynchronous chat has a social aspect to it and at times is informative. People are able to add new references or experiences or ideas. This depends on how well you post a discussion. Sometimes you can leave no room for feedback or be so far away from the topic that it becomes irrelevant for feedback. Most feedback however is uninformative and not really a discussion. Although it may be socially relevant and create social presence, it can create a burden of work.


The quality of the peer group is also important and it really only takes one difficult person in a group to alter the mood of the group towards work or otherwise.

Planning authentic assessment.

Some Questions to consider.
In what context is the assessment and the learning?
Who are the stakeholders ?
What are the individual issues for the evaluand ?(one being evaluated)
What values and criteria are important?
How will you do the data collection and analysis?
In what format will you report results?
Can you make it easy to do, time and cost efficient?
How accurate, valid, reliable is the assessment?
What are the cultural issues to consider?
(Williams, 2005)

What other issues can you think of?

.
Some ideas to enhance online discussions-

An introductory class activity and ice- breaker could incorporate a class activity that designs the assessment rubric. It may end up wasting a lot of time but would encourage ownership.

Give examples and information about the expected quality of discussions, netiquette, expected length, attachments, referencing, use of reflection and readings required.

Personalize the discussion with multimedia, photos and voice recordings.
Be aware that different cultures, languages and philosophies exist, especially in globalized classrooms. People have dial-up and different bandwidths and different levels of technological and computer literacy.

Social presence is important to foster for each member of the learning community.
(Zemsky & Massy 2004)

Further Reading:

Zemsky, R & Massy, WF (2004) Thwarted Innovation-what happened to e-learning and why? The Learning Alliance Report
Retrieved 12/5/2007
http://www.thelearningalliance.info/Docs/Jun2004/ThwartedInnovation.pdf

Dilemma 3

Dilemma 3
What are the best, summative or formative assessments?


Formative Assessment

This is the basis of learning as the assessments given here inform and form the structured learning pathways. Learning through mistakes, extra feedback or resources, rewards and shaping of behaviour are involved. The issue for the authentic formative assessment is that it is the cornerstone of learning, requires almost continual assessment and feedback.

Summative Assessment

Analysis of standards of achievement or the final outcome of a course with respect to a pre-determined rubric is called the summative assessment. It is much like the traditional assessment philosophy but can take the guise of an authentic assessment.
Without summative assessments there would be no standardisation, ability to reward those that achieved a grade. There are some courses that do not require summative assessments, particularly where there is no registration of attainment required.

In most cases both formative and summative assessments are required.

Which do you prefer?

Do we always need summative assessment?

Further readings:

Cavanagh, S ()Maths teachers encouraged to assess creatively Education Week 25(36) retrieved 13/5/2007 ERIC database USQ library
http://web.ebscohost.com.ezproxy.usq.edu.au/ehost/detail?vid=5&hid=3&sid=493709b9-2725-438a-88a6-841abb0eb883%40sessionmgr



Fox-Turnbull,W (2006). The Influences of Teacher Knowledge and Authentic Formative Assessment on Student Learning in Technology Education. International Journal of Technology & Design Education, 16 ( 1), pps. 53-77,
retrieved Ebscohost, USQ library 19/4/2007

Zaphiris, P &Zacharia, G (2005) Participatory Design of Interactive Computer-based learning systems in Eds C.Howard, J.Bpettcher, L.Justice, K.Schenk, PL Rogers & GA Berg Encyclopedia of distance Learning , Vol 3, USA Idea Group Reference, pp 1460-1466
Retrieved USQ Library Database 13/5/2007
http://reference.igi-online.com.ezproxy.usq.edu.au/content/details.asp?id=8174&keywords=summative%2C+formative%2C+assessment

Dilemma 4

Dilemma 4
How can you deal with plagiarism?

One dilemma is plagiarism (and cheating) and how to deal with it.
Increasing class sizes, lack of teacher/student face -to -face communication, loss of librarian/ student supervision, growing use of large banks of information has increased the risk of plagiarism. The cut and paste of Internet research has increased the problem perhaps more inadvertently for students than cheating.

Given the new collaborative learning, social constructivism and Internet approach to knowledge and information, is there really such a thing as intellectual property and individual thought?





There are online ‘paper-mills’ where for a fee you can buy a paper for an assignment, which is certainly wrong.









Isn’t it?





Where do we draw the line in the sand between literature search and plagiarism?

Heberling’s paper makes the point that for many reasons it is harder to cheat or plagiarise in online media, which Rowe’s paper refutes. Whats the truth?




Although a teacher can read posted thoughts and have an idea how good a student is, when suddenly atypical writing occurs or two people are handing in the same assignment, without a proctor, you can never be sure of who an author may really be.

Is asking others for help authentic learning or plagiarism?

What do you think?

Unfortunately cheating is common and universities are so busy raising funds that ethics and faculty desire for authentic work may be in conflict with administration. Leonard Mertz’s paper provides some interesting thoughts on this issue.
Is plagiarism the responsibility of the student, teacher or administration’s? Is it the fault of the Internet, computers or social culture that reinforces a Machiavellian approach?

Currently we are using 4 approaches to deal with cheating:
1 The ethical approach discusses values and virtue towards learning, not just obtaining a piece of paper.

2 The prevention approach. We can use a password login, changing the password at regular intervals. Several short assessments ,group submissions, small group-work, increase student/facilitator contact and open-book exams prevent cheating and plagiarism.

3 The policing approach uses currently available software for plagiarism. A set of rules and principles are established by universities and journal editors to discourage plagiarism. Punishments and legal recourse also controls cheating. Although the case of the student who sued the university for letting him plagiarise raises issues of who sues who.

4 The iMap which visually records the thought processes of literature search using diagrams, text, images, concept maps as a working record of process. It is much like a reflective journal but is more thorough in portraying thought processes.

Do you have any other ways to monitor plagiarism or cheating?

Further reading

Walden, K& Peacock, A (2006) The I-Map: a process centred response to plagiarism. Assessment and evaluation in higher education 31(2) pp201-214
Retrieved USQ library ERIC database 13/5/2007
http://web.ebscohost.com.ezproxy.usq.edu.au/ehost/pdf?vid=6&hid=3&sid=493709b9-2725-438a-88a6-841abb0eb883%40sessionmgr8


Heberling,M (2002) Maintaining Academic Integrity in Online Education Online Journal of Distance Learning Administration V(1)
Retrieved 12/5/2007
http://www.westga.edu/~distance/ojdla/spring51/heberling51.html


Anderson,C (2001) Online cheating – a new twist to an old problem Student Affairs 2 (winter) retrieved 13/5/2007
http://www.studentaffairs.com/ejournal/Winter_2001/plagiarism.htm

Rowe NC (2004) Cheating in online student assessment: beyond Plagiarism Online Journal of Distance Learning Administration VII(II) retrieved 13/5/2007
http://www.westga.edu/~distance/ojdla/summer72/rowe72.html

Leonard Mertz, D(2005) Is Plagiarism on the rise or just being Disregarded at Online Universities? Worldwide Learn retrieved 13/5/2007
http://www.worldwidelearn.com/education-articles/plagiarism.htm

Dilemma 5

Dilemma 5
What is an online portfolio?

A portfolio is a collection of best works selected for assessment. With some software, an electronic or online portfolio can be created.
The portfolio allows the student to select the works that will show that they have reached the learning objectives and allows them to redo work, update, and edit as they need. Online portfolios allow recording of voice, pictures, videos and other multimedia.
The student can feel secure that attempts that failed to achieve are not used in summative assessments. They are used for formative feedback and learning.
The works can be easily retrieved and used for display to others, like parents, school boards, classrooms.

They are authentic works that can receive rapid assessment and feedback towards learning and linked to learning outcomes. Completed portfolios can become learning tools for future students as they are easily stored and accessed.

The Electronic journal or e-journal records the student’s reflective practice. They can be used to build relationships and accessed by chosen others. They can form the basis for portfolio building, brainstorming, sharing work within a learning community and constructivism.

Is this what is really happening? What are your experiences in using e-portfolios?

Stages of development of a portfolio
Planning Stage Determine goals, audience, parameters for the portfolio assessment
Plan the content and sequencing


The Working/Process Portfolio Collect the multimedia pieces of work and assessment


The Connected/Product Portfolio. Share the portfolio for feedback from peers and teacher and use the feedback for improvement.


The Presentation/Showcase Portfolio. Formal presentation of works with respect for each person’s work and summative assessment.






Extra Reading List:

Wiseman, K (2005) Electronic portfolios in Eds C.Howard, J.Boettcher, L.Justice, K.Schenk, PL Rogers & GA Berg Encyclopedia of distance Learning , Vol 2, USA Idea Group Reference, pp 807-813
Retrieved USQ Library Database 13/5/2007

http://reference.igi-online.com.ezproxy.usq.edu.au/content/details.asp?id=8069&keywords=summative%2C+formative%2C+assessment

Barrett,H (2000)The electronic portfolio development process in Electronic Portfolios , American Association for Higher Education retrieved 10/5/2006
http://electronicportfolios.org/portfolios/aahe2000.html

Guhlin,M (2002) Electronic portfolios retrieved 10/5/2006

http://www.edsupport.cc/mguhlin/artifacts/writings/misc/99_2000/portfolios.html
/ac.pdf

Dilemma 6

Dilemma 6
How can an authentic project be assessment?

Projects have been around for a long time.
A Project is an interdisciplinary, long –term, focused, authentic model of social constructivist learning. They are fun, motivating, challenging and students play an active role.
The project is student centred and student directed. Students usually choose their own topics and parameters.
The content of the project is then meaningful for the student. It will be culturally relevant and respect student’s abilities and disabilities.
Projects are real-world about the student’s environment and thus meaningful for the student. This increases motivation and self-esteem. It also prepares people towards implementing their new learning in the workplace.
The project aims to connect academic life, work and home as well as the learning community.
A tangible product is produced that allows the student to use reflective practice, obtain expert feedback and develop higher order thinking, deeper appreciation with knowledge.
The project allows the student to increase problem solving skill, communication skills, social skills, cognitive skills, affective skills, Meta-cognitive skills and increase their technological literacy.
People can use their individual learning styles and diverse learning to best advantage.
However, projects can take more time than is allotted and not address all the curriculum needs. They require a large amount of time and work from teachers. Some teachers cannot cope with the technology and others are not used to the non-directive nature of project work. Some students may become too independent unless rules are established at the start. Often there is a lack of resources, administrative support, parental support.

What projects have you done?

Share some of your teaching experiences with projects.

Some more reading:

Railsback, J. (2002) Project based instruction: creating excitement for learning US Department of education, ERIC retrieved 12/5/2007
http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000b/80/27/e5/ac.pdf