TAAASS501A Lead and coordinate assessment systems and services

Develop and extend assessment expertise and practices

Figure 2 Develop assessment system and services

Figure 2 Develop assessment system and services

Enhance professional development of assessors

The identification of and reflection on personal development needs is as important for assessors as it is for any professional. Assessors need to be able to complete:

  • Needs analysis and priorities;
  • Communication within communities of practices and networks;
  • Personal reflection and build positive self-efficacy; and
  • Research on issues, current practices, balancing practices and trends;

Needs analysis and priorities

The training and assessor competency standards provide a ready-made tool for the identification of training needs for assessors. At this, the Diploma level of learning and performance, standards can be used as a means to identify an individual's job and career-related training needs. A systematic means to do this can make use of the following steps:

  • Establish the Competency Standards Relevant to the level of employment/occupational context;
  • Map competencies to specific learning requirements;
  • Confirm existing competence held by the individual;
  • Assess competency gaps that may impact job performance;
  • Assess competency gaps that may limit personal development;
  • Confirm individual and job-related training needs;
  • Isolate non-competency issues that impinge upon performance against job or work requirements.

The above process 'folds back' on itself. This is because it is often assumed that an individual who holds a competency can perform in all contexts, or that the competency is current or held to the required level of proficiency. Equally, there may be factors that prevent competency being demonstrated that are outside the control of the individual. No amount of development can necessarily promote competence in these circumstances. Figure 3 illustrates the training needs analysis and overall competency profiling process.

Figure 3 Using standards to identify an assessor's training needs.

Figure 3 Using standards to identify an assessor's training needs.
© The Institute for Working Futures. Used with kind permission

Closing a learning or performance gap between individual competency and position requirements can also have wider impact. For the organisation it may mean acquiring the competencies to compete; for the individual it may mean accessing improved career pathways.

Priorities for development can be set by deciding where the closing of a gap will impact:

  • Job performance;
  • Compliance requirements;
  • Personal career development needs;
  • Management/company preferences; and/or
  • Personal needs.

Forming and joining communities of practices and networks

Building relationships outside the place of employment or across groups or communities involves network communication. Network communication most often relies on a sense of shared purpose that extends beyond an organisation's immediate structure or culture. This may be driven by professional or personal needs.

The term community is being used to define not just individuals with a common geographic location, but also a group of individuals that share a purpose built on social norms, values and expectations. Such communities may exist within the organisation (teams) or across organisations and locations.

Sociology identifies that socially constructed realities are one way that individuals can find meaning within community structures. The values communities provide stimulates an interface between individual and collective interpretations of the world. Many types of communities exist in the vocational setting. The most common include:

  • Communities of practice;
  • Learning communities;
  • Communities of learners;
  • Knowledge communities;
  • Professional associations, work groups, etc.;
  • Educational (university, self-development);
  • Social (religious, schools);
  • Recreational (sporting clubs, hobby groups, gymnasiums).

While not denying the importance of the different forms of networks, in the context of this section communities of practice are very relevant. They are formed by individuals who most often share a commonality of purpose because of their work, education or professional experiences.

Communities of practice are emerging as communities or informal networks that link individuals anywhere and at any level of an organisation or society based on shared interests.

Communities of practice are, in effect, professional knowledge networks. These arise out of interactions and understandings created when people share information, formally or informally. It is one of the most important forms of knowledge within the business world, and one of the least understood. Networking, where professionals or communities of practice gather together or share a discipline, is most often viewed as the basis for networked knowledge. This is a limited view. The value of networked knowledge resides in its potential to utilise information from multiple sources to generate new and relevant perspectives. Networks are crucial in a world where it is impossible to allocate time and space to acquiring all the knowledge that may be important to an organisation.

A strong set of networks should include interactions with formal and informal groups outside the workplace. Networked knowledge also resides in the social fabric of the location, including characteristics such as political and social stability and the workforce's sense of well-being. Knowledge arising from these settings is often referred to as 'social capital' (see the Knowledge Management topic area).

Activity 1

Identify your networks

  • List the networks you are involved with outside work.
  • How many of these networks are professional and how many informal or social? Which networks contribute the most to your assessment activities? How?
  • Do any of the networks you are involved develop competencies that you require when completing assessment activities?
  • Consider why some companies are now paying for employees to become involved in networks as diverse as professional associations to gyms or literature classes?

Reading 1

DETYA (2001c), 'Networking for assessors', Guide 9 Training Package Assessment Guides, Australian National Training Authority (ANTA): Brisbane. Available at http://resourcegenerator.gov.au/loadpage.asp?page=TPAG.htm .

Activity 2

Reading 1 not only overviews how to participate in an assessor network, but also provides a guide to how to create networks for assessor professional development and peer support. Access this reading and complete the following.

Personal reflection and positive self-efficacy

An individual's perception and competencies will affect their ability to learn and perform. Understanding the link between personal reflection and self-efficacy is therefore an important consideration for assessors, both for their own and their students' development.

Self-efficacy refers to an individual's disposition or perception about their abilities to mobilise cognitive, motivational and behavioural facilities to successfully execute a specific task within a given context. Research has confirmed that the determinants of self-efficacy-physiological state, psychological arousal, emotional arousal, concept of own capability and task dimensions and risk-are relevant to learners engaging in a learning and assessment exercise.

Figure 4 Determinants of self-efficacy

Figure 4 Determinants of self-efficacy

Self-efficacy is an important consideration in how individuals approach and complete personal development but it should not be the sole basis for constructing learning or assessment processes. Personal development can directly leverage reflection and build self-efficacy. Self-efficacy is critical to establishing the motivation to successfully engage in personal development.

As depicted in Figure 4, psychologists have used self-efficacy to reinforce the process of reflection prior to a person entering into any new activity or making choices. People generally want to know and weigh their options, determine 'what is in it for them' (personal benefits and rewards) and determine the effort required to gain the rewards for their optimal choice (Bandura, 1986, 2001; Brown, Collins & Duguid, 1989; Schunk, 1981).

It is known that individuals who perceive themselves as highly efficacious have a dispositional perspective described previously as positive affect (PA). Their level of motivation permits them to better embrace different ways to learn and indeed, if deployed well, engage in more successful development activities. Individuals who perceive themselves as having low self-efficacy are likely to have increased negative affect (NA). This increases their anxiety, promotes disjointed efforts and often causes premature failure or non-completion of learning and development activities.

One way to build cognitive capacity and promote reflection is to encourage wider networks. Interactions in communities of practice and networks can stimulate improved access to information and professional choices. In turn, an individual's experiences, networks and beliefs can be used and drawn upon to positively guide future actions and interactions (Bandura, 2001:3).

Determining current assessment practices

Assessment practices are evolving constantly. Keeping abreast of such developments is often difficult. As trends in technology, work practices and learning patterns evolve, so do the type, range and mix of assessment activities.

Reading 2

Hyde, P, Clayton, B & Booth, R (5 April 2004), Exploring assessment in flexible delivery of vocational education and training programs , NCVER: Adelaide . Sourced January 2005, at http://www.ncver.edu.au/publications/1268.html

Balancing assessment approaches

The relationship between on- and off-the-job assessment is often ambiguous. In particular the distinction is blurred as a national competency standard, in the Australian context at least, has tended to concentrate on performance, not behaviours. This performance is constituted by knowledge and skills that may be assessed in a formative manner on or off the job. However, summative assessment must confirm that the standard of performance attained is that required at a level of employment. This suggests the workplace is the most logical choice for collecting evidence and making a final assessment decision on an individual's attainment of a competency.

Dangers

Problems with competency-based assessment can arise where assessment focusses on task performance only within the immediate context where it is being assessed. Another common problem identified with current approaches has included the use of the assessment process by the assessor to criticise, rather than encourage, trainees. Common barriers to a trainee's involvement in assessment being positive can occur if:

  • The purpose of assessment is confusing;
  • Anxiety is not addressed during the introduction of the training session or assessment activity;
  • The trainer/assessor uses language or a pace that either alienates or causes boredom;
  • The information given is irrelevant or unrelated to workplace requirements;
  • Body language used does not reinforce positive and participative involvement; and
  • The trainee/assessor uses culturally inappropriate language, activities or instructions.

The aim is to avoid subjective assessment . This is where personal opinion of the assessor may dominate the determination of whether the assessee has attained the competency to the required standard. Under competency-based approaches the preference is for objective assessment where assessment is based on agreed outcomes and, preferably, across a defined range of possible assessment methods.

Objective assessment requires you to make an assessment decision based on the evidence. Three key questions need to be asked prior to making a final assessment decision:

  • On balance can the assessee meet the stated performance requirements to a standard where they can do this task/job duty in the workplace?
  • Has all evidence been met or have there been areas of failure to accord with the requirements?
  • Overall is there a requirement to revisit or cover some evidence requirements at a later stage before awarding competence?

Objective assessment is, in reality, encouraged by the establishment of a standard that applies to a performance outcome. This description is consistently described with training seeking to deliver what is required in a real performance context. Competency-based assessment is the determination as to whether an individual has achieved the capacity to perform the standard, as described.

Activity 3

Reviewing assessment tools and methods

Complete the following chart and required responses alone or in a small group. Identify why the suggested assessment approaches are/are not appropriate.

ASSESSMENT METHOD SELECTION

ACTIVITY TO BE ASSESSED

SUGGESTED ASSESSMENT

OK/
NOT OK

WHY?

  • Resolve over-the-counter customer complaints

Off-the-job skills role-play and test by an expert

 

 

  • Drive a forklift

Off-the-job assessment by a single assessor against knowledge and skills criteria

 

 

  • Explain the use of a hammer drill

On-the-job discussion with workplace supervisor who is expert in the area

 

 

  • Maintain coalface safety equipment

Off-the-job assessment by facilitator (coach) using agreed criteria (checklist)

 

 

  • Review emergency evacuation procedures

Off-the-job or on-the-job group session of at least 5 participants using self-assessment linked to agreed criteria

 

 

  • Explain housekeeping for pharmacy shop floor

On-the-job group session or individual session with supervisor against criteria and follow-up knowledge test

 

 

  • Operate a wheat silo

On-the-job individual training session using observation and a skills test

 

 

Applying the principles of assessment in the workplace

It is important to revisit the basis upon which all assessment systems and services should be developed. The key principles of assessment suggest assessment should be valid, reliable, fair and flexible. These principles alongside others that are added by industries and companies (sufficiency, cost-effectiveness, authenticity, currency, equitable, etc.) all denote design values that evidence gathering, judgment and reporting mechanisms must support.

This is discussed in more detail in learning resources supporting TAATAS501A Undertake organisational training needs analysis.

Table 1 Key principles of assessment

Principle

Enabler

Barrier

Reliability

(How well is it able to provide a basis for judging competence despite who, where, or when it is used?)

  • Objective basis that promotes clear basis for assessment to judge standards of performance & attributes displayed by individual (assessee)
  • Clear expectations expressed through wording of performance criteria
  • Consideration of variations or how changes will impact performance
  • Use of formative assessment to build into summative outcomes
  • Process of assessment is simple
  • Subjective basis built into assessment that allows assessor scope for reinterpretation
  • Unclear relationship between assessment task and employment/ jobs
  • Narrow or prescriptive tools
  • Over-reliance on summative assessment
  • Process of assessment is complex

Validity/Sufficiency

(Does assessor have enough evidence to make a judgment?)

  • Interpretation of evidence is guided by tools
  • All aspects supporting the progress towards full performance are assessed
  • Assessment tools tied to outcomes not venue or location for evidence collection
  • On completion of assessment the individual is 'job ready'
  • Competency levels of the assessor are stated
  • Assessment of competency is not related to readiness for performance in a job
  • Tools that necessitate assessment occur only off-the-job
  • No distinction between on- and off-the-job assessment requirements
  • Failure to integrate and link all the assessment tools
  • No discrimination of special competencies an assessor may require

Flexibility

(Does the assessment task promote options and choices?)

  • Assessment is not seen as an intervention in either work or learning
  • Assessment presents options
  • Assessment promotes access
  • Assessment can occur regardless of age, race, religion, gender, etc.
  • Removes barriers that may occur if only a limited number of assessors can use the tool
  • The assessment tasks limit options for completion on or off the job
  • Assessment is hard to establish or access
  • Assessment limits access
  • Few assessors are qualified to use the tool
  • Process of assessment is complex

Authentic

(Is the evidence from the assessee?)

  • Collection of evidence can be traced directly to the assessee
  • Evidence can be gathered from multiple sources and not traced back to the assessee.

Currency

(Is the evidence current in terms of competency requirements?)

  • Assessment permits tasks to reflect current work practices
  • Current technology and equipment are integrated
  • Assessment is tied to the latest endorsed competency standards
  • Assessment reflects past paradigms or practices
  • Assessment is linked to a philosophy or approach not universally accepted (e.g. quality management, best practices, rational decision making)

Ensure equity in assessment processes and practices

. . . the policies and approaches that ensure vocational education and training is responsive to the diverse needs of all clients. Through the implementation of these policies and approaches, the benefits of participating in vocational education and training are available to everyone on an equitable basis including: women where underrepresented; people with disabilities; people from non-English speaking backgrounds; Indigenous Australians; and rural and remote learners (ANTA, 2001, AQTF: Standards for Registered Training Organisations ).

This quotation confirms the importance that any assessment process, activity or practice conform with the legal, organisational, moral and ethical requirements to provide equitable access.

Reasonable adjustment, sometimes called reasonable accommodation or allowable adjustment, is designed to ensure that all people are treated equally in the assessment process-this means that, wherever possible, "reasonable", adjustments are made to the assessment process to meet the individual needs of students (DETYA, 2001a, Kit to support assessor training ).

Under the Australian Registered Training Organisation (RTO) regime laid out in the Australian Qualifications Framework (AQTF) it is expected that 'reasonable adjustment' will be undertaken to accommodate equity needs. This suggests that assessment systems, activities and instruments must be modified, customised or revised to maximise equity and access for those individuals or groups with special needs.

The AQTF established compliance standards all RTOs must meet to gain registration. Standard 6 states:

. . . the RTO applies access and equity principles and provides timely and appropriate information, advice and support services which assist clients to identify and achieve their desired outcomes (Standard 6, ANTA, 2001, AQTF: Standards for Registered Training Organisations , available from the publications at http://www.anta.gov.au ).

Standard 8 states:

. . . the RTO's assessments meet the requirements of the endorsed components of Training Packages and the outcomes specified in accredited courses within the scope of its registration (Standard 8).

Table 2 is sourced from the VET in Schools and provides a useful list of ways existing assessment processes may be 'reasonably adjusted' to accommodate equity and diversity concerns for specific audiences.

Table 2 Possible strategies for adjusting the assessment process

Category

Possible issue

Strategy

Indigenous

  • Cultural background
  • Speaking
  • Reading
  • Writing
  • Numeracy
  • Confidence

 

  • Allow enough time to learn about the community and how this may impact on the assessment process, for example whether the age and gender of the assessor is an issue
  • Check that assessment strategies are inclusive and culturally sensitive, for example it may be more appropriate to demonstrate knowledge rather than talk about one's knowledge
  • Identify any events and obligations that may affect the timing of the assessment, for example funerals and family responsibilities
  • Use methods that do not require a higher level of language or literacy than is required to perform the job role-Indigenous languages may be appropriate for Indigenous community workplaces
  • Encourage participants to use alternative ways of demonstrating competence, for example drawing instead of writing, using a video to record demonstration of skills
  • Use support person or other support strategies
  • Use short sentences that do not contain large amounts of information
  • Clarify information by rephrasing
  • Confirm understanding, for example ask the candidate to rephrase questions
  • Read any printed information to the candidate
  • Use graphics, pictures and colour coding instead
  • Offer to write down, or have someone else write, oral responses given by the candidate
  • Ensure that the time available to complete the assessment, while meeting enterprise requirements, takes account of the candidate's needs

Disability

  • Speaking
  • Reading
  • Writing
  • Numeracy
  • Limited study skills and/or learning strategies
  • Discuss with the candidate and supervisor whether the disability is likely to impact on the learning or the assessment process
  • Use appropriate learning strategies (including suitable formats such as large print and audio)
  • Use support person (including note takers, interpreters) and other support strategies (including technical aides) as required to assist learning
  • Ensure that the time available to complete the assessment, while meeting enterprise requirements, takes account of the candidate's needs
  • Put a higher emphasis on formative assessment and collection of evidence over a period of time
  • Access to Recognition Process and self-assessment
  • Access to empathetic employers and use support person or other support strategies (including technical aides) as required in the workplace

Girls and young women

  • Confidence and valuing of skills learnt
  • Literacy and numeracy (NESB)
  • Workplace cultural issues
  • Access to SWL or School Based New Apprenticeships
  • Gender-inclusive learning environment
  • Use an assessor of the appropriate gender
  • Use appropriate assessment strategies, for example are there issues if a project requires men and women to work together?
  • Provide a mentor or support person
  • Access to Recognition Process and self-assessment
  • Access to gender inclusive processes and materials

Language, literacy and numeracy

  • Speaking
  • Reading
  • Writing
  • Numeracy
  • Limited study skills and/or learning strategies
  • Discuss with the candidate and supervisor whether language, literacy and numeracy are likely to impact on the assessment process
  • Use methods that do not require a higher level of language or literacy than is required to perform the job role
  • Read any printed information to the candidate
  • Make sure font size is not too small
  • Use graphics, pictures and colour coding instead
  • Offer to write down, or have someone else write, oral responses given by the candidate
  • Ensure that the time available to complete the assessment, while meeting enterprise requirements, takes account of the candidate's needs
  • Have a number of shorter assessments rather than trying to cram too much into one session

Non-English speaking background (NESB)

 

  • Speaking
  • Reading
  • Writing
  • Cultural background
  • Confidence

 

  • Discuss with the candidate and supervisor whether language, literacy and numeracy are likely to impact on the assessment process
  • Use methods that do not require a higher level of language or literacy than is required to perform the job role
  • Use support person
  • Use short sentences that do not contain large amounts of information
  • Clarify information by rephrasing
  • Confirm understanding, for example ask the candidate to rephrase questions
  • Read any printed information to the candidate
  • Use graphics, pictures and colour coding instead of, or to support, text
  • Offer to write down, or have someone else write, oral responses given by the candidate
  • Consult with experts to make sure that methods and materials are culturally sensitive, for example gender issues
  • Ensure that the time available to complete the assessment, while meeting enterprise requirements, takes account of the candidate's needs
  • Access to RPL and self-assessment

Adapted from ANTA, 2001, Guide- Kit to support assessor training, available at http://cms.curriculum.edu.au/the_cms/tools/new-display.asp?seq=6678

Activity 4

Cognitive skills used to guide and support quality assessment practice and address issues in assessment practice

Cognitive and learner-centred approaches to assessment lead to a consideration of learner-led learning (pull) rather than just instructor-led learning (push) approaches. Assessment can assist transform previous 'push' approaches to engage individuals in making learning meaningful to them, their applied context and the competencies sought by organisations. A self-discovery approach encourages individuals to adopt a 'sensemaking' approach to learning, problem solving and development of cognitive frameworks

Quality assessment practices can accommodate strategies that foster cognitive skills; for instance, experiential learning, simulations, project teams or games may be used. As reinforced further on in this section by Bloom's Taxonomy, cognitive skills can include:

  • Analysis skills;
  • Synthesis skills;
  • Interpretative skills;
  • Planning skills;
  • Evaluation skills;
  • Problem solving skills;
  • Critical thinking skills;
  • Knowledge transfer skills.

From a cognitive perspective alone, educators have increasingly questioned assessment based on the competencies that focus only on performance and not on critical thinking skills or underpinning knowledge. Figure 5 highlights how individuals learn and retain learning best.

Figure 5 Cognition—learning retention

Figure 5 Cognition-learning retention
(Chi, Bassock, Lewis, Reinmann & Glaser, 1989:148)

Differentiation of the types of learning and how learning outcomes are framed critically impact what and how assessment criteria are set or assessment tools are shaped. One way of separating out how educational outcomes can be assessed is to use an approach called the taxonomy of educational objectives (Bloom, 1956). This 'taxonomy' separates out three main 'domains' of learning.

Cognitive - concerned with knowledge (acquisition and application), understanding, thinking/creativity and intellectual abilities; for instance:

  • Define a term;
  • Describe a topic;
  • Read a technical drawing.

Psychomotor - or performance domain concerned with motor and skills application; for instance:

  • Type a letter;
  • Drive a car;
  • Communicate with staff;
  • Develop an x-ray film.

Affective - concerned with attitudes, motivation, self-awareness, interests; for instance:

  • Show concern for safety;
  • Demonstrate sensitivity to problems of the environment;
  • Demonstrate lifestyle preference(s);
  • Take responsibility for hygiene.

Bloom's taxonomy of educational objectives serves as a key reference point for the framing of assessment for non-competency outcomes, but also for assessing these outcomes within a competency-based approach. For instance, how well has the learning experience delivered the behavioural outcomes required for task performance? The following diagram depicts a summary of the cognitive skills. Such a breakdown of cognitive skills highlights how assessment activities or learning outcomes can be framed in a manner that isolates the type and level of complexity. Figure 6 summarises Bloom's taxonomy and is then followed by a table that details the subcategories of cognitive skills and the verbs and activities that may usefully guide the framing of assessment and learning outcomes. These give an indication of the way an assessor or manager can quickly review to see if cognitive skills are being adequately assessed in the instruments and documents being prepared within their organisation.

Figure 6 Complexity of learning and cognitive skills

Figure 6 Complexity of learning and cognitive skills

The benefit of the Bloom's taxonomy is that it enables instructional designers and teachers to implement pedagogically sound assessment methods. The taxonomy also promotes differentiation of levels of thinking and development of high-order thinking skills. Bloom's taxonomy ensures that learning activities, content and assessment tools incorporate metacognitive instructional aspects into their design and implementation. This supports the educative aspect of any learning, whether physical or electronic.

Table 3 Major categories in the cognitive domain of the Taxonomy of Education Objectives

 

Examples of general instructional objectives and behavioural terms of the cognitive domain of the Taxonomy used when framing the learning outcome or the assessment criteria

Descriptions of the Major Categories in the Cognitive Domain

Illustrative General Instructional Objectives

Illustrative Behavioural Terms for Stating Specific Learning Outcomes

Knowledge
Knowledge is defined as the remembering of previously learned material. This may involve the recall of a wide range of material from specific facts to complete theories, but all that is required is the bringing to mind of the appropriate information. Knowledge represents the lowest level of learning outcomes in the cognitive domain.

  • Knows common terms
  • Knows specific facts
  • Knows methods and procedures
  • Knows basic concepts
  • Knows principles

Defines, describes, identifies, labels, lists, matches, names, outlines, reproduces, selects, states

Comprehension
Comprehension is defined as the ability to grasp the meaning of material. This may be shown by translating material from one form to another (words to numbers), by interpreting material estimating future trends (predicting consequences or effects). These learning outcomes go one step beyond the simple remembering of material, and represent the lowest level of understanding.

  • Understands facts and principles
  • Interprets verbal material
  • Interprets charts and graphs
  • Translates verbal material to mathematical formulae
  • Estimates future consequences applied in data

Converts, defends, distinguishes, estimates, explains, extends, generates, gives examples, infers, paraphrases, predicts, rewrites, summarises

Application
Application refers to the ability to use learned material in new and concrete situations. This may include the application of such things as rules, methods, concepts, principles, laws and theories. Learning outcomes in this area require a higher level of understanding than those under comprehension.

  • Applies concepts and principles to new situations
  • Applies laws and theories to practical situations
  • Solves mathematical problems
  • Constructs charts and graphs
  • Demonstrates correct usage of a method or procedure

Changes, computes, demonstrates, manipulates, modifies, operates, predicts, prepares, produces, relates, shows, solves, uses

Analysis
Analysis refers to the ability to break down material into the component parts so that its organisational structure may be understood. This may include the identification of the parts, analysis of the relationship between parts and recognition of the organisational principles involved. Learning outcomes here represent a higher intellectual level than comprehension and application because they require an understanding of both the content and the structural form of the material.

  • Recognises unstated assumptions
  • Recognises logical fallacies in reasoning
  • Distinguishes between facts and inferences
  • Evaluates the relevance of data
  • Analyses the organisation structure of a work (art, music, writing)

Breaks down, diagrams, differentiates, distinguishes, identifies, illustrates, infers, outlines, points out, relates, selects, separates, subdivides

Synthesis
Synthesis refers to the ability to put parts together to form a new whole. This may involve the production of a unique communication (theme or speech), a plan of operations (scheme for clarifying information). Learning outcomes in this area stress creative behaviours, with major emphasis on the formulation of new patterns or structures.

  • Writes a well organised theme
  • Gives a well organised speech
  • Writes a creative short story (or poem, or music)
  • Proposes a plan for an experiment
  • Integrates learning from different areas into a plan for solving a problem
  • Formulates a new scheme for classifying objects (or events or ideas)

Categorises, continues, completes, composes, creates, devises, designs, explains, generates, modifies, organises, plans, rearranges, reconstructs, relates, reorganises, reviews, rewrites, summarises, tells, writes

evaluation
Evaluation is concerned with the ability to judge one value of material (statement, novel, poem, research report) for a given purpose. The judgments are to be based on definite criteria. These may be internal criteria (organisation) or external criteria (relevance to the purpose) and the student may determine the criteria or be given them. Learning outcomes in this area are highest in the cognitive hierarchy because they contain statements of all of the other categories, plus conscious value judgments based on clearly defined criteria.
  • Judges the local consistency of written material
  • Judges the adequacy with which conclusions are supported by data
  • Judges the value of a work (art, music, writing) by use of internal criteria

Appraises, compares, concludes, contrasts, criticises, describe, discriminates, explains, justifies, interprets, relates, summarises, supports

(Bloom, 1956)

Dangers

To some extent, knowledge and understanding underpin all learning and performance. Unfortunately the implementation of assessment in vocational and work-based programs has tended to undervalue cognitive skills. This focus has caused some problems with assessment approaches, including:

  • Inconsistency in standards of performance;
  • Difficulty developing instruments to consistently assess learning or performance outcomes to an agreed standard, irrespective of the mode of learning;
  • An assumption that performance of the skills suffices as evidence that the underpinning knowledge has also been acquired;
  • Design of assessment tools that ignore knowledge and focus on collecting evidence of physical, observed performance;
  • Design of assessment tools and methods that give little indication of reliable and valid transfer of knowledge to support performance in a range of cultural and work contexts; and
  • Criticism that assessed outcomes are too organisational specific and learners lack the ability to transfer competence to other workplaces or career pathways.

Ethical standards underpin assessment practice

Ethics shape individual and therefore organisational and societal values and standards. Concepts of ethical behaviour for an individual, a business and society may vary. An individual may hold views on standards and ethical behaviour that can evolve through both work and life contexts. In many cases standards and values that underpin work may conflict with the individual's ethical basis.

The definition of ethics is much debated. We firstly will define ethics by determining what it is not. We will particularly concentrate on the relationship between morals and ethics. We will then move on to study how damaging conflict can be avoided when identifying and reporting poor work-based standards and values.

Ethics, morals and values are often used interchangeably; however, ethical values are not necessarily the same as moral values. Therefore, from the outset we need to distinguish ethics from morals. Morals are standards of behaviour adhered to by a society concerning social mores and customs. Morals can thus change from generation to generation, from society to society, from culture to culture. What is communicated as morality today might be obsolete tomorrow. Ethics , on the other hand, concern those values beyond mores and customs that pervade human existence. Ethics concern the essence of human dignity and what is necessary to uphold it.

The following table illustrates how ethics can guide societal thinking:

Table 4 Comparative Protestant-Confucian ethical basis

Protestant Ethics

Confucian Ethics

Give rise to:

  • Hard working bring success
  • Frugality
  • Savings
  • Security
  • Value of money and personal worth
  • Enjoying rewards of hard work

 

 

Give rise to:

  • Paternalistic view on relations (husband-wife, ruler-subject, elder-younger, father-son)
  • Value of family
  • Education
  • Securing a future for the family
  • Hard working
  • Home ownership
  • Clean living

With the above chart we could also encounter similarities and profound differences with Islamic, Hindu or Buddhist ethical beliefs. Equally, all instil and generate different values and moral beliefs.

Activity 5

To illustrate how personal values influence judgments on ethical behaviour of others, complete the following question:

Do you support the following? Reflect on why or why not.

  • Ethnic cleansing;
  • Polygamy (having more than one wife/husband);
  • Clear-felling trees for paper manufacture;
  • Making animal sacrifices to please the Gods;
  • Speed cameras being used to collect revenue rather than reduce motor vehicle accidents;
  • Spending taxes to train a neighbouring country's military officers when they have unelected, unrepresentative regimes.

This section is not an in-depth study of ethics but we do need to appreciate that ethics guide collective action and that values, standards and morals that are based on ethics guide everyday actions of a workforce. The ethics of assessment often revolve around balancing individual, learning and organisational values. In some cases, unfortunately, ethical dilemmas arise that provide no clear-cut course of action. Actions that may appear ethical at the time may be interpreted later by others as unethical.

Developing accessibility and removing disadvantage in assessment systems and processes

When undertaking assessment, the assessor must ensure that the tools will not disadvantage the trainee. In particular they need to be non-discriminatory and permit the trainee to have full access and understanding of the evidence and the standard required to be assessed as competent.

Beyond the principles of assessment being fair, flexible, valid and reliable, assessees have basic rights. Table 5 from the Department of Employment, Education and Training, Assessment Practical Guide (1994:23) outlines some of the critical rights and considerations.

Table 5 Critical principles, rights and desirable practices in the assessment process

Principle

Right

Desirable Practices

Free Consent

People have the right not to be pressured in any way to be assessed

  • obtain free and informed consent;
  • advise in advance that a training program or course will have an assessment;
  • provide people with an outline of the assessments that are planned and their weighting or importance;
  • explain the purpose of assessment;
  • in the workplace situation, obtain consent forms from minors; and
  • either obtain consent forms for assessment where there is the possibility of harm or injury or ensure dialogue with trainee covers correct health and safety procedures prior to practice.

Confidentiality

People expect assessors to limit access and to maintain the security of information about themselves

  • limit access to information;
  • maintain security of identifiable information;
  • return information to candidates, where possible;
  • use numbers rather than names to further enhance anonymity.

The assessment process should be developed to ensure that all the following are unambiguous and clear to all involved in the assessment process:

  • The type and amount of evidence required;
  • Process for evidence collection and who will make the judgment as to sufficiency;
  • The assessment method and the role of the assessor;
  • Means or mode of setting an assessment task for the learning covered;
  • Arrangements for collecting, recording and evaluating evidence or data;
  • Counselling and appeal processes (as required);
  • Resources required by all parties; and
  • Documentation and reporting processes.

Activity 6

The aim is to make assessment resources ethical as well as easy to access and use. The type of assessment resources required can impact the ease of access. For instance consider the following:

 

Knowledge-based resources

Issues with access

E. g . b ooks

Availability, library access

Worksheets

 

Videos

 

Operating Procedures

 

Other (identify one that may impact your)

 

 

Skills-based resources

Issues with access

E. g . Facsimile machine

Availability, none in workplace, must do from office or training centre

Special clothing

 

Equipment (etc.)

 

Other (identify one that may impact you)

 

 

Physical environment resources

Issues with access

E. g . Guest speaker/Expert

Availability: is there one on site? If not, is there one in the company?

Training rooms

 

Safety hazards

 

Other (identify one that may impact you)

 

Activity 7


Readings
readings icon

Supporting
presentation
supporting presentation

Knowledge
quiz
knowledge quiz

Additional resources
additional resources