Background
The evaluation of learning has a strong influence on studying and learning. How students believe they will be evaluated steers what and how students study – not what the instructors tell them is important or what the study guide says. This is why we call our evaluation practice a ‘hidden curriculum.’ The understanding of students and instructors will not clash, nor will there be a hidden curriculum, if the instruction, objectives, and evaluation practices and criteria are in line with each other, as well as clear to both parties.
There are two rival objectives for the evaluation: a developmental one and an evaluating one. The developmental evaluation is geared towards promoting learning. It tries to find out how well students know the material they have studied, and to bring to light any shortcomings and gaps in the students' knowledge. The instructors gain insight about how to improve and change their teaching, in order to be able to promote their students' learning better. It is easier for students to redirect their studies according to the feedback than if they had no feedback.
The evaluating assessment concentrates on evaluating the final result of learning, the performance of a course. The evaluation attempts to assess the level of students' knowledge and skills in relation to the target level.
(The above is based on the university’s guide Yliopisto- ja korkeakouluopettajan käsikirja, chapter 11, WSOY 2003).
The evaluation of exams at the Department of Computer Science has usually followed the practice of evaluating assessment.
Objectives
Regardless of how evaluation of learning is carried out, it should focus on the main themes of the learning-objective matrices, since they have been set as the primary conceptual entities that should be learned during a course. This is a fact that should be made clear when communicating with students, when teaching, and when giving instruction. If the instructors are committed to this, then the students will also commit to learning the knowledge and skills involved in the main themes.
The lectures, exercises, projects, learning material, etc., must support the attaining of the learning objectives for the knowledge and skills of the main themes.
The practice of evaluating assessment can be considered well justified when it comes to separate exams, since they are organised specifically for students to prove the level of their knowledge and skills in the subject matter of the exam.
When evaluating the knowledge and skills in course exams and, in general, and course involving face-to-face teaching, some features of developmental evaluation should be included. Evaluation and immediate feedback should be incorporated into the course structure. This will promote learning, as receiving feedback on your own learning helps you steer your learning in the desired direction.
Recommendations for evaluation of learning during courses
1. The first assignment is given to the students immediately at the beginning of the course. The assignment is compulsory.
The deadline is set at the Monday of the second week of the course, at the latest, and each student receives feedback on their skills within two days of the deadline. The assignment can be about e.g. a topic from the first lecture, the pre-requisites for the course, or one of the main themes of the course or a popular text on one of the main themes from a textbook or other publication. One possibility is to ask for essay-type answers that require the students to assess themselves, ergo ‘what I learned,’ ‘what I did not understand,’ ‘what I knew from before and which parts of it I needed to review and which I know well.’ Self-assessment can be expected even if the assignment is technical and not an essay.
2. The objective is for students to receive feedback weekly during a course. Some examples of suitable evaluation and feedback methods:
a. The instructor evaluates and gives feedback on an assignment handed in by a study circle or single student. Each assignment handed in by a study circle or student includes a self-assessment, ‘what were the knowledge and skills I learned from this assignment?’
b. Study circles and students give each other peer feedback in writing. The instructor is informed about the answers and feedback.
c. Study circles and students give each other peer feedback orally. Each study circle or student gives a prepared presentation of the assignment at the exercise session, and another study circle or student acts as opponent.
3. Course exams
a. KThe problems should focus on the main themes of the course and resemble the assignments during the course.
b. Students are allowed ‘cheats’ during the exam, e.g. one sheet of paper (A4) where they have written down the things they want to remember.
c. Students must pass in each of the main themes. The person in charge of the course sets the evaluation and passing criteria for each main theme in the exam problems. One way of doing this is to set the first part of the problem according to the column 'approaches the learning objective' in the learning-objective matrix, and the second part of the problem according to the 'reaches the learning objective’ column.
d. The evaluation of the exam should be completed within two weeks, preferably even sooner. There are three categories for passing or failing a course: (i) pass, (ii) pass if the student complements their performance, otherwise fail, and (iii) fail. Students who have passed receive a grade.
i. Students who pass in all the main themes have passed the course.
ii. Students who fail one of the main themes will be informed quickly and given the chance to complement their performance. The complement may consist of an essay, an oral interview with or without additional assignments, etc. The complement has a deadline that must be met. Students who have complemented their performance successfully will pass the course. If not, they fail the course. Students are eligible for taking courses that have this course as prerequisite whether they have passed it or not.
iii. Students who fail in two or more main themes, fail the whole course.
4. Grading
All assignments are graded and students receive points for them that will affect the final grade. It is important to award points from the first assignment; e.g. the same number of points as one exercise session yields. One suitable way of dividing the points between the course exam and course assignments could be to half of the grade-affecting points come from assignments and half from the exam.
5. Resourcing evaluation and feedback
a. Resources for giving feedback can be freed e.g. by giving fewer lectures. You could consider whether it would be possible to organise the course so that it is based on one weekly lecture instead of two, for example. Then the learning focus would move towards the completion of assignments.
b. LThe larger amount of feedback and the extra time could be included in the coefficients for the exercise sessions.
c. The utilisation of peer feedback promotes learning, both when it comes to the substance itself, as well as giving and receiving feedback.
d.The department has had good experiences with using automatic tools to support learning. These tools include e.g. SQL trainer and TRAKLA. The feedback from automatic tools seems to have a positive influence on learning, since the knowledge and skills exercised with them appear to lead to good results in the exams.
e.. You can incorporate self-assessment as part of the course, since it can give students agreeable feedback on their own learning, though it is not easy to ensure the correctness of the learning through self-assessment. The point is that students already start to consider their own learning process and skills level during the course. You can refer to the learning objectives during lectures or exercise sessions, for example, and not just at the beginning of a course, to make students assess their own skills.