Design and Analysis of Algorithms (5 cu) Period I, Fall 2014 Summary and comments on course feedback November 10, 2014 (MK), revised December 12, 2014 (MK) Contents O Comments added in the revision I Basic information II Student feedback III Reflections by the lecturer IV Suggested changes V Response to verbal feedback O Comments added in the revision: Over 20 students were registered for the renewal exam. Apparently, only some of them attended the actual exam session, of which 16 returned at least one signed paper sheet. Eight (8) got 24 or more points, out of the maximum 60 (including exercise points), and so passed the exam; the rest 8 failed. The below paragraphs contain a revision of the lecturer's reflections. By Dec 12, no new course feedback was received via the anonymous course feedback system. However, the lecturer (MK) has received additional criticism from and via the powers at the CS department regarding the written reflections on the course feedback below (Sections III-V). The main message was that in some places the tone and the content are not appropriate, even if the factual matters are correct: the reflections should be more "inclusive" and less argumentative concerning "who to blame". Therefore, the lecturer hereby apologizes the parts of the text that have caused dissatisfaction. It should also be noted that the reflections only present the lecturer's personal opinions, viewpoints, and best knowledge, not the department's or university's opinions or policy. To be more concrete and to remove possible misconceptions, the lecturer would like to rephrase the unfortunate lines from the last year's reflections (presumably the main cause of dissatifaction, see the end of Section V) into a more constructive form that applies also to the present year's case, e.g.: "The failed answers [in the renewal exam] suggest that the DAA course was built on somewhat unrealistic expectations about the students' prior knowledge and skills in basic mathematical reasoning. To enable participation in the course for students with varying prior knowledge and skills, it has to be thought out by the teachers, how to ensure that the path from the BSc level courses to the DAA course becomes smooth enough." In other words, the lecturer had thought that certain things have been learned, for sure, in earlier courses, whereas some students may have thought that the skills they have achieved so far should, for sure, suffice for advanced level courses. This dilemma summarizes quite well the main concern also in the Fall 2014 version of the DAA course. If you are still unhappy with the phrasings above and find them offending, please let the lecturer know how you feel, in a way or another. I Basic information The course belongs to advanced studies and is mandatory to students in the specialization area of algorithms and machine learning. The course takes a traditional form: seven weeks of lectures, 4 hours per week; weekly exercise sessions; a 2.5-hour exam (held on Oct 23). The course follows very closely selected chapters of the course book (CLRS, 3rd ed.). Lectures are given mostly using a black board, essentially no lecture slides are made available. A large number of students (51) registered for the course, 42 attended at least some of the course activities. The course exam was attended by 35 students, of which 17 passed the course, with grade distribution (5**** 4* 3**** 2******* 1*). The renewal exam will take place on December 2 - at the moment, 20 students have registered for the renewal exam. From the exam and the exercises one could earn 0-54 and 0-6 points, respectively. For those that attended the exam, the total number of points were distributed as follows: 0- 8: ** 9-13: *** 14-18: ********* 19-23: **** 24-28: * 29-33: ******* 34-38: **** 39-43: * 44-48: *** 49-53: 54-60: * II Student feedback The course organizers thank for the feedback! The feedback has been carefully analyzed. By Nov 10, we received a total of 10 entries via the anonymous course feedback system. The variance among the entries is generally high for most of the five specific feedback questions (5=agree, 1=disagree): Clear objectives 5*, 4*, 3*, 2***, 1*** Supportive learning material 5, 4**, 3*, 2***, 1*** Supportive activities 5, 4, 3**, 2****, 1*** Reasonable assessment 5, 4**, 3*, 2*, 1*** Laborous course 5*****, 4****, 3, 2, 1* Grade as a whole 5, 4*, 3***, 2**, 1**** While the verbal comments also show high variability and opposite opinions, a couple of issues occur repeatedly. First, many thought the exercises were really hard. Second, some thought the exercise sessions were not helpful. Third, many thought the lectures were not useful, since one has to study the course book anyway. These thoughts and other verbal comments are commented in the sections below. III Reflections by the lecturer The feedback seems to stem from the students' heterogeneous background and somewhat unjustified expectations. It is also questionable how representative the feedback is of the whole student population that took the course. Namely, the sample size of 10 is very small compared to the number participants in the course, and it is much smaller than the sample size of 18 in the year 2013 feedback. In 2013, the course was given the best grade 5 in 3 feedback entries and grade 4 in 6 entries, the average being 3.4. The 2014 implementation of the course was essentially the same as in 2013, just a bit more fluent; the number of credit points has been increased from 4cr to 5cr, without adding any new material. Nevertheless, the main concern is naturally the high percentage of students that attended but did not pass the course. The question is: what could be changed? There are three perspectives: (1) Change the learning objectives, i.e., lower them significantly. (2) Change the learning activities significantly. (3) Ensure the students have the needed prior knowledge and skills. The first option is not realistic - it would be a different course then. The second option has some potential, say, to increase the acceptance rate from 50% to 60%. However, tailoring the teaching and learning activities to the lower tail of the distribution may well lead to decreased performance in the higher tail: e.g., the number of grades 3 and 4 may decrease. If we want a real change, the hope is in the third option. But, of course, the present course cannot and is not supposed to re-teach prior courses. A solution could be a well-thought "entrance exam". IV Suggested changes 1. At the very beginning of the course, test that the serious attendees of the course have sufficient prior knowledge and skills, e.g., roughly equivalent to grade 3 or higher from the BSc level Data Structures course. Others are encouraged to quit right away. 2. At the "lectures", consider allocating some time for interactive problem solving in various forms: in groups, individually, and by the lecturer. For example, in the first lecture hour, give the main ideas and definitions and perhaps a toy example; start the second lecture hour by a related problem solving task. This assumes the student have read the material from the book before the lectures! 3. Consider making some exercises easier to approach by breaking the solution into two phases (a and b). 4. Give exercise points only for solutions that are returned by email well before the session, and also give individual feedback on some of the returned solutions. V Response to verbal feedback Why are exercises so hard? Because experts have found the difficulty suitable for a MSc level course on design and analysis of algorithms. Indeed, the exercises are from the course book, a standard text book used in equivalent courses all around the world. There are two good explanations why a student may feel the exercises are hard: (1) The book is too a difficult read, due to lack of needed prior knowledge and skills. (2) The student does not invest a sufficient personal effort to study the material. Why are exercise sessions not that helpful? The purpose of the session is not to practice problem solving but to (a) "reveal" solutions and the thinking behind the solution, (b) give students an opportunity to ask questions, (c) control that exercise points are given to those that deserve them. There are several good explanations why a session may end up being not that helpful. First, students may mark a problem solved, even if they have not really prepared a solution or attempt they could describe to the audience. Second, if a student has not really tried to solve a problem, a quick revealing of a solution will be hard to follow. Third, students may not make any questions. Admitted, it is the course organizer's responsibility to *not forbid* students (1) preparing good solutions, (2) working hard on the problems before the session, (3) making questions. Why are lectures not that useful? Because the course follows closely a good text book, lectures can only have a limited added value. It is true that the lectures, as they were, are just one possibility and not necessarily the best use of the time. So far, the idea has been to go through and emphasize the main concepts, with a few more technical examples, and also let students ask questions. It is possible that many students would benefit more from more active participation, "doing," in lectures, not just by themselves on the exercises. One feedback line was: "this course was pretty much a book exam." This is true in that all the material needed is in the book, which is good, as it offers a way to pass the course by just taking a separate exam. However, actually the lecture course is a "book exam with benefits." Namely, in the lecture course you can find friends that are taking the course at the same time, which enables, e.g., making groups and discussing and solving exercise problems together. Lectures and exercise sessions also give an opportunity to ask questions and get questions answered. You do not have these benefits when you just study independently for a separate exam. Another feedback line: "The time usage, counted by hours per credit method is ridiculous." Well, how should we take a feedback like that? Namely, 27 hours per 1 credit unit is the official measure! It is based on the estimate of 1600 hours of work for 55 credits per year. What is wrong with that? Of course, if somebody tries to earn 55 credits in only, say, 4*7=28 weeks, then the load per week is high, about 57 hours. But would not it be ridiculous for a full-time student to study only 28 weeks per year?! In the same entry: "The demand that students spend 18 hours reading the book is just silly,..." Let's inspect the truthfulness of that comment carefully: The comment claims that there was a demand of reading the book 18 hours. However, the truth is that there were no such demand, nor even recommendation. Let's repeat the lines on the course web page: "For instance, 4 hours for the lectures, 2 for the exercise session, 8 for solving the exercise problems, and the remaining 5 hours for reading the book." There we see it: 5 hours, not 18 hours. And, that was just an example of possble time usage. Now, we let the reader of this draw conclusions about the how some of the feedback should be taken. In one feedback entry, the author accused the lecturer for blaming students lazy and blaming them of their poor success in the exam in the year 2013 response to the course feedback. Everyone can go and view the 2013 response on the course's web page. Presumably, the author refers to the following lines that concern the renewal exam: "The failed answers suggest that there are severe deficiencies in quite basic mathematical reasoning. The students seem not ready for taking the advanced level course. The instructor wonders how they could have passed earlier courses in math, in algorithms and data structures, and in programming." The question is, whether or not those lines represent the truth. Is it true that the students who fail do not have sufficient prior knowledge and skills? Well, the exams contain components that also measure prior K&S. There is evidence. The key insight is perhaps that, even if a student can earn a BSc degree by passing the needed courses with grade 1, it does not mean that the student is ready for MSc studies. Similarly, not everyone who has a MSc degree is ready (or will be accepted) to PhD studies.