Beyond Value-Added Teacher Assessment

One of the problems with the value-added approach to teacher assessment, which is probably also one reason for its failure to identify teachers who are consistently effective or ineffective across time, is its black box approach to the entire process. In other words it employs a strictly statistical strategy for differentiating between teachers without attempting to explain why the students of some teachers seem to learn more than the students of other teachers.

As it happens, however, we already know why . The explanation is found in the truly seminal piece of educational research called the “Beginning Teacher Evaluation Study”. Employing intensive, repeated observations of 25 second- and 25 fifth-grade classrooms, this study found that, on average, 2 hours and 15 minutes of the second-grade school day was devoted to academic activities (which were defi ned as instruction in reading, mathematics, science, and social studies), whereas 55 minutes was devoted to nonacademic activities (such as music and art), and 44 minutes was “wasted” on things such as waiting for assignments and conducting class business.

Taking math and reading as the two primary academic subjects of interest, the researchers found that, on average, the 25 second-grade teachers allocated 2 hours and 6 minutes per day to instruction. Their students were actually engaged in learning for 1 hour and 30 minutes (or 71 % of the time). What was even more telling, however, was the fact that the top 10 % (approximately) of the teachers allocated 50 minutes more to instruction than did the bottom 10 % , and their students were actually engaged in learning these subjects for about the same amount of extra time (50 minutes).

Although this may not sound like a great deal, it means that, in these two crucial subjects, some children could receive 150 hours more instruction during a school year than other students. And, since the average amount of time actually allocated to teaching these subjects was 2 hours and 6 minutes, this means that some children received 71.4 days more instruction than others, or a total of over 14 weeks of extra schooling !

To put all of this in context, the investigators contrast two hypothetically average students, one of whom (Student A) receives a grand total of 4 minutes per day of relevant instruction and one (Student B) who receives 52 minutes. Since these students are average, they would start the year at the 50th percentile on the standardized tests, yet by midyear Student A would decline to the 39th percentile, while Student B would improve to the 66th percentile! The authors go on to justify the feasibility of their analyses as follows:

It may appear that this range from 4 to 52 minutes per day is unrealistically large. However, these times actually occurred in the classes in the study. Furthermore, it is easy to image how either 4 to 52 minutes of reading instruction per day might come about. If 50 minutes of reading instruction per day is allocated to a student (Student A) who pays attention a third of the time, and one-fourth of the students’ reading time is at a high level of success [these authors defined “a high level of success” as instruction administered at an appropriate level of difficulty], the student will experience only about 4 minutes of engaged reading at a high success level. Similarly, if 100 minutes per day is allocated to reading for a student (Student B) who pays attention 85 percent of the time, at a high level of success for almost two-thirds of that time, then she/he will experience 52 minutes of Academic Learning Time per day.

So, the moral here is that massive differences exist in both the amount of instruction that different teachers deliver, as well as in the amount of relevant instruction students receive . (We’ve already mentioned some work 50 that found that the variability in the amount of instruction received by typical students on a school wide basis can be as much as 50 % , which borders upon a criminal offense in my opinion.)

So while I haven’t seen these studies even mentioned in the value-added literature, in my opinion they constitute the only theoretical rationale of which I am aware for why we should be able to differentiate teachers who produce more learning from those who produce less of it. And by simply monitoring classroom instruction by continuously recording it on digital cameras (assuming that provisions were made for constantly identifying opportunities for improvement and then providing sufficient professional development to show teachers how to teach more intensely) we could go a very long way toward either reducing teacher differences in performance or weeding out those teachers who consistently teach less. At the very least we could combine these data with value-added procedures, which in turn might improve the latter’s present woeful ability to identify teacher differences that were consistent over time.
Read More : Beyond Value-Added Teacher Assessment

Value-Added Teacher Assessment

Most commonly associated with William B. Sanders and his colleagues (originally at the University of Tennessee and now at the SAS Institute), one such approach is predicated on the proposition that if enough data on individual students are available over time, then this information can be used to predict these students’ test score gains in the future.

It therefore follows that, if all of any given teacher’s students’ test score gains can be predicted based upon these students’ past performance, then any discrepancies from these predictions represent that teacher’s effectiveness ineffectiveness for that particular year. Called value-added teacher assessment , this approach uses sophisticated longitudinal statistical modeling procedures to generate predictions regarding students’ test score gains for a given year.

It then defi nes any observed classroom performance that turns out to be better than predicted on the end-of-year test as the value added by the teacher of said classroom. (Again, what else could it be?) This approach has resulted in some relatively promising findings, especially for mathematics, to a lesser extent for reading, but apparently not so much for other subjects. Before considering these findings in any detail, however, it is worth noting that the model attempts to simulate the situation in which:

• Students are randomly assigned to teachers (which would help to decrease the individual differences in students’ propensity to learn between teachers’ classes that occur when students are assigned on the basis of their likelihood to gain more or less highly on standardized tests — such as occurs when parents request that their children be assigned to a given teacher based upon that teacher’s reputation or when a principal assigns students that he or she believes will prosper more with one teacher than another or when students are grouped/tracked based upon their ability level);
• Students are tested twice per year, once at the beginning of the year and once at the end (because the learning and forgetting that goes on during the summer is not under the control of the next year’s teacher but obviously affects how much children improve from the previous May’s testing to the next May’s testing — which in turn is used to judge that teacher’s effectiveness);
• Subtract the two test scores for each teacher to get a measure of how much his or her students learned during the year;
• Repeat the entire process the next year;
• Compare each teachers’ learning results across the two years after statistically controlling for as many factors not under the teachers’ control as possible (such as the amount of instruction students’ had previously received, and continued to receive, from their home learning environments).

Since these conditions are extremely difficult to implement (and information regarding children’s actual home learning environment is nonexistent) in the real world of schooling, Sanders and colleagues have made a valiant attempt to do the best they can with what is available to them. Their results have generated a great deal of excitement outside education (both President Obama and Malcolm Gladwell are huge fans), but unfortunately, although the value added researchers’ efforts are interpreted as showing that teacher effects are considerable in any given year, the results assessing the consistency of these effects over time are considerably less impressive.
Read More : Value-Added Teacher Assessment

Assessment In Education

What is meant by assessment in education? The term is widely debated but rarely defined. The word ‘assess’ is usually associated with words like measure, gauge, determine, evaluate, judge, weigh up, appraise, and so on. As discussed below, there are many versions and interpretations of assessment. Equally, there is an issue over who or what is being assessed. Is it the student’s learning or the teacher’s teaching? Is it a course, a curriculum or a method of teaching?

Why should we assess?
The reasons for assessing students vary widely. On a positive note, assessment can serve the following purposes:

• giving feedback to teachers and learners;
• providing motivation and encouragement; acting as both an arm-twister (a stick) and an incentive or reward for some students (a carrot);
• to boost the self-esteem of pupils (equally it can dampen it) and give a sense of achievement;
• as a basis for communication, e.g. to parents, governors or the outside world;
• as a way of evaluating a lesson, a teaching method, a scheme of work or a curriculum;
• to entertain (if done in the right way).
• Assessment performs many other functions in society which may not be viewed as positively as the six roles above:
• as a means of ranking pupils so that they can be grouped, streamed or segregated in some way;
• as a means of selection or filtering (sorting and sifting) for either employment or further education;
• to allocate students to a certain choice or pathway, e.g. a career, a new subject choice at the next level up;
• as a way of discriminating or choosing between students for other reasons.

Some different kinds of assessment
The wide range of purposes for assessment can be seen in the types of assessment that can be identified:

1. Diagnostic assessment (pre-testing): this is a form of assessment used to evaluate,  before and during teaching, every pupil’s knowledge, skills and understanding, in order to inform and improve the teaching that is to follow it. The pupils’ strengths and weaknesses can be gauged, as can their prior conceptions (see alternative frameworks) on the area to be taught and learnt. Diagnostic assessment is essential for a constructivist approach to teaching (see constructivism) and forms a good basis for differentiation, by enabling teaching to be ‘pitched’ at the right level and tailored to individuals’ needs.
2. Formative assessment, also known as assessment for learning: this occurs when assessment is seen as an essential part of the learning process (unlike summative assessment, which takes place after learning is complete). It is another way, like diagnostic assessment, of using assessment to look forward, to guide action and to shape future teaching and learning. Assessment for learning can include self assessment (in which pupils reflect on and evaluate their own learning) and peer assessment, in which they help to evaluate and think about each other’s learning. Formative assessment has received, quite rightly, increasing attention in recent years (from about 1998 onwards). It has been said to be especially helpful for ‘low achievers’ and in narrowing the gap between lower and higher achievers (Black and Wiliam, 1998a, 1998b), in contrast to summative assessment, which is said to increase this gap and to demotivate those who don’t succeed.
3. Summative assessment this occurs at the end of a teaching unit, a module or a course, such as GCSE or A-levels. Its purpose is usually to give a student a mark, grade or ranking. This form of assessment tends to receive the most publicity in terms of media coverage (school league tables), political debate (‘falling standards’), complaints from employers (‘we’re not supplied with the skills we need’) and criticisms from higher education (‘A-levels don’t discriminate between students at the highest levels’; ‘Alevels are too easy’).

The importance of variety in methods of assessment
One of the aspects of assessment said to be beneficial is the use by teachers of a wide range of methods and means. Teachers can assess through what they hear and see, as well as what they read. Assessments can be oral or written; they can be formal or informal; they can involve teachers’ observations as well as tests; they can consider co-operative group work as well as individual work; they can include coursework as well as tests and examinations. Assessment can involve a variety of outputs: spoken presentations, exhibitions, posters, or portfolios. Assessment can involve the use of ICT: word processing, desktop publishing,

PowerPoint presentations, or Internet searches. Running through all these varied means of presenting and assessing students’ work is Bloom’s taxonomy. This can be used as a checklist so that assessment can be seen to involve not only factual recall but also the use of synthesis and evaluation at the top end of the taxonomy. It should also reflect the affective domain (enthusiasm, motivation, attitude) as strongly as the cognitive (skill, knowledge and understanding).

Current and recurrent debates on assessment
The topic of assessment seems to generate a great deal of debate (hot air in some cases) from educationalists, politicians, universities, employers and parents. Certain issues are current and are certain to recur. Perhaps the main issue for teachers and for parents in the past 20 years has been the huge growth in the sheer volume of assessment. Pupils of all ages seem to have been subjected to an increasing number of formal tests at different
stages from 5 to 16. This may have pleased some parents but it has certainly worried others and their offspring. For every single teacher, the rise in the quantity of testing has created a huge demand on their time and energy. The growth era in the volume of assessment saw the popularity of the dubious adage: ‘You don’t fatten a pig by continually weighing it.’

Another major, recurrent debate has concerned ‘standards’ and, in most cases, allegedly falling standards. This has occurred as the percentage of pupils obtaining five ‘good’ GCSEs has risen above 50 per cent and the success rate at Advanced level has grown, with special attention being paid to the numbers gaining three A-levels at Agrade.

Some of the ‘elite’ universities have complained that they need a new means of discriminating between students in the top echelons of A-level; equally, employers have grumbled that standards of literacy and numeracy have fallen despite rising achievement at GCSE and equivalent examinations in Scotland and elsewhere. The standards debate is certain to continue—there can never be an absolute standard in education as there is, for example, for time or for length. The metal rod, exactly one metre in length, housed at a certain temperature in Paris, has no equivalent in education.

A long-standing and complex debate, which I cannot go into fully here, is the question of validity of tests and examinations. What do exams actually measure, apart from a student’s ability to do the exam? Can certain qualities, aptitude, potential as an employee or prediction of future success be inferred from examination and test results? The debate is analogous to the old question of intelligence testing: what do IQ tests measure other
than one’s ability to do an IQ test?

Finally, one of the most heated debates in education in the UK and elsewhere has arisen over the publication of examination and test results in the local and national media. Supporters say that this is necessary for parental choice and (even more unfairly) for naming and shaming certain schools. Critics have argued that the examination ‘league tables’ do little more than reflect the socio-economic class of the students’ families. My
own view is that any table of results should show the ‘added value’ developed by the school based on the entry point of the pupils and their ‘social capital’.
Read More : Secondary Education: The Key Concepts (Routledge Key Guides)