Fred Newmann, Walter Secada, and Gary Wehlage

The authors' premise is that innovative teaching techniques (e.g., cooperative learning, group discussions, hands-on experiments and videos) do not guarantee a change in the "intellectual quality" of what students are asked to do. For example, "a portfolio that shows a variety of student work over a semester might replace the final exam taken in one sitting, but the portfolio itself could be filled with tasks" devoted to remembering and listing isolated bits of information. Thus, the merit of any teaching technique should be judged by its ability to improve the "intellectual quality of student performance."
 

The document listed below is devoted to defining "intellectual quality," developing criteria

for judging the intellectual quality of the school work students are asked to perform, assessment tasks, and student work. The criteria were designed for use in mathematics and social studies and many examples in these subjects are included. But the authors encourage application to any content area. The criteria can, therefore, serve to define delivery and performance standards across the curriculum and across teaching methods.
 

Attached are the criteria for assessing the intellectual quality of student work in social studies. They appear to be appropriate for grades 7-12.
 

A Guide to Authentic Instruction and Assessment: Vision, Standards and Scoring, 1995
pp 94-101
Fred Newmann, Walter Secada, and Gary Wehlage
Document Service
Wisconsin Center for Educational Research
School of Education
University of Wisconsin
1025 West Johnson Street
Madison, Wisconsin 53706
(608) 263-4214

PART III: STUDENT PERFORMANCE
 

Overview and General Rules

Parallel, but somewhat differently worded, standards are presented for student performance in mathematics and social studies. The three standards reflect two of the general standards for authentic achievement as follows:
 

Construction of Knowledge: Analysis

Disciplined Inquiry: Disciplinary Concepts; Elaborated Written Communication
 

A. Scores should be based only on evidence in the student's performance relevant to the criteria. Matters such as whether the student followed directions, neatness, correct spelling, etc. should not be considered unless they are relevant to the criteria.
 

B. Scores may be limited by tasks which fail to demand analysis, disciplinary conceptual understanding, or elaborated written communication, but the scores must be based only upon the work shown.
 

C. Scores should take into account what students can reasonably be expected to do at the grade level. However, scores should still be assigned only according to "absolute" criteria in the standards, not relative to other papers that have been previously scored.
 

D. When it is difficult to decide between two scores (e.g., a 2 or a 3), give the higher score only when a persuasive case can be made that the paper meets minimal criteria for the higher score. If the specific wording of the criteria are not helpful in making this judgment, base the score on the general intent or spirit of the standard described in the introductory paragraphs of the standard.
 

Mathematics
 

Mathematics Standard 1: Analysis
 

Student performance demonstrates thinking with mathematical content by organizing, synthesizing, interpreting, hypothesizing, describing patterns, making models or simulations, constructing mathematical arguments, or inventing procedures.
 

This standard is intended to measure the extent to which the student demonstrates higher order thinking, i.e., thinking that goes beyond mechanically recording or reporting facts, rules, and definitions or mechanically applying algorithms.
 

The term "mathematical" analysis calls attention to the fact that the content or focus of the thinking should be mathematics. If analysis occurs about other subject areas, without a connection to mathematics, the performance would still score a 1 on this standard.
 

There are two guiding questions here: First, has the student demonstrated mathematical analysis? To answer this, consider whether the student has organized, interpreted, synthesized, hypothesized, invented, etc., or whether the student has only recorded, reported, or mechanically applied rules, definitions, or algorithms. If work is not shown, correct answers can be taken as an indication of analysis if it is clear that the question would require analysis to be answered correctly. Second, how often has the student demonstrated mathematical analysis?
 

4 = Mathematical analysis was used throughout the student's work.
 

3 = Mathematical analysis was used in 50 percent or more of the student's work.
 

2 = Mathematical analysis was used in less than 50 percent of the student's work.
 

1 = Mathematical analysis constituted no part of the student's work.
 

Example: If no work was shown and ¾ of the questions were judged to require analysis to get the right answer, and the student did all of those correctly, she or he would score a 3 on this scale. On the other hand, if the student got them all wrong, he or she would get a 1. In short, the scorer should estimate the percent of the total task that the number of correctly answered analytic questions comprised.
 

Mathematics Standard 2: Disciplinary Concepts
 

Student performance demonstrates an understanding of important mathematical ideas that goes beyond application of algorithms by elaborating definitions, making connections to other mathematical concepts, or making connections to other disciplines.
 

This standard is intended to measure the extent to which the student demonstrates use and understanding of mathematical concepts. Prior to scoring the student's work, the rater should identify what mathematical concepts, if any, a student must use and/or understand to succeed in the task. Low scores may be due to tasks which fail to call for understanding of mathematical concepts.
 

A guiding question for using this standard is, "Does the student show understanding of the fundamental ideas relevant to the mathematics used in the task?" Correct use of algorithms does not necessarily indicate conceptual understanding of the material. Such understanding can be demonstrated, for instance, by elaborating upon the concept through definition, or by making connections between the core concept and other related ones.
 

If work is not shown, correct answers can be taken as an indication of conceptual understanding, if it is clear that the task or question requires a conceptual understanding in order to be completed successfully.
 

Completion of the task is not necessary to score high.
 

4 = The student demonstrates an exemplary understanding of the mathematical concepts that are central to the task. Their application is appropriate, flawless, and elegant.
 

3 = There is substantial evidence that the student understands the mathematical concepts that are central to the task. The student applies these concepts to the task appropriately; however, there may be minor flaws in their application, or details may be missing.
 

2 = There is some evidence that the student understands the mathematical concepts that are central to the task. Where the student uses appropriate mathematical concepts, the application of those concepts is flawed or incomplete.
 

1 = There is little or no evidence that the student understands the mathematical concepts that are central to the task, or the mathematical concepts that are used are totally inappropriate to the task, or they are applied in inappropriate ways.
 

Mathematics Standard 3: Elaborated Written Communication
 

Student performance demonstrates a concise, logical, and well articulated explanation or argument that justifies the mathematical work.
 

Performance that meets this standard could include, or consist of, diagrams and drawings as well as prose. To score high on this standard, the student must communicate, in writing, an accurate, complete and convincing explanation or argument.
 

The score should not be based on the proportion of student work that contains explanation/argument but on the quality of mathematical communication, wherever it may occur in the work. We use the prefix "mathematical" to draw attention to the fact that the substantive content of what is written about must be related to mathematics.
 

4 = Mathematical explanations or arguments are eloquent, clear, complex, and complete. Mathematical communication is exemplary.
 

3 = Mathematical explanations or arguments are present. They are largely concise, clear, and well articulated; however, they may be slightly flawed or incomplete in minor ways.
 

2 = Partial mathematical explanations or arguments are present. They are incomplete because they have not been finished, they omit a major part of an argument, or they contain several mathematical or logical errors. Note: Simply showing work on algebraic problems (regardless of whether the answer is right or wrong) constitutes a low level of explanation and should be scored a 2 if no other part of the student's work on the task shows a higher level of communication.
 

1 = Mathematical explanations or arguments are absent or, where present, they are seriously incomplete, totally inappropriate, and incorrect. This may be because the teacher's questions have left no room for argument or explanation, e.g., fill-in-the-blank and multiple-choice questions.
 

Social Studies:
 

Social Studies Standard 1: Analysis
 

Student performance demonstrates higher order thinking with social studies content by organizing, synthesizing, interpreting, evaluating, and hypothesizing to produce comparisons/contrasts, arguments, application of information to new contexts, and consideration of different ideas or points of view.
 

This standard is intended to measure the extent to which students demonstrate cognitive activity that goes beyond mechanically recording, reporting or otherwise reproducing information. Analysis may include proposing generalizations and supporting them with evidence; articulating and testing different theories or points of view; synthesizing and categorizing by applying abstractions to more specific information (this could include comparing similarities and differences); considering implications and application of information in new contexts; raising broad questions that help to interpret more specific information; or interpreting the meaning of personal roles, ideas, or events. The essential question is whether students demonstrate construction of knowledge through thinking and the organization of information, versus reproduction of knowledge by restating what has been previously given to them.
 

The rhetorical form of students' statements might qualify as analysis (e.g., "The main reason for the American Revolution was taxation without representation"), but to score high on analysis, the student's work must appear to be reasonably original, not merely a restatement of some analysis that was given previously in a text or discussion. In assigning a 3 or 4, the rater should be reasonably confident that no significant portion of the response has been virtually copied from some other source (i.e., text or oral statements of others).
 

4 = Substantial evidence of analysis. Most of the student's work includes analysis. At least three statements indicate that the student has successfully generalized, interpreted, tested, or synthesized specific information.
 

3 = Moderate evidence of analysis. A central portion of the student's work includes analysis. At least two statements indicate that the student has successfully generalized, interpreted, tested, or synthesized specific information.
 

2 = Some evidence of analysis. A small, but not central, portion of the student's work includes analysis. At least one statement shows that the student has successfully generalized, interpreted, tested, or synthesized specific information.
 

1 = No evidence of analysis. Almost all statements consist of recording, or reporting specific information, without evidence of the student's organizing it or reflecting upon it; OR virtually all analysis offered is unsuccessful or in error.
 

In scoring analysis, the proportion of work that illustrates analysis is more important than the actual number of statements indicating analysis.
 

Social Studies Standard 2: Disciplinary Concepts
 

Student performance demonstrates an understanding of ideas, concepts, theories, and principles from the social disciplines and civic life by using them to interpret and explain specific, concrete information or events.
 

This standard is intended to assess the extent to which students use important ideas of the social disciplines to make concrete information and events more meaningful. Substantive concepts (e.g., depression, social class, culture) are often grounded in disciplinary knowledge from history and the social sciences, but many important ideas in social studies (e.g., justice, freedom, citizenship) may have no exclusive origins or associations within a single discipline. The main issue is the extent to which the student has used substantive social ideas to organize, explain, interpret, summarize, and extend the meaning and significance of otherwise discrete pieces of information.
 

Social studies concepts may be used even though they may not be stated explicitly, and this may vary with student grade level. For example, an elementary student's discussion of conditions or oppressed groups might indicate use and understanding of the concept of equality, without stating the specific word.
 

If the topic of the task is itself a substantive idea (e.g., revolution), students should get full credit for successful use of it. Give credit only for ideas that are used appropriately in the context of the assignment. No credit should be given for serious errors in application or interpretation. The phrase "social studies concepts" means a minimum of one social studies concept.
 

4 = The student has used social studies concepts to organize, explain, interpret, summarize, and extend the meaning and significance of otherwise discrete pieces of information. The use of the ideas illustrates exemplary understanding.
 

3 = The student has included social studies concepts to organize, explain, interpret, summarize, and extend the meaning and significance of otherwise discrete pieces of information. The use of the ideas is somewhat limited and/or shows some flaws in understanding.
 

2 = Social studies concepts are included, but their use is significantly limited and/or shows significant flaws in understanding.
 

1 = The work includes virtually no social studies concepts, or the use of any that are included shows almost no understanding.
 

The score for social studies Disciplinary Concepts should be based on the quality of use of social studies concepts, not on the proportion of student work that reflects social studies concepts.
 

Social Studies Standard 3: Elaborated Written Communication
 

Student performance demonstrates an elaborated account that is clear, coherent, and provides richness in details, qualifications and argument. The standard could be met by elaborated consideration of alternative points of view.
 

To use the criteria below, the scorer should identify specific points in the student work that are elaborated, and should make a judgment about the coherence of the overall framework in which various points are communicated.
 

4 = Exceptional: The writer provides substantial and accurate elaboration for two or more important statements. The details, qualifications, and nuances are expressed within an overall coherent framework intended for the reader, and relevant to the topic. The response is so rich as to be worthy of display as an outstanding example of writing in social studies.
 

3 = Elaborated: The writer provides some elaboration for two or three important statements or provides substantial elaboration for one important statement. In either case, the details, qualifications, and nuances are expressed within a coherent overall framework intended for the reader, relevant to the topic, and without major inaccuracies.
 

2 = Minimal: The writer provides reasonably accurate elaboration for at least one important statement.
 

1 = Unsatisfactory: The writer provides virtually no information or provides only disjointed details OR the writer provides discrete claims, broad generalizations, slogans, or conclusions, but none are elaborated.
 

When a task includes several parts, the score for elaboration should be based on the part(s) answered in prose.