Deeper Assessment: Why don’t we practice what we know we should do?

As part of my work as an innovation coach with the Virginia is for Learners Innovation Network, I collaborated with Julie Foss to present a four-part series on Deeper Assessment. This post is cross-posted with Advanced Learning Partnerships (ALP), a partner in the Virginia is for Learners Innovation Network who has graciously asked me to participate as a coach.

Driving Questions: What do we really want students to know and be able to do? How do we determine what’s important? To them? To us? To our subject areas? To our communities?

The four-part series on Deeper Assessment began with a high-level overview of using backwards design to decide what we really want students to know and do. Prioritizing learning outcomes is a common first step in designing a guaranteed and viable curriculum that prepares students beyond “the test.” Prioritizing standards helps determine vertical scope to prepare students for future courses in their sequence as well as to develop transferable skills across courses and subject areas. Ultimately, the highest priority standards help prepare students for “life,” whatever that may look like now, a month from now, or years from now.

The session started with a short clip from Ted Dintersmith’s Most Likely to Succeed of Linda Darling-Hammond sharing research on how fleeting inert knowledge is. Inert knowledge is information we memorize and repeat but never really use, and we lose about 90% of all inert knowledge we are exposed to. The clip from Dintersmith’s Innovation Playlist reports findings from a study conducted at the Lawrenceville Academy in New Jersey, where over two years, students were re-tested in the fall on their final science exams. Average scores went from a B+ to an F. Most students failed their exams after just three months. Moderator Julie Foss asked the 20+ participants in the session whether they might expect these same outcomes in their own divisions.

  • Justin Roerink, principal of the Hanover Center for Trades & Technology speculated that the trend might be found in a lot of classrooms. He queried, “How important is the material we are asking students to learn?” He further questioned the value of material we present to students if they don’t remember it a day, a week, or a month later. He encouraged increased use of hands-on assessments.
  • Stephanie Haskins from Staunton City Schools, noted that for a long time teachers have been caught up in the details of what needs to be taught rather than the big ideas of, “what is the important concept here?” She suggested that now is a good time to back up and forget about “all the details and all the bullets” and focus on the big ideas we really want students to understand. These big ideas will be far more memorable.
Priority Standards: You get what you aim for.

Moderator John Ross shared an overview of a backwards design process, one that many educators hear about but few use in practice, at least according to his experience. In that process, prioritized standards lead to those big ideas Stephanie Haskins references. Then assessments are developed first, based on those big ideas, and prior to considering any instructional materials or activities. That’s the crux of backwards design: design the assessment first. Using a “Fist to Five” formative protocol, participants were asked “How does this backwards design process resonate with current practice in your division?” The most common responses were 2s and 3s. 

  • Stephen Castle from Hanover County acknowledged “4 in theory…2 or 3 in practice” further noting that it’s important to dedicate time to build a truly collaborative PLC structure where teachers can do the work of prioritizing standards and determining a systematic and structured approach to addressing them. He acknowledged that despite these efforts, some teachers may still be reluctant to trust that taking a mastery-based approach is going to yield the results they want on “the test” at the end of the year. He suggested many teachers pull back from more authentic instruction prior to testing and rely on “drill-and-kill” to get information pushed into students’ inert knowledge, which we know from Linda Darling-Hammond, doesn’t stick.
  • Andrea Hand from Fairfax County Public Schools concurred with Castle and shared the idea of the tension between “have to have versus nice to have syndrome.” Some educators believe that one thing they “have to have” is good test scores at the end of the year, so they focus on that. Once they feel comfortable that will happen they can then work on the more authentic learning and others which, to them, fall under the category of “nice to have.” She suggests that the more we can advocate and align the “have to haves” with our collective beliefs around authentic learning it would guarantee students more of those experiences. 

This tension is real, but many of the participants felt that our unique circumstances have given us reason to get back to what we know works to provide more authentic instruction that prioritizes learning outcomes to the needs of students, not just trying to cram in inert knowledge students will likely soon forget. Many of the participants continued through the rest of the week with additional sessions on designing performance-based tasks, creating learning progressions that lead to standards-based rubrics, and exploring portfolios and other resources to capture evidence of student learning.

Check out the conversation at this YouTube link, as well as the agenda with downloadable resources, a link to the slide deck, and links to resources from the other three sessions.

Rethinking Rubrics: Rubrics that Make You Think

In 2010, my colleague (and mentor) Dr. Sharon Harsh was presiding over a meeting with staff from the Appalachia Regional Comprehensive Center (ARCC—our organization) and the Virginia Department of Education (VDOE), with whom we had collaborated for five years. She was summarizing the trends in education from the past decade or so and going out on a limb by making predictions of trends that were soon to influence education. She hit them right on the head, especially with one prediction: learning progressions will become prevalent and guide the work educators do at all levels.

Simply put, learning progressions describe the most likely steps people will take when developing new knowledge and skills. For example, before students can combine fractions with different denominators, they have to recognize what fractions are and understand what they represent. They have to know that a larger number in the denominator doesn’t mean it’s a larger fraction. Later they come to understand how different fractions are related—focusing on how to express two fractions with equivalent denominators, then unlike denominators. There’s more, but that’s a portion of the idea of how some concepts related to fractions progress.

Sharon got this so right! Learning progressions strongly influenced the way new standards were developed. And state departments of education, including VDOE staff in the present, are developing and sharing the learning progressions behind their standards so teachers can better understand how students master standards within and across a grade level. Teachers, too, are developing learning progressions at a finer grain that help them understand how students develop skills and knowledge within a single standard (like the idea of combining fractions above). I find learning progressions really intriguing, but I’m a little geeky like that.

Applying Learning Progressions

I’ve long used rubrics to support my instruction and to score student work. In the graduate class I taught, every activity used a rubric, and the students got all of the rubrics on day one and were encouraged to use them as they worked through activities. I’ve never really given multiple-choice tests. Ever. I’ve also helped a lot of teachers develop rubrics, especially when they need to assign some sort of score or grade to complex problems or projects. In many cases, a multiple-choice question isn’t the best option.

Below is an example of a rubric I created in the past. It’s typical of many I’ve seen. If you’re a student who wants to score well, you don’t make mistakes. As you make more mistakes, your score is lower. It seems logical, at first.

 

Learning Outcomes

Novice Developing Approaching

Expert

Grammar and mechanics of language The product contains numerous (7 or more) errors in grammar, punctuation, and/or capitalization of written text, or 3 or more errors in spoken language. The product contains several (4-6) errors in grammar, punctuation, and/or capitalization of written text, or 1 or 2 errors in spoken language. The product contains a few (1-3) errors in grammar, punctuation, and/or capitalization of written text, or no more than 1 error in spoken language. The product contains no errors in grammar, punctuation, or capitalization of written text, or no errors in spoken language.
Solve multistep problems with fractions The student does not show his/her work, presents incomplete work, or inaccurately presents work in regard to the guidelines. The student designs a solution that has more than one error in calculation. The student designs a solution that has no more than one error in calculation. The student designs a solution that meets the guidelines with no errors.

While this is a pretty typical rubric, it isn’t really very helpful for promoting learning. Why? It’s not the number of errors that’s important, it’s the kind of errors that students make that’s most important. If a student makes two or three errors, but there’s no clear pattern to them, it may just be a mistake because of a lack of time or sloppiness. That doesn’t tell me anything about what they do or don’t understand or how I need to re-teach them. But when a student makes consistent errors, like using “its/it’s” incorrectly over and over, or writing too many run-on sentences, or confusing larger denominators with larger fractions, then I know what to focus on. I needed something that showed me common errors, as well as that progression of how learners move from being a novice to mastering the standard.

Improved Rubrics

I’ve finally been able to connect that sage prediction that Sharon Harsh made with my own practice. Since standards are based on learning progressions, we should be monitoring where our kids are along those progressions. This helps not just teachers, but students too! Both can see what skills and knowledge they’ve mastered, where they need to go, and even suggestions as to what steps they might take to get there. Some might recognize that this is also a critical component of using formative assessment strategies to support learning, especially as proposed by Margaret Heritage (e.g., Where am I going? Where am I now? How do I get there?).

So over the past couple of years, I’ve been pushing myself to improve my rubrics. Instead of just counting errors, which tells me little about what my students truly know or can do, I’m now designing rubrics that describe the progression of learning students go through when mastering a content standard.

Please note: In the examples, the scoring categories are labeled as Learning Outcomes, but many teachers will recognize that the language used is drawn from actual standards, in these examples, the Virginia Standards of Learning, Common Core State Standards, and a WIDA ELD standard. So, in this way, the rubrics are actually standards-based. In fact, they’re probably more standards-based than any forced-choice assessment can be, at least for sophisticated learning outcomes.

Now when I work with teachers on complex problems or performance tasks, we co-develop rubrics that describe learning progressions. See the examples below created recently with some great teachers from the Crestwood School District in Dearborn, Michigan. These are rough drafts, but even at this stage I can see the progression learners go through for each of these learning outcomes. I learned this from these teachers, but every time I do this, the discussion we have about learning progressions is great.

 

Learning Outcomes

Novice Developing Approaching

Expert

Write opinion pieces on topics or texts, supporting a point of view with reasons and information.  The student’s product does not contain a clearly stated opinion or goes off topic and there’s no evidence. Possibly no reasons. The student’s product does include a clearly stated opinion, but lacks support through reasons that are expanded or supported by evidence from the texts. The student’s product does include a clearly stated opinion with some evidence, but the reasons lack coherence, may not be clearly sequenced or organized. The student’s product contains a clearly stated argument (or point of view) with reasons supported by evidence drawn from the texts and is clearly organized and coherent.
Students read informational articles on globalization to consider its impact on their lives (e.g., Internet, mass media, food and beverage distributors, retail stores).   The student’s product includes an opinion but does not include information from the articles. There’s no indication the student has or can read the articles. The student’s product contains phrases or some keywords from the articles but may not be explained or connected to a position related to their lives. The student’s product includes some examples from the articles but they may not support their position as it relates to their lives. The student’s product includes citations of examples from the articles that support their position and relates those citations to their lives.
Make a line plot to display a data set of measurements in fractions of a unit (1/2, 1/4, 1/8).  The student’s product does not create a line plot or creates something different from a line plot. The student’s product contains a line plot with simple fractions (e.g. ½ and ¼), with fractions out of order (because of denominator). Something’s out of order. The student’s product contains a line plot with points inaccurately plotted, so it does not match the data, though the fractions are in order. The student’s product contains an accurate line plot that displays the appropriate data and the fractions are in order.

 

Summative assessment is just one use of this type of rubric. Now that we’ve described learning progressions for these standards, these rubrics have multiple uses. Teachers can hand them out at the beginning of any unit, lesson, or activity that uses these learning outcomes so students know what they can do to get the grade they want. It saves teachers time because they don’t need to create rubrics for every activity, just for each standard. More importantly, students can use the rubrics to monitor their own progress. Schools wanting to move towards mastery learning or standards-based report cards can also use these types of learning progressions to truly describe what the difference between an A or a B (or other two grading categories) really means. It’s not just a score, it’s a point along mastery. Finally, this type of rubric is helpful when talking with parents. When parents want to know, “Why didn’t my kid get an A?” teachers can show parents exactly where their child’s current performance is along the progression and where they need to get to master the outcome (and get that A!). Maybe in the future, parents will ask, “How can I help my kid master the standards?” Maybe.

Supporting Formative Assessment with Technology

A few strands of my work have come together recently, and they focus on using technology to support formative assessment. This has been one of the most common requests recently from teachers/schools I’ve been working with and is the focus of two additional projects I’ve been working on. (You can skip directly to the tools here. Updated with two new tools on March 2.)

Through my work with the Appalachia Regional Comprehensive Center (ARCC), I’m collaborating with staff from the Virginia Department of Education to pilot a statewide cadre of teachers exploring formative assessment. Teachers from six schools across the state are working through training materials developed by Dr. Margaret Heritage and the Center for Standards and Assessment Implementation (formerly the Assessment and Accountability Center).

Margaret Heritage's book

Formative Assessment by Margaret Heritage

Dr. Heritage is probably the nation’s foremost expert on formative assessment and has been implementing and studying formative assessment strategies in multiple districts for years. (Check out her book on the topic!)

 

What Dr. Heritage does is provide concrete steps teachers can follow to embed formative assessment in their teaching. I agree with Heritage when she notes that formative assessment is not a single “thing” or event. It’s not a quiz. It’s not a test. It’s an ongoing dialog between teachers and students that ids designed to collect evidence of where students are in their journey of mastering skills and knowledge.

From my perspective, I overlay this idea of formative assessment being a process to the selection of relevant technology resources. An online quiz or a classroom responder (clicker) is itself not a formative assessment tool, unless used that way. For that reason, I’ve grouped a range of digital resources that can be used formatively, but you have to first identify the way you want to use it. That’s the trick to picking any technology for classroom use, actually. Figure out what you need to accomplish, then find a resource to match.

Check out the resources here

This is not an exhaustive list, and it was just updated this past week after my visit to Dubuque where I learned Infused Learning is on the way out and GoFormative.com is a new resource to be considered. I’d appreciate hearing from you about the digital resources you use and how you use them to support formative assessment so I can update this over time.