Deeper Assessment: Why don’t we practice what we know we should do?

As part of my work as an innovation coach with the Virginia is for Learners Innovation Network, I collaborated with Julie Foss to present a four-part series on Deeper Assessment. This post is cross-posted with Advanced Learning Partnerships (ALP), a partner in the Virginia is for Learners Innovation Network who has graciously asked me to participate as a coach.

Driving Questions: What do we really want students to know and be able to do? How do we determine what’s important? To them? To us? To our subject areas? To our communities?

The four-part series on Deeper Assessment began with a high-level overview of using backwards design to decide what we really want students to know and do. Prioritizing learning outcomes is a common first step in designing a guaranteed and viable curriculum that prepares students beyond “the test.” Prioritizing standards helps determine vertical scope to prepare students for future courses in their sequence as well as to develop transferable skills across courses and subject areas. Ultimately, the highest priority standards help prepare students for “life,” whatever that may look like now, a month from now, or years from now.

The session started with a short clip from Ted Dintersmith’s Most Likely to Succeed of Linda Darling-Hammond sharing research on how fleeting inert knowledge is. Inert knowledge is information we memorize and repeat but never really use, and we lose about 90% of all inert knowledge we are exposed to. The clip from Dintersmith’s Innovation Playlist reports findings from a study conducted at the Lawrenceville Academy in New Jersey, where over two years, students were re-tested in the fall on their final science exams. Average scores went from a B+ to an F. Most students failed their exams after just three months. Moderator Julie Foss asked the 20+ participants in the session whether they might expect these same outcomes in their own divisions.

  • Justin Roerink, principal of the Hanover Center for Trades & Technology speculated that the trend might be found in a lot of classrooms. He queried, “How important is the material we are asking students to learn?” He further questioned the value of material we present to students if they don’t remember it a day, a week, or a month later. He encouraged increased use of hands-on assessments.
  • Stephanie Haskins from Staunton City Schools, noted that for a long time teachers have been caught up in the details of what needs to be taught rather than the big ideas of, “what is the important concept here?” She suggested that now is a good time to back up and forget about “all the details and all the bullets” and focus on the big ideas we really want students to understand. These big ideas will be far more memorable.
Priority Standards: You get what you aim for.

Moderator John Ross shared an overview of a backwards design process, one that many educators hear about but few use in practice, at least according to his experience. In that process, prioritized standards lead to those big ideas Stephanie Haskins references. Then assessments are developed first, based on those big ideas, and prior to considering any instructional materials or activities. That’s the crux of backwards design: design the assessment first. Using a “Fist to Five” formative protocol, participants were asked “How does this backwards design process resonate with current practice in your division?” The most common responses were 2s and 3s. 

  • Stephen Castle from Hanover County acknowledged “4 in theory…2 or 3 in practice” further noting that it’s important to dedicate time to build a truly collaborative PLC structure where teachers can do the work of prioritizing standards and determining a systematic and structured approach to addressing them. He acknowledged that despite these efforts, some teachers may still be reluctant to trust that taking a mastery-based approach is going to yield the results they want on “the test” at the end of the year. He suggested many teachers pull back from more authentic instruction prior to testing and rely on “drill-and-kill” to get information pushed into students’ inert knowledge, which we know from Linda Darling-Hammond, doesn’t stick.
  • Andrea Hand from Fairfax County Public Schools concurred with Castle and shared the idea of the tension between “have to have versus nice to have syndrome.” Some educators believe that one thing they “have to have” is good test scores at the end of the year, so they focus on that. Once they feel comfortable that will happen they can then work on the more authentic learning and others which, to them, fall under the category of “nice to have.” She suggests that the more we can advocate and align the “have to haves” with our collective beliefs around authentic learning it would guarantee students more of those experiences. 

This tension is real, but many of the participants felt that our unique circumstances have given us reason to get back to what we know works to provide more authentic instruction that prioritizes learning outcomes to the needs of students, not just trying to cram in inert knowledge students will likely soon forget. Many of the participants continued through the rest of the week with additional sessions on designing performance-based tasks, creating learning progressions that lead to standards-based rubrics, and exploring portfolios and other resources to capture evidence of student learning.

Check out the conversation at this YouTube link, as well as the agenda with downloadable resources, a link to the slide deck, and links to resources from the other three sessions.

Why Word Problems Aren’t Real-World Problems

O.K., I should have probably stated, “Why word problems usually aren’t real-world problems,” but that’s not as catchy a headline. As someone who works with a wide range of teachers across the country, I do often run into the misconception that the typical word problem—at least the ones I found used in many classrooms across the country—represent “real-world” problems. For some reason, using words to present problems about baking and fractions or making change tip the scale towards authenticity in many educators’ minds. My goal is to push their understanding of exactly what is meant by a real-world problem.

The primary limitation of simply representing a problem with a single known (and desired) solution with words rather than algebraically or graphically (or musically, or…?) is that changing the representation system somehow elevates it to a real-world problem. But it’s just a different way to represent an academic exercise, not a problem. When a word problem can easily be translated into a different representation system that all result in the same answer, the goal is to determine if a student can use a known algorithm. Real-world problems are usually not that simple, even when dividing pizzas and donuts.

Exercises Problems
Simplistic, well-structured
Distractions have been eliminated
Complex, ill-structured
May contain noise or detractors
Academic setting Real-world connection
One correct answer May be multiple correct answers
Focus is getting correct answer Focus is on the process and strength of evidence

Adapted from Wiggins, G., & McTighe, J. (2005). Understanding by design (expanded 2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

Grant Wiggins and Jay McTighe do an excellent job of comparing a real-world problem to an academic exercise in their foundational Understanding by Design instructional design model. The implication here is that a problem is complex; there are options. Using a real world context is not enough if the problem becomes routine and simplified. Real-world problems often have no single correct answer. Sometimes they don’t even have correct answers, but some answers can be better than others. Problems focus on the problem-solving processes students use to develop their solutions, not just a correct answer. In the real world we make decisions about real-world problems using the best information we have available and justifying our decisions with some kind of evidence (What kind of car can I afford? Is college really worth it? Should we clone dinosaurs?). That means my solution to a real-world problem can be different from yours, but both can be used to demonstrate the process we took to come up with our solutions.

I don’t want to imply that simplified word problems are bad. Students do need them to develop foundational knowledge and practice skills. These types of problems are those that fit squarely into what many educators understand as Depth of Knowledge Level 2, in which students apply and practice basic skills, often decontextualized. These problems can use any type of representation system. But if students are only presented with a word problem at the level of basic application, they are not exposed to the cognitive demand implied by encouraging the use of real-world problems—requirements of content standards in many grade levels. Real-world problems require students to draw upon a repertoire of knowledge and skills in order to address a non-routine problem, because whose real world is all that routine? These problems are often associated with Depth of Knowledge levels 3 (strategic thinking) and 4 (extended thinking), and that’s the intent of most content standards that include mention of real-world problems.

Designing Online Learning

As a coach with Advanced Learning Partnerships, I have the privilege of working school divisions participating in the Virginia is for Learners Innovative Network (#VA4LIN). If you live in Virginia you may have heard about this sweeping initiative launched by the Virginia Department of Education with support from VaSCL, JMU, and Ted Dintersmith. ALP coaches are working with the schools and divisions in the network who have identified their own priorities for promoting innovation.

One of the divisions I’m working with, Bedford County Public Schools, has already initiated efforts related to personalized learning prior to joining the network and asked for some support with developing online and blended learning. Division staff want to leverage online technologies to provide more personalized support for both learning opportunities for students and professional development for adults in the system. I met with a group of ITRTs (Instructional Technology Resource Teachers), library/media specialists, and others in Bedford County on May 9 to review instructional design for online and blended learning. Our goal was to develop design specifications for educators in Bedford County to refer to when designing their own learning materials.

Educators from Bedford County Public Schools discuss designing powerful online learning.

I provided access to materials based on my book, Online Professional Development, Design, Deliver, Succeed! and subsequent publications. We reviewed key instructional design principles, considered how staff are currently addressing them, and considered how these principles might be presented in the first draft of a design specifications document. The key ideas we reviewed include:

  1. Know your audience.
  2. Define your learning outcomes.
  3. Assess your learning outcomes.
  4. Consider your visual design.
  5. Match media to your outcomes.
  6. Evaluate your learning.

You can access the slide deck with links to handouts and templates on Google Drive. Let me know if it’s helpful. Next steps for the, the participants include returning to existing online professional learning to review what has already been developed and determine how the specifications might impact that work.

Considering implications of instructional design principles on practice.