I’ve been facilitating inquiry team meetings and helping teachers make meaning from standardized assessment data in New York State schools for well over a decade now. Experience has uncovered the good, the bad, and the ugly when it comes to understanding and responding to the information we’re provided. I know that certain protocols inspire the development of far better hunches than others, and establishing clarity about the purpose of the assessments and the limitations of the data is critical. We perpetuate a good deal of mythology about the use of standardized assessment and data in the field, and unless we’re willing to call it what it is and do better, the kids we care about will never reach their full potential. In fact, a good deal of harm can be done.
These are the five biggest myths I consistently confront when I’m invited to facilitate data informed conversations inside of schools:
1. Standardized assessments are useless because they can’t measure imagination, passion, or creativity. This is false. In fact, standardized assessments weren’t designed to measure imagination, passion, or creativity anymore than a stress test was designed to determine pregnancy. If your team is eager to measure those things, very different measures, tools, and approaches should be used. It’s important to begin data analysis by establishing clarity about the purpose of the assessment, its design, and the strengths and limitations of the data it produces. It’s important to assess teachers’ capacity for quality data work as well. Reflecting on these initial findings can help facilitators design customized programs. Canned approaches rarely serve anyone well.
2. Standardized assessment data is more meaningful that any other data we collect. This is very flawed and very dangerous thinking. In fact, even well respected supporters of standardized assessment in education are questioning the quality of many current assessments, including those given in New York State. If test design is flawed, the data we receive are flawed, and the conclusions we draw from the data will be as well. We need to own this reality or the interventions that we pursue in our classrooms will not serve our students well.
3. Data team meetings are driven by standardized assessment data. First, I’ve learned the importance of distinguishing data team meetings from inquiry team meetings. Seem like semantics? Not so much. Even if your system hasn’t done the same, triangulation matters. In the past, quality standardized assessment data have offered important perspective about students’ performance as they’ve worked to meet standards, but they’ve never provided adequate information about what students know or how they can read, write, comprehend, reason, or problem solve. It’s been my experience that the most powerful data on the table are almost always qualitative, emerging from learning made visible during instruction, when teachers are trained to treat formative assessment as a verb instead of a noun.
4. Numbers and spreadsheets tell us important things about learners and learning. See point two above. Then consider this: how might our desire for quick, simple answers inspire our love of numbers and our attachment to spreadsheets? Data do not tell us anything. They serve as a catalyst for questioning and further inquiry. When data are flawed, the quality of the questions we ask and the inquiry work we engage in become that much more important.
5. Great teachers don’t need data. They’re intuitive and have a keen sense of their students’ needs. I’ve been teaching for a very long time, so I like to think that my perspectives about the students I serve are fairly informed. When I was coached to define data differently and make far better meaning from it, I began to recognize its potential to sharpen my senses and supercharge my intervention approaches, though. If the way that you’re using data isn’t making you better at what you do and growing your confidence and satisfaction, it’s possible that you need to ditch your approach rather than dissing the use of data.
The fact is that any time a teacher pauses long enough to gather evidence of student learning, data are being collected. When we check for understanding during a lesson, we’re gathering data. When we confer with writers during workshop, we’re gathering data. When we peek over a student’s shoulder as they use a microscope for the first time, we’re gathering data. When we sit knee to knee and ask a struggling learner what’s going on and how we can help, we’re gathering data. Regardless of whether or not we document our data, we’re constantly gathering them.
Whether we like it or not, we’re all data driven. It’s impossible not to be. We are teaching in very interesting times though. Now more than ever, we need to be as informed as we can be about the use of data in the field of education and promising practices that can help us continue to serve students well.
This is something I’ve become very passionate about over the last four years, and I have some great stories to tell. If this interests you, stop back over the next few weeks. I’ll introduce you to some very dedicated, creative, and generous teachers who are pushing my thinking about what data are, how we gather them well, and how we might use data to deepen our commitment to creative problem solving and inspired instruction, assessment, and curriculum design.
