Tuesday, September 17, 2013

**Disclaimer - This post does not reflect the views of #LivEd or @bill_m_4, only the views of myself (@ryanhorne0076). The hope is to start a robust discussion on the topic of DDI (Data Driven Instruction)**

Let me begin by saying I like data. I like data to help me make everyday decisions in my own personal life. I like data as an educator. Every teacher uses data in one form or another to assess their students’ learning. So, how much data is too much data?

First of all, this post is written with no data to support, or debunk, my statements. If you are looking for links on DDI (Data Driven Instruction) research, links to other blog posts on the same topic, or expert quotes on DDI, then you’ll want to look elsewhere. This is simply my stream of consciousness on the topic. The omission of data is done purposely. I am writing this post on data, without including any data, to try and prove my point.

Emily - SRI: 784, SMI: 658, ELA: 717. At first glance I feel as though I’m looking at a ten year old’s credit score, not their school learning accomplishments. What do these numbers say? What do they really mean? Perhaps more importantly, what are we NOT seeing when these numbers are the only information we see about a student?

When students are identified by their various test results, we de-personalize them. They become a walking collection of lexile levels, reading scores, and math percentiles. Many schools have gone “all-in” in the current hand that is DDI. The pendulum has seemed to swing so far that we no longer talk about students as people, but rather their most recent summative test results. I want to know more about my students.  I want to know HOW they like to learn. I want to know WHAT they like to learn. I want to know what INTERESTS them.

Why is this trend happening now? The technology that allows us to administer tests, collect results, and disaggregate data used to be so complex and expensive that it was reserved for the few experts who had both the statistical knowledge and sufficient funds to filter the data and purchase the infrastructure. Now, the technology is available to most educators in an easy to use format, provided by most major edtech companies, at competitive prices. Now many educators have a warehouse of student data at their fingertips with various amounts of training on how to read and interpret that data.

Sometimes in education, we are adverse to an educational trend because the trend is new to us, it makes us uncomfortable, we are unsure of the trend because of our lack of knowledge. We fear the trend because it is not the way we always have done things in the past. What if the trend invokes different intuitions? What if the trend feels deeply flawed or inherently wrong? Not because it’s different, confusing, or new, but because it simply feels wrong?

Our students are spending so much valuable, precious, face-to-face classroom time taking standardized tests. We cannot get this face-to-face time back. Students are unintentionally taught this is a big part of learning. Do some learning, pencil in bubbles. More learning, then click on the correct answer. So many students are being turned off to the learning process, as this has become the learning process.

How much of the DDI trend has been kick started and perpetuated by big money edtech corporations? There’s huge money in standardized testing. Huge money in warehousing data. How much of the DDI trend has been written into law by legislation lobbied by these corporations? Who is profiting the most from DDI? Our country? Our students? Various CEO’s?

How much data do we need to prove that a system is broken? How much data do we need to show what our students have achieved and what they are capable of? Does the data that defines our kids need to be warehoused on a server in Silicon Valley, or does the data that defines our kids need to be coming from the people who know them the best?


  1. There has to be many out there who disagree. Love to hear your thoughts. DDI is an important discussion to have.

  2. You ask some provocative questions! How much testing is too much testing? My thoughts are as follows, there needs to be some metric for which to measure the progress of a student. Standardized testing provides that metric because they are uniform and unbiased. They are applied to everyone and can give a good apples-to-apples raw analytic value. There is simply no getting around this! Everyone uses different measurements to judge things in their day to day life. How do we know which candidate is better for a job? Their resume, certifications, degrees, etc. We certainly don't take their word for it, or only judge them "from the people who know them the best". However you personally feel about it, numbers are still the best way to tell which of two things are bigger, and so we gather analytics about students and assign them a score. It may seem cold and mechanical, but so is the world!

    "So many students are being turned off to the learning process, as this has become the learning process." This doesn't have to be the case even when standardized tests are administered. Why does standardized testing mean standardized learning? One of my best study aids for the math section on the SAT's was taking physics. My physics teacher was amazing, and gave me a true passion for the subject. In working on physics problems, I came to appreciate mathematics and develop a true understanding for how they worked by assigning meaning to the numbers. I was then able to take these abstract concepts and apply them to "plain" math problems by thinking of how they relate to the real world. There was no physics standardized test we were being trained for. It was just pure learning fun and it had the side effect of making me better at related skills that were tested. Just because there is a test, does not mean you can/should only teach for that test. Just teach your subject and let the chips fall where they may.

    "I want to know HOW they like to learn. I want to know WHAT they like to learn. I want to know what INTERESTS them." Good, then do that! Engage with them, encourage them show them that learning is fun and that being a life long learner is extremely rewarding! But be honest. Having your knowledge tested does not end with graduation. Standardized testing is not a school age Phenomenon. As a software developer I have never been to an interview where I was not asked a series of "standard" questions. Passing the interview test does not necessary mean I will be a good fit, but failing it guarantees that I wont.

    Getting back to my original point. How much testing is too much testing? When the results no longer provide useful data points for measuring the progress or skill set of the individual being tested. The real question is... are we there yet? I don't know, I need more data ;-)

    1. I can see that you a very intelligent individual and I would wager that you found success in school and success on tests because of your intellect.

      I agree that we need to have metrics to measure student progress. However, my concern is with the type of data that is collected from these standardized tests. Do most standardized assessments test for lower order thinking skills or higher order thinking skills?

      which brings me to my next counter-point. Educators often discuss the importance of 21st Century higher order skills. If you are giving the interview, what are you most interested in? Let's use your software development career as an example. I want to know how much the interviewee knows about code, I want to know the various codes he/she can write. More importantly, I want to SEE examples of software that the interviewee has actually completed. Yes show me a resume with impressive credentials, but also show me your portfolio that has authentic examples of your work being used in day-to-day operations.

      In response to teachers teaching for learning, and not teaching to the test, we must unfortunately look at how recent legislation and how that impacts teachers and schools. Many states now tie in student test scores to teacher evaluation. Teachers feel forced to "teach to the test" in order to keep their job or receive a positive evaluation. Is it possible to teach in a "non-standardized" way as your physics teacher did and still have students score well on tests? Of course, but the current system in place labels these great teachers as "risk takers". Education as a whole is risk adverse. Don't we want it to be the norm that educators teach like your physics teacher? If so, high stakes teacher evaluations tied to student test scores does not promote this type of learning.

      For me, it's the unintended consequences of DDI that hampers our goal to produce 21st Century learners. Sure, what teacher wouldn't want to find out how there students best learn and what they want to learn about? It is increasingly more difficult to do this with so many tests occurring during this face-to-face class time.

      It's the message we hear from our leaders. We want to produce deep thinkers, problem solvers, self-directed lifelong learners. But students become so turned off to learning when they are exposed to a system of test after test, after test. This cycle unintentionally directs kids away from becoming self directed lifelong learners. They see the learning process meaningless, and often the best students, are the best students, because they become experts at "playing the game of school".

    2. I appreciate the comment! Actually I was mediocre in middle and high school. It was that very physics class (also a fantastic English teacher that I still keep in touch with) that woke me up. Then in college I really found my pace and truly became a life long learner.

      I agree with you that an abundance of standardized tests can lead to a teach-to-the-test attitude. I also agree with you that teachers like my physics teacher should not be considered outliers. However I feel that this is more of a dystopian outlook and not representative of what is going on at this moment. A world where creativity and individually are crushed by the testing demands of a broken school system would make a great intuition pump, but sounding the alarms now is still premature.

      In your counter-example you bring up the point of seeing the work of the candidate as opposed to judging them based solely on their resume and accomplishments. But this is itself another metric. Although it might not "feel" like a numeric comparison, it actually is. When I look at somebodies code, I don't just check if it runs. I judge many criteria and unconsciously assign a value to each of them. Code that is complex, but well commented is less valuable then code that is so clear and concise that it doesn't need comments. This is a standard that I judge by, However I doubt any candidate was specifically trained for this standard. It comes naturally with a deep understanding of the subject matter.

      A deep understanding of the subject matter... All too often this is missed by educators and students alike. I have known many "crammers" that think learning is only about passing tests. I have also known teachers that don't have a deep understanding of what they are teaching, which is why they teach to the test. If a teacher does not have a deep understanding of the material, how can they express the passion necessary for student to take interest? The issue is detecting and correcting this behavior on both sides.

      I am aware that one of the biggest issues teachers have with standardized testing is that the scores are directly tied to the evaluation process. In truth I am on the fence about this. While I agree we need some way of sifting the good teachers from the bad teachers, I don't believe this is it. There are simply too many variables in the equation that has nothing to do with the individual teachers. So whats the solution?

      (Actually I do have what I think to be a fair system for this, if your interested.)

    3. Couldn't agree more about the lack of "deep understanding" of subject matter in schools! A recent blog post on students & teachers not understanding the big picture and not seeing how lessons are connected to create that deeper understanding: http://dangerouslyirrelevant.org/2013/09/why-are-you-working-on-that.html

      I would love to read about the fair evaluation system you have!

  3. The lobby for standardized assessment has a lot of power behind it, and a lot of popular support from parents and legislators (IMO). I believe that although there is a huge movement towards quantifying student learning through the use of standardized measures, as teachers we still have the opportunity to engage students in learning that is meaningful and authentic to them.

    It can be an appealing notion to "teach to the test." Especially with the push from some members of the public toward remuneration based on our students' results. This scares me: the motivation to teach based on how students will be assessed will likely skyrocket if this actually happens.

    But standardized testing does not preclude student-based learning. Skills that students need to achieve on standardized tests CAN be taught through an approach that focuses on authentic learning experiences. In my opinion, it may take more time, and it will likely take a lot of effort on the part of the teacher, but it can be done. We as teachers need to stay focused on the values of education that are personally important to us: getting students to synthesize information, not just regurgitate it. When we design learning with this goal in mind, I believe that students will be able to adapt their understanding to new problems and new contexts, whether they appear on a standardized test or as a part of their lives later on.

    Teaching to the test will not achieve the same results for our students. At least, not according to the data I've seen.

    1. "Especially with the push from some members of the public toward remuneration based on our students' results. This scares me: the motivation to teach based on how students will be assessed will likely skyrocket if this actually happens."

      This IS happening on two levels. One, teachers' evaluations are tied to high stake test scores. Two, school districts are graded on their "performance" and compared to other districts by using this data. Many administrators feel forced to "teach to the test" so that their school will score well on their "school-wide report card" and teachers feel the need to "teach to the test" so that their evaluations turn out well.

      Again, unintended consequences of DDI. I do agree that teachers CAN teach students to use higher order skills AND their students will still score well on the tests, without resorting to teaching to the tests. the problem is that these teachers are labeled rogue teachers. It is the "hard" way to teach. It should be the other way around. Our education system should promote this type of teaching, not discourage it.


Subscribe to RSS Feed Follow us on Twitter!