Tag Archives: common core

Hold Content Accountable Too: a scalable Method

This post originally was published on Tom VanderArk’s “VanderArk on Innovation” blog on Edweek. It was also published on GettingSmart. The following is an edited version.

Specific programs and content, not just teachers and ‘teacher quality’, must be held accountable for student outcomes. A recent study published by WestEd shows how, given certain program conditions, cost-effective and rigorous student test score evaluations of a digitally-facilitated program can now be pursued, annually, at any time in any state.

Historically, the glare of the student results spotlight has been so intensely focused on teachers alone, that the programs and content ‘provided’ to teachers have often not even been recorded. Making the case for the vital importance of paying attention is this scathing white paperChoosing Blindly, Instructional Materials, Teacher Effectiveness, and the Common Core, from the Brown Center on Educational Policy’s Matthew Chingos and Grover Whitehurst.  The good news is: digital programs operated by education publishers for schools organically generate a record of where and when they were used.

Today’s diversity of choices in digital content – choices about scope, vehicles, approaches & instructional design – is far greater than the past’s teacher-selection-committee picking among “Big 3” publishers’ textbook series. This wide variety means content can no longer be viewed as a commodity; as if it were merely a choice among brands of gasoline. Some of this new content may run smoothly in your educational system, yet some may sputter and stall, while others may achieve substantially more than normal mileage or power.

It is important to take advantage of this diversity, important to search for more powerful content. The status quo has not been able to deliver results improvements in a timely manner at scale. And spearheaded by goals embodied in the Common Core, we are targeting  much deeper student understanding, while retaining last decade’s goals of demonstrably reaching all students. In this pursuit, year after year, the teachers and students stay the same. What can change are the content and programs they use; ‘programs’ including the formal training programs we provide to our teachers.

But how do you tell what works? This has been extremely challenging in the education field, due in equal measures to a likely lack of programs that do work significantly better, to the immense and hard-to-replicate variations in program use and school cultures, and to the high cost, complexity, and delay inherent in conventional rigorous, experimental evaluations.

But. There is a cost-effective, universally applicable way for a large swath of content or programs to be rigorously evaluated: do they add value vs. business-as-usual. The method is straightforward, requires no pre-planning, can be applied in arrears, and is replicable across years, states, and program-types. It can cover every school in a state, thus taking into account all real-world variability, and it’s seamless across districts, aggregating up to hundreds of schools.

To be evaluated via this method, the program must be:

  1. able to generate digital records of where/when/how-much it was used at a grade
  2. in a grade-level and subject (e.g. 3-8 math) that posts public grade-average test scores
  3. a full curriculum program (so that summative assessments are valid)
  4. in use at 100% of the classrooms/teachers in each grade (so that grade-average assessment numbers are valid)
  5. new to the grade (i.e. evaluating the first one or two years of use)
  6. adopted at sufficient “n” within a state (e.g. a cohort of ~25 or more school sites)

Every program, in every state, every year, that meets the above criteria can be studied, whether for the first time or to validate continuing effectiveness. The data is waiting in state and NCES research files to be used, in conjunction with publisher records of school/grade program usage. This example illustrates a quasi-experimental study to high standards of rigor.

It may be too early for this level of accountability to be palatable for many programs just yet. Showing robust, positive results requires the program itself be actually capable of generating differential program efficacy. And of course some program outcomes are not measured via standardized test scores. There will be many findings of small effect sizes, many implementations which fail, and much failure to show statistical significance. External factors may confound the findings. Program publishers would need to report out failures as well as successes. But the alternative is to continue in ignorance, rely only on peer word-of-mouth recommendations, or make do with a handful of small ‘gold-standard’ studies on limited contexts.

The potential to start applying this method now for many programs exists. Annual content evaluations can become a market norm, giving content an annual seat at the accountability table alongside teachers, and stimulating competition to improve content and its implementation.


Tagged , , , , , , ,

Future Vision 2025 Parents: the “Same” is no longer good enough

What future are we aiming at? This series of 6 posts, Future Vision 2025, describes some of my personal education mission milestones. These are not predictions, they are aspirational. They are framed as significant differences one could see or make by 2025. What’s noticeably different in 2025 when one examines students, parents, teachers, learning, assessment, media & society? How and when these milestones are reached are not addressed. Some milestones are indicated by the emergence of something ‘new’ (at least at robust scale), others by the fading away of something familiar and comfortable.

Parents 2025

Parents see that 2025’s learning environments for their children are different, and superior, to what they went through as students themselves – just as they expect to see that 2025’s medical options are different and superior to what was available 30 or more years previously.

Parents are noticing positive results for their children, including a background of more general satisfaction with school and learning, and higher confidence in their ability to learn challenging content – presented by teachers with new tools and in new ways.

They are even beginning to expect improvements in schooling. Rather than dig in their heels resisting classroom changes, insisting on schooling “the way I know it worked just fine for me,” parents are starting to believe in progress – because they finally see it working in their own family. They can see and hear their children apply more of their school day “book-learning” to their every day “real world”. Parents notice that for their children, school learning is making more sense, in ways it never did for the parent.

Some parents would even be upset if they found out that their child was receiving ‘only’ the good old familiar and comfortable lecture & practice that the parent remembers from their own schooling. Because they know it can and should be better for their child. Nevertheless, the inertia and familiarity of the same-old same-old keeps it the safe, default choice for education decision-makers.

So, grass roots movements have formed among highly-involved parents of school children to encourage and support teachers through the adoption and quality implementation of proven, though unfamiliar, techniques and tools – in stark contrast to the last many decades of those same highly-involved parents’ rigid adherence to the status quo, based on their conviction that whatever is new could only be degrading learning for their children.

Tagged , , ,

Common Core meme provocation and response – 9 ways to Go Faster!

The “Angry Mother Destroys Common Core” internet meme got my goat when I browsed the comments section. I grant that the actual meme problem was overbearing (it’s the “write a letter to Jack”  about subtraction error using the number line). But in the comments I saw a lot of complaints about how the number line approach was not the fastest way to the answer, and thus a confusing waste. Not those commenters’ faults in a way; they are just sticking with the one-right-way paradigm about math they were taught back in the day.

So I yielded to temptation and uploaded a response comment on the meme’s thread, sharing here for (I hope) your amusement:

Apparently to many folks, the Fastest way to get to The Answer is the Point of math. Here are 9 ways to go faster:

1) Memorize lots of formulas – as many as you can.
2) Once it seems to you like one of your formulas can be applied, run it.
3) Even for addition, use a calculator (We all have one in our phones, sweet!)
4) Download a powerful calculator that has lots of formulas pre-coded for you to just punch in a few #’s. Less memorizing!
5) Have your mom do the problem for you (that’s faster, right?)
6) Have your mom just give you the answer (now we’re getting fast!)
7) Have your smartphone solve it (it knows more than Mom!) http://www.nydailynews.com/…/new-photomath-app…
8) Don’t answer the question – heck what difference does it make if you answer someone else’s canned math question anyway?
9) Don’t even bother taking a math class and save yourself years of grief – other people know how to do the math that you can’t do with a calculator or app anyhow, right? And aren’t there lots of people to tell you they never broke out a binomial or even an ‘x’ in their worklife, ever, so what a waste?

Oh, and now that speed has been served, please enjoy your trip into the real world after school, where people get good pay if they can solve real problems via pattern recognition, efficient problem abstractions, and a myriad of hierarchical solution methods, using logical thinking. Within a few months of hiring on, managers know who can solve problems and who is weak. Calculators don’t cut it.

Oh yeah one more thing: in the real world there is no 1 right answer in the back of the book.

My kids will see your kids out in the world competing for jobs. Good luck to them all!

Tagged , ,

The Digital Learning Revolution is not Glossy. (Or LTE.)

The Digital Learning Revolution Will Not Be Glossy. Or LTE.

First posted on Sums&Solutions blog.
Part one of a multi-part series

The true Digital Learning Revolution has not yet arrived. If you go into a classroom and see every student with an iPad on wifi, full 1:1, you are not necessarily seeing a Digital Learning Revolution. Counting what type and how glossy and how many are the digital devices is not how you tell.

Because the Digital Learning Revolution is not about digitizing conventional learning. Nor even about increasing access.

It’s not about digitized problem sets – even if they are gamified. Not even if the problems are scored instantly; nor even if the problem sequence can be varied based on responses (aka “adaptive learning”). Textbook-like problems presented digitally, no matter how entertainingly wrapped in back-story, music, interesting side-bar links, procedural hints and immersive 3-D exploration, are still just this: use previously memorized patterns and procedures to get THE right answer.

It’s not about digitized asynchronous lectures. By their nature they are not interactive. They are passive. Yes, even if talking heads and filmed overhead grease pen scrawls have moved from VHS-access in the 70’s to YouTube-access 40 years later, lectures are not the Digital Learning Revolution.

And it’s especially not about the advent of the latest digital hardware vehicles. Tsunamis of digital hardware have washed into many classrooms, many times. From Apple IIe’s in the 1980’s to Apple iPad II’s in the teens. With interactive whiteboards somewhere in between. First off, the change in how most subjects were taught day to day was minimal. Worse, it did not become the “new normal” for students or teachers to even just use them day to day. There was no killer app. No deep penetration. No Digital Learning Revolution – yet.

Of course, revolutionizing the learning itself depends on the content IN the digital vehicles, a point powerfully made in this excellent white paperChoosing Blindly, Instructional Materials, Teacher Effectiveness, and the Common Core, from the Brown Center on Educational Policy’s Chingos and Whitehurst.  And if that content is just a digitization of the conventional, then no matter how glossy and retina-resolution the screen, no matter how anywhere or anytime or speedy the access, the learning will still be “conventional” learning. By the way, how well has a focus on conventional learning, a focus where the content is considered a commodity, done over the last four decades?

Note moreover, that a narrow view of digital content + student, without taking into account the teacher’s interaction with new content and a new learning process, is also not the Digital Learning Revolution. Because as Chingos and Whitehurst point out, the Digital Learning Revolution occurs at the intersection of the student, and the content, and the teacher. So new digital vehicles, even conveying radically different content (such as interactive videogames), or, rather, especially when carrying radically different content, will not achieve the Digital Learning Revolution … without a comprehensive re-tooling of teacher understandings, processes, and goals.

Beyond Hardware

What about the other major digital game-changer of the 21st Century, you say – what about digitized access? Searchable access to the world’s libraries of content? Anywhere anytime access to the cloud through cheap personal hand-held devices?

You are a participant in that access revolution. So, look around you, what is your experience? Have you experienced, or seen a Digital Learning Revolution? A communication revolution to be sure – connectivity is off the charts. And it’s certainly a revolution in “find something, cut, and paste”. A plethora of small, disconnected written nuggets delivering instant gratification for quick trivia questions. Consumption from the cloud is off the charts. But, when you are looking for depth, you have not yet seen a revolution of learning. As I blogged here re speed v. depth, and here re googling.

The digital access revolution did not bring the Digital Learning Revolution along for the ride.

Again, the key is content. And that a Learning Revolution must involve three interacting components: student, content, and teacher. As I blogged here re blended learning. A Learning Revolution requires the teacher for social, evaluative, motivational, and yes, human communication. The Digital Learning Revolution will require humans. The best sort of humans: teachers who help others grow and improve

In the next installment: well anyhow, we should expect digital content for free, right?

Tagged , , , , , ,

Common Core Pessimists: Consider the Incremental Advance on Information Quality

In response to Rick Hess’ blog “The Common Core Kool-Aid” lamenting the latest silver-bullet ed-reform groupthink. I agree with Rick’s consistent themes that “it’s about the quality” and incremental improvement.

So, from the perspective of my experience in a non-profit math content publisher and ed researcher, a simpler take on CC: improves quality/quantity of widely relevant, widely comparable, statistically valid, actionable information for feedback into improving learning systems nationwide. State-level standards/assessments quality have empirically shown wide variation such that the low quality end has generated suspect information (e.g sky high state proficiency rates vs. NAEP rates). Apparently it’s politically more likely for the standards/assessment system to be gamed at the state, vs. federal, level.

So, not a panacea, nor a catalyst or driver for any specific “reform” agenda, just an incremental but i.m.o. highly significant and valuable improvement on the current state-of-affairs in information quality, transparency, and widespread relevance. Would be wonderful if over time CC turned out to provide a higher quality “floor” under teaching-to-the-test, too.

%d bloggers like this: