Tag Archives: STEM

Hold Content Accountable Too: a scalable Method

This post originally was published on Tom VanderArk’s “VanderArk on Innovation” blog on Edweek. It was also published on GettingSmart. The following is an edited version.

Specific programs and content, not just teachers and ‘teacher quality’, must be held accountable for student outcomes. A recent study published by WestEd shows how, given certain program conditions, cost-effective and rigorous student test score evaluations of a digitally-facilitated program can now be pursued, annually, at any time in any state.

Historically, the glare of the student results spotlight has been so intensely focused on teachers alone, that the programs and content ‘provided’ to teachers have often not even been recorded. Making the case for the vital importance of paying attention is this scathing white paperChoosing Blindly, Instructional Materials, Teacher Effectiveness, and the Common Core, from the Brown Center on Educational Policy’s Matthew Chingos and Grover Whitehurst.  The good news is: digital programs operated by education publishers for schools organically generate a record of where and when they were used.

Today’s diversity of choices in digital content – choices about scope, vehicles, approaches & instructional design – is far greater than the past’s teacher-selection-committee picking among “Big 3” publishers’ textbook series. This wide variety means content can no longer be viewed as a commodity; as if it were merely a choice among brands of gasoline. Some of this new content may run smoothly in your educational system, yet some may sputter and stall, while others may achieve substantially more than normal mileage or power.

It is important to take advantage of this diversity, important to search for more powerful content. The status quo has not been able to deliver results improvements in a timely manner at scale. And spearheaded by goals embodied in the Common Core, we are targeting  much deeper student understanding, while retaining last decade’s goals of demonstrably reaching all students. In this pursuit, year after year, the teachers and students stay the same. What can change are the content and programs they use; ‘programs’ including the formal training programs we provide to our teachers.

But how do you tell what works? This has been extremely challenging in the education field, due in equal measures to a likely lack of programs that do work significantly better, to the immense and hard-to-replicate variations in program use and school cultures, and to the high cost, complexity, and delay inherent in conventional rigorous, experimental evaluations.

But. There is a cost-effective, universally applicable way for a large swath of content or programs to be rigorously evaluated: do they add value vs. business-as-usual. The method is straightforward, requires no pre-planning, can be applied in arrears, and is replicable across years, states, and program-types. It can cover every school in a state, thus taking into account all real-world variability, and it’s seamless across districts, aggregating up to hundreds of schools.

To be evaluated via this method, the program must be:

  1. able to generate digital records of where/when/how-much it was used at a grade
  2. in a grade-level and subject (e.g. 3-8 math) that posts public grade-average test scores
  3. a full curriculum program (so that summative assessments are valid)
  4. in use at 100% of the classrooms/teachers in each grade (so that grade-average assessment numbers are valid)
  5. new to the grade (i.e. evaluating the first one or two years of use)
  6. adopted at sufficient “n” within a state (e.g. a cohort of ~25 or more school sites)

Every program, in every state, every year, that meets the above criteria can be studied, whether for the first time or to validate continuing effectiveness. The data is waiting in state and NCES research files to be used, in conjunction with publisher records of school/grade program usage. This example illustrates a quasi-experimental study to high standards of rigor.

It may be too early for this level of accountability to be palatable for many programs just yet. Showing robust, positive results requires the program itself be actually capable of generating differential program efficacy. And of course some program outcomes are not measured via standardized test scores. There will be many findings of small effect sizes, many implementations which fail, and much failure to show statistical significance. External factors may confound the findings. Program publishers would need to report out failures as well as successes. But the alternative is to continue in ignorance, rely only on peer word-of-mouth recommendations, or make do with a handful of small ‘gold-standard’ studies on limited contexts.

The potential to start applying this method now for many programs exists. Annual content evaluations can become a market norm, giving content an annual seat at the accountability table alongside teachers, and stimulating competition to improve content and its implementation.

 

Advertisements
Tagged , , , , , , ,

The Digital Learning Revolution is not Glossy. (Or LTE.)

The Digital Learning Revolution Will Not Be Glossy. Or LTE.

First posted on Sums&Solutions blog.
Part one of a multi-part series

The true Digital Learning Revolution has not yet arrived. If you go into a classroom and see every student with an iPad on wifi, full 1:1, you are not necessarily seeing a Digital Learning Revolution. Counting what type and how glossy and how many are the digital devices is not how you tell.

Because the Digital Learning Revolution is not about digitizing conventional learning. Nor even about increasing access.

It’s not about digitized problem sets – even if they are gamified. Not even if the problems are scored instantly; nor even if the problem sequence can be varied based on responses (aka “adaptive learning”). Textbook-like problems presented digitally, no matter how entertainingly wrapped in back-story, music, interesting side-bar links, procedural hints and immersive 3-D exploration, are still just this: use previously memorized patterns and procedures to get THE right answer.

It’s not about digitized asynchronous lectures. By their nature they are not interactive. They are passive. Yes, even if talking heads and filmed overhead grease pen scrawls have moved from VHS-access in the 70’s to YouTube-access 40 years later, lectures are not the Digital Learning Revolution.

And it’s especially not about the advent of the latest digital hardware vehicles. Tsunamis of digital hardware have washed into many classrooms, many times. From Apple IIe’s in the 1980’s to Apple iPad II’s in the teens. With interactive whiteboards somewhere in between. First off, the change in how most subjects were taught day to day was minimal. Worse, it did not become the “new normal” for students or teachers to even just use them day to day. There was no killer app. No deep penetration. No Digital Learning Revolution – yet.

Of course, revolutionizing the learning itself depends on the content IN the digital vehicles, a point powerfully made in this excellent white paperChoosing Blindly, Instructional Materials, Teacher Effectiveness, and the Common Core, from the Brown Center on Educational Policy’s Chingos and Whitehurst.  And if that content is just a digitization of the conventional, then no matter how glossy and retina-resolution the screen, no matter how anywhere or anytime or speedy the access, the learning will still be “conventional” learning. By the way, how well has a focus on conventional learning, a focus where the content is considered a commodity, done over the last four decades?

Note moreover, that a narrow view of digital content + student, without taking into account the teacher’s interaction with new content and a new learning process, is also not the Digital Learning Revolution. Because as Chingos and Whitehurst point out, the Digital Learning Revolution occurs at the intersection of the student, and the content, and the teacher. So new digital vehicles, even conveying radically different content (such as interactive videogames), or, rather, especially when carrying radically different content, will not achieve the Digital Learning Revolution … without a comprehensive re-tooling of teacher understandings, processes, and goals.

Beyond Hardware

What about the other major digital game-changer of the 21st Century, you say – what about digitized access? Searchable access to the world’s libraries of content? Anywhere anytime access to the cloud through cheap personal hand-held devices?

You are a participant in that access revolution. So, look around you, what is your experience? Have you experienced, or seen a Digital Learning Revolution? A communication revolution to be sure – connectivity is off the charts. And it’s certainly a revolution in “find something, cut, and paste”. A plethora of small, disconnected written nuggets delivering instant gratification for quick trivia questions. Consumption from the cloud is off the charts. But, when you are looking for depth, you have not yet seen a revolution of learning. As I blogged here re speed v. depth, and here re googling.

The digital access revolution did not bring the Digital Learning Revolution along for the ride.

Again, the key is content. And that a Learning Revolution must involve three interacting components: student, content, and teacher. As I blogged here re blended learning. A Learning Revolution requires the teacher for social, evaluative, motivational, and yes, human communication. The Digital Learning Revolution will require humans. The best sort of humans: teachers who help others grow and improve

In the next installment: well anyhow, we should expect digital content for free, right?

Tagged , , , , , ,

“There’s no achievement gap in videogames” – Quentin Lawson

I don’t want to learn how to play most videogames. By videogames I am thinking involved console games like Call of Duty, or MLB the Show. As a 50-something, that may not be surprising as I’m past the “shoot-em-up” or “race-car” ages (well maybe not real race cars). But truthfully, I would enjoy being able to give my my teenage sons a decent playing partner. The thing is, I know there would be a long and challenging learning curve. Because the learning is discovery/exploratory. And it’s not trivial or short, there is a lot to pick up. For me, it would be both mentally challenging and take significant amounts of time. And I already feel I have enough mental challenge-per-week to sink a battleship. I’m not looking for more. And I can’t afford to have significantly more time, or energy, sucked out of my days.

My point is: learning a console videogame like these is not easy, it takes focus, it takes effort, it takes mental agility, it takes perseverance, it takes time. Sorta like learning anything complex.

And here’s the point of this blog post: it would be ludicrous for anyone to propose that there’s an achievement gap for children to learn videogames. It would also be ludicrous to say that there is an engagement issue – at least for males with the games I mentioned. And it would be beyond incredible to say that kids have a lack of perseverance at solving the game’s problem scenarios.

I attribute this observation to Quentin Lawson, Executive Director of NABSE, the National Alliance of Black School Educators. I was demo’ing for him how all math concepts could be introduced a visual puzzles on a computer, which could be interacted with and animated to understand how to, for example, add fractions. He saw how this was like a videogame and, with young Black male students in mind, noted in an offhand way that “there’s no achievement gap in videogames,” so this could level the playing field. I have been quoting Quentin ever since.

Because how could anyone imagine that success for any child in learning any videogame could depend on:

  • their parents’ education level
  • their parents’ wealth
  • their neighborhood
  • the quality of their friends
  • how much their parents could “tutor” them on the game
  • their own success in school so far
  • the language they speak at home
  • or any other “subgroup” factor

It would be ludicrous; at the least I can’t imagine any such attributes being used by anyone as excuses why children couldn’t win at the game.

So, if productively engaging with challenging core content, like algebra, in a deep and mathematically rigorous way, that requires learner interaction and experiential learning, that starts easy and is gradually scaffolded, that develops problem-solving, perseverance, and confidence in ability to “win”, can be made into a videogame-like experience, then teachers can build-upon, cement and interconnect that mode of learning into deeper understanding and skills, without concern for any digital content achievement gap.

Tagged , , , , , , , , , , , ,

Ixnay the Disruption: A Least Disruption Strategy to Scale Fast in Schools

Q: Which education startup (company with less than $10MM in funding / revenue) will be the most disruptive in the coming decade and why? (Quora.com post)
A: My vote for Least disruptive provider, and why least disruption is a strategy to Scale Fast in schools (disclosure: the shop I work at): MIND Research Institute. Least disruption: any district/site/teachers/students can weave a revolutionary program easily into the same pedestrian way they do business now (including videogaming by students). In other words, a program that can technically scale nationwide in a few years – governed only by district uptake.
Leverage point: the instructional materials teachers and students use. Instructional materials (old-school: textbooks) are teachers’ tools to help students achieve their objectives. Provide a vastly more powerful, highly engineered, modern tool (picture: swap out hand-saws for electric chain-saws at the urban tree farm) along with a systems-process for integrating and using that tool (picture: don’t forget recharging stations, safety training & goggles, saw maintenance). A revolutionary tool can:
  • be simple for teachers to learn to use, and, can deepen teacher’s content knowledge as they use it,
  • be universally accessible by and productive for any student;
  • enable immersively interactive experience-based learning (consider: what is happening at the moment the student is learning and how interactive is that moment),
  • ensure that students can not just cram and memorize meaningless, disconnected fact-trivia or monkey-typing-procedures.

So, given minimized system disruption, but the addition of an easy-to-use, powerful tool and simple process changes, the bulk of a slow-to-change market can change quickly. MIND Research makes supplemental K-pre-algebra digital math content for 1:1 and teacher direct instruction in schools, delivered thru workstation/tablet/whiteboard. The digital content is visual (no language at first) interactive animated math puzzles, videogames as courseware

Tagged , , , , , , , , ,
%d bloggers like this: