Tag Archives: innovation

Hold Content Accountable Too: a scalable Method

This post originally was published on Tom VanderArk’s “VanderArk on Innovation” blog on Edweek. It was also published on GettingSmart. The following is an edited version.

Specific programs and content, not just teachers and ‘teacher quality’, must be held accountable for student outcomes. A recent study published by WestEd shows how, given certain program conditions, cost-effective and rigorous student test score evaluations of a digitally-facilitated program can now be pursued, annually, at any time in any state.

Historically, the glare of the student results spotlight has been so intensely focused on teachers alone, that the programs and content ‘provided’ to teachers have often not even been recorded. Making the case for the vital importance of paying attention is this scathing white paperChoosing Blindly, Instructional Materials, Teacher Effectiveness, and the Common Core, from the Brown Center on Educational Policy’s Matthew Chingos and Grover Whitehurst.  The good news is: digital programs operated by education publishers for schools organically generate a record of where and when they were used.

Today’s diversity of choices in digital content – choices about scope, vehicles, approaches & instructional design – is far greater than the past’s teacher-selection-committee picking among “Big 3” publishers’ textbook series. This wide variety means content can no longer be viewed as a commodity; as if it were merely a choice among brands of gasoline. Some of this new content may run smoothly in your educational system, yet some may sputter and stall, while others may achieve substantially more than normal mileage or power.

It is important to take advantage of this diversity, important to search for more powerful content. The status quo has not been able to deliver results improvements in a timely manner at scale. And spearheaded by goals embodied in the Common Core, we are targeting  much deeper student understanding, while retaining last decade’s goals of demonstrably reaching all students. In this pursuit, year after year, the teachers and students stay the same. What can change are the content and programs they use; ‘programs’ including the formal training programs we provide to our teachers.

But how do you tell what works? This has been extremely challenging in the education field, due in equal measures to a likely lack of programs that do work significantly better, to the immense and hard-to-replicate variations in program use and school cultures, and to the high cost, complexity, and delay inherent in conventional rigorous, experimental evaluations.

But. There is a cost-effective, universally applicable way for a large swath of content or programs to be rigorously evaluated: do they add value vs. business-as-usual. The method is straightforward, requires no pre-planning, can be applied in arrears, and is replicable across years, states, and program-types. It can cover every school in a state, thus taking into account all real-world variability, and it’s seamless across districts, aggregating up to hundreds of schools.

To be evaluated via this method, the program must be:

  1. able to generate digital records of where/when/how-much it was used at a grade
  2. in a grade-level and subject (e.g. 3-8 math) that posts public grade-average test scores
  3. a full curriculum program (so that summative assessments are valid)
  4. in use at 100% of the classrooms/teachers in each grade (so that grade-average assessment numbers are valid)
  5. new to the grade (i.e. evaluating the first one or two years of use)
  6. adopted at sufficient “n” within a state (e.g. a cohort of ~25 or more school sites)

Every program, in every state, every year, that meets the above criteria can be studied, whether for the first time or to validate continuing effectiveness. The data is waiting in state and NCES research files to be used, in conjunction with publisher records of school/grade program usage. This example illustrates a quasi-experimental study to high standards of rigor.

It may be too early for this level of accountability to be palatable for many programs just yet. Showing robust, positive results requires the program itself be actually capable of generating differential program efficacy. And of course some program outcomes are not measured via standardized test scores. There will be many findings of small effect sizes, many implementations which fail, and much failure to show statistical significance. External factors may confound the findings. Program publishers would need to report out failures as well as successes. But the alternative is to continue in ignorance, rely only on peer word-of-mouth recommendations, or make do with a handful of small ‘gold-standard’ studies on limited contexts.

The potential to start applying this method now for many programs exists. Annual content evaluations can become a market norm, giving content an annual seat at the accountability table alongside teachers, and stimulating competition to improve content and its implementation.


Tagged , , , , , , ,

Transforming the Education Market: Look to Non-Profits

This piece was originally published on the Huffington Post on May 3.

The convergence in the last 20 years of advancements in computer, cognitive & neuro-sciences has made game-changing educational programs a possibility. In the area of mathematics education, inventors can now see a path to give teachers powerful yet easy to use, radically different digital tools to get all students to be proficient in math and even algebra. The breakthroughs go beyond math for math’s sake, beyond proficiency on tests, to the fundamental purpose of math education for all students: every person possessing the powerful thinking, analytic and problem-solving skills that mathematical literacy promises. To put a number it, a 2011 Organisation for Economic Co-operation and Development study estimated that even modestly increased student math skills would add over $40 trillion to the U.S. economy over current students’ lifetimes.

But the K-12 education market is a poster child for unhealthy markets. Products are developed to politicized market specifications, which are far below their potential. Digital solutions from the past 30 years have not worked, so expectations are low. Curriculum is treated as a commodity, and not even evaluated. Not only do most educators not seriously expect teacher tools or content to make a game-changing difference, but K-12 purchasers are looking for approaches familiar to what they experienced in school. The market even lacks understanding of the indicators of program quality and effectiveness necessary to meet the market spec of standardized test success, so simple or secondary features hold sway instead.

In this market, I believe the leaders achieving radically higher educational goals are mission-driven, not-for-profit organizations. Non-profits, almost by definition, need support beyond current market forces. Visionary and savvy business social investments, though relatively small change to $1B businesses, can be and should be vital support for continuous invention and prove-out.

Many business leaders are looking to “help education” through their corporate social responsibility strategies. Unfortunately, based on my experience inside a non-profit education researcher and publisher, what most businesspeople will be looking for is just marginal improvement. There is an unconscious acceptance of the familiar, of “inch deep” learning rather than breakthrough deeper learning. There is a focus on increasing speed and reducing cost. There is a lack of appreciation for the vital role of the teacher.

So, for those visionary businesses and foundations interested in accelerating a quantum leap forward for all teachers and students in K-12 education outcomes (their future workforce/customers), I’d like to suggest that the following considerations are crucial:

1)  Look for non-profits pursuing radically different approaches. Digitizing existing content and approaches is just more of the same, even if ported onto a glossy-screened touch-tablet. People are actually inventing new learning tools, content and processes — like inventing powered flight. Real transformation is going to use teaching and learning models that seem and look and feel radically different from how you learned. Search for that.

2) Look for truly scalable approaches. These three questions will help you evaluate a program’s scalability:

i) Are they applicable to all teachers and students, from gifted to struggling to English learners?

ii) Are they aimed at the heart of the problem right now: at all schools and teachers and training processes as-is? Think Los Angeles, New York, Chicago, Houston and D.C. public schools?

Don’t just focus on and invest in the fringes of the school market, which will take years if ever to reach the majority of students in the community, and the nation. Each year, 4 million more children are passing through an untransformed pipeline.

iii) Can they scale up fast and without limit, driven ultimately by non-philanthropic funds? That is, do the economics of the solution enable eventual demand and resources from the main market, i.e. government public schools, to adopt, scale and sustain?

While some non-profits aimed at breakthrough transformation will be in the early stages of research and invention, you can also look for others with solid evidence that their program delivers results, and that there will be market demand and that it is economically scalable.

There are examples of businesses following all of these principles in their social investment in education. Corporate foundations, CEOs and chairs of Cisco, Broadcom, Emulex, Microsemi, PwC, Bank of America, Chevron and others  have come together to support a breakthrough math program using instructional software to tap students’ visual reasoning. You would not recognize math taught this way, it is that different.

The lowest performing elementary schools in Orange County were provided grants to launch this math program. Over 80% of those schools are now participating. A 45,000-student district serving predominantly economically disadvantaged English learners, Santa Ana Unified, went district-wide and closed its “achievement gap” with the California state average. And the targeted schools at a county-wide level are greatly outpacing similar schools in Academic Performance Index growth. The proven results have attracted district funding at over 1,000 additional sites. Over 500,000 students are being served, and scale-up is economical.

For any company interested in helping education, keep an eye out for the pioneering non-profits that fit this profile. Your social investment will then be poised to go beyond “help,” to transform and scale to millions.

Tagged , , , , , ,

Ixnay the Disruption: A Least Disruption Strategy to Scale Fast in Schools

Q: Which education startup (company with less than $10MM in funding / revenue) will be the most disruptive in the coming decade and why? (Quora.com post)
A: My vote for Least disruptive provider, and why least disruption is a strategy to Scale Fast in schools (disclosure: the shop I work at): MIND Research Institute. Least disruption: any district/site/teachers/students can weave a revolutionary program easily into the same pedestrian way they do business now (including videogaming by students). In other words, a program that can technically scale nationwide in a few years – governed only by district uptake.
Leverage point: the instructional materials teachers and students use. Instructional materials (old-school: textbooks) are teachers’ tools to help students achieve their objectives. Provide a vastly more powerful, highly engineered, modern tool (picture: swap out hand-saws for electric chain-saws at the urban tree farm) along with a systems-process for integrating and using that tool (picture: don’t forget recharging stations, safety training & goggles, saw maintenance). A revolutionary tool can:
  • be simple for teachers to learn to use, and, can deepen teacher’s content knowledge as they use it,
  • be universally accessible by and productive for any student;
  • enable immersively interactive experience-based learning (consider: what is happening at the moment the student is learning and how interactive is that moment),
  • ensure that students can not just cram and memorize meaningless, disconnected fact-trivia or monkey-typing-procedures.

So, given minimized system disruption, but the addition of an easy-to-use, powerful tool and simple process changes, the bulk of a slow-to-change market can change quickly. MIND Research makes supplemental K-pre-algebra digital math content for 1:1 and teacher direct instruction in schools, delivered thru workstation/tablet/whiteboard. The digital content is visual (no language at first) interactive animated math puzzles, videogames as courseware

Tagged , , , , , , , , ,

Common Core Unleashes Innovation and Choice in Curriculum

For the last 10 years I’ve been part of a team developing and deploying a highly distinct and innovative visual approach for teaching and learning math via a supplemental software program and accompanying education process. It’s now being studied by an IES grant, and we have seen replicable, significant, positive results at scale. We are convinced that curriculum is not a commodity, that new and innovative instructional design and neuroscience-based learning environments will greatly assist teachers to productively engage and advance all students, and thus that availability and scale-up of new curriculum is vital to improving student outcomes.

From this perspective of a curriculum innovator seeking to scale, the possibility of the U.S. market significantly re-fragmenting after the Common Core “consensus” I find very distressing. Having spent the last 10 years scaling up one-state-at-a-time through wildly different and changing state standards (even in math!) and assessments, the thought of Common Core unraveling translates into more slow slogging to get a new solution deployed and proven.

Yes, the paradox from my view is that common standards and assessments will help bring about a wider choice of innovative curricula (design/approach), because of the reduced barriers of time and cost to scale-up and prove. The inverse, a retreat from Common Core, by any state, will mean less access in that state, or later access, to innovation from any but the biggest brands and most deep-pocketed publishers.

Now, if you are a conscious believer that curriculum is a commodity well, then, who cares. We’ve got plenty of curricula. Some of it free. But consider  if 21st century curriculum (design/approach) turns out to be a major difference between scaling up success or never getting there…

Sure, the whispered predictions of a monolithic “common curriculum”, even if the word “framework” is attached, could also be terrifying to small innovators. But the “common” can and will stop at common assessments. I don’t believe the market, or CCS, or Common Core assessments, will drive a monolithic curriculum. In my view, a single standard, or assessment, does not dampen the enthusiasm to develop curricula. Many curricula can, should, and will be built regardless of whether there is 1 spec or 50, 1 yardstick or 50. Just look at the intrastate variety now; obviously a “single” state standards and assessment does not drive into place a single curriculum. But, again, if you unconsciously believe curricula are interchangeable commodities, then it would seem more likely to you that assessment would determine curriculum.

So: if one is for a larger variety of curriculum (content/approach) choice and a market where the publishers (for-profit or non-profit) are maximally incented to continuously innovate and improve their offerings, then one should favor CCS and inter-state-comparable assessments. Seems like people of most political stripes would agree that a larger market better drives the creation of innovative new products that increase quality and reduce cost.

Why should the U.S. market not aim to attract investment in innovation and new products/programs commensurate to the its size? Why require unnecessarily high costs of development and marketing a new program/product to serve a 50-different-states market? Why would we want to retreat to a future of 50 different and un-normalized yardsticks so that anything that actually works has to prove itself 50 times over?

I think the apparently widely held belief that ed-reform is all about (i.e. on the back of) teachers, and that it doesn’t therefore matter that much what approach or curriculum they are given/choose, is a) wrongheaded and underinformed, b) going to take way too long to get us where we need to get on a national scale, and c) tremendously clouding the logic of this debate.

Curriculum (content/approach) is not a commodity! High quality, rigorous studies are not always into the future doomed to come up with the tired old “no significant effects” conclusion for almost every program. Independent of teacher quality, we also need a market that values and stimulates continuous investment in more/better content and tools for teachers, from which they and their districts are free to choose.

Tagged , , , , ,
%d bloggers like this: