Monthly Archives: December 2011

If something Actually Worked in Education… 4 things to Look For

“Education” is a huge conversation, but almost all of the conversation is about the education problem. There is not a serious or mature conversation about solutions.

Meanwhile lots of resources and, as important, attention, are going into an ever-changing, wide variety of programs which, it is hoped, will be part of some solution.  How do we know when we’re looking at any given program, whether it can have game-changing impact on K-12 schools, or has no chance? I call the former, programs that “Actually Work”.

Here I identify and describe four characteristics of any program that Actually Works: Scope, Results, Robustness, and Scalability. Failure at any one of these is failure to Work.

1 – Scope

A program that Actually Works must cover or affect an entire course curriculum. For example, let’s think of mathematics. A program that Actually Works can be as small as one grade level, or one course. But the scope can’t be less than that — it can’t be just a specific concept — because you need to be able to Tell if something Actually Works, and that takes an end-of-course assessment. And what Actually Works needs to help educators and students with what they are held accountable to: summative assessments. So, in mathematics, a cool interactive visualization of parabola parameters on a graphing calculator doesn’t meet the Scope requirement.

2 – Results

To Actually Work, a program needs Positive Results. As stated above in Scope, the results need to be measured using a summative assessment. The Positive Results need to be essentially this: for educators and teachers using the program, all the students become Proficient in the given subject area. I’m willing to accept Proficient as defined by states as meeting state standards on a state standardized test, and “all” as 90%+, but a more general way to define Proficient is: functionally competent to create work product, or solve problems. So, do these Positive Results need to pass a clinical, random-assignment gold-standard evaluation to scientifically attempt to identify the Cause? No. For a program that Actually Works, the results will speak for themselves, robustly at scale in any real world setting, and there need be no argument about what caused the results. Empirically, on implementing the program, you get the Positive Results. In a wonderfully luxurious later phase of this education conversation, when multiple programs Actually Work, we can fret about what we don’t yet know. We can fret over whether extra hours made the difference, or what component came from human or material or process components of the program, or motivation, or factors external to a school. We can fret over whether one program that Actually Works has been properly compared to another program that Actually Works. And we can fret over exactly how do we evaluate the answers to those questions. But the breakthrough step is to have “one” exemplar program that Actually Works.

3 – Robustness

If a program Actually Works, it will work for any educator and any student.  A program that only works for some educators — for the most motivated, or the most experienced, or the most skilled — does not Actually Work. A program that Actually Works needs to work for a first-year teacher with low confidence, right? I mean if it doesn’t work in that situation, then we have a Big Problem, right? And a program that Actually Works has to work with every student subgroup. In the area of math, for example, that means that it works for students who are multiple grades below grade-level. It works for students who have been testing low. It works for students who don’t get much help at home on math. It works for students who have low confidence in their math abilities. It works for students who are suffering from low motivation to do well. And it works for students who are low in language arts proficiency, and students who are learning English.

And the educator or student, for whom the program Actually Works, can be in any school, any school district, in any city and in any calendar year. In other words, Positive Results should be replicable across geography and time.

4 – Scalability

If a program that Actually Works can’t Scale, then it is of no use. By Scale I mean reach millions of students. And there are three aspects of scaling: Implementability, Economics, and Speed.

First Scalability Aspect: Implementability. The program needs to be able to be implemented across the dominant education model. In other words, it must fit within the existing structure of facilities, people, and time. The program should *not* then be requiring some additional changes, such as re-tooling school buildings; re-tooling school hardware; re-tooling school personnel; requiring home technology, or changing the time or location of education. Another consideration for scalability is that the program must not only be do-able by teachers, but also be embraced by teachers. Anything less than a full embrace by teachers will result in failure, failure to achieve Positive Results, either from outright rejection of the program, or by spotty or sub-par implementation of the program.

Second Scalability Aspect: Economics. The program needs to fit within current school economics. This means both for initialization of the program as well as for sustaining the program. For start-up, there is not only the program outright cost but also the costs for site facilitization (see above) and especially in teacher time for training. Outright cost needs to be in the neighborhood of current instructional materials costs per student. Facilitization costs need to be near zero. And teacher training time costs need to also be near zero, as PD days and substitute days become artifacts of the past. Sustaining costs (e.g. annual renewals) need to be low enough to survive tough priority battles during even the toughest budget years (see above: teachers must embrace and highly value any program, for it to be Scalable).

Third Scalability Aspect: Speed. If there is a program that Actually Works with all of the above, to be worthwhile it must also have the characteristic of quick scale-up. By quick I mean that it must have the capability, if the market demands it, of adding millions of students and their teachers per year to the program. In a country with 49 million students and 3.5 million teachers in K-12, anything less than this means we have no solution at all.


My Premise: Learning is Sub-Optimal by a Factor of 1000x

Here is my basic premise, and driver: it is possible for the value of global human capital – that is, people problem-solvers and wealth-generators – to be increased by orders of magnitude.

The posts in this blog are motivated by this belief, and will explore it and illustrate it using dominantly my personal experience in context of the work at MIND Research Institute.

The billions of people on this planet are the solution to their problems, not the problem itself. Solutions come through people learning knowledge and skills which they can then creatively apply to problems. And learning can be:

  • 10 Times Faster
  • 10 Times Cheaper – for 10 Times more Reach
  • 10 Times More Effective, Deeper, and Better-Retained

I said people as “wealth” generators, and I mean wealth in the broadest terms: yes money, but also jobs creation, personal fulfillment, creative arts, free time, community health, personal happiness and satisfaction in the family and workplace.

By Faster I mean: reducing the number of minutes-on-task, and of calendar months, to reach a given performance level.

By Cheaper I mean: reducing Dollars per Learning.

By More Effective I mean: improving success rates for those who commence learning anything.

By Deeper I mean: gaining a substantial conceptual framework, and skills in recognizing patterns, which enable solving of non-standard, sophisticated, tricky problems. This is not a typical goal. Here’s a counter-example: a deep understanding of history is not the Jeopardy-winning ability to extract or match factoids of date, place, and name quickly out of one’s brain (note: a computer program called Watson recently beat the best human Jeopardy champions — of course).

By Better Retention I mean: retain beyond the formal assessment period. Whatever you don’t retain for effective re-use over the long-term (years), was a waste of time and resources to learn, and a rationalization of being able to re-learn it more quickly is extremely weak to the point of useless. Comedy show Saturday Night Live’s character Father Guido Sarducci makes this point, hilariously in “The Five Minute University.”

Tagged ,

Higher Ed Cost: No Redline on the Tach

Recent California Higher Ed Headlines:

Orange County Register Nov. 23– Pay for retired CSU professors seen as ‘double dip’: Retirement has been good for Cal State Fullerton Professor xxx. As a full-time professor of mechanical engineering during 2008-09, xxx made $106,000 for teaching eight courses. Last year, after retiring, he made $146,600 – for teaching four classes at the same school.

L.A. Times Dec. 2 – UC regents approve pay hikes for 12 staffers: The administrators and attorneys received raises of between 6.4% and 23%. The action renews debate about efforts to retain important talent as the university considers further fee increases. UC President Mark Yudof said the raises were a necessity to keep good employees on board.

I made a crucial financial mistake early in my career – of not going directly into the secure embrace of government employment, where unlike in the private sector, there is no bottom-line governor on compensation and benefits. I could have had early retirement in my 50’s, lifetime health, accrued vacation/sick day payouts, and a guaranteed lifetime 6-figure pension (if I made it to the Exec level) equivalent to having a multi-million dollar nest egg.  Instead I’m currently leading a non-profit with an unmatched 401k plan. Quality people will work for less than what is offered in government jobs. Look at the number of applicants per job opening for public safety jobs. But the market does not govern government compensation.
I believe government subsidy of higher education costs for students who graduate and join the workforce, getting higher-paying jobs thanks to their subsidized college training, is a great social investment in multiple ways. But taxpayers can’t afford to annually chase absolutely unsustainable increases in cost, exemplified by stories like the above. The cost of higher ed is inexorably rising to a ceiling: the maximum total sum that the payers (government subsidy and parents and student loans) are evidently willing to pay — apparently a rather high number. Why wouldn’t UC fees for parents eventually rise to a private-school-like $25k-$40k per year? No reason; this is predictably where we are heading in California. At which point affordability for the middle class is laughable, and we are really screwed as a society.
As a beneficiary of a subsidized ($702/year) UC education in a more economically sound era, I believe that government subsidies to public higher ed should continue, that they should ensure that a college education is extremely affordable, but that they should come with thick strings attached on cost. I don’t believe that a quality undergraduate education can not be delivered at lower cost than now. The cost to the taxpayer, parent, and student should have a ceiling, indexed to grow no faster than inflation, reducing uncertainty. Universities could choose to take or leave this deal, and then figure out how to live within a fixed budget. Perhaps UCLA and UC Berkeley choose to forego subsidies — the private route — while many other campuses take this deal. These are smart people, and delivering a quality undergraduate education on a budget is not a moon shot.

%d bloggers like this: