Bureaucrats shouldn't be your audience for measurement and accountability
They will come asking for TPS Reports. It’s our job to kindly ignore them.
Have you ever found yourself saying, “you know, the one thing we need in high school sports are more third party, standardized reporting rubrics to prove their effectiveness?” Me either. But the following quote comes from a recent report on strategies to “update the traditional high school sports model”:
If sports are so great, prove it. Step one is defining athletic program standards that a school can be evaluated against, ideally by a third party. Step two is documenting how sports experiences contributed to the educational and health outcomes of students, through exit surveys and other tactics. Step three is presenting the impact of programs in a form that can compete with other worthy causes.
Please, don’t do this.
While I emphatically disagree with this approach, I agree with the report’s assertion that this is being driven by “[c]orporations, foundations, philanthropists, and government [who] all recognize the need to address the major problems of our time… But they need the evidence of positive outcomes at scale to invest.”
It’s actually refreshing to see the bluntness with how this is presented. There are no platitudes about this effort being “for the children” or generalities about the moral need to measure sports. Bureaucrats want it, so the report argues it should be given to them.
Yes, the tail is wagging the dog — but should it be that way?
This is an all-too-common problem in education. Talk to any edupreneur who dares to try a different approach to education. One of the biggest barriers you will hear is their pressure to adopt conventional testing measures to prove their value to bureaucrats, college admissions officers, and others whose frame of reference is the type of school that the educator is trying to create an alternative to for families.
If all the options feel the same, what’s the point of options?
For instance, a few years ago a paper was published titled, Three Signs That a Proposed Charter School Is at Risk of Failing. The researchers asked, “what if we could predict which schools are likely not to succeed—before they’ve even opened their doors?” After reviewing more than 600 charter applications, the researchers determined that one of the top three warning signs for a potential charter was “A child-centered curriculum: Charter applications that propose to deploy child-centered, inquiry-based pedagogies, such as Montessori, Waldorf, Paideia, or experiential programs.”
Are Montessori schools failing students, or are they not easily boxed in by the metrics that charter authorizers are looking for? This is the fundamental question driving the flaws in our approach to accountability in America. I imagine many of the parents of students in the estimated 3,000 Montessori schools across the U.S. would be shocked to learn that failing students is baked into the Montessori approach. As the report acknowledges, “Many of these child-centered schools aren’t ‘failing’ in the eyes of their customers.”
The problem is we’ve lost sight of who the audience is.
The Manhattan Institute recently commissioned three reports “to understand how state policy is affecting microschooling,” with profiles of activities in Idaho, Arizona, and New York. In the summary essay written by Andy Smarick, I provided this reflection:
Adam Peshek, a senior fellow at Stand Together Trust and an expert in state-level education policy—particularly the policies related to school choice—highlighted the opportunities and challenges related to government involvement in the small-school movement. He compared today’s surge in microschooling with the early, innovative days of charter schooling (prior to today’s more regulated charter-school sector), when educational entrepreneurs with new ideas found a way to create new schools. If funding weren’t an issue, microschooling would benefit from staying in the private sector, where flexibility is greatest—meaning that school founders and leaders are empowered to make decisions related to school size, schedules, age spans, and curriculum. Many things that today’s microschool founders hope to accomplish do not fit neatly into statutes and regulations crafted years ago. Although currently, because of the pandemic, some public systems have experimented with pods and small-school environments, they have done it out of necessity, as a stopgap measure. Because of various rules and parents’ expectations, schools will revert to traditional ways after the pandemic.
I’m not saying this because I have some deep seated aversion to charter schools. In fact, I think they are a great option (maybe the best option) for many, many families. But we would be kidding ourselves if we portrayed the sector as the laboratory of innovation that many envisioned it would become decades ago.
Today, if you’re a scruffy-haired 20-something educator with a passion to try something new – like many early founders of today’s largest charter orgs were – the idea of getting the capital, legal support, political acumen, marketing, and everything else needed to launch a new charter school is overwhelmingly daunting. Even if you can, the effort represents such an absurd level of administrative work that it’s hard to see how the effort wouldn’t be better directed at actually serving kids.
There are new models cropping up all over the country — pods, hubs, microschools, hybrid homeschools, and many more. We cannot repeat the mistakes of the past if we hope to continue to see the innovation we’ve witnessed in recent years.
They will come asking for TPS Reports. It’s our job to kindly ignore them.
Would it be fair to say that the central problem with the reporting you are railing against is that they are vanity metrics? Where vanity metrics are those that fail to reflect anything of real value. The poster child for vanity metrics is the old "hit counter" on a web site. There is nothing useful about that information, though its makes the naive webmaster feel good.
As opposed to actionable metrics which are central to the most important outcomes that are being sought. When a metric is actionable there are clear and specific actions that can be taken to improve the case of underperformance. In the case of sports there are clear and specific actions that will lead to an increase in the score of a particular team or player.
I presume that you would be good with actionable metrics if they were available for education, right?
Enjoy,
Don Berg
Deeper Learning Advocates