Reposting the CURMUDGUCATION blog:   Segregation, Choice and Education + Grading Good-faith Gibberish

CURMUDGUCATIONThe slightly-cranky voice navigating the world of educational “reform” while trying to still pursue the mission of providing quality education.



Segregation, Choice and Education

Posted: 07 Apr 2016 11:26 AM PDT

The National Education Policy Center just released a research report from William J. Mathis and Kevin G. Welner on the question, Do Choice Policies Segregate Schools?

Spoiler alert– the short answer is “Yes.”

A review of the research and literature led Mathis and Welner to conclude that while some choice schools are integrated, charters largely are very segregated. That segregation can be by race, poverty, dual language learners (ELL), and students with disabilities. While black students are generally either under-represented or over-represented in charter schools, the researchers found that poor, ELL, and students with disabilities are under-enrolled in charter schools. Within a choice system, both segregation and the achievement gap grow.

However, before charter foes leap on those results, there is this to be considered:

Even without school choice, America’s schools would be shockingly segregated, in large part because of housing policies and school district boundaries. School choice policies that do not have sufficient protections against unconstrained, segregative choices do exacerbate the problem.

In other words, charter-choice systems may be making things worse, but they certainly didn’t create the problem. Or to put it yet another way, when anti-charter folks say that charters are creating massive segregation problems, they are correct. And when charter fans say that housing-based choice is creating segregation problems, they are also correct.

There may be valid arguments in favor of some charters in some situations, but the “we will fix segregation and close the achievement gap” argument is not one of them. Charters clearly do neither of those things.

Public schools aren’t getting it done, either. But in order to look for real solutions, we need to stop pretending that fake solutions are actually working.


Grading Good-faith Gibberish

Posted: 07 Apr 2016 08:55 AM PDT

Les Perelman is one of my heroes for his unflinching exposure, time and time again, of the completely inadequacy of using computers to assess writing.

Perelman and his grad students create BABEL

, (the Basic Automatic B.S. Essay Language Generator) a program that can generate brilliant gibberish. Diane Ravitch, education historian and activist, took a stab at using BABEL and got, in part, this:

Didactics to subjugation will always be an experience of humankind. Human life will always civilize education; many for diagnoses but a few of the amanuensis. Myrmidon at absurd lies in the search for reality and the realm of reality. From the fact that denationalization excommunicates the denouncements involved of civilizations, humanity should propagate absurd immediately.

This scored a mere 4 out of 6. Apparently Ravitch, as an older American, suffers from being the product of our earlier status quo education system. If only she’d been exposed to the Common Core.

The software that scored her essay is PEG writing, and the site has some lovely FAQ items  , one of which Ravitch highlighted.

It is important to note that although PEG software is extremely reliable in terms of producing scores that are comparable to those awarded by human judges, it can be fooled. Computers, like humans, are not perfect.

PEG presumes “good faith” essays authored by “motivated” writers. A “good faith” essay is one that reflects the writer’s best efforts to respond to the assignment and the prompt without trickery or deceit. A “motivated” writer is one who genuinely wants to do well and for whom the assignment has some consequence (a grade, a factor in admissions or hiring, etc.).

Efforts to “spoof” the system by typing in gibberish, repetitive phrases, or off-topic, illogical prose will produce illogical and essentially meaningless results.

In other words, PEG knows it doesn’t work. It also assumes a great deal in assuming that students writing pointless essays on boring subjects for baloney-filled standardized tests are “motivated” writers. Can the software accurately score motivated gibberish? Can the program distinguish between frivolous garbage and well-meant garbage?
Probably not. As noted in PEG’s response to the question of how the software can evaluate content:
However, analyzing the content for “correctness” is a much more complex challenge illustrated by the “Columbus Problem.” Consider the sentence, “Columbus navigated his tiny ships to the shores of Santa Maria.” The sentence, of course, is well framed, grammatically sound, and entirely on topic. It is also incorrect. Without a substantial knowledge base specifically aligned to the question, artificial intelligence (AI) technology will fail to grasp the “meaning” behind the prose. Likewise, evaluating “how well” a student has analyzed a problem or synthesized information from an article or other stimulus is currently beyond the capabilities of today’s state of the art automated scoring technologies.
PEG bills itself as a “trusted” teaching assistant that can help relieve some of the time pressures that come from having many, many essays to grade. But I can’t trust it, and it’s unlikely that I ever will.
This is the flip side of Common Core reading, an approach that assumes that reading is a batch of discrete behaviors and tricks that are unrelated to any content. Here we assume that writing is just a series of tricks, and it doesn’t really matter what you’re writing about, which is a concept so bizarre that I can barely wrap my head around it. Use big words– even if they have nothing to do with the topic of the essay. Use varied sentence lengths– but don’t worry about what the sentences say.
PEG, like other similar services, offers as proof of its reliability its closeness to human-rendered scores. But that happens only because the human-rendered scores come from a rubric designed to resemble the algorithm of a computer, not the evaluative processes of a human writing teacher. In other words, you make the computer look good by dumbing down the humans used for comparison.
Pearson’s continued fascination with AI-directed education, as well as the news that PARCC will use computer essay grading in four of its six states– these are Bad News, because computer software is simply not up to the job of evaluating writing in any meaningful way. BABEL is just one more demonstration of how completely inadequate the software tools are.

P.S. My favorite line from my own BABEL efforts:

Charter, frequently to an accusation, might innumerably be grout for the allocution.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s