What Bad Assessment Looks Like, Go Math! Style….

As many of my loyal (and not so loyal) readers know, among the things I’m attempting this year is to help a K – 3 public school in an impoverished area of the Bronx implement an alleged “CCSS aligned” curriculum. I’m not a fan of most math curricula I’ve seen, and even the best require some kind of adaptation to be effective with the classroom population Go Math!, however, is among the worst of the worst I’ve encountered, and in my short tenure attempting to adapt it, I find myself mostly doing to the work of informing teachers what not to do, which includes correcting mistakes in terminology and developmental appropriateness.

In this post, I’m going to be discussing assessment: much of Go Math! relies on “end of chapter” exams that focus on multiple choice questions that are either poorly worded or completely inappropriate. One of the criticisms leveled at Go Math! has been its almost pathologic focus on procedures, without giving adequate coverage of concept development and higher order thinking (despite the fact that the graphic designers sprinkled the workbooks with “HOT Problem” logos that are not very “high” or require much in the way of “thinking.”)

The latest howler came in the 3rd grade chapter on area and perimeter. Most of the multiple choice questions don’t actually require any kind of thinking beyond remembering to count boxes when the word “area” appears in the question, or counting line segments when “perimeter” appears, which includes this task:

Can you figure out why this is a cruddy question?

Can you figure out why this is a cruddy question?

Okay, it’s not bad enough that most of the test asks exactly the same type of question over and over again, but if you really wanted to assess whether a student knows the difference between finding the area of a shape and perimeter, wouldn’t it be a good idea to provide shapes where the area and perimeter are not the same? 

My suspicion began when out of the 40 tests I looked at, each and every student got the question correct. Okay, it’s not a very difficult question to begin with, but when I looked at the work provided, I noticed that half the students had numbered the squares inside, and the other half numbered the line segments around the outside and came up with 22! Perhaps I’m a little misinformed here, but what exactly is this question assessing? (Let’s set aside the fact that Jake’s bedroom is 22 square meters, or 237 square feet, which makes it larger than the Superior Queen Room at the SoHo Grand in NYC.)

This is not to say that multiple choice tests are all evil; however, if you’re going to go through the trouble of wasting a child’s time with these type of assessments, at least make them interesting and meaningful. This is a garbage question that does nothing to help a teacher assess a student’s understanding of perimeter and area, and whatever 20-something year old editor who allowed this to appear should be fired, or confirmed as a replacement for Arne Duncan.

Posted in bad assessment, Go Math! | Leave a comment

Just to be clear….

mannvsduncan

Posted in Uncategorized | 2 Comments

Is this the most ignorant article EVER about Common Core Math?

“Vox” is a website that claims to explain “everything you need to know in two minutes,” but it took me a lot less time than that to figure out this may the stupidest article about the Common Core math program I’ve ever read. Full disclosure: this is only true if what you mean by “read” is “look at this hideously stupid graph which the author used to support her point.”

And here it is!

stupid math graph

I’ve never actually heard of the “Program for International Assessment of Adult Competencies,” but I’m pretty certain CCSS or non-CCSS trained 7th graders know that a graph that lists the outcomes of a scored measurement without stating the highest possible score is immediately suspect. I’ll get to some assumptions later, but basically this is telling us that whatever sample of US adults took this test did 88% as well as the adults in the top scoring nation, Japan. I think that’s pretty damned good, considering the United States is second to the world in poverty, leaving Japan in the dust by over 10 percentage points (and I’m sure Japan uses a much higher economic benchmark for poverty than we do here in the US.) Of course, we all know that poverty is the single greatest predictor of poor school performance.

But let’s make an assumption: suppose the top score on this alleged exam was, what, 300? That means that the Japanese adults scored 288 out of 300, while the United States adults score 253 out of 300, which puts the Japanese at 96%, leaving us in the dust with a competency of 84.3%, which whittles the 25 point difference to less than 12 percentage points. This leaves aside that it is completely unknown whether the sample was somewhat random (Japanese engineers versus United States journalism majors, perhaps?) or even large enough to be statistically significant.

Oh, and incidentally, whoever made this graph would have lost a significant number of points on the 3rd grade New York State math exam because he/she did not label the “0″ on the horizontal axis, nor include an incremented scale…

The article goes on to feature two videos from the websites, LearnZillions and Khan Academy, which share the dubious distinction of being the epitome of what “bad” math teaching looks like. There are also quotes from Dan Meyer, a one-man publicity machine who believes he speaks for all math teachers, despite the fact that he spent exactly 5 years teaching in an actual classroom.

Finally, this article is yet another example of the “waking up on third base” phenomena, which posits that everything that you see in a Common Core math curriculum is the direct result of the implementation of the Standards. Nothing could be further from the truth: all of the items described on in the article have been documented, published and taught since the NCTM published its curriculum standards a quarter of a century ago. If you’ve been teaching math using a textbook that was published in the last 20 years, you’ve probably seen all this stuff before including, with all deference to Mr. Colbert, the infamous description of a “number sentence.” Telegram for Mr. Colbert: 1989 is writing to tell you to “LOL!”

I love it when people who know little to nothing about education (Libby Nelson, the author of this piece of junk, is a journalism major who has written for Politico and something called Northwestern Magazine) try to dip their toes into a controversial subject like the Common Core and then end up getting eaten alive by people like me. Actually, I don’t love it, but it does make me angry because a formerly respected organization like the NCTM actually tweeted the link to this worthless piece of codswallum, and smelling something rotten, I just had to follow the scent.

 

 

 

Posted in Common Core State Standards, Khan Academy | 2 Comments

Yo, @LouisCK : We Need To Have A Dad-to-Dad Talk About Math, Homework & the CCSS

Dear Louis CK:

Okay, I’m a fan and I love your work. End of fawning admiration. I want to talk to you dad-to-dad about your daughter’s homework, the Common Core, and the fate of the world. Seriously, I do.

I’m sorry your daughter got a math homework assignment that was so confusing and led to discord in your  tranquil home. I’ve been a math teacher for the past 30 years, and if your daughter had been in my class, I’m sure she would have come home with these types of assignments on a regular basis. It’s not that I want to torture your child or make you feel dumb. It’s just that part of the unspoken agreement between schools and parents is that children must get homework, and the more challenging it is, the better. This is because we’re currently in an educational era where teachers can get away with developmentally inappropriate assignments by citing the “fact” that we now have “high standards,” and that we’re trying to “challenge our students to do their best.”

This runs counter to the fact that homework has never been shown to be effective. If that isn’t bad enough, helping your child with homework does not lead to better long term outcomes. Sure, it may make you feel as if you are being a “good parent,” but there are lots of ways to be a good dad; unfortunately, getting involved in school just doesn’t happen to be one of them.

Let me cite my own personal history: I have a 22 year old daughter. When she was in elementary school, I took it upon myself to help her with her math homework. I mean, who would be better than a math teacher with an MS in elementary mathematics education? What could go wrong?

A lot, it turned out. First, as an “expert” on mathematics teaching, I was appalled at 90% of the homework that my daughter had to do. I frequently found it confusing, inappropriate and just plain dull. This did not endear me to my daughter or her teachers. Once, while I was attempting to help my kid with an activity that required her to order fractions, I was told that I didn’t know what I was talking about. This was followed by a door slam.

After several incidents like this, I finally waved the white flag and admitted defeat: imagine how humiliating it was to admit that the teacher who spent his days bringing great math to his students couldn’t do it with his own kid. At the same time, I had a realization: mathematics, despite its seeming neutrality, is the most emotional subject taught in school, for a variety of reasons. The most salient one is that it is the subject that we use to gauge our own intelligence. Other subjects, like language arts and social studies are “interpretive,” so there is certain amount of “wiggle room” when answering questions. By way of contrast, mathematics is about logic and process, and when you start engaging with your kid in these two domains, you’re asking for trouble.

My solution was to step down: I told my daughter that she should spend a reasonable amount of focused time doing her math homework, and if there were questions that were confusing or difficult, she would give it the old college try and then bring it into school. Her teachers were invariably understanding and she never felt singled out, as other kids in her class also found math confusing. It’s just the nature of mathematics: deal with it. If we tried to make it easier, it would no longer be mathematics.

Take my advice: let your kid do her math homework on her own, and if she finds it frustrating, tell her she’s most likely not alone, make her a nice cup of chamomile tea, sit on the couch and read a book together. That’s what good dads do.

Which brings me to the Common Core: I agree, the Common Core State Standards are idiotic, but not for the reasons you may think. My disagreement is more than the fact that they are vague, poorly written and inappropriate; my problem is that they will never accomplish what they claim. They will not create a new generation of new “college and career ready” children, they will not make them more likely to get good jobs, and they will not make them good citizens in this poorly functioning democracy. In fact, it will not lead to a great future for our kids, or for anybody else for that matter. This is because no matter whether CCSS sticks around or dies a slow death, the future for our children is going to suck, and when our grown children are fending off the marauding hordes and baking away during the 140º summers, the issue of the viability of the Common Core State Standards is going to be as relevant to our survival as the current size of a Kardashian butt.

The real danger of the CCSS is that it distracts us from the real problems of the world, which includes the plots hatched by mendacious assholes like the Koch Brothers who are busy polluting our environment, stripping away our basic rights and pushing poor slobs like us into financial ruin. The advocates of the CCSS have made lots of noise about  “raising the bar” and all sorts of other nonsense relating to world competitiveness (which I have covered here), but in the end, it’s probably best to pay no attention to that man behind the curtain. Seriously, speaking dad-to-dad, you’ve got better things to do with your time. Now would you get back to making us laugh?

Posted in Common Core State Standards, Math Anxiety, Research | Leave a comment

“Waking up on third base” & Why CCSS Boasts Are Hollow

Jim Hightower, whom many of us in the over-40 set will remember as the plainspoken former Texas politician, once described former “president” George H.W. Bush as someone who “was born on third base and thinks he hit a triple.” Hightower was speaking about Bush the Younger’s proclamation of accomplishments that were, in fact, “inherited” through his family connections.

I feel the same way when I hear the constant comments from the CCSS supporters who are claiming that education has been “transformed” as a result of the implementation of these most dubious of “standards.”

EngageNY, New York’s own propaganda organ for the CCSS, and John King, New York’s own “Commissioner of Education,” have entertaining Twitter feeds that are continuously guilty of the “waking up on third base” claim. Let’s look at some of my favorites:

Jonathan King’s claim: “It was great to see how hands-on and comfortable students at JD George are with Common Core standards.”

My comment: Really? Compared to what? Do you have any proof that the students were not “hands on” and “comfortable” before Common Core standards were implemented? Many CCSS proponents seem to have a “before CCSS” and “after CCSS” view, as if one day everybody woke up and realized “hey, we haven’t been allowed to teach interesting lessons under the ‘old’ system. But CCSS has arrived and liberated us! Huzzah, huzzah!”

I’ve been in education for the past 30 years, and believe me, if you’re a well trained teacher who has been keeping abreast of contemporary methodology and practice, the CCSS will not seem “new” or “novel.” (They will, however, seem confusing, arbitrary and inappropriate.) Furthermore, the CCSS may actually inhibit teachers from doing all the higher level thinking they did before CCSS arrived to save U.S. education, especially if your school adopted one of those cruddy “Common Core Aligned” textbooks that focus on low levels of thinking.

Here’s another howler, this time from the EngageNY Twitter feed:

“Parents are hearing that their children feel more prepared for college with Common Core.” 

I didn’t have to get more than 2 seconds into the linked video to shut it off and whisper under my breath: THE PLURAL OF ANECDOTE IS NOT DATA, THE PLURAL OF ANECDOTE IS NOT DATA! On what basis is this claim even remotely true? Was there a double blind experiment performed in which a large sample of students from non-CCSS schools and CCSS implemented schools were compared to see who had better success rates at their comparable colleges? Were the subjects “pre-tested” to compare the difference between “before CCSS” and “after CCSS” cohorts? Where there linear regressions, and Kruskal-Wallis tests performed, maybe an ANOVA here or there? Can this claim withstand any claim of “rigor?” Even more important, would a student who was not educated using CCSS be able to smell the rampant BS in this video?

Well, I was educated in the New York State public school system (Jericho High School, class of ’77), decades before CCSS was a glimmer in Bill Gates’ eye, and somehow I learned the skills necessary to “write arguments to support claims with clear reasons and relevant evidence” (that would be CCSS.ELA-LITERACY.W.6.1.A, in case you were wondering….) I intend to apply that standard to this Twitter statement, which, if you are reading this blog post, should convince you that I, in fact, possess these “before CCSS” skills.

Let me proceed:

The “sample size” in this video is all of 2 students, one of whom is in college, while the other is in 8th grade. And just so you don’t think there was any “sample bias” here, the “expert” who is telling us this story is… wait for it… their mother! The “evidence” presented is thin at best, and is based on an “opinion” rather than an actual “fact.” Furthermore, it is my claim that this may be an example of “cherry picking,” and not a representative sample, based on the fact that this was promoted by the leading propaganda source for CCSS in New York State, EngageNY. Using my non-CCSS implemented writing and analytic skills, I can conclude that this Twitter statement is lacking any sort of veracity.

There are a few teachers out there who truly believed that the CCSS has “improved their game,” but if I was one of them, I would not want to admit to that, the implication being that I was one of those “plug and play” educators who never taught anything beyond the basic skills (unless you were trained in the “back to basics” movement….) In my field of mathematics education, the NCTM Standards and Focal Points are old hat: we knew how to design and implement curricula that were based on conceptual understanding, problem solving and procedural fluency decades before the shining light of CCSS shone upon us. Seriously, I don’t understand how CCSS proponents can claim that anything in the standards are “forward thinking,” unless you define “forward” as “backwards.” In an era where the word “literally” is a synonym for “figuratively,” this might be a little true.

Posted in Common Core State Standards, Junk, Uncategorized | 2 Comments

Now that she’s got the kids, where to find teachers?

If you remember a previous post, I pointed out that the Success Academy Charter Schools was advertising for students in some very interesting places, which included my neighborhood of Park Slope, as well as a website where I publish educational materials. How happy I was that the deadline passed, and  no longer have to look a these ads begging me to fill out a form so that my child can enter their “lottery.” (Yeah, like I would buy a ticket for that contest, even if it was free….)

Well, now that Eva has gotten all her eager young students ready to grace her halls this coming fall (including the building where she is going to displace a program for handicapped students who will have to travel miles away), it looks like she’ll be needing teachers to educate those eager young minds. So what do I see when I log into my TeachersPayTeachers account? You guessed it:

 

Want to join the team? Eva's got room for you (well, not if you're too expensive, that is...)

Want to join the team? Eva’s got room for you (well, not if you’re too expensive, that is…)

 

and this!

ad2

It gets even better when you arrive at the web site: prospective applicants are told the following:

At Success Academy, we make history every single day. Our schools rank in the top 1% in math and top 7% in reading among all New York schools. 100% of all scholars passed the 2013 science exam with 99% earning the highest possible rating. We provide an exploration driven curriculum for grades K-8, focused on the whole child. Chess, art, and other specialized subjects bring excitement to the school day and foster a lifelong love of learning. Our communities are calling for more, and we are meeting this urgent demand with rapid expansion.

If that doesn’t qualify as hyperbole, I don’t know what does, especially because much of what is in this paragraph is not actually “true.” But then again, everyone loves working with a winner, so why not invite all those eager, underemployed, impressionable and disposable youths to do this:

Are you ready to join this movement and make an indelible mark on education reform across New York City?

That is, if the “movement” is funded by such right-wing “philanthropies” like the Walton Family Foundation or pseudo-educational foundations intent on compromising public education like Bill & Melinda Gates, not to mention the billionaire hedge-fund managers who crashed the economy and threw tens of thousands of people out of work. Yes, this is the type of ‘movement’ I want to join!

I’m truly tempted to “apply now,” if only to see whether I could actually game something beyond a cursory “thank you for your application; we’ll contact you if you have any positions open in your area.”

Of course, being a 54 year old man finishing his 30th year in education in a “high needs” area (math and science), I’m a most unlikely candidate for any position at Success Academy, where young, inexperienced teachers probably fit their needs a bit better. I probably couldn’t put up with the demands of the work day (at 54 years old, teaching for 8 hours with just a lunch break is most likely beyond my stamina), and being bossed around by an “education manager” who has 2-3 years in “people management” is probably going to grate on my nerves (especially when I’ll have to spend most  of my time improving test scores and making sure kids look ‘straight ahead’ at all times.)

On second thought, I’ll keep my day job.

 

Posted in charter schools, Uncategorized | Leave a comment

Moron VAMs & Non-Cognitive Ability

In a recent post, I summarized and explicated on research conducted by C. Kirabo Jackson, a professor at Northwestern University, who published a paper that examined the use of test scores to measure the effectiveness of a teacher using the “value added model,” which is better known as VAM.

In this post, I’m going to continue where I left off last time, as well as present Jackson’s findings using visual representations.

To begin, Jackson shows that by analyzing data from the National Educational Longitudinal Study from 1988, that there are two types of outcomes from teaching: cognitive and non-cognitive. These are summarized in the table below:

A comparison of cognitive vs. non-cognitive ability.

A comparison of cognitive vs. non-cognitive ability.

Jackson cites extensive evidence showing that teachers have differing effects on both cognitive and non-cognitive abilities of their students, and that together these two effects these can be represented as a  “vector” – that is, each one acts on the long term outcome on an individual student in combination.

Incidentally, “grades” are different from “test scores” in that when a teacher issues a grade to a student , it usually comprises different aspects of the student’s work, which would include things like completing assignments, taking part in activities, and showing up to class. As such, it is much more representative of a student’s cognitive and non-cognitive abilities. What is interesting is that test scores are only moderately related to grade point average (GPA). According to Jackson, there is only a 36% correlation between test scores in math and English and a student’s GPA. Even more interesting is the fact that test scores are completely unrelated to socio-behavioral outcomes. Okay, let’s just repeat that in bold: test scores are unrelated to socio-behavioral outcomes.

Knowing this, we can understand that teachers vary in their abilities to increase their student’s cognitive and non-cognitive ability; similarly, students also have their own vector, which is represented by their cognitive and non-cognitive ability. Therefore, the effect of student “a” being in a class taught by teacher “b” is the sum of their two vectors. Pretty simple, yes?

Some examples of  “teacher ability vectors” (my terminology) appear below:

three different teacher ability vectors

The left graph shows a teacher who improves both a student’s cognitive and non-cognitive abilities equally; the middle graph show a teacher who is very effective at raising test scores, while being ineffectual at raising non-cognitive behaviors. The right graph shows the performance of a teacher who has a great effect on non-cognitive ability, while not improving test scores a great deal. Which leads us to the question: which teacher is the “most effective?”

Which leads us to the following thought experiment: if you had to choose between the teacher in the middle graph and the one on the right (assuming the one on the left is quite rare), which one is more highly correlated with things like fewer arrests, more college-going and improved wages? If you’ve been following me, you know that the data show that improvement in these important outcomes are more highly correlated with teachers who improve non-cognitive ability. According to Jackson, if we went by test scores alone to judge teacher effectiveness, more than half of the teachers who improve these long term outcomes may not be identified as being “effective.” This is because the teacher effects on college going and wage improvements is as much as three times larger than can be identified by using test scores alone.

If you are a educator, parent or policymaker, you should be outraged at this conclusion, which can be represented using this graph:

Screen Shot 2014-04-17 at 12.05.37 PM

All of which begs the questions: if you are a teacher whose job performance is being judged using a “value added model,” you may choose to shift your resources to keep your job:

is this sa good idea?

Indeed, this seems to be the focus of all “school reform” since “President” George H.W. Bush slammed us with “No Child Left Behind,” and President Obama followed up with the equally egregious “Race to the Top.”  Folks like Arne Duncan like to rattle away at something called “accountability,” and in our mindless pursuit of higher test scores, something will be sacrificed. As Jackson’s research suggests, focusing on test scores could likely lead to more students dropping out of high school, fewer going to college and worst of all, lower wages as adults. Wait, is this what educational reformers really wanted?

Posted in Research, Value-Added Model | Leave a comment

Discrepancies #1

Maria Montessori vs. Michelle Rhee: who is the "real" reformer?

Posted in Uncategorized | Leave a comment

One of many nails in the VAM coffin….

If you’ve read the American Statistical Association’s position on the dangers of evaluating teacher performance based on the “Value-Added Model,” you’re probably wondering how they arrived at this very sobering conclusion. As Albert Einstein was alleged to have stated, “Not everything that counts is countable, and not everything that is countable counts.” In this case, AMSTAT took that advice to heart and so strongly inveighed against VAM that they essentially labeled it a form of statistical malpractice.

Associate Professor C. Kirabo Jackson, the most understated hero to decimate VAM.

In this post, I’m going to examine one of the studies that no doubt had a profound impact on the members of AMSTAT that led them to this radical (but self-evident) conclusion. In 2012, the researcher C. Kirabo Jackson at Northwestern University published a “working paper” for the National Bureau of Economic Research, a private, nonprofit, nonpartisan research organization dedicated to promoting a greater understanding of how the economy works (I’m quoting here from their website.) The paper, entitled “Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina”  questions the legitimacy of evaluating a teacher based on his/her students’ test scores. Actually, it is less about “questioning” and more about “decimating” and “annihilating” the practice of VAM.

I downloaded the paper and have been reading it for the past few days. Jackson clearly has done his homework, and this paper is extremely dense in statistical analysis which is rooted in data collected by the National Educational Longitudinal Study 1988, which began with 8th graders who were surveyed on a range of educational issues as described below:

On the questionnaire, students reported on a range of topics including: school, work, and home experiences; educational resources and support; the role in education of their parents and peers; neighborhood characteristics; educational and occupational aspirations; and other student perceptions. Additional topics included self-reports on smoking, alcohol and drug use and extracurricular activities. For the three in-school waves of data collection (when most were eighth-graders, sophomores, or seniors), achievement tests in reading, social studies, mathematics and science were administered in addition to the student questionnaire.

To further enrich the data, students’ teachers, parents, and school administrators were also surveyed. Coursework and grades from students’ high school and postsecondary transcripts are also available in the restricted use dataset – although some composite variables have been made available in the public use file.

The survey was followed up in 1990, 1992, 1994 and 2000, which means that it began when students were just about to begin their high school career, and then followed up when they were in 10th and 12th grades, and followed them through post-high school, college and postgraduate life. It is one of the most statistically valid sample sets of educational outcomes available.

What should be noted is that Jackson is not an educational researcher, per se. Jackson was trained in economics at Harvard and Yale and is an Associate Professor of Human Development and Social Policy. His interest is in optimizing measurement systems, not taking positions on either side of the standardized testing debate. Although this paper should reek with indignation and anger, it makes it’s case using almost understated tone and is filled with careful phrasing like “more than half of teachers who would improve long run outcomes may not be identified using test scores alone,” and “one might worry that test-based accountability may induce teachers to divert effort away from improving students’ non-cognitive skills in order to improve test scores.”

But lets get to the meat of the matter, because this paper is 42 pages long and incorporates mind-boggling statistical techniques that account for every variable one might want to filter out to answer the question: are test scores enough to judge the effectiveness of a teacher? Jackson’s unequivocal conclusion: no, not even remotely.

The first thing Jackson does is review a model that divides the results of education into two dimensions: the cognitive effects, which can be measured by test results, and the non-cognitive effects, which are understood to be socio-behavioral outcomes, which when combined, determine adult outcomes. To paraphrase the old Charlie the Tuna commercial, it’s more than whether we want adults that test good – we also want them to be good adults. Clearly, Jackson is aiming a little higher than those who would believe that test scores are the end result of “good teaching.” He’s focusing on what “non-cognitive” effects a teacher can have on a student, which includes things like diminishing their rates of truancy and suspensions, improving their grades (which are different from test scores) and helping increase the likelihood that they will attend college.

Which poses the less than obvious question: if teachers have an effect on both cognitive and non-cognitive outcomes, are they correlated or independent? That is, if a teacher is effective in raising test scores, will that lead to less truancy, fewer suspensions, better grades and less grade retention? Even more interesting is the idea that teachers could be more effective on one scale while being low on the other: is it possible for a teacher to be very effective at improving a student’s non-cognitive functioning while not having an effect on his/her test scores?

By page 4, Jackson’s paper starts to draw blood: using the results of the NELS 1988, Jackson concludes that a standard deviation increase in non-cognitive ability in 8th grade is associated with fewer arrests and suspensions, more college-going and better wages than the same standard deviation improvement in test scores. It’s almost as if Jackson is telling us, “hey, 8th grade teachers: want to improve your students future life? Spend less time on test prep and more time helping them show up at school, staying out of trouble and improving their actual grades.” 

This alone would be enough of a takeaway, but this incredibly dense paper continues to hammer away at any thought that test scores are meaningful in any way: in the same paragraph, Jackson states that a teacher’s effect on college-going and wages may be as much as three times larger than predicted based on test scores alone. HFS! Oh, and just to make things more interesting, it is followed by this statement: “As such, more than half of teachers who would improve long run outcomes may not be identified using test scores alone.”

To summarize, we’re only in the middle of page 4 of this paper, and we’ve already learned the following:

a) Teachers have an effect on both cognitive skills of their students, and non-cognitive skills of their students. The first leads to higher test scores, the second leads to more college going, fewer arrests and better wages.

b) In 8th grade, non-cognitive achievement is a better predictor of college going and higher wages, as well as fewer arrests and suspensions, than test scores.

c) A teacher’s effect on these “non-cognitive” outcomes is as much as 300% greater than can be measured using test scores.

But wait, there’s more!

Okay, I’m only below the middle of page 4, and already I’ve read three conclusions that essentially kill off any legitimacy to judging a teacher’s effectiveness based on test scores, and the good stuff has even gotten started!

What Jackson is up to in his paper is something bigger, way bigger: it would be possible to argue at this point that somehow cognitive and non-cognitive skills, while both responsible in some part to positive adult outcomes, are still  correlated; that is, if you improve the test scores, the other non-cognitive stuff will come along as a bonus. This is where Jackson goes for the jugular, and, as is typical of research papers, he essentially “buries the lead.”

“This paper presents the first evidence that teachers have meaningful effects on non cognitive outcomes that are strongly associate with adult outcomes and are not correlated with test scores.” (Emphasis mine, italics his, by the way.)

I have to stop with this blog post here (but I promise to do more deciphering of this paper in the next few days.) My only question at this point would be: why hasn’t anybody explained this to Arne Duncan, perhaps through the use of hand puppets and a mallet?

Posted in Value-Added Model | 7 Comments

The Myth of School Diversity

I was traveling on the 6 train this morning, which runs up the East Side of Manhattan. I got on at Broadway/Lafayette, and because it was rush-hour, the car was crowded, but more crowded than usual. On the train were a gaggle of middle-schoolers who were having themselves a good old time, and it did not escape anybody’s notice that they all happened to be Asian. This would not be surprising as the 6 train does run right through Chinatown. But observing them got me to thinking about diversity in schools.

Stuyvesant High School, one of the most selective high schools in the city, is 72% Asian. As I watched these “yutes,” I knew that a good number of them would be attending Stuyvesant at some point in the future, and it got me thinking about the recent report that New York City has among the most segregated schools in the United States. These students live in a very tight-knit community where families share the same language, the same culture and the same ideas about what a “good” education looks like. They’ve “gamed” the Stuyvesant admissions test by setting up cram schools in their community, and no doubt every one of these students have friends or relatives who went to Stuy and their parents no doubt aspire to have them attend the same school. They know their children will be among other children like themselves, and they no doubt know the teachers and administrations who work there.

With 72% of the school populated by Asian students, it may appear that Stuy is not a very “diverse” school. This belies the fact that being “Asian” could mean many things: when we label someone as Asian, we could be talking about someone from China, Japan, Korea, India or Pakistan as well as the incredible number of islands in the Pacific Ocean, each of which has it’s own distinct language and culture. It also belies the fact that Asians can be first, second or third generation, or that they could be trans-racially and/or trans-nationally adopted. They could be Buddhist, Hindi, Muslim, Christian and, yes, even Jewish. This is not to mention the fact that there is economic diversity within the community, which includes those who live in the poverty that is found in sections of Chinatown to those who commute in from the wealthy suburbs of Long Island. Beneath the “visual” homogeneity of Stuyvesant High School lies a vast mosaic of diversity that remains invisible.

My daughter, who is now 22 years old, was never interested in attending Stuyvesant, and none of her friends were interested either. She attended a fairly high performing middle school, and her intention was to attend a high school that would prepare her for college. However, her middle school drew children who were interested in the arts; the school had an excellent reputation for it’s photography, music, drama and dance program, and many of the students went on to attend the best schools in the city which focused on performing and visual arts.

The high school my daughter attended, the Beacon School, appeared to have a very diverse population: during her time at Beacon, my daughter regularly brought home friends who were from all parts of the city and from many ethnic, racial and religious backgrounds. However, beneath this visible “diversity,” there was quite a bit of homogeneity: the families all appreciated the fact that Beacon was a very progressively minded school that sent its students to places like Cuba and post-Katrina New Orleans in the name of social justice. I can’t imagine there would be many parents with strong conservative political views who would be comfortable sending their child to Beacon. At the same time, if you were deeply interested in science or math, Beacon wouldn’t be a good place to spend your high school years. While the sciences were taught, there was no push for the kind of high-flying research projects that are found at Stuyvesant and Bronx Science.

It’s very interesting watching how diversity has become the new buzzword in the field of education: every private school website has a page about “diversity” in their school, which touts the fact that the school is committed to having a range of students in their classes, but what constitutes diversity is quite elastic. A private school may have “visual” diversity in that there are students of different skin color, but the reality is that they are quite homogenous in that the families belong to the same economic class. On the other hand, we can somehow assemble a school of students in different economic classes like Stuyvesant, where 41% of the students are eligible for free or reduced priced lunch, yet it appears to be incredibly homogenous in the vast majority come from Asian backgrounds.

The problem is that homogeneity occurs for many reasons that have nothing to do with racism, classism or any of the other type of discrimination. In many schools, segregation comes about as a result of geography: I work in a school in the South Bronx once a week. it is 60% Hispanic and 40% Black, which makes it seem like a very homogeneous population. But as we all know, there is considerable diversity within the Hispanic population, which means we have children from the Caribbean Islands, including the Dominican Republic and Puerto Rico, or North America, Central America or South America. The Black students come from all over Africa: some are Senegalese, others are from Somalia, and still others are from Ghana, Ethiopia and Eritrea. But there’s considerable homogeneity here: the students all within walking distance from the school, and they are all poor. Very poor.

My conclusion is this: schools are inherently segregated places. This is while families want their children to attend schools with diverse populations, they also want their children to be around others who have similar values to their own, which leads to homogeneity. They may feel that their children should be around other children who share the same cultural or linguistic background, they may want their children to go to school with the same families who live in their neighborhood, they may have the same “aspirational values,” or, in the most extreme scenarios, they want their children to only associate with those who share the same socio-economic class, the same religious affiliation or the same political philosophies. At the same time, consideration must be given to the school’s environment (“no excuses” discipline vs. caring culture), or its reputation for arts or sciences.

From what I  can conclude after this subway ride up the East Side of Manhattan (all the Asian middle schoolers got off at 59th street, btw), schools are, by their nature, homogenous, and the solutions we’ve tried in the past have not moved the needle into more diverse territory. In fact, they have only made schools more segregated in one way or another: “magnet schools” segregate children according to their interests, “gifted and talented” programs segregate children according to their “measured intelligence” (which is highly unstable), and charter schools are well known to exclude those with learning disabilities and limited English proficiency (their “no excuses” discipline systems and almost pathological focus on standardized test prep also keep away those with progressive educational philosophies.) Parochial schools segregate children according to religious affiliation (and even degree of that affiliation), and expensive private schools segregate children according to socio-economic status.

In the end, I have no answers, only an observation and a conclusion. But that does not mean there is no hope: just because schools cannot be diverse does not mean they should not be diverse. At the same time, we have to put a lot more thought into what is meant by diversity, why it is important and what can be done about it.

 

Posted in Uncategorized | 2 Comments