HISD - College Ready?

Ericka Mellon wrote an article about the performance of high school students at HISD schools on a "college ready" assessment in the Houston Chronicle.

The article describes advances HISD has made in the number of "college ready" students on the TAKS Exit Level exam. This is defined (as far as I can tell) by a student receiving a scale score of 2200 in each of the Math and Reading sections of the exam. My first two questions are:

• Do these scale scores correlate with any future success, such as college performance or acceptance?
• Scale scores are a way of applying a "fudge factor" to try to normalize score reporting across different years' tests (just like the SAT scores, for example); are we sure they represent something meaningful? Can they easily be adjusted year to year to affect the numbers?

Here's an example of the scale score issue: One year, a student can get 23 of 36 questions correct, and receive a scale score of 2100. The next year, the test is determined to be "easier", so a student needs 26 of 37 correct for the same scale score. What's not clear to me is that the composition of the 23 or 26 correct affects the student's scale score. On the first test, if a student shows complete mastery of a most topics, and gets 13 wrong representing a 0% mastery of two core topics, can that be the same as a student who misses 13 questions, some here and there, but with a decent grasp of all the concepts? Can you compare two students who get 23 correct - one who gets 23 of the 24 "easy" questions right, and none of the 12 "hard" ones, and a student who gets 23 correct, a mix of hard and easy questions? Does it make sense to map a scale score to just a raw score, or should the questions or their distributions be weighted? You should refer to the TEA web site documenting the conversion of raw scores to scaled scores on the TAKS.

Never mind. Let's say students who meet the scale score test are all equally "ready for college". One of the documents Ms. Mellon attaches at the bottom of her article shows the achievement levels per high school in HISD. The numbers are interesting. DeBakey has an impressive record of preparing their students for the TAKS exit, the best in town. Almost all their kids score at least proficient in Math and Reading. Bellaire last year saw 82% of their kids "pass" in Math, 76% in Reading. Carnegie, 95%/94%, and that's way up from 87%/72% (!!) in 2007. I love the HSPVA numbers, which kind of buck the trend of doing better in math than reading: 84%/96%. Lamar's numbers are 65%/63%.

What does this mean for a parent trying to decide which HS is right for their kid? On the one hand, if you are not worried about your kids passing these standards, maybe these aggregate numbers aren't that important to your individual case. On the other hand, I worry that schools have been or will be looking at these numbers, setting campus goals, and then expending a large number of resources trying to get those numbers up. Although that's not a bad reaction (again, assuming these metrics actually measure something meaningful), in practice, I fear this means that fewer or no resources at those schools will be focused on the students who are in no danger of missing these goals - the advanced kids who could also use more attention to better develop their own skills and interests. I worry more and more that campus educational resources are a zero-sum game, and when the balance shifts inordinately to focusing on bringing the bottom students up, the top students get less attention. What should that balance be?

As parents, should we focus on sending our kids to schools with the best records, assuming their staff already feels confident in their students' performance and can focus on deeper or broader curricula? Or should we worry that schools at the top are there because they're focusing so many resources on passing these tests, and our children may languish?

Health care call in

February 12 is the National Call-in Day for HR 676, which is the House resolution implementing a national single-payer health care system. Please call the Congressional switchboard: 202-224-3121 and ask for your representative's office. You can also call (202-456-1414) or fax (202-456-2461) the White House.

Single payer health care is different from "universal health care" because it eliminates the need for private health insurance plans. This helps realize a few important effects:

• Eliminate the inefficiencies of private plans. The 25% - 30% of our premiums which go toward profit and marketing can be used instead to provide care to more individuals.
• Remove the mis-alignment between profit and health. Insurance companies, to make a profit, routinely deny care to their customers, in what often seem like arbitrary decisions. A governmental single payer plan will need to determine what procedures and items to pay for, but it will at least be nationally consistent, a single set of rules for everyone. Yes, there will be a public organization or committee responsible for those decisions - but such deliberations should take place transparently, with the potential for oversight from all of us who are affected. Compare that to the decisions now made by your insurer - who makes them? Are they in your best interests? Do you feel they are concerned to make sure you're healthy, or do they deliver just enough care so you don't leave, and they can maximize their profit?
• Improve doctor choice. Current insurance plans exclude doctors and hospitals, forcing you to choose providers "in the plan". With a single payer, everyone is in the plan.
• Improve the competitiveness of American firms. Firms employing American workers have to pay inflated premiums (see the inefficiencies point above) for health care for their workers. Firms in almost every other developed country have their workers covered under some state-organized plan. Sure, health care has to be paid for somehow; payroll or per-worker taxes are the most likely tool to use. But when everyone's covered, and inefficiencies are squeezed out of the system, it should end up that the cost per worker, for the same level of care, is less under a single-payer plan than in our current environment.
• Potentially mitigate billing errors and issues. I'm on thin ice here because I have no experience dealing with Medicare or Medicaid. Do they pay on time, and in full? Dealing with a single payer may make it unnecessary for a practice or hospital to have a large staff dedicated to collecting payment from insurers. The process may be streamlined, saving time and money for everyone involved.
• Implement a single formulary for prescription drugs, allowing the negotiation of reasonable prices for such a huge market commitment. Yes, this would set a single price for each drug; it would have to be reasonable enough for drug makers to be able to fund research into new drugs. It may end up that the market shifts a bit; drug research and drug manufacture may be split up under some compulsory licensing scheme. A drug researcher would take on the costs of developing new drugs, typically with access to federal funds; once a drug (or device) is developed and tested, the organization would recoup the costs from licensing the resulting patents to manufacturers. A robust market in manufacture could help drive the cost to the consumer down.

A health care market under HR 676 would potentially look dramatically different from what we have now; some companies and segments will be unnecessary and disappear, and others will change dramatically. That's fine with me, because it seems the system we have now is fundamentally broken, and is not delivering the care we deserve for the dollars we're pouring in.

Get out of jail, free

The Houston Chronicle has an article about technology at the Houston Municipal Court system. The court's computer network has been hit, hard, by a virus - the report suggests Conficker but the chief technical director for the City doesn't think so. The effects include a shutdown of the muni court on Friday, and a suspension for arrests for Class C misdemeanors.

To me it seems clear there are a number of lessons to learn:

1. Keep your critical computers away from the Internet. There are computers which are necessary for your department to function, and then there are the computers people use for less critical purposes. Think strongly about keeping an "air gap" between your critical infrastructure and the Internet.
2. Avoid an operating system monoculture. Another hard thing to do - it's a trade-off between being easy to manage (the Southwest Airlines approach) and being robust. Having even 20% of your computers running something other than Windows can mean the difference between an inconvenient virus attack, and shutting down your courts for the weekend.
3. Migrate off Microsoft Windows. It's a bug-ridden virus magnet.The sooner you move away from MS Windows on your critical infrastructure machines, the sooner you become more resistant to viruses and worms.

Houston Chronicle - loose reporting

Mike Glenn of the Houston Chronicle reported yesterday about an HISD teacher who was escorted off campus because a drug-sniffing dog identified her vehicle. Inside, there were some unidentified pills, for which the teacher could not immediately produce a prescription. Students were in tears as they heard of her being taken into custody by district officials.

I'm disappointed by this article. This isn't reporting; it's repeating an unsubstantiated allegation against an HISD teacher. Until there's proven wrongdoing, this event should be treated as discretely as possible. The paper should only publish this kind of information when it turns out to be an example of an employee who is under performing, is dangerous to students, or who will be disciplined by the district. I would be encouraged if the Chronicle were to adopt such a policy going forward.

To compound this particular affront, Roberts is a small enough school that its identification of the teacher, while attempting to walk the line between descriptive and anonymous, is in fact sufficient to let everyone know who it is. We all hope this will turn out to be a misunderstanding, quickly resolved; however, the publicity the Chronicle has afforded this case has served to add damage to the reputation of a long serving and much beloved teacher, who we hope to see back in the classroom soon.

Another fun map

Here's another fun map, showing how many students travel 10+mi at various magnet programs. Larger boxes mean more students.

I'm not sure how useful this one is; it's just a fun one to look at.

Where do the 10+mi riders live?

I got an early holiday present on Thursday: a response to my open records request to HISD for data on the magnet students riding 10+ miles to their programs. I was pleasantly surprised; I'd not received an acknowledgment, and was about to send a "snail mail" follow up to my original email request.

Part of what I found out can be seen graphically here at Geocommons.com, a web site that lets you upload geo-coded data and see it plotted on a Google (or Yahoo or Open Street Maps) map. You can have several "overlays" if you like, each corresponding to a data set. The data on that map shows how many students travel 10+ miles from each zip code - it's their home code, not the destination. I also received a long list of every school and the number of 10+ mile riders, and I got a small table explaining the racial breakdown of the same group:

Race	Percentage of 10+mi riders	Percentage of District
White	10%	8%
African American	48%	28%
Hispanic	34%	60%
Asian	8%	3%
Native American	<1%	<1%

In my request, I asked for income level bands, which they don't have; but I did get the response that 84% of the 10+ riders qualify for free/reduced lunches (this was also reported in the Houston Chronicle today). I also asked for the same distribution information for the magnet program as a whole, and for the district; I got the district racial breakdown from a different source, which also pointed out that district-wide, 79% of the students qualify for free or reduced lunches.

I worry that reducing or eliminating services for the 10+mi riders will make magnet attendance difficult for 84% of them (2822 students). They are likely to have the least flexibility in their schedules or access to transportation.

Resouces:

• geocommons.com
• batchgeocode.com calculates lat/long coordinates for a batch of addresses
• swivel.com lets you graph various data sets, but I didn't really get the hang of it. Looks cool though.

Feeding the Hungry

At First UU Church Houston a few years ago, our Parent's Group started a project to create meal packs for the hungry in Houston. We were not trying in a broad way to feed the homeless; instead, we were trying to give congregants an alternative to giving money to street solicitors. In this way, we thought people might find it easier to be more generous in the moment.

With the packs themselves, were looking to accomplish a few goals:

1. Packs should comprise non-perishable items, so they can be stored at home and taken with you in the car/on your bike so you can give them to people who are in need;
2. The nutrition should be balanced, with a good amount of protein;
3. There should be plenty of liquid, important especially in the summer;
4. Items should be edible even with bad gums or teeth (soft foods);
5. The packs should be inexpensive and provided at-cost to purchasers so they feel comfortable buying many and handing them out.

With those in mind, we managed to create the following packs for about $3 each (by purchasing items in bulk):

• A quart sized "zip" locking bag
• A paper napkin and plastic spoon or spork
• A postcard with a map of Houston and locations/phone numbers of aid agencies
• A liter/quart of water
• A juice box
• A tuna-fish based lunch pack (the most expensive item, but the most nutritious)
• A pudding
• An applesauce
• A pack of cheese its or other snack
• A pack of cheese or peanut-butter stuffed crackers
• A granola bar

We managed to stuff almost 2000 calories of long-lasting food with a good mix of nutrition, while providing water and kind of fun snacks. The packs were easy to shop for, simple to assemble, and are satisfying to give out. Because we were able to recoup the cost of the packs each time, the project itself was self-sustaining after an initial "investment" from the social action group of the congregation. By taking pre-orders, we were even able to expand the production at various times.

I'm proud that my daughter's Girl Scout troop did a similar project this year at Thanksgiving. Because of fund-raising rules, they were not able to recoup the costs from outside the troop; still, they considered it a valuable project and invested their collective dues into making five packs for each girl.

This is not a project which tries to address the causes of homelessness or hunger; it's not a program which will help people get off the streets. Those are additional, sustained efforts which need to take place as well. But as we know, structural change doesn't happen overnight; in the meantime, we can make these small efforts while we work on the large problems.

Houston ISD December Board Meeting

Hi Folk(s?)

Prior to today, there was a widespread assumption that the Houston ISD Board would address the issue of transportation for Magnet students at their December meeting. However, when you look at the agenda published for the meeting, there's no mention of that issue as a topic for discussion or decision.

It's been reported that the decision will happen at the January meeting; but, as always, read the agenda first.

That said, there is a group of parents who will be attending the meeting to demonstrate their position on the issue. The plan is to meet at 6:00pm at

HISD Administrative Headquarters Hattie May White Building - Board Auditorium 4400 W 18th St, Houston, TX 77092-8501 Hwy 290 @ Loop 610

Houston ISD ASPIRE: Good idea, bad implementation?

Houston ISD has implemented a new evaluation system called ASPIRE, developed in collaboration with Battelle for Kids, based on the SAS Educational Value-Added Assessment System (EVAAS). This is a new initiative to track student progress year-over-year (longitudinally), instead of comparing this year's third grade class to next year's and last year's (a cross-sectional study). The idea behind the new evaluation is that if you track the same population over time then you can see how different teachers improve their progress year after year. It's a nice idea, but as far as I can tell, this implementation has at least two major flaws. I'd appreciate comments from educators and statisticians either confirming or rebutting these observations; I'm neither, and I like to hear from experts.

Before I outline what I consider are ASPIRE's weaknesses, I'd like to go on record as a supporter of the concept at least in theory. It's clear that the old method of measuring a teacher's "performance" year after year, with a changing population each year, is unfair to the teacher because it does not control for possibly wide swings in their class's demographics. If one year a teacher has an overall eager student body, and the next, one that comes in with a lack of skills or a lack of focus, the teacher's "performance" will vary. The exit scores for each of those classes will differ; one year he or she will look like a success, and the next, possibly a failure.

The implementation of ASPIRE is an attempt to measure a class's incoming and outgoing level of achievement, and determine what if any effect the teacher has on the students. On first sight, that seems reasonable; however, the implementation at HISD falls short for the following reasons.

The first potential flaw in the system is that schools with an advanced student population will show little to no year-over-year improvement. This is an effect of the tests chosen for the ASPIRE metrics: the Texas Assessment of Knowledge Skills (TAKS), and a normed test (Stanford or Aprenda). Neither of these tests can differentiate among the students in the 99th percentile, and that effect may even be true for a larger population (perhaps to the 95th percentile). If a child stays in the 99th percentile year after year, or bounces around within the 95th-99th percentile band, they show no (or negative!) progress according to those tests. It's not fair to penalize that child's (or that class's) teachers because the test can't measure student progress at that level. The same goes for the TAKS; it's designed to measure a very basic level of subject mastery; for schools with advanced students, that mastery happens as a matter of course, and possibly even early in the school year. What happens in the classroom supplemental to that (a deeper investigation of the subject, a broader survey of related topics) is not measured in ASPIRE.

When HISD implements "performance based pay" on such systems, they intend to truly reward the teachers who take struggling students and help them reach new levels of accomplishment during the school year. However, they run the risk of leaving behind the teachers who are teaching the average or advanced students, and that's not fair. By publishing this "value add" data on the web, they actually give the misleading impression that the schools with more accomplished students actually have flat or failing performance.

Let me be clear: this is not a problem for all student populations. For students who are not advanced, an improvement year after year would be meaningful if it were measured correctly.

That brings me to the second potential flaw: the data may not be reflecting what ASPIRE needs to measure. The first input is TAKS performance, a measure of mastery in various subject areas. This is probably a good indicator when used for reading (comprehension) and mathematics, which are measured every year; those are subjects where each year builds upon the student's previous knowledge, and an improvement may signal a significant change in understanding. The other areas measured (science and writing) are less obviously incremental, and aren't tested every year; and other parts of the curriculum (history, art, music, foreign language, health, for example) aren't measured at all.

The other test used as input is either the Stanford or the Aprenda (for Spanish speaking students). Unfortunately for this effort, these are nationally normed tests, useless for measuring student progress. Very briefly, a norm referenced test is one in which the student is assigned a rank against their peers that year; the questions are not assessing subject matter knowledge, but are instead chosen as differentiators between students. To see the effect of the first characteristic, just think of how a student will score differently based on a different set of kids taking the test; the same student could be in the 70th, 80th, or 90th percentile depending on who else is taking the test. Clearly, this is not simply measuring achievement; while a large part of how well a student does on the test depends on their knowledge, a significant factor is the set of other students, over which they have no control.

The second problem with normed tests is more subtle. The questions on the test are not chosen to assess what a student knows; instead, they're effectively chosen for how "tricky" they are, so they expose a difference between sets of students. The purpose of a normed test is to rank all the test-takers along a continuum of scores; you can't do that if there are a large number of questions on the test that everyone gets right or wrong. On the TAKS, which is a criterion-referenced assessment and is measuring comprehension and mastery, it's OK for everyone to get all the questions right; that means that all the students in Texas have mastered that subject that year. The normed tests are not serving that same purpose; such a result on the Stanford or Aprenda would be a serious failure.

The final issue with ASPIRE involves the more general debate about whether these standardized tests are actually providing a relevant measure of student accomplishment, and accurately reflect the effects of good or poor teachers (as opposed to a good or bad curricula, inappropriately homogenized pedagogical methods, external factors such as days lost to weather, etc.). You clearly cannot improve a system such as public education without being able to measure it; however, there's a valid debate over whether we know how to describe and measure the effects of a successful education. Until we get to that point, I'm supportive of attempts to assess educational effectiveness, and skeptical of punishing or rewarding teachers simply by using the results of those potentially ineffective efforts.

The idea of tracking each student population's progress longitudinally (year-over-year) and measuring their improvement is a good one; however, I'm disappointed that HISD and Battelle seem to have gotten the implementation wrong with ASPIRE. I can't tell if they use the TAKS and Stanford/Aprenda metrics simply because that's what they have at hand (and they don't want to change the tests, or add new ones), or if it's just because they fundamentally don't understand how poorly the tests measure what they're trying to track. Perhaps ASPIRE will get better over time; it may also be that my analysis above is flawed in one or many ways. If I'm way off base, I'd love the reassurance of being proven wrong.

More portal trouble at HISD

Frankly, I'm embarrassed. Houston ISD does many things right, but their web technology developers seem to be unable to create an online application which works for everyone. I wonder if it's because they exclusively use Microsoft tools for development and deployment?

We received a flyer in the mail recently which describes the Houston ASPIRE effort, and points parents to their web link. Your mileage may vary; in fact, it's probably likely to work for you if you use MS Internet Explorer, or are on an MS Windows system. I have access to neither, so I can't verify that setup. It sure doesn't work for me in my Firefox-based browser on my Linux system.

Never mind of course that the whole implementation of ASPIRE has its own set of weaknesses. I'll address that idea in a later post. I'm just frustrated that HISD only seems to be making a half-hearted effort to engage parents and the public online.