Universal Prekindergarten: Evidence from the Field

Welcome Carnival of Evidence-Based Parenting readers!  Today’s Carnival focuses on preschool education. I will keep an ongoing list of links from other participating blogs at the end of this post.

This is Part Three of my preschool series.

In Part One, I asked: Is Preschool Necessary? The answer? No, if there is enough enrichment, play-based activities, and speaking to your child at home. Which means the majority of middle-to-upper class families likely don’t need preschool. So what about more disadvantaged kids?

Enter Part Two, in which I asked: Does Head Start Work? The answer? To a point, yes. Like any preschool, it provides modest gains in kindergarten-1st grade. The problem is the 3rd/4th grade fade-out, which affects almost any preschool-type learning for almost any socio-economic group. So, short-term – yes, it often is helpful. Long-term – the jury is out.

So here we have Part Three. We are spending exorbitant amounts of money to educate disadvantaged kids through Head Start and other programs, with spotty results. We are also now advocating universal prekindergarten, even though the evidence shows preschool in general provides spotty results. There are confounding factors throughout, including the way one measures achievement, the varying degrees of quality in Head Start programs across the nation, and the quality of the schools the children enter for elementary.

Here we ask: So what is the measure of quality? What makes one program better than the other? And do we trust these measures of quality?

The Oklahoma Experiment vs Georgia vs Florida

First, let’s clear up a misstatement in President Obama’s State of the Union speech:

Tonight, I propose working with states to make high-quality preschool available to every child in America.  Every dollar we invest in high-quality early education can save more than seven dollars later on – by boosting graduation rates, reducing teen pregnancy, even reducing violent crime.  In states that make it a priority to educate our youngest children, like Georgia or Oklahoma, studies show students grow up more likely to read and do math at grade level, graduate high school, hold a job, and form more stable families of their own. (my bolding)

Somewhat disingenuous.  The bolded parts are in reference to very specific preschool programs  such as the Perry Preschool Project and the Abcerdarian Experiment. Hardly a large scale universal prekindergarten program such as Oklahoma’s. No universal prekindergarten program has been able to prove that students are more likely to graduate high school, hold a job, or form more stable families – as a matter of fact, they are too new anyway.

Despite that misstatement, Oklahoma (Tulsa, specifically) does boast a “quality” Head Start program as well as a “quality” prekindergarten program. Due to the presence of these programs and a 74% enrollment rate for Tulsa prekindergarteners, Tulsa-area prekindergarten and its Head Start program is a subject of a multi-year Georgetown University research project.

However, first, we must define quality, as it is a contentious measure.  What most researchers call the ‘accepted’ measures of a program’s quality are measuring input factors – the amount of money spent per student, the student-teacher ratio, teacher pay and certification, and the facilities in which the children learn. Input then, rather than output.

Let’s look at input first.

Input

Two standards of quality are the environment rating scales developed by the Frank Porter Graham Early Childhood Center at the University of North Carolina- Chapel Hill.   The first is the revised Infant/Toddler Environmental Rating Scale (ITERS-R) and the second is the Revised Early Childhood Environmental Rating Scale (ECERS-R).  Each features seven subscales: space and furnishings, personal care routines, listening and talking, activities, interaction, program structure, and parents and staff.

A third standard of quality is that of the National Institute for Early Education Research (NIEER). These are the ones cited the most often in articles lauding Oklahoma’s program (which scores 9/10 on the scale). The ratings are for: Comprehensive Early Learning Standards; Teacher has BA; Specialized training in pre-K; Assistant teacher has CDA or equivalent;  At least 15 hrs/yr in-service; Class size 20 or lower; Staff/child ratio 1:10 or better; Vision, hearing, health, and one support service; At least one meal; and Site visits.  Georgia scores 10/10.  Florida, on the other hand, scores 3/10. A score of articles published in Oklahoma (including one by This Land Press) cite this scale as measure of Oklahoma’s program being better than Florida’s, evident also in the money Oklahoma has spent on their program versus Florida.

However, despite the fact that NIEER lauds programs such as Oklahoma’s and Georgia’s, while scorning Florida’s, the NIEER standards are a much inferior form of measuring classroom inputs than the ECERS/ITERS scale. ECERS/ITERS have been linked to positive classroom performance through at least the second grade of school, and is measured by a trained observers’ analysis of how a classroom functions, as opposed to the dry measures of NIEER.

To properly judge a program, one would need a mixture of an ECERS/ITERS scale and measuring actual results into kindergarten and beyond.  The NIEER standards can serve as yet another measure of a program’s quality, but the general fallacy found in the early education argument today seems to be using the NIEER standards as the only measure of quality. Hence, the mistaken belief that Florida’s program is automatically inferior to Oklahoma’s and Georgia’s.

In Tulsa, Oklahoma’s case, CAP Head Start, Educare, and the state-run prekindergarten program are quality based both on the NIEER criteria and the ECERS criteria.

Head Start in Tulsa County

Head Start in Tulsa is administered by Community Action Project of Tulsa County (CAP). CAP took over the Head Start grant in 1998 (incidentally, when the universal prek program was written into law). Prior to CAP’s arrival, Head Start was administered by the now-defunct Tulsa Community Action Agency, which had severe financial and management problems.

CAP Head Start is lauded as a model program by the national office of Head Start. Among its characteristics:

– A requirement that all lead teachers have a bachelors degree and are early childhood-certified (in contrast, National Head Start requires 50% of lead teachers to have a BA by Fall 2013).

– Pay and benefits equal or greater to prekindergarten teachers in the Tulsa Public School system (and TPS prekindergarten teachers are paid the same rate as elementary school teachers)

– A class ratio compliant with National Head Start standards (10:1)

– Collocation of early childhood centers near elementary schools; in some cases centers are in the schools. In other cases classes are mixed billets – some students are Head Start and some students are state-funded prekindergarten students. Finally in other cases Head Start serves the three-year-olds and the public school serves the four-year-olds.

– Seamless collaboration with Tulsa Public Schools, Sand Spring Public Schools, and Union Public Schools. This includes constant tweaking of best practices in order to create a unified approach to early childhood education in the county. It also means coordinated school calendars, and collaboration on direct services such as breakfast and lunch.

– Partnerships with other entities such as Educare, the George Kaiser Family Foundation, and Family and Child Services

– Financing beyond federal Head Start dollars – Federal, state, and local grants; private donations; low-interest loans; and tax-exempt bond financing.  Funding includes money from the George Kaiser Family Foundation, Early Head Start, Head Start, Title I, the State of Oklahoma Pilot Early Childhood Program (now called the Early Childhood Program), individual donations, the Tulsa Area United Way, and the Oklahoma Department of Commerce. CAP of Tulsa County Head Start also receives 90 percent of the funding per student that would otherwise flow to TPS if there were no collaboration.

-Tailored service at different sites depending on need, all in collaboration with the public school. This includes wrap-around whole family services, nutrition services, and special education services.

– Data-driven practices – CAP does its own internal policy research, collaborates with other think-tanks and nonprofits on research, assists Georgetown University in its research project, and outsources research to the Early Childhood Department at Oklahoma University -Tulsa.

– Many of CAP’s buildings are state-of-the-art, designed to encourage learning and exploration.

– CAP also has missions other than Head Start, albeit complementing the efforts of Head Start. This includes a wholesale family and community advancement approach to poverty alleviation.

– Wholesale effort to encourage interaction by parents. No busing, so parents interact with the teachers. Parenting classes, career training, and financial services.

State-funded prekindergarten

The state-funded universal program was basically sneaked into a bill in 1998, by a lawmaker who ostensibly was to fix what was known as the ‘four-year-old’ problem – small school districts who were enrolling students younger and younger in order to keep state funding.  The bill, not widely read line by line, passed easily. The elements of the state-funded program:

– Allows for collaboration with entities such as Head Start under the 1998 law, as long as both parties agree.

– Maintains a 10:1 class ratio.

– Lead teachers must have bachelors degrees and certifications in early childhood.

– Teachers are paid a regular public school wage.

– At the time of the Georgetown study, TPS received approximately  $3,700 per student for every student enrolled full-time in pre-K (and approximately $2,000 per student for every student enrolled half-time in pre-K).

– As of this writing, only four school sites in Tulsa Public Schools do not have any prekindergarten program; however, there are spillover programs available at early childhood development centers (ECDC) that include Head Start funds as well. There are magnet programs available via lottery. Some of the programs are part-time, some are full. Some are traditional school year; some are continuous calendar year.

Educare

Educare was founded in 2003 in Chicago by the Buffet Early Childhood Fund and the Ounce of Prevention Fund. The first site opened in Tulsa in 2006; there are now three in Tulsa, the most in any city (a fourth Oklahoman Educare is in Oklahoma).

Educare serves children from 6 weeks old to five years old.   It is another Head Start provider in Tulsa, although it generally uses Early Head Start funds for their younger children and state funds for the four-year-olds. It also receives funding from the Kaiser Foundation (its biggest donor) and the former State of Oklahoma Pilot program.  Like CAP, Educare is data-driven and outsources much of its data analysis to OU. Its buildings also are designed with learning in mind; I had the pleasure of touring Tulsa Educare III with beautiful breakout spaces, play areas off each wing, and child-sized restroom facilities. The whole scene was very reminiscent of another tour I took recently of Riverfield Country Day School, a Reggio Emilia school.

Educare also has a low teacher/student class ratio, master teachers with BAs, and programs for parental career advancement (called Educareers). Since much of its quality control is based on direct classroom observation, it has observation rooms in which an educator and evaluator can watch a classroom without bothering the students.  To qualify for Educare, a parent must be DHS qualified and be working; therefore, unlike with CAP students, most of the parents are working and need full-time care. Like CAP, a goal of the program is to show parents that they and their children matter.

Output

So, we agree all of Oklahoma’s programs are considered quality in terms of input factors. In that sense, they are far superior to the national average and to Florida’s programs. But, as Chester Finn pointed out in his book Reroute the Preschool Juggernaut, why are we not looking at result-based accountability? Why automatically dismiss a program because it spends less money and because it has less-than-desirable inputs?

Oklahoma

Educare: The Frank Porter Graham Institute at the University of North Carolina Chapel Hill are in the midst of an implementation study and a randomized control study of five Educare sites. The implementation study found the longer children were in Educare, the better their results upon entering kindergarten. Upon entering kindergarten, long-term Educare students were up to their middle-class peers’ level of development. Of course, results upon kindergarten entry are only half of the story, and unfortunately, Educare does not yet have data on the later years.

CAP Head Start vs Tulsa Pre-K:

Head Start vs. Pre-K is a great example of how money does not necessarily imply superiority.

One noticeable fact is that the state-funded prekindergarten is noticeably cheaper per pupil than Head Start. So, as the Georgetown study authors state: “From the public officials’ perspective, Head Start is superior to state-funded pre-K only if its multiple benefits are high enough to justify its higher costs.”

So is it?

To answer that question, we must first acknowledge that Head Start has more than just an educational mission, and that its school ‘readiness’ program has a very broad definition of what constitute readiness. This contrasts with the state-funded prekindergarten’s exclusive focus on education.  The study’s authors argue that if Head Start shows increased social benefits and health benefits for its students versus those enrolled in Tulsa’s prekindergarten program, then Head Start may still have a place in the Tulsa early educational landscape.

According to the study, both programs had positive impacts on the children’s cognitive development. However, in pre-reading and in pre-writing, the state-run prekindergarten program was clearly superior.

Of course, Head Start only serves disadvantaged students, whereas Tulsa Public Schools serves the entire population. However, the researchers accounted for that by also looking at Tulsa Public School students who qualified for the free lunch program. The difference persists – TPS is better when it comes to pre-writing and pre-reading indicators.

In fact, the state-run prekindergarten leads to significant improvements for students with parents speaking Spanish at home in reading and writing skills.

For math, both programs equally prepared the children for kindergarten. And for social-emotional development, in general the TPS pre-K students were less timid and more attentive than a control group, whereas Head Start students showed no progress. However, when comparing TPS free-lunch students vs. Head Start students, there were not many differences, indicating some of the timidity and inattention may be related to outside demographic factors. There were however still some differences in attentiveness, with TPS ‘winning’ in that score.

In terms of health, the researchers indicated Head Start wins in that round, indicating Head Start therefore still has value in terms of promoting children’s health and therefore has a place in Tulsa’s early education landscape.

Possible explanations for the differences between Tulsa Prekindergarten free lunch students and CAP Head Start students:

Let’s talk pre-reading and pre-writing skills first. The researchers suggest the TPS program is superior to the Head Start program in pre-reading and pre-writing skills because it focuses more on direct educational instruction, whereas the Head Start program focuses more on fantasy play. That very well may be true; if students are explicitly taught to learn letter-sound combinations and how to hold a pencil to form a word, then they will perform better on a test.

However, does that automatically mean the TPS program is ‘better’ than Head Start in teaching literacy? Perhaps not. As I pointed out in an earlier post, most evaluations of pre-reading and pre-writing ignore the five pillars of literacy. Phonics and holding a pencil are only part of what helps a child read and write. Role playing, telling a story out loud, and similar activities are also necessary parts of pre-literacy; in fact, most studies indicate that in the preschool years, these play-based activities are more suited for developing pre-reading skills than direct instruction in phonics.

Despite my grave concern on the mismeasuring of literacy, we will use the word-letter identification and tracing/writing skills as proxies for improvement in literacy in order to compare results across ages and across state programs.

In terms of math, the researchers are flummoxed – TPS teachers spend more time on math than Head Start teachers do. Then why are the results similar? The researchers state (my bolding):

Perhaps children learn math more through well-chosen examples than through repetition,
in which case time on task could matter less for math than for language instruction. It is
also the case that, on average, CAP Head Start teachers had 2.5 math courses as
undergraduates, as opposed to 1.9 math courses for TPS pre-K teachers (p < .10), which
could translate into higher quality math instruction despite less time spent on this domain of learning.

Yes, they may have hit the nail on the head without even realizing it. ‘Direct math instruction’ may not be the best way to teach math. What the researchers see has fantasy play may also have elements of math instruction in it, cleverly taught in a way young children actually learn.

Which means, despite the fact that I bristle at cheering on Head Start over any other program, I continue to fret over how we measure learning.

Next is their analysis of social skills. TPS seems to have the comparative advantage on attentiveness. It may seem like a small thing; however, as the researchers point out, attentiveness is an important predictor of school performance. A possible explanation of TPS’s advantage, they state, is that most of the TPS students attend the same school as kindergarteners as the one they attended as preschoolers. The school, therefore, is a very familiar environment.

This, of course, calls into question the state of art facilities CAP and Educare boast. Although the sites are collocated or near the elementary schools, they are not in the same building.

Finally is the study’s belief that Head Start has a comparative advantage on the health of the child.  Not really my purview, but it’s important to note their measure of health was how many times a child saw a dentist in the past year. Makes sense that Head Start, with its on-site consultations and then referrals would lead to better dental health.

Long-term output in Tulsa

That’s all well and good in the short term. What about that dreaded third grade slump?

Unfortunately, there is no Tulsa-specific Head Start data on third graders. However, there have been two studies on Tulsan third graders who attended the state-run prekindergarten.

The early cohort showed no measurable gains by third grade, a common story.  For the second cohort, a few years later, there were measurable results for boys in math.

That result is celebrated by the study’s authors and by proponents of Oklahoma’s prekindergarten system. Detractors, of course, would be skeptical of the cheer surrounding one success among one subgroup in one subject.

Regardless of how to view this change, the question of why the second cohort has a success whereas the first showed no change is easily answered. The first study was done in 2000-2001 and the second in 2005-2006.  As the authors explain,

For the early cohort, Oklahoma’s universal pre-K program was only in its third full year of operation. TPS and other school systems had to implement the pre-K program relatively quickly, making many choices about hiring, curriculum, and professional development. Even if all these choices were sensible, such a large shift in policy takes time to implement effectively. Thus, one difference between the two cohorts may be that the pre-K program was more mature and running more smoothly in 2005-06 than it was in 2000-01. Parallel program adjustment may also explain the differences across cohorts. For short-term learning gains to be sustained, changes are likely needed in K-3 instruction – what is taught, when it is taught, and how it is taught. If K-3 teachers do not adjust curriculum and teaching strategies to account for the growing presence of children who experienced preschool, then comparison group children are likely to catch up while learning gains of treatment group children are likely to stall. Reports from the field in Tulsa suggest growing awareness of school readiness improvements, which were well-publicized in Tulsa and which many teachers reported noticing themselves (Community Service Council of Greater Tulsa 2002).

Lessons learned from Tulsa

So what can we learn from Tulsa? First, a quality program is going to lead to better short term results. The input quality matters; unfortunately, it’s hard to judge which input matters. Those familiar with NIEER standards would point out that Oklahoma hits 9 out of 10 of its quality control measures. However, Oklahoma also hits the majority of the ECERS/ITERS measures, which are not as related to expensive measures such as higher teacher pay and instead related to how classroom instruction is conducted. The two are likely related, but the absence of high pay does not necessarily commute to the absence of good classroom instruction.

Second, despite these quality inputs, by the traditional measures, the outputs still are faltering. Even massive participation in a universal system – which arguably changed the culture of the early elementary years – appears to result in only modest gains. However, as I point out numerous times, the issue there may be with how things such as reading and writing are taught, thanks to No Child Left Behind and the Common Core. It is telling that math instruction seems to be the same across Head Start and TPS students, and that math is the one subject in which we see progress in third grade boys.

We’ll get into what Oklahoma means, policy-wise, in part four. In the meantime, let’s quickly move on to two other state programs and see how they fare.

Georgia

Georgia, if you remember, scores a hearty 10 out of 10 on the NIEER factors. Universal Pre-K classes are provided by a wide range of approved facilities, including public schools, Head Start centers, private child care centers, faith-based and other nonprofit centers. A wholescale study was done on its program. What the study found was the availability of universal prekindergarten helped disadvantaged children  living in small town and rural areas, increasing both reading and math test scores at fourth grade as well as the probability of their being on grade for their age.

In addition, in 2001, research began examining Georgia Head Start versus the Georgia universal prekindergarten (GPK) program. In a 2003 paper, the researchers found GPK students showed at least as much improvement as private preschool students and the same as Georgia Head Start programs – so there was really no comparative advantage of one program over another.

Lessons Learned:

The main takeaway from the Georgia study is that before universal prekindergarten, those students did not have access to many preschool programs. Therefore, the increased availability of preschool led to results.

The other takeaway, however, is that those who do have access to a preschool do not necessarily need a universal program.

Florida

Prospect is one paper that immediately puts Florida in the bad column, as does This Land Press. The reasoning is that Florida spends to little on each child and doesn’t require its teachers to have college degree in early education. This Land Press continues that Florida only meets three out of ten NIEER standards.

So what does Florida have that we can emulate?

Input

– Class ratio and class size limits

– although it does not require the  teachers to have college degrees, they must have Child Development Associate certificates and be trained on literacy

Output

-It strives to be a prekindergarten program, not childcare.

-It has standards that they update every three years, focusing on social/emotional development, motor skills, literacy, and pre-math skills.

– It strives to be results based. It cares less about the input (in terms of the regulations it imposes) and more about the output.

– The Florida Voluntary Prekindergarten (VPK) program is mostly not run by the public schools. Private childcare and preschool operators, including faith-based programs, comprise the majority if the providers, although school districts could sign up to be providers if they wished. It is easy to become a VPK provider, but not as easy to stay a VPK provider – non-performers are dropped from the program. And, its education standards are robust, and aligned with the public schools.

– When choosing a provider, parents have complete transparency regarding their compliance with education standards and accountability.

Lessons Learned:

Quite simple, and obvious. Input is important when judging a prekindergarten program’s quality – but we can’t ignore output. And one way to have a good-output program is to have standards aligned with the public schools, require transparency, and penalize for lack of results. That idea can be controversial precisely because of the continued niggling question – are these the right standards to ask of early childhood education providers?

Next up: Part Four – “Why Johnny is Broke” – the policy proscriptions that follow from these data.

Check out these other Carnival posts!

The Early Education Racket (Melinda Wenner Moyer)

Preschool Should Be Less School and More Play? (Momma Data)

Preschool at Home? Let the Children Play! (School of Smock)

Mixed-Age Preschool: Benefits and Challenges (Science of Mom)

Picking a Preschool (Momma, PhD)

Universal Prekindergarten: Evidence from the Field (Six Forty Nine)

What Can We Learn from a Single Preschool Study? (Red Wine and Apple Sauce)

Preschool, Shmeschool (Fearless Formula Feeder)

9 thoughts on “Universal Prekindergarten: Evidence from the Field

  1. Pingback: Welcome to the Carnival of Evidence-Based Parenting! First Edition: Preschool | Science of Mom

  2. Pingback: Preschool at Home? Let the Children Play! - School of Smock | School of Smock

  3. Pingback: What can we learn from a single preschool study? « Red Wine & Apple Sauce

  4. In other words, it’s complicated! You illustrate so well the complicated task of evaluating early education programs. It’s completely understandable from the standpoint of a child’s “best” interests, limited time and money, etc that these interventions are multi-faceted (with often different goals) and thus provide a challenge for comparing, contrasting and targeting the exact features produce results.

  5. Thank you, and I think this fine analysis deserves a wider publication. Have you considered submitting it elsewhere?

    Polly, my Friend, ‘complicated’ is insufficient and dismissive, I think. Seems like those who believe in universal preschool will do anything to say that the results are not in while an analysis like this clearly shows that ‘universal’ is unsupportable.

    “Each features seven subscales: space and furnishings, personal care routines, listening and talking, activities, interaction, program structure, and parents and staff.” Just think, what if something like these criteria were applied to homes of children just after birth? (on the order of you cannot take a baby from the hospital if you do not have a car seat) A form of home intervention is provided by the early intervention portion of the Individuals with Disabilities Education Act (IDEA or the law dictating special education). Far from perfect and lacking any outcome measures as criteria for program evaluation, I do believe EI programs change families, save the lives of children and foster advancement of babies into the specialized preschool programs.

    Also, consider exploring an in home parent support program named Avance, started in San Antonio.

  6. I teach in a Tennessee Voluntary Pre-K classroom and all of my students come from homes considered by federal guidelines to be financially at-risk. This year will complete my eighth year and I can state without a doubt that the 160 children that have successfully navigated my program have entered Kindergarten ready to be successful life-long learners. With the day-to-day research I have accumulated in the past eight years, I am a advocate for universal Pre-K so that all students can receive the same benefits as my students. That being said, I do believe that teachers will need to be trained in best practices for 4 year olds and be monitored to make sure that all students receive the maximum benefit from this first year of school.

  7. Pingback: Why Johnny is Broke: Marrying Market-Driven Principles with Government Excess in Early Education | Six Forty Nine

  8. Pingback: Technology and Literacy: Are Screen Time Restrictions Necessary? | Six Forty Nine

Your thoughts?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s