Collecting Data to Promote Positive Change

by Marie Masterson, PhD


This document may be printed, photocopied, and disseminated freely with attribution. All content is the property of the McCormick Center for Early Childhood Leadership.

Try entering a classroom or family child care setting at children’s eye level. What objects or activities are waiting to invite children’s exploration and inspire their curiosity? What interactions strengthen the love of reading, communication with others, and feelings of competence? Are children solving problems and collaborating with others? Are there safe and meaningful relationships among peers? How do adults facilitate engagement and learning?

This perspective – evaluating the experiences of children – motivates the work of the McCormick Center Quality Assessment Teams. Our focus involves the daily life of children who spend 35-40 hours a week in family child care, community-based programs, school-based preschool, and after-school settings. Are they getting all that is needed to prepare them to be caring and kind, ready for school, and ready for life? Our assessment teams are dedicated to helping program leaders and teachers understand the many ways their work influences children and families.

As part of quality improvement systems, assessment teams that can deliver reliable, accurate data are essential. Reliable and valid data collection means that an assessment conducted on a Monday provides the same congruent results as an assessment conducted on a Thursday. Whether one assessor or another observes a specific setting, both will evaluate what they see and hear using the same lens and criteria resulting in consistent scoring and feedback. Professional assessors view each early childhood setting through a shared lens of criteria, such as the Classroom Assessment Scoring System (CLASS), the Environment Rating Scales (ERS), or the Program and Business Administration Scales (PAS and BAS).

To be prepared for excellence, our teams are trained by the authors or professional instructors for each scale. Once initial reliability is attained, we continue to practice, review, discuss, and compare notes. This rigorous and detailed focus ensures that each assessor takes the same perspective when they view a setting. We practice scientific standards of reliability, where observations are double coded to ensure accuracy – meaning that both assessors who evaluate a setting at the same time do so in the same way.

As part of the observation and assessment process, the team takes detailed notes, records conversation samples, and considers the way teachers engage children in learning. They review the physical setting to be certain it provides access to stimulating materials. They listen and watch interactions throughout all aspects of the day, including teaching times, snacks, meals, self-care routines, and outside activities. They watch to see that teachers are tuned in and actively supporting engagement. They listen for the quality of language being used and look to see that teaching is child-centered and individualized. They listen to hear that teachers build on children’s questions and engage them in meaningful conversations about materials, activities, and play. By watching and listening, the team learns a great deal about high-quality teaching.

Our profession recognizes the primacy of play in children’s learning and development. Play is the way young children explore, examine, test their ideas, and develop their skills. During observations, assessors consider the way materials are prepared to promote complex dramatic play. They look for interest areas that provide compelling tools and materials that reflect and connect to children’s real-life experiences. Importantly, they look for culturally and linguistically responsive teaching, which builds on the strengths of the family, neighborhood, and community.

When states focus on building aligned systems of support for the early childhood profession using quality assessment, everyone benefits. First, quality assessment stabilizes the system by providing cohesive approaches to evaluation and data collection. As a result, conversations can take place among and across programs, as well as across systems. Accurate data collection and feedback support quality improvement in programs in the following ways:

  • As programs prepare for an assessor’s visit, they take pride in being part of quality improvement processes. They communicate this priority to teachers and families.
  • As programs participate in quality assessment, they learn how accurate evaluation provides needed data for reflection. This process activates a growth mindset and promotes active collaboration in continuous quality improvement.
  • Scientifically valid feedback provides concrete information for coaches and program staff who work together to set action steps and goals for quality improvement.
  • Children benefit from programs that participate in quality improvement efforts anchored meaningful data.
  • Families can make informed decisions about where to enroll their children based on reliable and accurate information about a program’s quality.

Accurate data collection and feedback also provide a needed anchor for research, program evaluation, and systems building. Questions to be explored include the following: What parts of high-quality teaching matter most for children? What kinds of coaching and training supports have the greatest impact? How can program staff partner with families to have the greatest influence on their children’s learning and development? These research questions inform state and national decisions and priorities for the field.

What are next steps? Researchers are exploring specific aspects of teaching quality. For example, they may find that an intensive focus on language and literacy, early math skills, or approaches that promote self-regulation are the factors that have the most impact on children’s development and learning. It may be that increments of quality make a difference up to a certain point, but that higher-level quality settings make an impact depending on the approaches used. While researchers work to clarify these issues, the field can be sure that high quality data collection can help provide needed answers.

Please contact the McCormick Center for Early Childhood Leadership to explore training opportunities to strengthen your skills and encourage the quality improvement processes of your program. We are here to support your success.

Marie Masterson, PhD, is the Director of Quality Assessment at the McCormick Center for Early Childhood Leadership at National Louis University. She is a national speaker, child behavior expert, researcher, and author of multiple books and articles that address high-quality teaching, early care and education, and parenting.