This document may be printed, photocopied, and disseminated freely with attribution. All content is the property of the McCormick Center for Early Childhood Leadership.
This resource is part of our Research Notes series.
Since 2009, the McCormick Center for Early Childhood Leadership at National Louis University has offered Aim4Excellence National Director Credential, an interactive online professional learning experience that focuses on the core leadership and management competencies that early childhood leaders need. To date, over 2,000 participants (directors, administrators, teachers, and family child care professionals) have completed one or more modules and over 1,300 have earned the Aim4Excellence National Director Credential. It was the first national director credential recognized by the National Association for the Education of Young Children (NAEYC) as meeting the alternative pathway training requirements for directors of centers seeking program accreditation. Nearly one-third of participants receive college credit for finishing the program. The content of Aim4Excellence is aligned with many state professional development credentials and the Aim4Excellence credential is embedded in two state quality rating and improvement systems.[1]
The nine online modules that comprise Aim4Excellence not only provide the basics of early childhood program administration-finance, program operations, and human resources management—but also the essential knowledge and skills that directors need to empower staff and lead organizations that adapt to changing conditions. Administrators learn to apply principles of effective leadership to create compelling visions for their programs, become agents of change, walk the talk of ethical behavior, and embrace the paradoxes of their roles. Each module is the equivalent of approximately 16 clock hours (or 1 semester hour) of instruction. The nine modules are:
Recently, the McCormick Center completed an evaluation of Aim4Excellence to assess its effectiveness. Findings from the study suggest Aim4Excellence participants were successful in completing the modules and their early childhood programs improved during the time they were enrolled in the program. The evaluation report describes the characteristics of Aim4Excellence participants and the programs they represented, completion rates, and change over time in administrative practices while directors were enrolled in Aim4Excellence. A full copy of the report is available here.
This study examined 1,372 individuals enrolled in Aim4Excellence, between January 1, 2014 and December 31, 2017. Participants worked for 555 different organizations, in 48 states and Canada. The racial composition of the participants was 58% White/Caucasian, 24% Black/African American, and 13% Hispanic/Latino persons. The average age of Aim4Excellence participants at enrollment was 42 years, ranging from 21 to 76 years-old. Although there was a broad range in participants’ educational background, nearly 60% of them had a bachelor’s degree or higher.
Since the Aim4Excellence program is administered online, data were collected directly from participants during the registration process and during their engagement in the program through two online sources. Descriptive statistics were generated on the participants and outcomes data were analyzed to determine completion rates and assessment scores for each module. A total composite score was created by adding each of the module’s assessment scores together.
To study the change over time in administrative practice, on-site assessments were conducted, using the Program Administration Scale (PAS),[2] at child care centers in four states (Delaware, Arizona, New Mexico, and Illinois) when Aim4Excellence participants first began the program and again when they completed it. The PAS is designed to reliably measure leadership and management practices of center-based programs using a 7-point rating scale (inadequate to excellent), with 25 items grouped into 10 subscale categories. Matched pre- and post- data were available for 30 of 58 centers assessed. Paired sample t-tests were used to analyze the average change in leadership and management practice.
Participants provided feedback by completing an evaluation survey at the end of each module and after finishing all of the modules. Participants rated the difficulty of each module on a 3-point scale: 1=piece of cake, very easy; 2=challenging but not overwhelming; 3=very difficult and challenging. They also rated their satisfaction on the content and organization of each module. The surveys used a Likert-type 5-point scale that ranged from 1=strongly disagree to 5=strongly agree. The survey responses were analyzed and descriptive statistics were computed. The results were examined for trends across the modules and compared with learning outcome results.
Participants completed several scored assignments throughout each module to assess their learning. An overall score of 70% was considered passing on these Evidence of Learning assessments. Average scores ranged between 87% (Module 3) and 96% (Module 5). Table 1 shows means, standard deviations, and percentage of participants who passed the Evidence of Learning assessments for each module.
Evaluation surveys indicated that participants were sufficiently challenged in their learning and highly rated their experience in Aim4Excellence. In eight of the nine modules, over 80% of respondents rated the modules as challenging but not overwhelming. Participants also rated their satisfaction with various aspects of the content and organization of the program. Across all modules, participants provided the highest ratings when asked whether the module challenged them to consider new and different viewpoints (M=4.44, SD=0.73) and provided the lowest ratings when asked whether the video segments, Internet links, and audio pieces worked smoothly (M=4.20, SD=0.98).
Results also suggest that when directors participate in Aim4Excellence administrative practice improves in their programs. Pretest and posttest results showed that Overall PAS scores increased .44 points (on a 7-point scale) or .19 standard deviations. Four of the subscales improved significantly: Center Operations, Child Assessment, Marketing and Public Relations, and Technology. Table 2 shows the change in the average overall PAS scores and nine subscale scores. Statistics include the pretest and posttest means and standard deviations, as well as the statistically significant change (t score) from the beginning to the end of the training.
Analysis of the 21 PAS items show significant changes in Staff Orientation; t(27) = 2.87, Facilities Management; t(27) = 2.38, Assessment in Support of Learning; t(27) = 3.05, and Use of Technology; t(27) = 2.64. Effect sizes were small to medium, ranging from d = .40 to d = .58.
Because Aim4Excellence is administered online, the availability of reliable and consistent data was greatly enhanced, with a robust sample of 1,372 participants over a four-year period. We can be confident in the results of this study because the large number of participants were from across the U. S., represent diverse demographic groups, and work in over 500 early childhood programs.
Persistence, as a measure of performance, was found to be strong with at least a 97% completion rate for each module. Results indicated that a majority of participants (51%) completed all nine modules. The average scores on the Evidence of Learning assessments were very high (ranging from 87% to 96%). The change over time study suggests that leadership and management practice improves in programs when their administrators participate in Aim4Excellence, particularly in the areas of orienting staff, managing facilities, assessing children’s learning, and using technology. Participants were very satisfied with their experience in the program and indicated they expanded their knowledge and expertise, were challenged to consider new and different viewpoints and found the resources interesting and informative.
[1] Aim4Excellence is embedded in the Iowa and North Dakota QRIS.
[2] Talan, T. N., & Bloom, P. J. (2011). Program Administration Scale: Measuring Early Childhood