VAL-Ed is designed, developed and tested to be both reliable (i.e. provide accurate measurement) and valid (i.e. measure leadership behaviors that lead to improved student achievement). To accomplish these goals, we are following a multi-stage development process that involves cognitive labs, pilot tests, and field tests. At each stage of the design and develop process, the properties of the instrument are being investigated through empirical study and expert review. The process is guided by the American Educational Research Association's 1999 Standards for Educational and Psychological Testing. Some of the steps that have been taken to ensure sound psychometrics include:

Item & Response Scale Development
All items and response scales were developed to be aligned with the ISLLC standards. Framework and assessment were critiqued by panels of education leaders and researchers.

Item study
To establish content validity, items were independently sorted by three respondents into our 36-cell framework. Responses indicated strong content validity and guided further item revisions.

Pilot/cognitive Interviews
To investigate assessment format, item clarity, and item importance, two rounds of cognitive interviews were performed with panels of practicing educators. Responses indicated practitioner preferences and understanding about assessment format and item-level language.

Nine-school pilot test
To examine how VAL-ED works in a field setting, an urban district was recruited to participate in a pilot. Data from the pilot will enable us to begin establishing reliability for the twelve scales and construct validity for the assessment using factor analysis.

Cognitive Interviews of Online Prototype
To examine the utility of an online prototype of the VAL-ED and test changes made after the nine-school pilot, cognitive interviews were conducted with teachers, principals, and supervisor from three schools. Responses indicated a preference for the online assessment and helped identify important issues to address in future development.

Bias review
The instrument will be submitted to a bias review committee of urban district leaders to ensure the assessment does not advantage respondents for anything other than the leadership behaviors we seek to measure.

Field test
In Spring, 2008, the instrument will be field tested in approximately 300 schools. Responses will be used to establish validity and scale reliability.

Proficiency standards
A panel of education leaders will be convened, using a holistic/body-of-work approach to set "national" proficiency standards based on pilot study/field test data.

Study to Evaluate Performance Level Descriptors
A panel of education leaders was convened to review the performance level descriptors created for use with teh VAL-ED.

 

For more about the psychometric properties of the VAL-ED assessment or dowload reports, please see teh featured publications section of this website.