The A*StarSM Audit ServiceThe A*StarSM Audit service is a comprehensive review of all classroom groups based on collected test results. Each class is measured according to the normal variation of its peers and a detailed analysis is performed on each irregular group. The audit is processed by sophisticated, patented, computer software. The review procedure and report elements are described below. Review ProcedureReview of the TestThe Audit review evaluates the test, not for what it measures, but for how well it makes the measurement. In this regard, the review examines the internal consistency and range of measurement; the suitability of the measurement for critical decisions (pass/fail, certification, program assignment, etc.); and the capability of the measurement to support an analysis of the standardization of the test administration. Review of the Test AdministrationThe Audit review applies an innovative, patented method of group test item response pattern analysis to evaluate the consistency of the test administration with the standardized procedures established by the test publisher. The review determines normative group response patterns and normal pattern variation. Response patterns of all individual test taker groups are evaluated against the appropriate norm and measured in terms of the normal variation. Exceptional groups are further evaluated for the cause of their deviation from the norm. Review of Irregular GroupsTest taker groups (by classroom, school, test center, etc.) with unusual test item response patterns are evaluated for the likelihood of inadvertent errors in test administration and potentially improper influence on test taker responses. Irregular Groups & Subject GroupsThe response patterns of irregular groups are highly unlikely to occur in a normal test administration. Most have subset, 'subject groups' that indicate the cause of the irregularity. The members of subject groups are identified and their item responses subjected to a probability analysis. The subject group response pattern is then compared to response patterns known to arise from particular patterns of proctor influence. The A*StarSM Audit ReportThe A*StarSM Audit report is provided as both a written document and as an electronic file with user friendly software to support further search and analysis. The Written ReportThe written report provides a psychometrician's analysis of the test, the test administration, and the significance of any problems found by the review. The report further provides a comprehensive listing of all proctor groups, the individual group Audit scores, and the group standings on the test and several measures of conformity with expected test performance. Individual client options determine areas of further analysis and the depth of the written review. The Electronic FileThe electronic file provides a versatile resource for the client to access the audit information for organizational use. The file includes a comprehensive listing of all test administrations and test taker groups along with A*StarSM measures of reliability and, where warranted, estimation of assessment problems. The client may utilize the electronic file to initiate a review of client selected test taker groups by group identification, by group characteristics or by group test results. Individual test takers may be identified whose scores are unlikely to support critical (i.e. pass/fail) decisions. The client may merge other client file information to permit, for example, a review of new proctors, or other organizational information. With multiple reviews conducted over time, the Report file may provide evaluation of test taker gains, or changes in instruction, recruitment, selection or other organizational programs. Characteristic Patterns
|