9 Training and Professional Development
Chapter 10 of the Dynamic Learning Maps® (DLM®) Alternate Assessment System 2015–2016 Technical Manual—Science (Dynamic Learning Maps Consortium, 2017) provides a complete description of training and professional development for DLM assessments, including the required test administrator training and the optional instructional professional development made available.
This chapter presents updates to the required test administrator training content as well as participation rates and evaluation results from the 2022–2023 optional instructional professional development modules. This chapter also describes three new science professional development modules.
9.1 Updates to Required Test Administrator Training
Training is required annually for all educators who administer the DLM alternate assessment. In 2022–2023, the training was updated with additional features to support educator learning.
Each training module for new test administrators now includes an optional note-taking guide that trainees can use to stay engaged and focus attention on the training topics. Following the presentation, trainees can check their answers in their notes using an answer key. Additionally, an optional interactive activity is provided at the end of each module to help trainees further confirm their understanding of the module content. Trainees can also download each module’s slides and transcripts for reference. These optional components of the training are included with the facilitator resources for facilitators to use at their discretion.
The training for returning test administrators consists of a single module and does not include a note-taking guide or an interactive activity, but returning trainees are given the option of downloading the module’s slides and transcript for reference.
The training for both new and returning test administrators concludes with a “Helpful Reminders” document that trainees can download and keep for reference if desired. The purpose of this document is to focus trainees on salient highlights of the information addressed throughout the training.
9.2 Instructional Professional Development
The DLM professional development system is built to support educators in their efforts to teach science to their students who participate in the DLM system. In addition to providing instructional support, the instructional professional development modules also teach educators about the DLM system. While the modules were originally intended for educators who administer DLM assessments, system data suggest that preservice educators, related service providers, parents, and others also access and complete the modules.
To support state and local education agencies in providing continuing education credits to educators who complete the modules, each module also includes a time-ordered agenda, learning objectives, and biographical information on the faculty who developed the training modules. There are a total of 10 science modules, which are described in section 9.2.1.2 of this chapter.
The 10 modules are available in both self-directed and facilitated formats. The self-directed modules are available online, on-demand. The interactive modules include a combination of video-based content, embedded activities, and, for participants who would like to receive a certificate documenting their successful completion of the module, a pre- and post-test. These certificates are sent directly to each participant’s email when they achieve a passing score on the post-test.
During the 2022–2023 school year, DLM staff worked in cooperation with the professional development team at the University of North Carolina at Chapel Hill to develop three new science professional development modules. Two of the modules were released during 2022–2023, and the third module will be released for the 2023–2024 year.
The new professional development modules are intended to help teachers learn about and teach specific science and engineering practices:
Science and Engineering Practice 2: Developing and Using Models– defines different types of scientific models and gives examples of how to use them in science instruction to engage students in academic science learning.
Science and Engineering Practice 3: Planning and Carrying Out Investigations– focuses on supporting teachers, including an example activity, in having students engage in carrying out scientific investigations.
Science and Engineering Practice 4: Analyzing and Interpreting Data– focuses on how students can analyze and interpret data and how to engage students through accessible activities.
Each module also includes guidance for teaching science content through differing levels of complexity using DLM Science Essential Elements and linkage levels. The writing and production process of the science modules was consistent with the approach used with existing English language arts (ELA) and mathematics professional development modules to ensure a more streamlined DLM learner experience.
9.2.1 Professional Development Participation and Evaluation
There are two ways in which test administrators and educators may complete professional development modules: required test administrator training and optional professional development. Some states require one or more of the optional professional development modules to be completed as part of the required test administrator training. States can require that certain modules be completed by new test administrators, returning test administrators, or both. Test administrators completing professional development modules as part of the required test administrator training access the modules through the Moodle training site where the rest of the required test administration training is located. The second way in which professional development modules are completed is through the DLM professional development website. The modules on the professional development website can be accessed by anyone and completed at any time. Participants completing modules on the website are administered a short evaluation survey following each module. The evaluation survey is not included in the required test administrator training. With the exception of the evaluation survey, the content of the modules is identical in the required test administrator training and the professional development website.
9.2.1.1 Required Professional Development Participation
A total of eight states required at least one professional development module as part of their required test administrator training. The modules included in the required test administrator training are required of all relevant test administrators (i.e., new or returning, as specified by the state). For example, a test administrator who only administers science assessments may still be required to complete a module on instruction for ELA. Table 9.1 shows the number of modules required, by state, for new and returning test administrators, as well as the total number of modules completed in 2022–2023. In total, 9,122 professional development modules were completed by 2,572 new and 2,409 returning test administrators as part of the required training.
|
New test administrators
|
Returning test administrators
|
||||
|---|---|---|---|---|---|
| State | Required modules | Test administrators | Required modules | Test administrators | Total modules completed |
| Delaware | 2 | 115 | — | — | 230 |
| Maryland | 4 | 512 | — | — | 2,048 |
| New Hampshire | 1 | 143 | — | — | 143 |
| New Mexico | 2 | 339 | — | — | 678 |
| Oklahoma | 5 | 510 | 1 | 1,222 | 3,772 |
| Rhode Island | 2 | 111 | 1 | 166 | 388 |
| West Virginia | 1 | 194 | — | — | 194 |
| Wisconsin | 1 | 648 | 1 | 1,021 | 1,669 |
Table 9.2 shows which modules were required for new and returning test administrators across all states choosing to include professional development modules in the required training. For example, the Principles of Instruction in ELA module was required for new test administrators in three states and was required for returning test administrators in one state.
| Module | States requiring for new test administrators | States requiring for returning test administrators | Total modules completed |
|---|---|---|---|
| DLM Claims and Conceptual Areas | 2 | — | 624 |
| DLM Essential Elements Overview | 5 | — | 1,585 |
| Effective Instruction in Mathematics | 2 | — | 1,014 |
| Individual Education Programs Linked to the DLM Essential Elements | 2 | 1 | 783 |
| Principles of Instruction in ELA | 3 | 1 | 2,685 |
| Who are Students With the Most Significant Cognitive Disabilities? | 4 | 1 | 2,402 |
9.2.1.2 Optional Professional Development Participation
Table 9.3 shows the number of individuals who completed optional professional development modules through the DLM professional development website, as well as the total number of test administrators from each state who had a student assigned to them for the DLM assessment. In total, 341 modules were completed in the self-directed format from August 1, 2022, to July 31, 2023. Since the first module was launched in the fall of 2017, a total of 3,843 modules have been completed on the professional development website.
| State | Participants | DLM test administrators | Total modules completed |
|---|---|---|---|
| Alaska | 0 | 120 | 0 |
| Arkansas | 2 | 749 | 2 |
| Delaware | 5 | 144 | 11 |
| District of Columbia | 1 | 101 | 2 |
| Illinois | 7 | 1,969 | 37 |
| Iowa | 6 | 512 | 21 |
| Kansas | 1 | 547 | 6 |
| Maryland | 3 | 977 | 14 |
| Missouri | 0 | 1,127 | 0 |
| New Hampshire | 0 | 194 | 0 |
| New Jersey | 7 | 1,852 | 120 |
| New Mexico | 1 | 437 | 23 |
| New York | 2 | 2,736 | 5 |
| North Dakota | 0 | 158 | 0 |
| Oklahoma | 0 | 938 | 0 |
| Pennsylvania | 2 | 3,318 | 2 |
| Rhode Island | 2 | 182 | 8 |
| Utah | 6 | 729 | 27 |
| West Virginia | 0 | 329 | 0 |
| Wisconsin | 16 | 1,019 | 50 |
| Non-DLM state and other locations | 6 | — | 13 |
| Note. Participant counts may include individuals who are not educators or test administrators (e.g., pre-service educators). |
To evaluate educator perceptions of the utility and applicability of the modules, DLM staff ask educators to respond to a series of evaluation questions on completion of each self-directed module. Educators responded to questions about the importance of content, whether new concepts were presented, the utility of the module, and whether educators planned to use what they learned. Educators respond using a 4-point scale ranging from strongly disagree to strongly agree. During the 2022–2023 year, across all modules, educators completed the evaluation questions 84% of the time, with response rates ranging from 33% to >99% per module. The responses were consistently positive, as illustrated in Table 9.4. Across all modules, 78% of respondents either agreed or strongly agreed with each statement.
To evaluate the consistency in the ratings for each module, we calculated Cronbach’s (1951) alpha from the four items for each module using all ratings from fall 2017 through the 2022–2023 year. Across all modules, alpha ranged from .94 to >.99, with an average value of .97, suggesting high internal consistency in responses.
9.3 Conclusion
The DLM system makes training and instructional professional development modules available to test administrators and educators as well as state and local education agency staff. The required test administrator training is designed to ensure that test administrators understand the DLM system to administer the assessments with fidelity. Complementary to the required training are the professional development modules, which provide participants learning opportunities to support instructional practices aligned with content measured by DLM assessments. In 2022–2023, participants provided consistently positive feedback regarding the importance and relevance of the professional development modules, although participation was low relative to the educator population. Finally, three new modules were developed, two which were released this year and one which will be available to educators in future years.