Listening Assessment Design and Development Workshop and Symposium
The series of workshops consisted of a pre-project training session, 2 workshops, and a symposium.
Where: Synchronous virtual workshops, Arizona State University (ASU) (in-person format), Big Ten Conference Center in Chicago, IL (hybrid format).
When: Online virtual training on January 12 & 19 and January 26 & 27, 2024; In-person training at ASU March 8 & 9, 2024 (with material repeated virtually March 11, 13 & 15 for participants in SE Asia); Chicago May 31 – June 1, 2024.
What: SEALC offered a series of in-person and virtual workshops in the spring of 2024 with the goal of designing listening assessment tests for 6 SE Asian languages at the intermediate and advanced proficiency levels as defined by ACTFL.
The pre-project workshop was a synchronous virtual event on January 12 and 19 at which participants were offered a review of the basic principles of teaching for proficiency and were introduced to the key principles and concepts of language assessment.
Workshop 1: Comprehension in the Context of Listening and Identifying Appropriate Listening Inputs was a synchronous virtual event for all participants in the US and SE Asia on January 26 & 27. Participants learned the target functions of intermediate and advanced listening texts. Participants examined examples of intermediate and advanced level listening texts prior to identifying and selecting three texts in their own languages at the two proficiency levels. Each pair of participants brought three listening texts for the two levels to Workshop 2.
Workshop 2: Listening Assessment Development was offered twice, once in-person at Arizona State U. (March 8-9) and once virtually for those teaching in SE Asia (March 11, 13, & 15). This workshop described the target functions for the intermediate and advanced levels to be elicited, relating them to the ACTFL guidelines. Using the text template provided, the participants created preliminary prompts and tasks for the listening test questions . Then the workshop trainers, Catherine Baumann and Ahmt Dursun, reviewed the work of each group. . Ahmet Dursun explained how to create rubrics for evaluating the newly developed tests. An ancillary benefit of this workshop activity is the fostering of collaborative relationships and a community of practice among the US-based and SE Asian-based language instructors. This professional network will serve the field of SE Asian language teaching well into the future.
At Arizona State University there were 32 participants representing 5 SE Asian languages (Burmese, Filipino, Indonesian, Thai and Vietnamese) and coming from 16 institutions (Arizona State U., Brown U., Cornell U., Harvard U., U. Hawaii at Manoa, Johns Hopkins U., U. Michigan, Northern Illinois U., Ohio U., Portland State U., U. Pennsylvania, UC-Berkeley, UC–Los Angeles, UC–San Diego, U. Washington, and U. Wisconsin-Madison).
The virtual workshop for those working in SE Asia had 32 instructors, representing 5 languages (Filipino, Indonesian, Lao, Thai, & Vietnamese) and 12 institutions: (in the Philippines Central Luzon State U.; in Thailand Chiang Mai U. and Chulalongkorn U.; in Singapore National University of Singapore; in Japan Ritsumeikan Asia Pacific University; in Indonesia State University of Malang; in Vietnam the IELTS Workshop in Hanoi and Ton Duc Thang University in Ho Chi Minh City; U. Hong Kong. ).
March – May collaborative work
During these two months, the participants worked virtually in pairs and developed two listening assessment tests, one each for the intermediate and advanced levels. The tests were reviewed by the trainers and revisions were made prior to the pre-piloting of the tests when the groups took each other’s tests and offered comments. Revised tests were submitted to SEALC.
Symposium
The final Symposium which was held at the Big !0 Conference Center in Chicago on May 31 & June 1 was a hybrid event. All groups made presentations on their Listening Assessment tests based on the questions below. Following each presentation, there were questions, comments, and recommendations from the assembled group. The in-person attendance was 38 and the virtual attendance was 26.
- What challenges did your group encounter when designing and developing the tests? Did linguistic or cultural issues play a role?
- What did you learn when you took the test designed by the other group? How did that inform your own test design skills?
- How has participation in the listening proficiency assessment development impacted your teaching? Your testing?
- Are there any areas you’d like to research or investigate in the future?
Deliverables
50 Listening Assessment tests at the intermediate and advanced levels for Burmese, Filipino, Indonesian, Lao, Thai, and Vietnamese have been submitted to SEALC. These tests will be made accessible to all who participated in this series of workshops.
The Trainers
Catherine Baumann and Ahmet Dursun, language assessment experts from the University of Chicago, provided the high quality, intensive training for all the workshops. Catherine Baumann, director of the University of Chicago Language Center, is a certified ACTFL tester and trainer, and consults at the secondary and tertiary levels on language program design, assessment, and curricula development. She oversees all programs at the University of Chicago Language Center. Ahmet Dursun is the Director of the Office of Language Assessment, where he is responsible for researching, developing, and managing the University’s language assessment programs. He is co-founder of the Language Pedagogy Innovation Initiative (LPII) at the University of Chicago where he has been designing and leading workshops in second language assessment and pedagogy. His research and publications on computer-assisted language teaching, testing research and practice, and test validation have appeared in multiple peer-reviewed journals, edited volumes, and books.
