BME7900 Seminar Series – Kerrie Douglas, PhD
We welcome Dr. Kerrie Douglas from Purdue University as our next speaker in our seminar series. Dr. Douglas is an Associate Professor of Engineering Education at Purdue University.
Development and Validation Studies for the Professional Skills Opportunities Survey
Abstract: Industrial demand for engineers who are well-rounded professionals has been the emphasis of many engineering accreditation standards, professional organization reports, and research topics since the mid-1990s. In surveys of what employers are most looking for in new engineers, professional skills, such as communication, leadership, and teamwork frequently rank higher than technical skills. Common approaches to assessment of professional skills tend to fall into two categories: self-report of competency or an assessment of one specific aspect of one professional. Thus, there are limited approaches to understanding holistically how students are developing professional skills. The purpose of this talk is to present a theoretical framework for approaching how students develop into having effective professional skills, the development of the Professional Skills Opportunities (PSO) survey and present validity evidence for how to appropriately use the PSO scores. In particular, we asked: 1) To what extent do the items written to capture specific aspects of professional skill factor as theoretically conceptualized? 2) To what extent does the PSO measure the same construct when students from differing gender identities, racial identities, academic cohorts and university classification respond to items? The PSO was administered at 13 universities across the US, including 3 R1 research institutions, 3 undergraduate teaching focused-institutions, 3 Minority Serving or Hispanic Serving Institutions and 4 HBCUs. In total, we received 2,246 responses. We conducted confirmatory factor analysis to examine the items function according to the theoretical structure and two types of measurement invariance studies – one from an Item Response Theory perspective and the other utilizing Classical Test Theory. The results of the analyses support the use of the PSO across institutions to understand the opportunities that engineering students have to develop professional skills.
Bio: Kerrie Douglas is an Associate Professor of Engineering Education at Purdue University and Co-Director of SCALE, a large Department of Defense workforce development project in secure microelectronics. In that role, she leads the education and workforce development across 19 universities. She also leads the Science and Ethics for Educational Data (SEED) lab. Her research is focused on improving methods of evaluation and assessment in large-scale engineering learning contexts. She works on problems of validity, equity and how to make inferences about diverse groups of engineering learners. She has been Primary Investigator or Co-PI on more than $50 million of external research awards. In 2020, she received an NSF RAPID award to study engineering instructional decisions and how students were supported during the time of emergency remote instruction due the COVID-19 pandemic. In 2021, she received the NSF Early CAREER award to study improving the fairness of assessment in engineering classrooms.
