Assurance of AI Enabled Systems – TBD


This artificial intelligence course introduces state-of-the-art methods for developing testing and evaluation plans for AI-driven systems and addresses novel challenges these systems present.

Want a discount? Become a member!

SKU: 80057264-spring2023-1 Category:


In this artificial intelligence course, test and evaluation experts from Johns Hopkins will share how cutting-edge test and evaluation plans are developed for AI-driven autonomous systems. Participants also will engage in a variety of interactive in-class exercises to get hands-on experience applying the latest in AI-based autonomous systems test and evaluation.

Complex systems driven in whole or in part by artificial intelligence (AI) are fast becoming ubiquitous, across a broad range of applications. These systems operate in extremely complex environments with overwhelmingly large combinations of inputs and variables. Engineers must test and evaluate the systems to ensure they are achieving the desired level of assurance, safety, and trustworthiness.



Andrew Ball
Andy Ball is the human and complex systems design strategist in JHU/ APL’s Design Thinking Group, where he leads cross-functional teams in the delivery of amazing solutions in the service of national security and space exploration.
Steve Biemer
Systems Engineering Program Instructor
Steve Biemer is a systems engineer for the Johns Hopkins University Applied Physics Laboratory. He works with both the Department of Defense and the Department of Homeland Security defining and conducting analytical end-engineering assessments of systems, platforms, architectures, and networks.
John Callahan
Chief Technology Officer
Dr. John Callahan is Chief Technology Officer (CTO) at, a leading biometric authentication company. He recently served as the Associate Director for Information Dominance at the U.S. Navy’s Office of Naval Research Global (ONRG) London office.
James Caroland
James is an active-duty Navy Captain in the Cyber Warfare Engineer community.  His 28-year career in the Navy has included various cyber leadership and technical roles at all levels in areas such as offensive and defensive cyber operations, cyber policy and strategy, capability development, and training and education.  He is currently the Chair of the…
Anton T. Dahbura
Anton (Tony) Dahbura is the co-director of Johns Hopkins University’s Institute for Assured Autonomy and executive director of the Johns Hopkins University Information Security Institute.
Michael DiRossi
Principal Research Engineer
Mike is a Principal Research Engineer at the Johns Hopkins University Applied Physics Laboratory, where he provides leadership, strategic vision, and technical oversight over a portfolio of cybersecurity research projects.
Ashutosh Dutta
Senior Scientist and Chair for Electrical and Computer Engineering for Engineering Professional Program
Ashutosh Dutta is currently a senior scientist and 5G chief strategist at Johns Hopkins University Applied Physics Labs (JHU/APL). He also serves as chair for Electrical and Computer Engineering for Engineering Professional Program at JHU.
Frank Fratrik
Frank Fratrik is the senior director of safety solutions at Edge Case Research, where he manages a group of system safety engineers who provide system safety management and engineering expertise across a diverse customer base of developer, users, and assessors.
John Gersh
John Gersh is a principal cognitive engineer in JHU/APL’s Intelligent Systems Branch, where he focuses on human-machine teaming.
Will Gray-Roncal
Principal Research Scientist
Will is a Principal Research Scientist at the Johns Hopkins University Applied Physics Laboratory, with expertise in data science, neuroscience, artificial intelligence, precision medicine, and learning research, including leadership of the CIRCUIT Program ( His research spans diverse domains ranging from undersea to outer space; a principal focus has been on mapping the networks of…
Joseph Greenstein
Joseph Greenstein is a lecturer in the Department of Biomedical Engineering. He teaches courses that cover topics in precision medicine, biomedical data sciences, and computational physiology and medicine.
Erin Hahn
Erin Hahn is a senior national security analyst and principal professional staff member in JHU/APL’s National Security Analysis Mission Area, where she supervises a group of analysts working on broad issues related to technology development and implementation.
Chad Hawthorne
Chad Hawthorne is a principal investigator and autonomy researcher at JHU/APL and has 20 years’ experience developing autonomy software for unmanned maritime systems.
Scott Hendrickson
Experimental Optical Scientist
Scott Hendrickson leads a physics group in the Research and Exploratory Development Department at the Johns Hopkins University Applied Physics Laboratory (JHU/APL).  This group focuses on a range of topics including quantum information, electromagnetics, and biomedical imaging. He currently helps lead projects focused on quantum information hardware development in partnership with government and industry.
Jenny Kelley
Chief Scientist
Jenny Kelley serves as the Chief Scientist of the Cyber Warfare Systems Group within the Asymmetric Operations Sector at the Johns Hopkins University Applied Physics Laboratory. Jenny came to the Lab from Morgan Stanley, where she championed initiatives in Threat Intelligence, Vulnerability Management, and Resilience. Prior to Morgan Stanley, she served as a Senior Technical…
Kevin Ligozio
Kevin Ligozio serves as the technical director and assistant group supervisor of the Tactical Intelligence Systems Group within the Asymmetric Operations Sector at the Johns Hopkins University Applied Physics Laboratory.
Matthew Montoya
Matthew (Matt) Montoya is an advisor, instructor, professor, academic program director, and researcher at Johns Hopkins Engineering for Professionals Systems Engineering, Healthcare Systems Engineering, and Lifelong Learning programs.
Barton Paulhamus
Bart Paulhamus is the chief of the Intelligent Systems Center at Johns Hopkins University’s Applied Physics Laboratory.
Alyson Phillips
Alyson (Aly) Phillips is the chief of staff for the Network Specialty Care Team at ChenMed, a value-based primary care and specialty care provider that delivers high-touch, affordable VIP health care to seniors with Medicare Advantage.
Jane Pinelis
Chief of the Test, Evaluation, and Assessment
Dr. Jane Pinelis is the Chief of the Test, Evaluation, and Assessment branch at the Department of Defense Joint Artificial Intelligence Center (JAIC). She leads a diverse team of testers and analysts in rigorous test and evaluation (T&E) for JAIC capabilities, as well as development of T&E-specific products and standards that will support testing of…
John Piorkowski
John Piorkowski serves as the Chief AI Architect and supervisor of the Applied Information Sciences Branch within the Asymmetric Operations Sector at the Johns Hopkins University/Applied Physics Laboratory. 
Gregory Quiroz
Project Manager and Visiting Scientist
Dr. Gregory Quiroz is a project manager in the Research and Exploratory Development Department at the Johns Hopkins University Applied Physics Laboratory (JHU/APL). In addition, he is a visiting scientist in the Department of Physics and Astronomy at JHU. His current research focuses on quantum system characterization and control protocols, and how these techniques can…
Alan Ravitz
Alan D. Ravitz is chief engineer in JHU/APL’s National Health Missing Area and is chair of the Whiting School of Engineering’s Engineering for Professionals MS program in Healthcare Systems Engineering.
Lynn Reggia
Lynn Reggia is the supervisor of the Human Machine Engineering Group within JHU/APL’s Air and Missile Defense Sector.
Sarah Rigsbee
Sarah Rigsbee is a senior human-centered design and innovation strategist and senior professional staff member at JHU/APL and is the lead human-centered design strategist for JHU’s Institute for Assured Autonomy (IAA).
Pedro Rodriguez
Pedro A. Rodriguez is the principal technical leader of multiple deep learning projects at JHU/APL, where currently he focuses on developing and deploying deep learning algorithms at the tactical edge for the U.S. Army and the Joint AI Center (JAIC).
Sri Sarma
Associate Professor
Dr. Sarma is an Associate Professor in the Institute for Computational Medicine, Department of Biomedical Engineering, at Johns Hopkins University.
Lia Scarince
Lia Scarince leads program strategy and project execution initiatives within the National Health Mission Area at JHU/APL, where she leverages 20 years’ experience at the front lines of public, private, and military health systems.
Aurora Schmidt
Aurora C. Schmidt is a project manager in JHU/APL’s Research and Exploratory Development Mission Area, and her research interests include sensor networks, estimation and coordination problems, signal processing, compressed sensing, optimization, multi-target tracking, control theory, and information and decision-making.
Christina Selby
Christina Selby is a senior professional staff member and section supervisor at JHU/APL, with expertise in developing and analyzing mathematical methodologies to solve critical problems that are not well understood.
John Slotwinski
Senior National Security Analyst and Project Leader
Dr. John A. Slotwinski is a Senior National Security Analyst and Project Leader in The Applied Physics Laboratory’s (APL) National Security Analysis Department, where he leads and performs studies on national security topics for the U.S. Government. Prior to that he was Additive Manufacturing Section Supervisor in the APL Research and Exploratory Development Department. In…
Tamim Sookoor
Tamim Sookoor is a researcher at JHU/APL, where his research interests include cyber physical systems (CPS), cyber security, the Internet of Things (IoT), and machine learning.
Adam Watkins
Adam Watkins is a principal staff member of JHU/APL with over 15 years’ experience in autonomy and robotics.
Tony Wei
PhD Candidate
Tony Wei is a third year Ph.D. candidate in Biomedical Engineering at Johns Hopkins University in Dr. Sridevi Sarma’s Neuromedical Control Systems Lab.
Rai Winslow
Raj and Neera Singh Professor
Dr. Winslow is the Founding Director of the Institute for Computational Medicine at The Johns Hopkins University School of Medicine and Whiting School of Engineering.
Dan Yaroslaski
Dan Yaroslaski is a senior professional staff member in the Tactical Intelligence Systems group within the Asymmetrical Operations Sector at Johns Hopkins Applied Physics Laboratory.
Kaliya Young
Young co-founded the Internet Identity Workshop in 2005 to bring together technologists who want to see decentralized identity come into being. This community is credited with creating internet standards such as OpenID Connect and OAuth and an initiation ground for collaborations that have led to multi-million-dollar projects. In 2010 she was recognized as a Young…
Reed Young
Reed Young is a member of the senior professional staff in the Research and Exploratory Development Mission Area at JHU/APL, where he serves as the program manager for Robotics and Autonomy.