As in every other aspect of our lives, artificial intelligence (AI) is finding its way onto university and college campuses across Ontario. Given the rapid advancements of AI, this shouldn’t surprise us. Still, it does, time and again. It’s a sad reflection of the opaque veneer often associated with these new technologies that we don’t see coming until they’re at work.
Last week, the discovery of facial recognition enabled vending machines by a University of Waterloo student sparked widespread discussion and concern about the use of potentially privacy-invasive technologies at Ontario’s post-secondary institutions. Because my office is currently looking into complaints about this incident, I am not at liberty to speak about it. We will let our investigation reveal the facts as they are and carefully consider whether the use of these vending machines complies with Ontario’s public sector privacy law as it currently stands.
One case I can speak freely about is my recent investigation into the use of AI-enabled proctoring software at McMaster University. Following a complaint by a student who wished to remain anonymous, my office opened an investigation into McMaster’s use of Respondus exam proctoring software that began during the pandemic and has continued since.
This software includes two programs. Respondus LockDown Browser locks down certain functions of a student’s computer during online exams so they can’t conduct an internet search, access other files on their computer, use the copy-paste function, message, or screen share with others. Respondus Monitor collects sensitive biometric information and monitors students’ movements and behaviours through audio-video recordings, using AI to detect instances of potential cheating.
Our investigation examined the university’s use of the software in the context of the Freedom of Information and Protection of Privacy Act (FIPPA). We found that the university is lawfully authorized to conduct and proctor exams to ensure their academic integrity. Nothing legally prevented them from doing so online, both during and post pandemic. This is a legitimate aim in a context moving more towards remote learning and heightened risks of cheating associated with modern digital tools. But, our analysis did not end there.
While the Respondus LockDown Browser collects minimal personal information, Respondus Monitor gathers more sensitive data, including biometric information, and makes consequential inferences about students’ movements and behavioural conduct through AI technology, raising significant privacy concerns.
Our investigation found that the university’s collection of students’ personal information was technically necessary for this exam proctoring software to function properly and, in this respect, complied with FIPPA. However, the use of students’ personal information by Respondus Monitor, the university’s inadequate notice to students about the purposes for this data collection, and the university’s insufficient safeguards to protect student’s personal information through its contractual arrangements with the company, were found to contravene the act. Most concerning was the company’s non-consensual use of students’ audio and video recordings, including via third party researchers, to improve system performance and enhance its services.
In the investigation report, I recommend that McMaster University introduce stronger measures to protect students’ personal information in the context of online exam proctoring and ensure an approach that balances academic integrity and student privacy rights. I also went on to make additional recommendations to address the broader privacy and ethical risks associated with the use of AI. I asked the university to report back to my office within six months on the implementation of these recommendations.
Recognizing the realities of my mandate, and in the absence of broader compliance tools under current law, issuing recommendations is all I could do. To be clear, McMaster University has been cooperative during the investigation. I have no reason to doubt the university will take my recommendations seriously. However, the fact that I cannot assure Ontarians that my recommendations will be acted upon and must rely on the goodwill of public institutions is simply not adequate. In today’s environment of significantly heightened risks associated with technology, including growing threats to cybersecurity. and potentially invasive uses of AI, Ontario students — and all Ontarians — deserve better assurances than that.
Once again, I urge the Ontario government to modernize FIPPA and its municipal counterpart, MFIPPA. Our province needs a more robust privacy protection regime with stronger measures needed to address the increased risks of a rapidly evolving digital world. I also urge the government to finalize its Trustworthy AI Framework and make it binding on Ontario’s public sector, including the broader public sector, to ensure that technological innovation unfolds in an ethically responsible manner for the benefit of all Ontarians.
Educators and policymakers can foster an environment where technology enhances the quality and integrity of the educational experience without compromising the foundational principles Ontarians hold dear. We must ensure that the legacy of Ontario’s education system continues to be one of innovation, integrity, and inclusivity.
It’s been nearly 200 years since Ontario founded its first university. Much has changed since then. That change must continue if we are to prepare future leaders to embrace and improve the world that awaits them. We owe it to all students — particularly those who dare to question and hold their institutions accountable — to ensure their education does not come at the cost of their privacy or other human rights.
In the words of Dr. Martin Luther King Jr: “The function of education is to teach one to think intensively and to think critically. Intelligence plus character — that is the goal of true education.”
This post is also available in: French
