Dr Danny Liu had been an Associate Lecturer at the University of Sydney (USyd) for just under a year when he was given the enormous task of coordinating a few large first year biology units. Liu estimates he had around 1000 students under his care, and no real way to keep track of them.
“We used spreadsheets, but it very quickly became hard to manage,” Liu admits. “I thought there must be a better way.”
USyd’s Education Portfolio website describes Liu as a “biologist by training, programmer by night.” He is not a data scientist, but it is these nightly activities that saw Liu try to solve the administrative headache of monitoring potentially thousands of students in a unit of study. With the assistance of Professor Adam Bridgeman, the then Director of First Year Studies in Chemistry, Liu developed learning analytics software which was the first of its kind in Australia: the Student Relationship Engagement System (SRES).
SRES was initially used to ease the grunt work in academia. Instead of noting attendance and participation on scraps of paper or bulky malformed spreadsheets, Liu uploaded this information into one streamlined database. But as he experimented with the software, he realised SRES was more than just an administrative tool.
A problem facing universities today is that they store so much student data, but have no idea what to do with it. Canvas, the learning management system, records everything from attendance and test scores to how many times we login and post on the online discussion boards. Liu realised that if SRES could integrate and analyse all of this existing data, it could have major implications for how staff understand and assist their students.
“Teachers look at their thousands of students and think ‘oh my goodness I know 20 per cent are struggling, but I just don’t know who, I wish I could sit down and look at my data and figure it out but I don’t have the time’.”
SRES solves that problem. The software itself is a blank database. Teachers must input or select information (a task difficult in the Blackboard era, but is now easily done with Canvas). Canvas has an open application programming interface which allows any software to automatically integrate with the system, so long as the user has the correct access rights. When SRES is integrated with Canvas, teachers select particular data (for instance, attendance rates, test scores) to run through three machine learning algorithms—decision trees, cluster analysis, and association rule mining.
Decision trees involve input variables and output variables, which are set by teachers. For example, a teacher can set the input variables as attendance, test scores and time spent online, and the output variable as the final grade.
The algorithm will take these input and output variables and generate a ‘tree’-like data visualisation, where branches represent tracks. These branches may show, for example, that students who receive less than 50 per cent in their first three tests tend to be on track to receiving a credit, whereas scoring above 50% puts them on the track to a high distinction.
“A teacher can look at these decision trees and basically say, it seems like the machine is telling me that from my data, this input variable is the one that most affects a student’s mark,” explains Liu. “Attendance might not matter, but how they went in the first three tests does.”
Continuing the example, a teacher can then use a built-in messaging engine to send a personalised email to all students who received less than 50 per cent in their first three tests, letting them know what support services and resources are available for struggling students. They can also send an email to those who received above 50 per cent, congratulating them on their mark and wishing them the best for the next couple of assessments.
Sending these emails, Liu says, has a huge effect on the way academics interact with their students. It creates a personal connection that was hard to achieve when individual students’ progress could easily fade into the background of a large cohort, if there was no way to keep track.
“SRES will only do things that teachers say are pedagogically important,” emphasises Liu. He manages the SRES databases along with a few coworkers, but they do not monitor teachers’ use of the software. Liu wants SRES to empower teachers, and for him that necessitates giving teachers autonomy over the database. They select the information, and they choose the algorithms to use.
“A teacher can know because of the way they designed the unit, that, for example, if a student doesn’t come to the first three tutorials, or if they fail a test, that they probably need some support,” says Liu. “A machine doesn’t need to tell them that.”
SRES user Alexander Page is a PhD candidate in Sociology and current Head Tutor for SCLG1001 and SCLG1002. He does not use the machine learning algorithms, preferring to use SRES as a “background pedagogical tool to reach out, communicate with, and assist […] students.
Page’s introduction to SRES was similar to Liu. He was put in charge of a cohort that was too large for him and his small teaching team to handle. “Many students can potentially slip through the cracks. SRES though—through quick collection, analysis, and use of current student data—helped us contact particular student groups very easily.”
“Contacting non-submitting students gave as an opportunity to assist students in distress and allowed us to direct them to professional resources such as Disability Services, Special Con, CAPS, and the like,” says Page. “I think this is the most important [benefit] we can focus on as a Faculty.”
Guien Mao is a SRES user and Scholarly Teaching Fellow from Civil Engineering in the Faculty of Engineering and IT. She started using SRES last year in Soil Mechanics and shares Page’s enthusiasm.
“As a coordinator of a class of 300+, it gets a lot harder to meet everyone in your class,” says Mao. “SRES is really about giving us greater opportunities to engage with students and develop ties with them, so that they feel comfortable with asking questions and have more chances to have that sweet ‘aha’ moment!”
***
In the first few years of operation, SRES was mainly used by Science academics who were co-ordinating large cohorts. In early 2017, SRES moved beyond USyd when the University of New South Wales, the University of Melbourne, and Monash University decided to pilot the software.
With the growth of the tool, SRES is now at a turning point—and Liu is anxious to keep it as a pedagogical aid that helps teachers reach out to their students.
“There is an inherent risk in all learning analytics software [being used ineffectively]. What we learn is that it’s really up to the teacher and how they approach it—their personality starts to creep in.”
“For SRES, before [staff] even learn how to use it, we tell them why you should be using it. We say, ‘wouldn’t it be great to support your students?’”
Another problem facing SRES is its sustainability as an independent project, run by Liu and his team of educational innovators. These USyd staff are passionate, but developing SRES is not their main role and they receive no university or external funding. When I ask Liu, who has worked closely with Canvas, if he would ever consider selling the software, he is hesitant. “Autonomy is important to our philosophy.”
When Liu discusses SRES, it’s clear he feels a moral duty to help students succeed. SRES is one of the fastest growing learning analytics tools that most students have never heard of. Roughly one in seven USyd staff use SRES to monitor students and Liu prefers to maintain its anonymity—not to mask sinister side effects like privacy intrusions, but so that students don’t misconceive the software as just another de-personalised, automated tool.
“Some lecturers wanted automated emails,” Liu confesses. He refused the request on the basis that it removes human interaction from the software. He emphasises SRES is first and foremost a tool to help teachers reach out, not to help outsource the emotional work of supporting students.
What SRES does for students is perhaps less quantifiable than improving USE scores or attendance records. When some lecturers brag about the fail rates of their courses, or when it’s easy to feel disillusioned in higher education, SRES and its popularity is a reassuring reminder that our teachers care.
