Smart ALEKS

Arizona State University’s College Algebra classes have looked a little different these past few years. If you entered a classroom in the fall of 2015, you’d likely find 100 to 140 students listening to the professor lecture. Today, walking into an algebra classroom, you’d see those same 100 students all on their computers, with a professor and five undergraduate assistants roaming the class to give personalized help.

The experiment, an implementation of the adaptive courseware model ALEKS, has paid off, administrators say. The rate of students who achieve a C or higher in College Algebra, which was 57 percent in 2015, is now up to 79 percent.

The administration chose to implement ALEKS, a 20-year-old piece of software now owned by McGraw-Hill, to what they saw as its full capability. Though the software can be used just for homework, at Arizona State the courseware has replaced the textbook, the exam and the lecture.

ALEKS works by first assessing students on an initial “knowledge check.” Depending on how students do, they start at different places in the curriculum.

“The system will say what you’re ready to learn versus what I’m ready to learn,” said Scott Virkler, chief products and operations officer at McGraw-Hill Higher Education. “Then it’s providing reports back to the instructors and showing the progress the students are making over time or not.”

The courseware instructs students using an integrated textbook and decides whether they can move on based on their performance on quiz questions. To demonstrate mastery in a particular skill, a student must solve five relevant problems in a row. They cannot move on to another skill before mastering the one at hand. Knowledge checks along the way make sure students are retaining content they've mastered.

“For almost the entire history of our educational experience, we’ve been using what we think of as a progress model,” said Dale Johnson, director of adaptive learning initiatives at Arizona State. In the progress model, if a student had been assessed on a topic and failed, the course moves on and that information isn’t covered again, he said. “It left [students] in an awkward situation, since most topics are cumulative, so you’re constantly building your knowledge scaffold.” In the mastery model, it’s more important to reach proficiency in a topic even if it takes one student longer than another.

Exams are given out by ALEKS to students when they reach certain milestones, say, five course objectives mastered, or 10. Students who do well on the initial knowledge check, Johnson said, can take the first exam the first week and complete the course in five to six weeks if they choose. If there’s enough time in the semester, students could even take another course.

For students who progress more slowly, the university allows them a “stretch semester” -- another 16 weeks to complete the coursework.

“In a traditional course, they would have to go all the way back to zero and start over,” Johnson said. “But in the stretch model, we let them start where they left off and continue on the next semester. And ALEKS gives us the ability to track that so we know if they are retaining that knowledge.”

Instructors use the ALEKS data to determine which students are not making progress and give them individual attention in class, Johnson said.

“With our lecture model, we were always teaching to the mean. But that leaves most people outside of the target audience,” he said. “You’ve got the slower students who are overwhelmed because you’re teaching too fast, and you’ve got the smarter students that are bored because you’re teaching too slow. So you’ve got huge disconnects in the traditional model that we’ve corrected.”

When the administration switched the algebra classes over to this model in fall 2016, they saw a 5 percent increase in the success rate. Additional incremental growth brought the rate up to 79 percent last year, and with the stretch semester up to 84 percent.

Every demographic group the university studied -- which included groups based on gender, Pell Grant eligibility, family educational attainment and minority status -- succeeded at a higher rate in the new model. Johnson noted that differentials did still exist between those groups, though, with men still doing better than women, for example.

Though the new model was also implemented in the university’s College Math class, the gains in that course were more modest -- a tick up to 88 percent success rate for a class that had always had one above 80 percent.

Benefits of Adaptive

Louise Yarnall, a senior research social scientist with SRI Education, said that every adaptive courseware product is unique, and ALEKS might have some advantages in its internal design. “Those core elements of immediate feedback, remediation access -- very targeted to you -- and also that baseline test. Those three pieces, I think, we’re seeing some sign over time that these are really important elements to design into adaptive courseware.”

Past research on adaptive courseware, such as a research report Yarnall co-wrote in 2016, observed much more modest results than those found at Arizona State.

The majority of analyzed implementations led to no discernible impact on course grades or course completion. She said that while the two ALEKS implementations in that study did not demonstrate significant changes in grades, researchers observed greater instructor and student reported satisfaction. “[Students] were saying, ‘I actually think and feel that I am learning more than I would in a regular class,’” she said.

Though Johnson said ASU has submitted a research report for peer review, its researchers were not able to incorporate many experimental controls, such as randomized assignment, into the data collection. Last year, the course had 5,872 students.

Johnson offered his own advice for other administrations looking to implement the courseware. “I don’t think you can be successful if you just bolt ALEKS on a lecture course, because you’re not taking advantage of the capabilities of the technology to personalize the learning path at scale,” he said. “You have to go all in. You can’t do this a little bit.”

Getting to a point where faculty members were willing to give up lecturing took years, he said. “You have to reach a point where you can trust the technology, and our faculty did.”

Although faculty members are often concerned that technology will be used to cut them out of the educational model, or at least reduce their numbers, Johnson stressed that the new model does not require fewer professors and has actually resulted in more undergraduate assistants being hired. The faculty also worked to decide on the course objectives before implementation.

“We’re actually investing as much or more in the instructional process than we were before,” Johnson said. “When students are working in ALEKS, they have to feel supported. They can’t feel isolated, and that’s why we make them come to class and work on the technology in class. We don’t want them sitting in a dorm room feeling like they’re lost in cyberspace.”

Currently, the university covers the cost of the ALEKS license for students, though if they did have to pay, the cost would be less than the cost of the textbook students were previously asked to use, Johnson said. The university is piloting implementation in precalculus classes, but the software doesn’t offer any higher-level math.

Yarnall said that a major hurdle to clear in further research on adaptive courseware is the tendency of companies and colleges to not want to share their data.

“To advance the field, there really has to be more give around those proprietary considerations and a little more focus on the bigger picture,” she said. “How can we arrive at a really good design and a set of implementation guidelines that everyone can use?”