University Discontinues In-class Professor Evaluations, Implements New Online System

This semester, Frostburg State students no longer took part in the traditional in-class professor evaluation process, and instead logged on to Blackboard to review their teachers and their courses. In an effort to modernize, increase efficiency, and save time and money, the university switched to an institution-wide online evaluation platform.

43.7 percent of students completed the online evaluations, lower than the 73 percent of students completing the paper evaluations in fall 2014.

Prior to this semester, students who attended classes at FSU would complete a course evaluation form at the conclusion of a course in December. A faculty member other than the class instructor proctored this evaluation process, and then collected the forms. This semester, however, evaluations were no longer administered in class, but instead could be accessed as online surveys through email or Blackboard. The professor and course evaluation surveys have two key functions: to provide feedback on how to improve future classes and to provide a numerical score of professor performance that can be included in annual faculty evaluations.

Dr. Michael Murtagh, coordinator of Frostburg’s Master of Science in Counseling Psychology program and Chair of the Faculty Senate, was involved the process of evaluating and ultimately overhauling the evaluation system. Speaking on the limitations of the original, in-person system he noted, “A couple of things were a little problematic, one of the things was that it took a chunk of time away from the class, so that’s that much more time students didn’t get for the course they were paying for the other concern was that it had been calculated [by the FSU Office of Assessment and Institutional Research] that between actual cost and time spent to do it the way it was being done, if we switched over to doing it online, that would save the university just under $40,000.”

Dr. Sydney Duncan, Associate Provost and former Chair of Frostburg State University’s English Department, was one of the initial developers of the plan to convert to an online evaluation system. She expanded on the challenges posed by facilitating the old system, adding, “we’re talking hours and hours of work for administrative assistants. It was a very labor-intensive process. It was also very expensive to do that way”

Dr. Duncan explained that the consideration for a transition to an online system began last academic year in the FSU Council of Department Chairs, which she was leading at the time. Many universities nationwide have converted to online evaluation systems, and therefore different program models and practices were available for the council to examine. The group read published research on online evaluations from academic journals such as Assessment and Evaluation in Higher Education, and from studies conducted at specific universities.

Both Murtagh and Duncan explained that after the council decided to recommend the switch to an online evaluation system, an open assembly was held and approximately 50 faculty attended and gave input. Following support voiced at the assembly, the Faculty Concerns Committee, a branch of the Faculty Senate, tackled the project of developing a comprehensive plan for the new system. After much discussion and deliberation, a finalized policy was brought before the entire Faculty Senate and approved.

“This past spring, while this [the work by the Faculty Concerns Committee] was going through, there were about ten faculty in the Department of English who piloted the online evaluations just to see how it worked and how it integrated… it worked great” Dr. Duncan added, saying that early test helped identify most of the potential technical glitches with the new online surveys.

FSU’s Office of Assessment and Institutional Research “will be doing our regular data analysis” once the window for students to complete their evaluations closes, according to Sara-Beth Bittinger, Director of the FSU Office of Assessment and Institutional Research.

“Our analysis will begin next week [using scores from the online evaluations], and then we will come up with the reports that are then distributed to the department chairs and the administrative aids on or around the middle of January,” Bittinger said. Converting to an online system means that the staff in the Office of Assessment and Institutional Research no longer has to spend hours compiling and scanning paper evaluation forms.

For over a decade, the same online evaluation system now in place for all courses, Courseval, has been used to evaluate classes taught online. Therefore, the implementation of the new system was simply a matter of expanding the process to all courses taught in classrooms.

Since the new online evaluations remained optional, the primary concern among some students and faculty has been that the program may see a decrease in response rate. While many students appreciate the increased time and privacy the new online system gives them, all parties involved want to see an evaluation process that is comprehensive and accurate. Students and professors alike worry that few responses may lead to evaluation results that are not as representative as those obtained using the old model.

Duncan addressed this concern and stressed that an initial drop in response rates will likely improve over time. “We’re going to see a drop in response rate. That’s pretty typical when people first move to online evaluations. What the literature seems to show is that over the first couple of years, as students get used to doing them this way, response rates creep back up,” she explained.

According to Bittinger, approximately 73 percent of Frostburg students completed in-class professor evaluations on paper at the end of the fall 2014 semester. By 4:00pm on Friday, December 11, with several hours still remaining before fall 2015 evaluations closed online, the Office of Assessment and Institutional Research was reporting 40.6 percent of students had completed them. That rate is higher than some early estimates, which were drawn from studies completed at other universities.

According to Murtagh, the Faculty Senate decided not to make the new evaluations mandatory at this time because most faculty were uncomfortable with the idea of forcing students to complete the surveys. The challenge with incentivizing the process, he explained, was that results could be skewed if one professor or department rewarded students who completed evaluations more than another. He reiterated that, “regardless of whatever happens, if there was a substantial drop-off in the amount of people who participated, or if there’s a dramatic shift in scores, whatever comes out there will be review and discussion [by the Faculty Senate on the new online evaluation system].”

Duncan believes the key to raising response rates with the online system is to better educate students about the importance of completing the surveys and providing the university with quality feedback on faculty and the courses they teach. “Hopefully your teachers are telling you ‘please fill out the evaluations, and here’s why…’ because it’s important for students to know why they matter and how they’re used,” she said.

The Bottom Line contacted several other member institutions of the University System of Maryland for comment on their course evaluation systems. Towson University confirmed that it has a campus-wide online evaluation process and Salisbury University reported that many of its specific departments use an online system, but neither school replied to requests for specific response rate statistics.

Allegany College or Maryland, a two year college located in nearby Cumberland, Maryland, did, however, provide The Bottom Line with the response rates from its online faculty evaluations, which it has been using since the 2013 Spring semester. According to Scott Harrah, director of ACM’s equivalent to FSU’s Office of Assessment and Institutional Research, 67.4 percent of students at the college completed online evaluations during the system’s first semester, and the program has averaged a 61.1 percent response rate overall since its initial implementation.

Any students with questions, concerns, or feedback about Frostburg State University’s new online faculty evaluation system is encouraged to contact Murtagh at mpmurtagh@frostburg.edu.