• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!


Giannamore Critique 4

Page history last edited by Kathleen Giannamore 7 years, 7 months ago Saved with comment


More Than Student Surveys and Design Rubrics

Anthony A. Piña and Larry Bohn

Sullivan University System



This study sought to answer the question – how do we assess the quality of instructors teaching courses that they did not design? This is a growing problem as online programs grow and course designers move toward centralized design. The sample population for the study included 140 online learning professionals (split evenly between faculty and administrators responsible for online programming) and 114 students enrolled in online courses. Participants were surveyed to determine the methods currently used to evaluate the quality of online teaching in higher education institutions.

Mean scores and standard deviation were calculated for each of the indicators of instructional quality. A comparison among administrators, online faculty and students’ results on 9 indicators of quality teaching were compared.  While the study showed no significant difference between faculty and administrators’ rating of indicators, there was a significant difference between faculty/administrators and students.


The opening statement in the summary of this article sums up the major problem with assessing online teaching:  “As online education continues to grow, so do the number of online courses being taught by those who did not develop the courses. However, the most popular rubrics for evaluating the quality of online courses tend to focus upon the course’s design, not upon the actions of the instructor teaching the course.” (Piña, A. A., & Bohn, L. 2014)

Because the validity of student ratings of instruction has been questioned over and over due to possible bias caused by numerous issues such as instructor popularity and student academic success or lack thereof, to name a few – many studies have recommended that multiple methods be used to assess online teaching. In search of a clear answer regarding the validity of the student evaluation of online teaching quality, these researchers focused on quantitative measures of readily available instructor actions and/or behaviors that were either easily observable in online courses or were collected using the schools learning management systems. While the information provided was useful, it just scratched the surface of what really needs to be evaluated.  Easily observable actions by the instructor are easily evaluated but as the study itself pointed out – there are many more in-depth questions that need to be answered such as: What constitutes a higher verses lower quality course announcement?; How do instructors leverage their expertise to add value to an already designed online course?; What are the indicators of an instructor who successfully facilitates online discussion?

There didn’t seem to be any glaring limitations to this study and so it may well serve as a starting point for further investigations. It becomes quite clear that other methods of online teacher evaluation should be explored in conjunction with student evaluations, but professional evaluation systems that are primarily structured to evaluate course design are clearly not the full answer to the question this study is seeking to answer.


Piña, A. A., & Bohn, L. (2014). ASSESSING ONLINE FACULTY: More Than Student Surveys and Design Rubrics. Quarterly Review Of Distance Education, 15(3), 25–34.


Comments (1)

Chip Ingram said

at 11:07 am on Oct 18, 2016

Much of the material in your Evaluation section should really be part of the Summary, and it helps answer some questions I had while reading the summary (e.g. why is the study important?). Moving that up ot the first part and then adding detail about the methods and results would have helped a lot here. That would leave the Evaluation pretty short, however, so I would like to see more details about how well the study was done, whether it was capable of answering the questions posed, etc.

You don't have permission to comment on this page.