The Score
Nov 15, 2022
The Score on Academic Integrity - Dr. Sonny Ramaswamy, president of NWCCU
playlist_add
Dr. Ramaswamy discusses what accrediting organizations look for and where academic integrity comes in.
-
-
More episodes from The Score
Sep 8, 2023
The Score on Academic Integrity – Garret Merriam, Associate Professor of Philosophy at CSUS
In recent years, it seems that the radio dial on ethics is moving up and down the spectrum. Ethical behavior, intentional or not, is at the root of cheating. This episode of The Score explores how our guest, Garret Merriam (@SisyphusRedemed), an Associate Professor of Philosophy at Sacramento State University in California, responded to widespread cheating on a final exam in his Introduction to Ethics course.
High points of the conversation follow. Note: Removal of filler words and minor edits have been made for clarity.
Kathryn Baron (01:33): Would you tell us what happened in your Intro to Ethics class?
Garret Merriam (01:42): I came to suspect that some students in my class might've been cheating on my final by Googling the answers on the final. I teach a course that's fully online, has almost a hundred students, and with that much material, that many students going on, it simply isn't possible for me to create novel finals every semester, as much as I would like to do that. I reuse large portions, though never the entire thing, of my final. And so, I found that by Googling the questions on my final, you could come up with a student who had uploaded a copy of the final with many of the correct answers to the questions.
I made the request of the website, called Quizlet, that they take it down, and I was very pleasantly surprised that they did so promptly and quickly. I was under the impression, I was assuming that they weren't going to respond, but they did. I was very grateful for that, very professional of them on their side of things.
And then after that, a part of me, perhaps somewhat of a devious part, I suppose, decided to run a little experiment. Part of my research is in experimental philosophy, and I like running experiments, and so I decided to see what would happen if I uploaded a copy of my final with the right questions but the wrong answers.
Garret Merriam (03:01):…After the final was complete, I ran a statistical analysis and found out that approximately 40 of the 96 students cheated on the final.
Garret Merriam (3:35): And this understandably created a bit of havoc both for me, for my students, for my department, and a number of people who became a part of this conversation going forwards.
Kathryn Baron (03:47): When you learned that a student had put the test up on Quizlet, how did you know that the students in your current class had copied it?
Garret Merriam (03:54): What initially led me to be suspicious was a mistake that I had made earlier in the semester. Every week, I upload a reading and a reading quiz, and the idea is they do the reading, and they take the reading quiz just to make sure to put a little pressure on them to incentivize them to actually do the reading. And one week I neglected to upload the reading, but did upload the reading quiz, and then a few hours later I realized my mistake and I went, and I uploaded the reading. But when doing so, I noticed that some of the students had already taken the reading quiz and had gotten a perfect score on it.
Garret Merriam (04:37):….That was hardly proof of anything, but it was enough to make me suspicious. It was enough to make me concerned that something would've been going on. So, I Googled those quiz questions, and sure enough, I found the copy of them on Quizlet.
Kathryn Baron (05:49): I read that you contacted the students suspected of cheating. How did that go?
Garret Merriam (6:04):…I put together sort of a blank form letter in which I contacted them and said that I have reason to believe that they had cheated on the final and a few more details without tipping my hand completely. And I sent that out to all of the suspected students.
And somewhere in the ballpark of about two thirds of them got back to me right away and confessed and said that yes, they had cheated, they were apologetic, some of them made excuses, others just asked for understanding and forgiveness, and about one third of them denied it.
And then about half of that third then turned around within 24 hours and even before I got back to them and said, "Okay, you know what? I actually, no, I changed my mind. I'm going to confess.” So, all of this very much reassured my confidence that my method was working here. And of the remainders, some of them, as far as I know to this day, still insist on their innocence. I'd handed things over to the administration at my university.
Kathryn Baron (07:59): Do you have any input into what action the university takes?
Garret Merriam (08:03): I get to determine the penalty as far as my class is concerned. All of the students who did this at the very least got an F on the final.
Kathryn Baron (10:33): I have heard of instances where some professors think, "Well, that would never happen in my class," and I'm wondering if you received any feedback like that, sort of implying that you must have done something not quite right as a teacher for students to cheat.
Garret Merriam (10:49): It's certainly tempting to think, and obviously there is some truth to that. The room for this kind of thing is going to vary depending on a lot of details about a particular instructor's class. To take the most obvious example, if you're not reusing material like I was, then you're not going to be encountering this particular problem.
While none of my colleagues gave that particular response, if there's anyone out there listening, I can certainly imagine that that might be a justified response. However, at the same time, there can be a kind of certain amount of arrogance and maybe laziness that might come along with that too, to think that the problem is something specific about the individual instructor, in this case me, rather than something that is a little bit more systemic.
Again, I want to give credit to professors and other instructors who have found ways to effectively discourage cheating, but I would also say you shouldn't rest on your laurels and recognize that it is, I think, a best practice to double-check your methods and your sources and to find out in any way you can, whether or not there actually is academic dishonesty going on. You should not simply assume that you are one of the fairly small percentage of instructors who has managed to stamp out academic dishonesty in their ranks completely.
Garret Merriam (12:13): The irony of cheating on an ethics final is something that was not lost on me, and I tried to impose that recognition on all the students who I communicated with as well.
Kathryn Baron (12:21): You did reach out to other colleagues and peers around the country on the online philosophy journal called the Daily Nous, that's spelled N-O-U-S, which I read is ancient Greek for intellect or understanding. What feedback were you looking for and did you get it?
Garret Merriam (12:38): It actually started on Twitter. I have a fairly modest Twitter presence, but a lot of fellow philosophers follow me, and I follow them. And so I post about the experience and Twitter being Twitter, everything was condensed and a lot of detail was washed out, so I think a lot of people didn't possibly fully understand exactly what I did and what my reaction to it was. So, someone with a larger following retweeted it with criticism and a lot of people started to jump on and accused me of engaging in dishonesty myself. The most common criticism is a kind of entrapment, that I encouraged or enabled students to cheat and then punished them for doing so.
Garret Merriam (14:07): I wanted to try to filter the audience down to people who at least had some experience with the kind of thing I was talking about.
…It became a very, very populous discussion, which I was fascinated to participate in, and the results were somewhat mixed. I think a lot of the people, once they got the full picture, recognized that I hadn't engaged in anything majorly morally problematic, and in particular the charge of entrapment was ill-placed. At the same time, several people did criticize, a…
playlist_add
Jul 24, 2023
The Score on Academic Integrity – Pete Van Dyke, Amazon Web Services
On this episode of The Score, we look at cheating from a different angle than we have before. Our guest is Pete Van Dyke, the Certification Security Program Manager at Amazon Web Services, the office responsible for minimizing cheating among people taking professional certification exams.
Kathryn Baron (01:57): Would you describe what you and your office do?
Pete Van Dyke (02:00): We divide our time among three different activities. One is looking at people that steal our exam content and post that online or charge money for that online. Those are known as brain dump websites. You'll probably hear me talk about that a couple more times today. The second thing that we do is we look at what are known as proxy testers. So, individuals or organizations that take exams for candidates charge them a fee for that, and then through remote control of the computer screens take an exam for them. And then the third thing that our team works on are individuals who misbehave during their exams. So, whether that's accessing a cell phone or hidden notes or having a third-party present, people that misbehave on exams…
Kathryn Baron (07:16): If I'm taking one of these exams, what can I expect before I'm cleared to actually begin the test?
Pete Van Dyke (07:22): Well, we present our exams in two different formats. One is at an in-person test center. So, we have literally thousands of in-person test centers all across the globe. If you were to take an in-person exam, you would schedule that. You would go in and there's a live proctor who would observe you as you take your exam but once COVID hit, the second modality for us, which is online proctored exams became very popular. And an online proctor exam, you don't have to go to a test center. You can take that right in the confines of your own home, and you don't have to interact with people live. What happens for online proctoring is that there is an online proctor located somewhere else in the world who is observing up to 16 or 18 people taking in an exam at one time, and they make sure that they're not misbehaving.
So, if you were to take an online proctored exam, there's an entire formal check-in process. So, we verify that the government issue ID is the same person as the person taking the test. You don't want someone who looks like me taking the test under the name of someone who looks like you, Kathryn. There's a very detailed room scan by video to make sure that there aren't any learning materials, that there aren't any secondary computers or electronic devices, any note-taking materials, pens, paper et cetera in the area.
And then there's also a systems check. So, the test delivery provider looks at that and sees what kind of programs are running in the background to make sure that there's nothing that would allow a candidate to record the testing experience and then steal content from the actual exam.
Kathryn Baron (09:07): So, what have people done to try to trick the security measures? Are there any anecdotes that stand out for you?
Pete Van Dyke (09:15): It's really limited only by creativity. So, for online proctored exams, because you don't have a human being in the same room, people attempt to cheat that system in lots of different ways. They may try to record the session, either audio record or video record. They may surreptitiously have notes and access notes during the exam. It's not unusual for someone to try and have a third person, a third-party individual in the room with them to help with the exam and indicate which questions have which answers. And we've seen evidence in the past of people using things like recording devices built into eyeglass frames or even using earbud type communicators so that someone can communicate with them what the correct answer is for items.
Perhaps one of the more interesting things that we've had when someone takes an exam with a proxy tester, the proxy tester loads software on their machine that allows them to remote control, take control of the desktop as they're using it. So, during that hour or hour-and-a-half that they're taking the exam, the candidate pretends to be taking the exam while someone thousands of miles away is actually taking the exam for them.
One of the funnier instances that we've had of exam misbehavior, we had a candidate that actually fell asleep during his exam. His head was leaned over, and he was snoring very loudly for about a 10-to-15-minute period. Yet, his exam continued to move forward because the exam proxy tester didn't realize that the candidate was sleeping, and he was just moving forward as had been planned.
Kathryn Baron (10:59): What are some of the less obvious red flags that the proctor will look for?
Pete Van Dyke (11:20): So, when you take this exam, you can see the webcam capture as it's running to make sure that your face is completely visible, and your shoulders are visible. But if you were to place that just slightly outside, if you continuously look, say down into the right or down into the left, that would be an indicator that there might be something there that that candidate is using to cheat on the exam. Leaning partially off-screen would be the same type of violation or exiting the exam completely.
We don't allow for breaks on our exam, even bathroom breaks. So obviously getting up and leaving for a minute or two and then coming back is a sign that there's at least the very strong possibility that the candidate was accessing information that they weren't allowed to have during the exam.
Kathryn Baron (13:53): Have security measures increased in recent years due to an increase in the products that enable cheating and the increase in online exams due to COVID?
Pete Van Dyke (14:53): As the pandemic continued, about a year-and-a-half, two years into it, we saw about 85% to 90% of our exams being taken via online proctoring and only 10% being taken in test centers. So that created a whole new environment for us. Obviously, if you don't have someone standing in front of you, it's easier to misbehave, it's easier to try and hide things, and it's easier to have another person in the room that's hidden from camera view. So, we had to adapt to all of that. And the proxy testers are very, very sophisticated.
Kathryn Baron (15:40): Well, how organized are they, the proxy and all the other companies? Is it difficult to find them and maybe put them out of business or anything like that?
Pete Van Dyke (15:50): Well, the challenge for us, we're a US-based company and a lot of the brain dump websites and a lot of the proxy testing organizations operate in countries outside of the United States. So, it then becomes very lengthy, very expensive and very difficult to pursue any type of legal action against these individuals in countries that may not even support that type of a lawsuit. So, it's very challenging.
The proxy tester networks themselves work a lot like a multi-level marketing campaign. They advertise all over the web. So, if you're on Facebook and a Facebook group about certification exams, it's not unusual to see multiple posts a day with people offering to take exams for you. We've seen this on LinkedIn. We've seen it all over the place, even on Etsy, believe it or not, and eBay.
You have one level of their organization that is recruiting potential customers. You have another level that works with them and negotiates pricing and details, and then you have a very sophisticated technical side of their organization that actually makes the proxy test happen by taking over candidates' computer screen and taking an exam for them. Industry-wide, we estimate that this is a multi-$100 million-a-year business. It's not unusual for a proxy tester to charge as much as $1,200 above and beyond the cost of an exam for someone to have an exam taken for them.
Kathryn Baron (17:19): You were talking about stealing the test questions earlier, and how do people do that? Is it that the people who are the proxies, they c…
playlist_add
Jun 2, 2023
The Score on Academic Integrity – Dr. Roy Swift, Executive Director of Workcred
This episode of The Score features Dr. Roy Swift, the Executive Director of Workcred, an affiliate of the American National Standards Institute. He also served as executive director of the National Board for Certification in Occupational Therapy. This appointment followed a 28-year career in the U.S. Army Medical Department, where in his last position, he was chief of the Army Medical Specialist Corps in the Army Surgeon General’s Office with policy responsibility for Army occupational therapists, physical therapists, dieticians, and physician assistants throughout the world.
Kathryn Baron (7:22): I'm wondering then if that disconnect is perhaps part of what leads to academic integrity problems in college and the frustration that you mentioned students often have?
Dr. Roy Swift (07:34): I do think K-12 is the foundational component to success in post-secondary education, in academia and Higher Ed. I believe it is crucial to individuals being able to make the right choices. There are several issues in regards to the system and preparing success in the post-secondary system. One is helping people understand how to learn to be able to identify resources, to build self-confidence in people. There is our need to move to more of a competency-based approach in Higher Ed, that's transparent and can signal to the work world or the government or whoever that what the person not only knows, but what the person can do. The current transcript is not helpful in this regard. And the reluctance of faculty to move to competency outcomes versus general course descriptions is problematic. I feel the issue is that the competency approach does put more pressure on the faculty to produce what they say they are producing because it is more transparent, and the assessment tools have to be more precise. The other disconnect is the lack of employability skills. The college is not teaching the behaviors that are expected in the workplace. Something as simple as coming to class on time, participating in class and being an active learner and working in teams often are forgotten.
Kathryn Baron (10:36): You mentioned a paper in an earlier conversation we had that you co-led on the integration of credentials, and I'm wondering if you can just tell us what were the primary takeaways from that and were you at all surprised by anything that you found when you were doing that work?
Dr. Roy Swift (10:54): Yes. Recently, I participated with the Higher Learning Commission, which is one of the national accreditors of universities and colleges. And because they are very interested in looking at the whole issue of credentials and how credentials may be integrated into a higher education system, industry credentials in this regard. The title of the paper was Institutional Accreditation at the Crossroads Drivers for Change, and it had four main themes. One was at the landscape and pressures on Higher Ed, employers and accrediting bodies are going to be increasingly to produce a product that is able to function at higher levels probably because of all the technology that is going on today will have to produce a very different kind of individual. Most people think technology will run people out of jobs, but it really looks like that what is going to happen is that it's going to force and put pressure on producing people with higher level of knowledge in the ideas of robotics and artificial intelligence.
Dr. Roy Swift (13:36): ….there are over 8,000 industry certifications right now. And every week they develop more and more industry certifications, and it is one in which they can be complimentary.
But unless we understand, like I said at the beginning, the credentialing system and how they may interface and complement one another, we are going to develop competing systems. Which may not be the best way of thinking about these various because each credential tends to have a lot of strengths. And so, we should use the strengths of each credential to be able to see how they can be integrated. Our particular group, Workcred, is looking about the embedding of certifications into degree pathways. We think the two can complement one another because certification is about competency assessment.
Dr. Roy Swift (17:14): So, people who are trying to design Higher Education need to take a more systems thinking about what is the work world telling them? What is the government telling them? What do students desire? What's the environment that we should be doing? And so, we take it from the, let's just say the national system to the state systems, to the academic systems, to the subsystems of provost, deans, faculty, students, and understanding how those layers exist. Unfortunately, too often the K-12 system, which we talked about as being the foundation, is not producing individuals that have the psychological ego strength to face many of the issues that students are facing. And that threat, I think, does cause students to do things that may be unethical, such as cheating.
Dr. Roy Swift (26:12): Well, I think the first step is building more precise learning outcomes, competencies, whatever word that people feel comfortable in using, because I think that many times, I hear students talk about the unknown, oh, I don't know what he's going to ask. I don't know what the expectations are. Competency gives more structure to the student as to what the expectations are in this regard. Competency-based assessment is really a more straightforward method of evaluating whether a skill has been achieved. And the students feel more secure when they know what it is that is expected of them, expected in the course, and expected on the assessment.
Kathryn Baron (30:50): ….what are your thoughts on what can be done in the training and workforce development industry to minimize cheating or even to, I don't know, curb the impetus to cheat itself?
Dr. Roy Swift (31:04): Well, I do think it's important to go back to transparency, relevance, and competency. I would say those are three main elements that has to be looked at. I would go back to my thoughts about competency-based education. When it's about competency, and you can repeat the assessment until you've achieved the level of competency. There's less reason to cheat. Let's say, okay, I failed. I'll go back and relearn. That's a skill that's taught in the military. I used to teach at the Academy of Health Sciences. It's called something different now in the army. But one of the things that always happens in the military is that teach, test, reteach, retest. And generally, students are given several times to be able to achieve it because the military believes it's about competency.
It might take them two or three times that one person can do it on the first time, but it doesn't mean that the person who took three times isn't just as good with that competency. So, I think we have to take that sort of an attitude in Higher Ed, instead of this, wow, we're going to fail you, and that's it. There's no other chance in this ring. And it sets up a more feeling of freedom to fail. And don't we tell people we learn by our failures?
playlist_add