I made a joke with Beth a few days ago about how I wished AI would grade my essays for me. And then I really thought about that, and now I can’t stop. I mean seriously if AI can write the students’ papers for them, what about us? Why can’t it grade papers too? Well, guess what. It can. I tried it. Shhh. Now don’t tell on me. I used both ChatGPT and Microsoft’s tool built into Bing. GPT is much better.
First off, it was a lot of work, probably equal to if I’d just graded them myself, but the benefit was far better for the student. It spat out paragraph long feedback on what was wrong, how the paper did or didn’t meet each rubric standard, and how to improve the work. I was simply amazed. If I were to leave that kind of feedback on a paper, it would take me days to grade instead of hours. As it is, I’ve already moved to grading by recording my feedback, so I can say more and make it easier for students to understand what I’m saying, but this was amazing.
But like a said. It wasn’t easy or quick. I first began by asking the AI to find a rubric for a news article using a summary lead and following the inverted pyramid style. It quickly pulled up a rubric that was adequate, but it was only on a 10-point scale. I used it anyway to see how it would work. Then my next prompt asked Maud, remember, I named her Maud in my last post. I asked her to grade this news article using that rubric, and pasted in a student’s news article. Surprisingly, it worked. She put the two prompts together and graded the article. It was okay, but there wasn’t much detail and it didn’t really meet the standards I was looking for. I decided instead to ask Maud to build me a rubric and added the essential things and the total points I needed. She created this.
Okay, that was interesting, but could she grade a news article using that rubric? I wondered. Yep! She gave the article a 47/60, provided the tally from the rubric AND added this feedback:
- The article follows the inverted pyramid structure.
- The information provided is relevant to the target audience.
- The article includes quotes from reliable sources.
- The lead is weak and could benefit from being more focused.
- The article could use more clarity in its presentation of information.
- The article could be more concise.
- The article could be more explicit in identifying biases or opinions.
Suggestions for Improvement:
- The lead should be more specific and focused on the key point of the article.
- The article should be more clear and concise, with shorter paragraphs and simpler language.
- The author should explicitly identify any biases or opinions and explain how they might influence the article’s content.
- The author should consider additional sources to provide a more complete understanding of the issue.
Crazy, right? I probably would have scored it a bit higher, but I noticed Maud didn’t say anything about grammar and punctuation in her feedback and I found a couple of specific errors, so I asked her about them. I typed in, “What about grammar and punctuation?” She replied:
As an AI language model, I don’t have the capability to fully analyze grammar and punctuation, but I can tell you that there are a few errors in the article. For example, in the sentence “This issue goes for not only students but members of the community who live downtown,” the phrase “goes for” should be replaced with “applies to.” Additionally, there are a few instances where punctuation could be improved for clarity and readability. Overall, however, the article is well-written and understandable.
She was right. In another instance, Maud pointed out two run-on sentences in the article and offered a suggestion for how to fix it. Amazing.
Yep, I’m obsessed, but that took a lot of work. In fact, I paused playing so I could quickly grade those news articles and get them back to students. This was going to take all day. First, Maud would forget my rubric after a few follow-up questions about the article she graded, so I’d have to start all over. Then sometimes she would ignore my rubric and just give a paragraph analysis of the article or she would leave out key elements. Basically, she was not consistent and I had to keep retraining her. She gave good feedback, but not the same feedback for each student.