As we’ve seen in the last few weeks, writing dialogue is an interesting area to explore the limits and capabilities of AI. Since GPT is a language module that predicts text based probabilities, it has trouble with things like subtext. It can achieve this with some prompting… but it will be a predictable subtext.
AI will not write any ground-breaking dialogues. While prompt-engineering can improve outputs, most serious writers won’t be happy with the results.
This week, I tested out using AI as a form of commenting on the dialogues my students wrote. While AI has difficulty writing engaging dialogue itself, I found it offered helpful comments and even detailed suggestions.
How I Did It
Once my students posted a story, I cut and paste it into ChatGPT with the following prompt:
You are professor in rhetoric and writing teaching a course on AI & Digital Storytelling. You recently assigned students to write 250 word micro stories that incorporate dialogue. Here is how we define good dialogue:
I also pasted the 4 tips for writing dialogue that we used to both write and generate different versions of dialogue.
I suspect that most of the suggestions were often random and rote, but they served to spark new ideas and help improve the students' drafts. Let’s face it… even our own comments can seem random or rote.
The dialogue in this micro story is realistic and authentic, and it conveys a sense of monotony and routine in the laboratory. The use of scientific terms and jargon enhances the believability of the conversation, and the characters' personalities come through in their dialogue.
One suggestion for improvement would be to break up the dialogue with some action and description to help the reader visualize the setting and the characters. For example, when the protagonist settles into their seat, some description of the lab equipment and the sounds and smells of the lab would help to ground the reader in the scene.
Additionally, the conversation could benefit from more conflict or tension to drive the story forward. While the protagonist's internal conflict is hinted at, there is not much external conflict between the characters in this scene. Including some disagreement or tension between the two undergraduates would help to create a more engaging and dynamic conversation.
This was, by far, the best comment we received. Notice how the AI can connect qualitative comments to specific contexts of the story… for example, the laboratory setting.
But this was also one of the best micro stories of the week. The better a student’s story, the more useful the AI’s response seemed to be. In most cases, better means more human control of the text. This student used microgenerations in strategic places. Her micro story was not auto-generated.
AI vs. Human Feedback
Human comments still hold their value because they are contextual, targeted, and reasoned. However, these comments also take time… time most teachers don’t have for a single assignment. This is where AI can come in, freeing up time for teachers to engage with their students on a deeper level.
With the right prompting and content, AI can deliver useful comments to students, as long as they understand that they are meant to trigger insights… not instructions to follow exactly. But that is also what I tell students about my own comments. Because I give them grades, though, they tend to see my comments as instructions and not insights.
Trained correctly, AI can deliver similar insights to my students without the power dynamics of my teacher role getting in the way. AI isn’t really commenting — its just making a really good guess at what I might tell a student. The student can decide if it is useful or not.
AI can’t replace educators. Human interaction around writing is still incredibly valuable. So valuable that we should use AI to make getting to that interaction easier. AI can help to inspire creativity and foster new ideas in students, allowing teachers to focus on those areas where human input is truly irreplaceable.
That said, there are still many issues with using ChatGPT for feedback. Submitting micro stories is one thing, but other kinds of work may have privacy issues.
Also, it takes a lot of time and expertise to “manually” use AI to generate feedback. In many cases, it actually might be easier to simply write out your comments or paste something from you library of comments.
That said… perhaps we should train students to get their own AI feedback 🤔.