GPT-4's Bad Poetry
I am really into human-computer collaboration. I write generative music based on fractals, and I co-wrote a zine last year with GPT-3. I'm not the person who is going to say that computers will never write poetry. What I do have to say is that I'm pretty disappointed by GPT-4's literary skills, specifically.
GPT-4 can write seemingly anything-- it can roll with just about any topic or style and produce a poem or story or essay. What it hasn't done for me so far is produce writing that I enjoy reading. I don't know if this is a low bar or a high bar. It's really just my personal preference.
I enjoyed the process of writing with the old GPT-3. Its writing sometimes went off into robotic loops, and yes sometimes it was offensive or just weird, but it was often surprising and I found the process engaging. A friend and I collected a lot of interactions and we turned our favorites into a zine. We got feedback that the writing was "weird" and "inhuman"-- fair enough. But it was a project I had fun with.
I had it in mind that I would produce issue 2 of the zine with GPT-4, but I just can't get into it. GPT-4 is locked down and limited, as many people have noted. I keep running into it saying that it is just an AI and doesn't have thoughts or feelings. While this is true, the interface of chat makes it natural to use I-you language. And I just don't seem to like the output as much.
Here's a poem that GPT-3 produced that I liked well enough to add to the zine:
The Nature of Mark Making
a poem by Cassandra CrunklevichInk gather at the nib
Drip then travel the paper
Drag and swirl around the bends
Where down meets up
And drawing becomes writing
Something once separated
Becomes one
Different meanings
The same forms
Flow and continue
Beyond the page and into the world.
This one I found evocative. I was still in the mode of being continually amazed that a computer could write at all. It fascinated me and I wondered where it was getting these images from.
Here's what GPT-4 did with the same prompt:
The Nature of Mark Making
a poem by Cassandra CrunkWithin the realm of whispered lines,
A scribe begins her toil,
Each stroke, each curve, a tale unwinds,
Upon a virgin soil.Cassandra Crunk, she casts her spells,
In graphite, ink, and hue,
The Nature of Mark Making tells,
Of worlds both old and new.A rhythm beats, the dance begins,
With pen held poised and deft,
Each twist and turn, a spark that spins,
A tapestry of heft.
...it goes on for 8 stanzas
This one has rhyme, meter, check: that's something GPT-3 couldn't do well. Responds to prompt content: check. But it's overwhelmingly cheesy, pompous and highfalutin. I'm embarrassed for it. And I guess I'm no longer amazed that it can write at all, so I have a higher standard now.
I mean writing poetry is hard, and rhyming poetry is especially hard. I suppose it is not to be expected that AI would necessarily excel at it. I suppose it's like, I mean one thing you might want out of a writing AI is for it to write things you want to read. That's clearly not the only standard but that's one standard. And for me at least, GPT-4 doesn't cut it. Poetry is bad. Stories are all the same. Essays read like Wikipedia. It is amazing and impressive that it can write grammatically, clearly, and on any topic. Is originality too much to ask or just not what it was optimized for? Maybe a bit of both.
Anyway I hope that there will be future models which are better at creative writing. I don't think this will necessarily happen automatically: the "helpful assistant" idea of what's good is probably fundamentally opposed to creativity. Someone will have to build AI with a different vision.