Against my better judgement, I was on the cesspool that is Elon’s X, the Mos Eisley of the web, when I came across something that made me feel rather upset. While that’s kind of the norm with the type of vapid trash that floats across that platform’s timeline, this was something that struck a chord with me, touching on art and “exploring it through AI.”
Before I get started, I want to be clear: I’m not coming from an anti-AI perspective. I don’t think that AI is of the devil & should be wholly avoided. There are incredible applications for AI as a technology. As someone working in VFX, the power of depth from monocular images, deepfake technology for facial replacement, and various AI use for relighting are all rather incredible. Software development can and has already used AI in hugely transformative ways, for codebase analysis, PR review, general automation, etc.
However, this post, by one Aleksander Holynski, was describing one of the many “features” that their new Genie3 model allowed users to do. In the attached video, he showed what looked to be Edward Hopper’s Nighthawks, except it was all wrong.
For those who need a refresher, this is the original piece by Hopper:
Back to Holynski’s post - in his attached video, the “camera” moves, showing different “perspectives” of that corner restaurant. At one point, the camera peers towards the back of the diner and into the supposed kitchen, later, it looks across the street, down the alley behind…
Let me pause here.
While I’m not going to attempt to categorically define what art is, one major component in my definition of art is that it involves an artist and comprises of their choices and taste. It’s every bit of what they decide to omit from the painting, song, or film as it does what they include. The photographer frames the mountain between the two houses in the foreground, but tilts the camera down to avoid the power lines above. The producer in the studio choses to route the singer’s vocals through a particular FX chain for a distinct texture, sweetening the sound so that it layers over the instruments in an impactful way. And the director choses to rearrange the sequence of a few shots for greater impact, which allows the next scene to be omitted, since the intention of the beat conveys much more elegantly through subtext.
I think what is really rubbing me the wrong way with Holynski’s post is that the subtext seems to imply that by “exploring” the artwork through this generative model is in some manner a way to engage with or better understand the art itself. It’s not.
I think it’s hard to immediately understand it in this visual context, because from the untrained eye, it does look like the three dimensional, almost-tangible world that Hopper painted Nighthawks from. But Hopper made deliberate choices: he placed the two individuals leaning on the counter towards us, the singular individual facing away. He had the restaurant worker leaning down, as if to grab something from below the counter. He set the direction in which he was facing in a manner implying he was mid-conversation with the couple. Hopper made so many choices.
This kind of “generative exploration” immediately shirks all of that. Within only seconds of the camera moving, Genie3 decides to slap the ugliest, incomprehensible scrawl that is a shoddy attempt at the diner’s branding on the wall, not in the original painting.
That sits above what LLM is trying to suggest is a kitchen… that is, if you can even call it a kitchen, not hinted at in the slightest in the original painting. Is it another part of the diner that patrons get seated at? The street running perpendicular to the one the painting’s perspective comes from is just some void of an alley, and looking down the main street to the right shows the “city” abruptly ending, with only lampposts fading off in the distance, none of this is in the original painting. The repeated, copy-pasted buildings are generic and nondescript.
There’s some kind of art gallery (a pawn shop, or just a horrific collection of smudges?) next door to the diner, of course, not in the original painting.
You’ve probably detected a theme.
Remember what was implied in the original painting - the couple facing us? That couple facing us was actually just a lady-less, leg-less, single dude, with his torso magically growing out of the counter, according to Genie3. Suddenly, the restaurant worker is just another (grey-haired, his hat is gone) patron. Never mind that any human looking at the painting for a single second could tell that was clearly not the case.
Imagine this with other forms of art:
- With Klepto 4.2, you can watch a generated, personalized blooper reel from The Godfather!
- Introducing Goop, allowing you to hear what Michael Jackson’s raw vocal stems for Smooth Criminal might-could-actually-probably-don’t sound like thanks to generative AI.
- Our latest LLM, ShatPP3, generates additional chapters to Tolstoy’s War and Peace!
I’m honestly just confused… what’s the point? Why do this? Sure, it’s flashy, visually attractive, but if you give it a second of thought, it has absolutely no meaning. And there seems to be a general trend with the use of AI: Altman conflating the image processing on today’s iPhones with fully generated content, with his general point being “why does it matter” and “it’s basically the same thing.” Even Holynski’s other uses for Genie3 involve revisiting places he’s lived via “a world he built to look like his hometown”, which just looks like a lard-flavored, lifeless, half-assed counterfeit of Google Streetview. Come on.
I’m going to stop here, before I get too carried away. A few times as I’ve been writing this, I’ve wondered if this guy is even real. Maybe it’s just an LLM-run, rage-bait, engagement-farm account. Who knows.
If you want to look at real art, check out Edward Hopper’s 1942 oil painting, Nighthawks.