I’m now starting to be convinced that the only maybe-possibly-sustainable use case of AI is going to be ad-driven entertainment/chatbots/“virtual girlfriend” type apps, as well as low-effort social malware such as spam, astroturfing, and phishing, running on very old non-reasoning models (and small diffusion models for generating the “photos” of your artificial girlfriend).

I say this because a trillion+ dollars have been spent on making generative AI productive enough to replace humans, and so far, there has been zero use cases that are even plausible. The cost per inference for reasoning models is absolutely exploding to literally hundreds- or thousandfold over non-reasoning models, and there is little evidence that anything works as claimed.

Low-effort chatbots, though, have been shown to effectively prey on the addictive nature of staring into an interactive funhouse mirror, and has the potential to generate more “engagement” (i.e. ad impressions) than traditional algorithmic social media.

Unless there is a significant advance in semantic modeling and inference, this is as good as it gets: we are now seeing the limits of predictive pattern-matching models. Improvements in “reasoning” are quite clearly asymptotic—a huge increase in token output and processing time only produces marginal improvement in output quality, and is not worth the extra money in virtually all cases.

Entertainment bots and spam, then, is the future of today’s generation of generative AI. Instead of scrolling through endless human-generated content on social media and being tracked and shown ads, people will be scrolling through endless machine-generated content on chat media and being tracked and shown ads.

Spam automation needs only the crappiest of models, because they rely on baiting people who can’t smell an SMS scam full of (intentional) misspellings. Social media astroturfing doesn’t need sophisticated reasoning models to reply “great, everyone line up like sheep again” to every post about COVID.

If entertainment bots actually catch on and become financially viable, then I think the next big step for the generative AI business would be client-side inference. Because these bots can run on older and smaller models, hardware advances may one day make it viable to run a virtual girlfriend entirely on a phone without setting its battery on fire. If that happens, server demand will drop precipitously as bot makers offload their cost onto millions of smartphones that contact the servers only to gatekeep access, send surveillance telemetry, and retrieve fresh ads.

In fact, I think client-side inference running smaller, non-reasoning models to do simple, low-stakes, non-critical tasks will become the vast majority of the AI use case long-term. It seems to be the only one that makes financial sense.

• #opinion
• #AI