The end of hypertext
The old internet was like reading a book. AI-enabled internet will be like talking with its authors.
The most shocking part of the Bing and Bard search demos is what didn’t happen. The chatbots answered all kinds of user queries… then the user never clicked through to another website.
Website publishers are asking, “Are we in danger?” Their business model relies on showing ads to visitors, but of no one clicks through they get no ad impressions, which means no money. If these “zero-click" searches continue, Bing won’t have anybody left to scrape answers from.
RIP HTTP
Publishers of static content - from the New York Times to recipe blogs – are right to fear. AI search results are the culmination of a long trend of enriching results so the user never needs to visit the underlying site.
But search is fundamentally limited. It can only find things that already exists.
For all the questions Google can't answer, we pull for answers from the source: other human beings. Quora, Reddit, Facebook Groups, Discord channels, etc. let us request new information.
But it’s not easy. You have to find the right corner of the internet with the right experts. Even then, maybe the right person isn’t online, doesn’t see your question, or can’t be bothered to answer.
Generative AI enables an entirely new protocol for information search, retrieval, and generation that lets publishers create bespoke content on demand.
The Lady of the Data Lake
Most entrepreneurs would love to pick Paul Graham’s brain for advice, but few get that chance. Instead, we can read his blog to try to absorb his advice across all his content.
Paul Graham himself might not be available to chat, but Paul Graham GPT lets you interact with the full corpus of his writings as if having a conversation. If the old internet was like reading a book; the new internet will be like interacting with minds.
PGGPT only accesses the articles from PG’s blog. But it’s an easy step to add much more data from
Tweets
Interviews
Public corporate records
Webpages of startups PGs invested in
“Data lakes” (to borrow a term from big data consultants) are repositories of such structured and unstructured data. Every website, brand, community, and individual can pool their data from which AI can answer original queries.
Google can index Paul Graham's blogs, but it won't have access to all the underlying data that powers his simalacrum.
This still feels like sci-fi, and it’s not quite uploading our consciousness. There are challenges around data curation, context windows, hallucinations; but the result is an ever improving approximation of talking to a person.
Thinking bigger
Embedding one expert is a great start. Soon this will be native to every Substack.
Next let’s embed the top 10,000 posts from r/startups.
Reddit jokes about the “hive mind”, but building an AI interface for each subreddit would let you interact with an amalgam of experts on any subject.
If I want Paul Graham GPT and Warren Buffet GPT as advisors, that’s an option. Maybe they charge for access. Since the end-user only interacts with the AI model’s output, it’s very hard to pirate the service without the underlying data. This could be a godsend for publishers and content creators.
Update: New tools have been released providing a “Squarespace/Wordpress for AI” that lets anyone build a chatbot from their own structured and unstructured data. The future’s coming fast.
An old dream
The internet we know consists mostly of text. It’s the T in HTML. But GPTs aren’t a collection of static texts like Wikipedia; they’re incomprehensible statistical models only readable by machines.
Tim Berners-Lee, inventor of HTTP and the World Wide Web, wrote in 1999:
I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers. A "Semantic Web", which makes this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The "intelligent agents" people have touted for ages will finally materialize.
Twenty-four years later, GPTs have progressed to the point that this dream is becoming reality. All manner of data will live in separate pools, with AI interfaces providing interoperability and human readability.
Instead of scraping pages directly, top-level search like Google might open a conversation with the right chatbot. Less “here’s a book with the answer”, more “I know just the guy you should talk to.”
AI search is exciting, but it’s really the last chapter of the old internet. The generative internet’s story has only begun to be written.