Unlocking Keywords: Generative AI and the Future of Search

How will generative AI change the way we search for information in the future?

Srivatsan Venkatesan

and

Ramesh Ravishankar

May 26, 2023

Hello, friends! 👋

Get ready for a thrilling ride as we dive into the captivating world of AI.

Today, we have an exciting story of competition, innovation, and the race to dominate the AI-powered search landscape.

So, what's the story behind Google's relentless pursuit to regain its throne?

Microsoft CEO Satya Nadella, about Google competing with Chatgpt, famously said,

"I hope with our innovation they will want to come out and show that they can dance. I want people to know that we made them dance."

This quote captures the intense rivalry and desire for innovation in the AI space.

You can watch the 39s video of the interview 👇👇👇

Let's rewind to 1995 and revisit Bill Gates' insightful interview with David Letterman to gain further perspective on this dynamic landscape.

In this interview, Gates discusses the potential of the emerging internet. Fast forward to today, and we can replace "internet" with "Generative AI" making the interview relevant as it highlights the transformative power of AI technology.

You can watch the 59s video of the interview 👇👇👇

The events that unfold are nothing short of dramatic 🌟 🌟

Google's AI event received mixed reviews, and a significant blunder resulted in a substantial market cap loss. But Microsoft isn't exempt from mishaps either, highlighting the high stakes of this AI competition.

In the midst of it all, Microsoft, traditionally with a smaller share of the search market, aims to challenge Google's dominance. Their integration of GPT into Bing disrupts the status quo and aims to attract more users. According to a study by StatCounter, Bing's market share increased from 2.8% in January 2022 to 3.1% in December 2022, following the integration of GPT. The impact is undeniable.

However, Google, the reigning champion, isn't backing down.

With its firm grip on the search business, commanding nearly 93% of the market, Google is determined to defend its territory. Each percentage point of search advertising market share, as highlighted by a study from eMarketer, amounts to a staggering $2 billion in revenue. This year, search advertising is projected to generate $237 billion in revenue. These figures underscore the immense value at stake.

To counter Microsoft's advances, Google makes its strategic moves, investing $100 million in Anthropic, another competitor to ChatGPT. The battle lines are drawn, and the future of search hangs in the balance.

Now, the question arises:

What does this fierce competition mean for the future of search?
How does generative AI play a pivotal role in shaping the landscape beyond traditional keyword-based approaches?

Join us as we embark on a thrilling journey to unravel the significance of generative AI in search technology and explore the untapped potential that lies ahead.

And if you haven't caught up on our previous blogs, here's a quick recap:

Let's set our course and explore how generative AI reshapes the search landscape, moving us beyond traditional keyword-based approaches.

Here's a sneak peek at the topics we'll be covering:

Google’s AI Evolution 🔍
Keywords or LLMs for Search?
Generative AI and Search 🌟
Leveraging Generative AI: Opportunities for Startups 💼

Let's get started.

Google’s AI Evolution 🔍

In 2016, CEO Sundar Pichai highlighted the important role of machine learning in Google's products, leaving a lasting impression.

But here's the surprise: Google, a company seemingly built for AI, would be caught off guard to such an extent. After all, Google has been wowing us with its AI prowess at Google I/O for years.

That year, Google invented the transformer, the key invention undergirding the large language models that power ChatGPT, the product that seemed to fluster Google so much over the past six months.

What was impressive about this year’s Google I/O is that it managed to combine what was compelling about Google’s last several I/Os — its clear AI capabilities and the products in which to manifest them — with the urgency and aggressiveness that are exactly what you would hope to see from a company feeling threatened for the first time in years.

Gmail evolved from “Smart Reply” to “Smart Compose” to “Help Me Write”

Google Photos evolved from “Find Photos” to “Magic Eraser” to “Magic Editor”:

This was a very clever way to reinforce the idea that Google has been at this AI stuff for a while, and it’s true! It was also a reminder that one of Google’s big advantages mirrors Microsoft’s: the company has many user-facing products to surface AI capabilities in genuinely useful ways.

Pichai noted that Google had 15 products with over 500 million users:

What's interesting is how Google is integrating AI into everyday software seamlessly.

It's no longer just a standalone destination; it's becoming an integral part of the tools we use daily. From Office and Google Docs to the operating system (Android) and graphics editors like Adobe
AI is woven into major web platforms' surfaces.

While Google may have faced some challenges in AI capability, they still have a tremendous advantage due to its control over billion-user platforms. They will find ways to integrate AI without direct costs, leveraging alternative methods like ads to subsidize the capabilities.

OpenAI faces the existential risk, not Google. They'll catch up and will have the reach/subsidy advantage.

And it doesn't end there. This so-called "competition" from open source will be free labor. Any winning idea is ported into Google's products on short notice.

Thanks, open source!

Keywords or LLMs for Search?

In the ever-changing world of search, a crucial question arises:

Keywords or LLMs? Traditional vs. cutting-edge.

Let's see how they impact search.

Keywords have long been the backbone of search engines, enabling users to input specific terms and receive relevant results. This approach has served us well, providing quick access to information and allowing us to navigate the vast expanse of the web. However, keyword-based searches have their limitations. They rely on users formulating precise queries and often struggle with understanding context and intent.

Now, let's talk about Large Language Models. Like OpenAI's GPT-3 and Google PaLM, LLMs can generate human-like text and directly answer queries.

Users can ask a question instead of searching for specific keywords and receive a comprehensive answer. This shift from keyword-based search to answer-oriented interactions can revolutionize how we discover and consume information.

Speaking of Search, let's explore a few use cases that showcase the power of Large Language Models (LLMs) in transforming the search experience:

Craving Thai food? Ask, "Where's the best Pad Thai nearby?" and get personalized recommendations.
Need the IRS phone number during tax season? LLMs can provide the correct number or guide you through tax-related inquiries.
Debugging code? Ask a coding question and receive a direct answer or step-by-step solution.

These examples show how LLMs can deliver tailored answers, making Search more interactive and accurate. However, it's essential to consider the suitability of LLMs for different types of searches. While they excel at complex questions, specific factual or localized information may benefit from keyword-based searches.

The debate between keywords and LLMs for search is important as we navigate the future of information retrieval. Striking the right balance between the precision of keyword-based search and the contextual understanding of LLMs is crucial to meet users' diverse needs and provide them with a seamless search experience.

So, can Google effectively leverage LLMs to unlock the transformative potential of search? Let's explore this further in the next section.

Generative AI and Search 🌟

OpenAI's ChatGPT and the growing interest in generative AI have shaken the world of search engines. It's an exciting development, but let's be clear—ChatGPT is still playing catch-up in the search engine game, sitting in the second-tier rank alongside other information discovery services. It's no surprise that Google continues to dominate the field.

Just take a look at the graph below:

As you can see, Google maintains its stronghold with a substantial market share, surpassing ChatGPT and other search engines. While ChatGPT has gained attention and showcased its capabilities, it's still a relatively minor player compared to the search giant.

But let's not underestimate the potential of generative AI and its impact on the future of search. The emergence of ChatGPT and similar technologies highlights the ever-evolving nature of search engines and the increasing role of AI in delivering personalized and relevant search results.

That's why Google is taking a hybrid approach, combining the power of generative AI with its tried-and-true search algorithms.

They understand that not all searches are about commerce, and there's value in providing generated text right at the top of search results.

To access AI-generated snippets in search results, opt-in to the Search Generative Experience (SGE) within Search Labs. Google's algorithms determine when AI answers are more useful than regular results. Sensitive topics like health and finances are excluded from AI interference, prioritizing user privacy and accurate information.

Of course, for searches that do have commercial possibilities, ads will still take the spotlight:

So, what does this mean for the future of Google Search? Well, it's all about AI, but perhaps not in the way you might expect.

Google isn't completely shifting its focus to chatbots or transforming its homepage into a messaging system like ChatGPT. Instead, they're putting AI front and center in the most valuable space on the internet—their search results

In conclusion, Google's emphasis on accuracy and factual information in search AI and its cautious approach in sensitive contexts reflects a commitment to providing reliable results. While LLMs can sometimes mislead, Google acknowledges the need for responsible use. The introduction of AI snippets for beneficial search results and their exclusion when unnecessary further highlights Google's efforts. As we look to the future, the development of Google Project Magi, a conversational and personalized AI search engine powered, opens up exciting possibilities.

Only time will tell how Google will utilize Bard-based Google AI search, Google Project Magi, and generative AI in the future. 🌟

SIDENOTE: Bing is now the default search for ChatGPT

Breaking Down the Moats: OpenAI vs. Google

SemiAnalysis published a leaked Google document revealing a startling admission: "We Have No Moat, And Neither Does OpenAI." This thought-provoking paper argues that the race between OpenAI and Google to develop the most advanced language models is overshadowed by the relentless progress made in the open-source community.

Key Points:

Document leak reveals lack of competitive advantage (moat) for both OpenAI and Google.
Open source community emerges as a hub of rapid innovation
Tremendous progress in just a month, with various advancements building upon each other
The scaling problem solved, enabling widespread tinkering and experimentation
The barrier to entry significantly lowered, empowering individuals to contribute and drive progress in AI research

Explore Simon Willison's Wefor blog for in-depth commentary on this.

Leveraging Generative AI: Opportunities for Startups 💼

AI Search startups are revolutionizing how we search for information by harnessing the power of machine learning and natural language processing. Their goal is to make finding relevant information easier, regardless of its format or source. These startups improve the search experience and enhance our understanding of the information we discover.

In the rapidly growing AI Search market, several startups are pioneers in reshaping how we interact with information. Let's take a look at some of the top AI Search startups:

Aiimi: Revolutionizing search with AI solutions.
Casetext: Legal research platform powered by AI. (Raised $45.8 million)
Alphasense: AI-driven business insights for better decisions. (Raised $120 million)
Syte: Visual AI technology for enhanced search experiences. (Raised $40 million)
ViSenze: AI-powered visual search and image recognition. (Raised $20 million)
OpenAI: Advancing AI research and language models. (Raised $11 billion)
Coveo: AI-powered search and recommendations for enterprises. (Raised $227 million)
Algolia: AI-driven search and discovery solutions. (Raised $184.2 million)
Yext: AI-powered search and knowledge management platform. (Raised $177.2 million)
Donde: E-commerce search, acquired by Shopify.

These startups are leveraging AI in unique ways to improve the search experience for users. Aiimi focuses on data and information management, Casetext caters to the legal profession, and Alphasense aids businesses in making informed decisions.

While the AI Search market is still in its early stages, these startups are poised to impact how we interact with information significantly. Their innovative approaches and cutting-edge technologies are reshaping the search landscape and setting the stage for a more efficient and intuitive way of accessing knowledge.

As the AI Search market expands, these startups will play a vital role in shaping the future of information retrieval and user experiences. Their dedication to leveraging AI opens up exciting possibilities for startups and users alike, propelling us into a new era of search-driven innovation.

In conclusion, the future of Google Search lies not in Google Bard but in the power of AI holds the key. Search results will blend generated information with web links, transforming Google's business and potentially revolutionizing the web itself.

This transition aims to trade the traditional "10 blue links" for a centralized knowledge hub, providing users with a comprehensive and trustworthy search experience.

So, let's remain curious and open-minded as we navigate the exciting realms of technology and search.

If you enjoyed this article, don't forget to subscribe to our newsletter for more insights and growth tips.

And follow us on 🐦Twitter: @rramesh25 and @storcube.

Remember to share this post with anyone who might find it valuable.

Happy reading!!

Until next time 👋

- Sri & Ramesh