What is AJAX in SEO?
7 min read
AJAX (short for Asynchronous JavaScript and XML, a term coined by Jesse James Garrett on February 18, 2005) is a client-side web development technique where a browser uses JavaScript (via the modern fetch() API or XMLHttpRequest) to exchange data (typically JSON) with a server in the background and update parts of the DOM dynamically without a full page reload. In technical SEO and AEO, relying solely on client-side AJAX for core content is dangerous: while Googlebot can render automatic on-load AJAX requests, it never clicks "Load More" buttons, and AI crawlers (GPTBot, ClaudeBot, PerplexityBot) do not execute client-side AJAX at all.
What is AJAX & How Does It Work in Modern Web Apps?
In a traditional server-rendered webpage, clicking a filter or pagination link forces the browser to request an entirely new HTML document from the server. AJAX decouples data fetching from full page reloads:
- A user interacts with the page (or the page finishes loading its initial HTML shell).
- JavaScript executes an asynchronous background HTTP request to an API endpoint using
fetch('/api/items?page=2')orXMLHttpRequest. - The server returns a lightweight JSON payload (despite the "XML" in the historical acronym, modern AJAX almost exclusively uses JSON).
- JavaScript parses the JSON response and injects the new HTML nodes directly into the live Document Object Model (DOM).
The History of AJAX in Google Search: From #! Hashbangs to Evergreen Chromium
How Google handles AJAX has evolved dramatically over the past two decades:
- The 2009-2015 Hashbang (
#!) Era (Deprecated): In 2009, Google proposed an AJAX Crawling Scheme where URLs containing#!(hashbangs) were rewritten by Googlebot to?_escaped_fragment_=to fetch a static HTML snapshot. Google officially deprecated this scheme in October 2015 and completely stopped supporting_escaped_fragment_in 2018. - Today’s Evergreen Googlebot (Web Rendering Service): Today, Googlebot runs an evergreen headless Chromium browser. When Googlebot crawls a page, it first parses the raw server HTML response (Wave 1), and then queues the page for JavaScript execution (Wave 2), where it executes automatic on-load
fetch()andXHRrequests.
3 Ways AJAX Still Destroys SEO & AI Search Visibility Today
Even with Google’s modern Chromium renderer, improperly architected AJAX causes massive indexing failures across B2B SaaS blogs, resource centers, and documentation sites:
| AJAX Anti-Pattern | What Googlebot WRS Does | What AI Crawlers (GPTBot / PerplexityBot) Do | The Engineering Fix |
|---|---|---|---|
1. "Load More" / Infinite Scroll Without <a href> Links |
Fails completely. Googlebot is stateless and never clicks buttons or triggers scroll events that require user interaction. | Fails completely. Only sees the first page of items present in raw HTML. | Include crawlable <a href="/blog?page=2"> pagination links alongside infinite scroll using history.pushState(). |
| 2. Client-Side Tabs / Accordions Fetching Content on Click | Ignores hidden API data. Content fetched via AJAX only after an onClick event is never indexed. |
Ignores hidden API data. | Render all tab/FAQ text in the initial server HTML and toggle visibility via CSS (display: none/block). |
| 3. Empty CSR App Shell Fetching Article Text via AJAX | Delayed two-wave indexing; risks Soft 404 if the AJAX API times out or is blocked by robots.txt. |
100% Invisible! AI crawlers do not execute JS; they see an empty <div id="root"></div>! |
Use Server-Side Rendering (SSR) or Static Site Generation (SSG) for all public pages. |
How to Build 100% SEO & AEO-Compliant AJAX Pagination and Filtering
Progressive Enhancement Pattern (HTML5 History API + SSR)
1. Every Filtered or Paginated State Must Have a Unique, Routable URL: Never update page content via AJAX while leaving the browser URL unchanged! Use the HTML5 History API (window.history.pushState({}, '', '/case-studies?industry=healthtech')) and ensure that if Googlebot requests that URL directly from the server, the server returns a 200 OK with the pre-rendered HTML.
2. Always Use Real <a href="..."> Tags for Navigation: Instead of <button onclick="loadNextPage()">Load More</button>, write <a href="/blog/page/2" onclick="loadAjaxPage(event, 2)">Load More</a>. Human users get an instant AJAX transition (via event.preventDefault()), while Googlebot follows the standard href attribute effortlessly!
3. Never Block AJAX API Endpoints in robots.txt: If your page fetches data from /api/v1/pricing-data during rendering, blocking Disallow: /api/ in robots.txt prevents Google’s Web Rendering Service from loading your pricing table.
Frequently Asked Questions
Can Googlebot crawl and index AJAX content in 2026?
Googlebot can index AJAX content ONLY if the AJAX request fires automatically on initial page load without requiring any user interaction (like clicking a button or scrolling), and if the API endpoint is not blocked by robots.txt. However, Server-Side Rendering (SSR) is always safer and faster.
Can ChatGPT (GPTBot), ClaudeBot, and PerplexityBot read content loaded via AJAX?
No! The vast majority of AI and LLM web crawlers fetch only the raw initial HTTP HTML response and do not execute client-side JavaScript or AJAX requests. If your article or pricing data is loaded entirely via client-side AJAX, your brand will be invisible to AI answer engines.
Why does "Infinite Scroll" or a "Load More" button hurt blog SEO?
Googlebot does not click "Load More" buttons or simulate scrolling to the bottom of infinite-scroll pages. Unless you pair infinite scroll with crawlable <a href="/blog/page/2"> pagination links and history.pushState(), Googlebot will only ever discover the first 10 posts on your blog.
Should I still use #! (hashbangs) or _escaped_fragment_ for AJAX SEO?
Never! Google officially deprecated the #! / _escaped_fragment_ AJAX crawling scheme in 2015 and retired support in 2018. Modern web applications must use clean URLs with the HTML5 History API (pushState) and Server-Side Rendering.