Schema Markup and llms.txt in 2027: The Technical Side of AI Search

Great content still needs to be readable by machines. If a crawler cannot reach a page, cannot tell what it is about or cannot tell who stands behind it, the page is unlikely to show up in search results or AI answers. This guide walks through the setup we use on mediasoli.com.

Step 1: Check crawler access in robots.txt

Open yourdomain.com/robots.txt. Make sure nothing blocks the crawlers you want. The main ones in 2027:

Crawler	Who runs it	What it feeds
Googlebot	Google	Google Search and AI Overviews
Google-Extended	Google	Gemini model improvement (opt-out token)
Bingbot	Microsoft	Bing, Copilot and ChatGPT search results
OAI-SearchBot and ChatGPT-User	OpenAI	ChatGPT search and browsing
GPTBot	OpenAI	Model training
ClaudeBot, Claude-SearchBot, Claude-User	Anthropic	Claude training, search and browsing
PerplexityBot and Perplexity-User	Perplexity	Perplexity answers
Applebot	Apple	Siri and Spotlight suggestions

Blocking training crawlers is a business decision, but blocking search and user-triggered crawlers removes you from AI answers. Security plugins and CDN bot settings sometimes block them without telling you, so test after any change.

Step 2: Get indexed in Google and Bing

Verify your site in Google Search Console and Bing Webmaster Tools. Bing lets you import your verified sites from Search Console in a few clicks. Submit your XML sitemap to both, and enable IndexNow (Yoast SEO and many hosts support it) so new and updated pages reach Bing quickly. This matters because ChatGPT search leans heavily on Bing’s index.

Step 3: Add the right structured data

Structured data, usually written as JSON-LD, describes your content using the shared Schema.org vocabulary. These are the types that matter most for AI search:

• Organization or LocalBusiness (or a subtype such as ProfessionalService): name, logo, description, contact details, address, service area, founder, services and social profiles in sameAs.
• Person for authors: name, job title, photo, description, worksFor and links to profiles such as LinkedIn.
• Article for blog posts: headline, dates, author and publisher.
• FAQPage for question and answer sections.
• Service and Offer for what you sell.
• BreadcrumbList and WebSite for site structure.

Connect everything into one graph

A common mistake is having two or three schema blocks that describe the same business with different names or details, for example one from an SEO plugin and another hand-written one. Use stable @id values and reference them: the article’s publisher points to your organization, the author points to the person, and the person’s worksFor points back to the organization. On our site we merged everything into a single Organization plus ProfessionalService entity for exactly this reason.

Validate it

Run key pages through Google’s Rich Results Test and the Schema.org validator. Fix errors first, then warnings that affect important properties.

Step 4: Publish an llms.txt file

llms.txt is a proposed standard: a plain Markdown file at yourdomain.com/llms.txt that gives AI tools a short summary of who you are and links to your most important pages. It costs almost nothing to add. Keep expectations realistic, though. There is little public evidence that it changes citation rates on its own, so treat it as a tidy extra on top of crawlability, content and schema.

A good llms.txt includes a one-paragraph description of the business, key facts such as location, founder, services and contact details, a list of main pages with one-line descriptions, and your best articles. Ours rebuilds itself automatically whenever we publish or update content, so it never goes stale.

Step 5: Keep it fast

Speed affects crawling and user experience, and slow, script-heavy pages are harder for some crawlers to render. Aim for green Core Web Vitals and strong Lighthouse scores. Read how we moved mediasoli.com to 100 in every Lighthouse category in our Lighthouse case study.

Technical AI search checklist

• robots.txt allows Googlebot, Bingbot, OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User and PerplexityBot.
• Site verified in Google Search Console and Bing Webmaster Tools, sitemaps submitted.
• IndexNow enabled.
• One connected schema graph with Organization, Person, Article and FAQPage.
• Author pages and bios with real credentials.
• llms.txt published and kept current.
• Fast mobile performance and no render-blocking bloat.

Technical work gets you eligible. Content and reputation get you chosen. For that side, see our GEO playbook.

Frequently asked questions

Does schema markup help with ChatGPT and Gemini?

Schema helps search engines understand your pages, and AI assistants rely on those search systems. It also removes ambiguity about who you are and what you offer, which supports accurate AI answers.

Should I block GPTBot?

Blocking GPTBot stops your content from being used for OpenAI model training, but it does not block ChatGPT search, which uses OAI-SearchBot. Decide based on your comfort with training use, and keep search crawlers allowed.

Is llms.txt required?

No. It is optional and experimental. It is worth adding because it is cheap and harmless, but it does not replace good content or structured data.

Which plugin should I use for schema on WordPress?

Yoast SEO and Rank Math both output a solid schema graph. Extend it with your business details and author information rather than adding a second, separate schema plugin.

Want us to set this up?

Mediasoli handles technical SEO, schema and AI crawler setup for WordPress, Shopify and Wix sites. Start a project and we will audit your site’s machine readability.

Sources

• Google Search Central, Introduction to structured data
• OpenAI, Overview of OpenAI crawlers
• llmstxt.org, The llms.txt proposal
• IndexNow protocol