Show AI crawlers what you want them to see – with Arnout Hellemans
Failed to add items
Add to basket failed.
Add to wishlist failed.
Remove from wishlist failed.
Adding to library failed
Follow podcast failed
Unfollow podcast failed
Show AI crawlers what you want them to see – with Arnout Hellemans
-
Narrated by:
-
By:
Summary
Arnout Hellemans advises that you don’t have to provide AI crawlers with full access to all of your content.
Arnout says: “People really need to look at their technical setup.
With that, I mean the rendered version versus the raw HTML – especially with a lot of AI crawlers not rendering yet, and also because I've seen a lot of discrepancies in that area.”
Is the rendered version typically quite different for different search engines?
“As with a lot of things in SEO, it depends.
Sometimes, when websites are built using JavaScript frameworks, the content is actually different in the raw HTML versus the rendered HTML.
Titles might be different. There might not be schema markup, there might be different headings, etc., because those can be changed by the execution of JavaScript.
That can severely impact the discoverability of your page.”