In this episode of Chat With Sean, I’m joined by Adam Gent, co-founder of Indexing Insight, a tool built to help SEOs understand how Google is indexing and de-indexing pages at scale. With over 15 years of experience in SEO across both agency and in-house roles, Adam shares deep insights into one of the most misunderstood areas of search: indexing behaviour and crawl management.

We explore how Google decides what stays in the index and what gets removed, why indexing is becoming even more critical in the AI search era, and how large-scale websites can lose visibility without realising it. Adam breaks down real-world case studies, including how technical issues can silently remove hundreds of thousands of pages from search.

Adam discusses:

- How Google’s indexing system actually works behind the scenes

- Why pages are actively de-indexed (and not just “not indexed”)

- The importance of crawl demand, engagement, and internal linking

- How Indexing Insight identifies indexing issues faster than traditional tools

- Why sitemap structure matters more than most SEOs realise

- How AI search increases the importance of being properly indexed

Topics covered include:

- Crawl current vs crawl previously indexed explained

- Why Google never emails you about active de-indexing

- The role of demand, engagement, and internal links in indexing decisions

- How to structure sitemaps for large-scale websites

- Splitting sitemaps by page type, demand, and business value

- Case study: how a T-Mobile indexing issue led to major traffic changes

- Why Google Search Console hides key indexing signals

- The 130-day indexing rule and 190-day crawl “forgetting” threshold

- How core updates can directly impact indexing behaviour- Building dashboards to communicate indexing issues to stakeholders

- Using input metrics to improve indexation outcomes

- Why SEOs need better business cases to influence developers

Thank you to our partner, SEO Gets, the easy-to-use analytics and reporting tool for your GSC & GA4 data. Be sure to check out the tool and use code search10 for 10% off: https://seogets.com/