Finding the "crawled but not indexed" status in Google Search Console can feel frustrating, especially when you're not sure what caused it or how to fix it. While we know the general reasons why this happens (content quality, technical issues, etc.), the reality is that different types of websites face unique indexing challenges.
In this guide, we'll walk through the most common real-world scenarios where pages get crawled but not indexed, and I'll show you exactly how to solve each one. Whether you're running an e-commerce store, a blog, or a business website, you'll find practical solutions you can implement right away.
Scenario 1: E-commerce Product Pages That Won't Index
E-commerce sites are particularly vulnerable to indexing problems. It's not uncommon for online stores to have 30-40% of their product pages stuck in "crawled but not indexed" status.
Why This Happens
Similar Product Variations: When you have products that differ only by color, size, or minor specifications, Google sees them as near-duplicates. For example, if you sell "Blue Running Shoes Size 8" and "Blue Running Shoes Size 9" on separate pages with almost identical descriptions, Google will likely index only one.
Out of Stock Products: Many stores leave old product pages live even after items are discontinued. These pages often have minimal content and no clear value to searchers anymore.
Faceted Navigation Issues: Filter pages create thousands of URL combinations. When users click "sort by price" or "filter by color," your site might generate URLs like "/products?color=red&size=large". Google crawls all these variations but usually won't index them because they're considered duplicate content.
Thin Product Descriptions: Product pages with only a few sentences, manufacturer descriptions copied from other sites, or just bullet points without substantial unique content often get crawled but skipped for indexing.
How to Fix It
Consolidate Similar Products: Instead of creating separate pages for each variation, use a single product page with dropdown selectors for size, color, and other options. This gives you one strong page instead of many weak ones.
Handle Discontinued Products Properly: You have three options here. First, if the product is temporarily out of stock, keep the page live but add "Out of Stock" messaging and link to similar available products. Second, if it's permanently discontinued with no replacement, redirect it with a 301 to the most relevant category page. Third, if you have a newer version, redirect to that new product page.
Use Robots.txt for Filters: Block filter and sort URLs in your robots.txt file to prevent Google from crawling them in the first place. Add rules like:
User-agent: * Disallow: /*?*sort= Disallow: /*?*filter= Disallow: /*?*color=
Enhance Product Descriptions: Write unique, detailed descriptions for each product. Include specifications, benefits, use cases, and any information that makes your page more valuable than a competitor's version of the same product. Aim for at least 300-500 words of original content per product page.
Implement Product Schema: Add structured data markup to help Google understand your products better. This includes price, availability, reviews, and other attributes that can differentiate similar products.
Scenario 2: Blog Posts Stuck in Indexing Limbo
You've published a new blog post, waited a few weeks, and it's still showing as crawled but not indexed. This scenario frustrates content marketers constantly.
Why This Happens
Recently Published Content: Google doesn't always index new content immediately, especially on newer or lower-authority sites. Sometimes it's just a waiting game.
Thin Articles: Blog posts under 500-800 words that don't thoroughly cover the topic often get skipped. If 20 other sites have already covered the topic more comprehensively, Google has little reason to index your shorter version.
Poor Internal Linking: If your new post isn't linked from anywhere on your site, Google has no way to determine its importance or relevance within your content structure.
Cannibalization Issues: If you have multiple blog posts targeting the same keyword or topic, Google might choose to index only one and leave the others in limbo.
How to Fix It
Be Patient (But Proactive): For genuinely new content on newer sites, give it 2-4 weeks. During that time, make sure the page is in your sitemap and request indexing through Google Search Console.
Expand Your Content: Compare your article to the top 10 results for your target keyword. If most of them are 2,000+ words and yours is 800, that's a clear signal. Add more depth, examples, data, and unique insights. Quality matters more than length, but comprehensive coverage is important.
Build Internal Links: Link to your new post from at least 3-5 related articles on your site. Use descriptive anchor text that includes your target keyword. Also link from high-authority pages like your homepage or main category pages if relevant.
Consolidate Competing Posts: If you have multiple posts about similar topics, consider merging them into one comprehensive guide. Use 301 redirects from the old URLs to the new consolidated post. This gives you one stronger page instead of several competing weaker ones.
Check Search Intent: Make sure your content matches what people are actually looking for. If everyone ranking for your target keyword has "how-to" guides and you wrote an opinion piece, that mismatch might explain why Google isn't indexing your page.
Scenario 3: Category and Archive Pages
Category pages, tag pages, and date-based archives often end up crawled but not indexed, especially on content-heavy websites.
Why This Happens
Thin Category Pages: If your category page is just a list of post titles with no additional content, Google might consider it low-value.
Too Many Tags: WordPress sites and blogs often have dozens or hundreds of tag pages, each with just 1-2 posts. These pages offer minimal value to users.
Duplicate Content Across Archives: Monthly and yearly archive pages often contain overlapping content, which Google sees as duplication.
Pagination Issues: If you have category pages with multiple pages of results, Google might crawl but not index the deeper pages (page 2, 3, 4, etc.).
How to Fix It
Add Unique Content to Categories: Write a 200-300 word introduction for each category page explaining what visitors will find there. Include relevant keywords naturally and provide value beyond just listing posts.
Reduce Tag Bloat: Audit your tags and delete or merge ones that have fewer than 5 posts. Be more selective about tagging going forward. Consider adding a noindex tag to tag pages if they don't drive meaningful traffic.
Noindex Low-Value Archives: Most sites don't need date-based archives indexed. Add a noindex tag to these pages through your CMS or SEO plugin. They can still serve users who browse your site but won't clutter Google's index.
Handle Pagination Properly: Use rel="next" and rel="prev" tags on paginated series to help Google understand the relationship. Alternatively, use a "view all" option that loads all content on one page (if the page count isn't too high).
Scenario 4: Landing Pages and Campaign Pages
Marketing teams often create special landing pages for campaigns, ads, or promotions. These frequently end up crawled but not indexed.
Why This Happens
Temporary by Nature: Landing pages created for specific campaigns often have short lifespans and minimal integration with the rest of your site.
Duplicate Content: If your landing page duplicates content from your main site (because you're testing messaging for ads), Google will likely skip indexing the duplicate version.
No Internal Links: Landing pages designed only for paid traffic often aren't linked from anywhere on the site, making them appear less important to Google.
Intentionally Hidden: Sometimes you don't want these pages indexed at all (for example, if they're meant only for specific audiences arriving via email or ads).
How to Fix It
Decide If You Want Them Indexed: First, ask yourself if these pages should be in Google's index. If they're only meant for paid traffic or specific campaigns, it might be better to add a noindex tag intentionally.
Make Them Unique: If you do want them indexed, ensure the content is substantially different from other pages on your site. Use unique copy, different angles, or additional information that makes the page valuable on its own.
Add Internal Links: If the landing page is part of your long-term content strategy, link to it from relevant blog posts or category pages. This signals to Google that the page has value within your site structure.
Plan for Lifecycle: Before the campaign ends, decide whether to keep the page live (with updates), redirect it to a permanent page, or take it down and let it 404. Don't just abandon campaign pages when they're done.
Scenario 5: Multilingual or Regional Pages
Sites with multiple language versions or regional variations often see indexing problems with their localized pages.
Why This Happens
Hreflang Issues: Incorrectly implemented hreflang tags confuse Google about which version to show users in different regions.
Near-Duplicate Content: If you have English-UK and English-US versions with minimal differences, Google might treat them as duplicates and index only one.
Auto-Generated Translations: Machine-translated pages without proper quality control often read poorly and get skipped for indexing.
Regional Pages Without Differentiation: Creating separate pages for different regions without actual regional content differences (just changing a city name) results in duplicate content.
How to Fix It
Implement Hreflang Correctly: Add proper hreflang annotations to tell Google which language or region each page targets. This goes in your page headers or XML sitemap and helps Google understand the relationships between versions.
Differentiate Regional Content: Don't just duplicate content across regions. Include region-specific information like local phone numbers, addresses, testimonials from local customers, or content relevant to that specific market.
Invest in Quality Translation: Machine translation is fine as a starting point, but have native speakers review and improve the content. Natural-sounding text in each language performs much better.
Use Canonical Tags Properly: If you have very similar English versions for different regions and only want one indexed, use canonical tags to point to your preferred version. Better yet, merge them into one international English page if the differences are minimal.
Scenario 6: Forum and User-Generated Content
Community forums, Q&A sections, and sites with user-generated content face unique indexing challenges.
Why This Happens
Low-Quality Posts: Not all user contributions are valuable. Short questions with no answers, spam posts, or off-topic discussions don't deserve indexing.
Duplicate Questions: Multiple users asking the same question in different words creates duplicate content across your forum.
Thin Threads: Discussion threads with only 1-2 replies might not have enough content to warrant indexing.
Session URLs: Forums often generate session-specific URLs that create duplicate content issues.
How to Fix It
Set Quality Thresholds: Only allow indexing of threads that meet minimum standards. For example, require at least 3 replies or 500 total words before a thread is eligible for indexing. Implement this through conditional noindex tags based on thread metrics.
Merge Duplicate Threads: Have moderators or automated systems identify and merge duplicate questions. This creates stronger, more comprehensive content instead of multiple weak threads.
Add Editorial Content: Supplement popular threads with editorial additions. Add summaries, expert commentary, or structured information that makes the page more valuable than just raw user comments.
Clean URL Parameters: Configure your forum software to use clean, static URLs. Block dynamic session parameters through robots.txt or use canonical tags to point to the clean version.
Scenario 7: Older Content Falling Out of the Index
Sometimes pages that were previously indexed lose that status and show up as "crawled but not indexed." This scenario can be particularly concerning because it represents a loss of existing rankings.
Why This Happens
Content Became Outdated: Information that was accurate and valuable when published might now be outdated, causing Google to reconsider its value.
Algorithm Updates: Google's quality algorithms evolve. Content that met indexing standards years ago might not meet current standards.
Site-Wide Quality Drop: If you've added a lot of low-quality content recently, it can affect Google's perception of your entire site, including older pages.
Decreased Link Equity: If pages that were linking to your content removed those links, your page might have lost the internal or external link signals that previously helped it stay indexed.
How to Fix It
Update and Refresh: Go through older content and update facts, statistics, and examples. Add new sections covering recent developments. Update the publication date to signal freshness.
Improve Depth and Quality: Compare old articles to currently ranking pages. If standards have risen (longer content, better formatting, more visuals), upgrade your older content to match.
Restore Link Equity: Add more internal links pointing to these pages from newer, higher-performing content. This signals to Google that the pages are still relevant and valuable.
Conduct Content Audits: Regularly review your content library. Merge outdated posts, delete truly worthless pages, and refresh valuable content. Quality over quantity always wins.
When Not to Worry About Crawled But Not Indexed
Not every page needs to be indexed. Here are scenarios where this status is actually fine:
Thank You Pages: Pages users see after submitting forms don't need to be in search results.
Checkout and Cart Pages: E-commerce transaction pages should not be indexed.
Internal Search Results: Pages showing search results within your site should be blocked from indexing.
Test Pages: Staging or test pages should definitely not be indexed.
Duplicate Language Versions: If you've properly set up canonicals to one primary version, having others not indexed is correct.
In these cases, you should intentionally use noindex tags or robots.txt rules rather than hoping Google just won't index them.
Taking Action: Your Next Steps
Here's how to tackle crawled but not indexed issues systematically:
- Identify Your Scenario: Log into Google Search Console and review the URLs listed as "crawled but not indexed." Group them by page type (products, blog posts, categories, etc.) to identify patterns.
- Prioritize Important Pages: Not every page needs immediate attention. Focus first on pages that were designed to rank and drive traffic.
- Apply the Relevant Solution: Based on the scenarios above, implement the specific fixes that match your situation.
- Monitor Progress: After making changes, track the affected URLs over the following weeks. Request re-indexing for your most important pages through Google Search Console.
- Prevent Future Issues: Put processes in place to avoid these problems going forward. This might mean content guidelines for writers, technical standards for developers, or regular audits of your site structure.
Final Thoughts
The "crawled but not indexed" status isn't always a crisis. Google crawls far more pages than it will ever index, and that's okay. The key is making sure your important pages (the ones that should drive traffic and conversions) get indexed successfully.
By understanding these common scenarios and applying the targeted solutions, you can dramatically improve your indexing rate. Focus on quality over quantity, maintain good technical hygiene, and regularly audit your content to keep your site healthy in Google's eyes.
Remember that indexing is just the first step. Once your pages are indexed, you still need to work on ranking them. But without indexing, you don't even have a chance to compete. Fix these issues first, then move on to the fun work of climbing the rankings.
Need a faster way to get your important pages indexed? IndexPro.app can help speed up the process and ensure your content gets the attention it deserves from Google.