Ask Scott Fox: How to Get Google to Index All My Web Site Pages?

Dear Scott Fox,
My new web site has been indexed by Google but I’ve noticed that some of my most important pages are nowhere to be seen (mainly the main categories of my site). How do I get the search engines to recognize a particular page and fill in these gaps in my site’s search engine results?
Thanks,
Billy
Click Millionaires Member
County Durham, UK

Scott Fox's Answer

Hi Billy,
The search engines will index any pages that they find. But they may only choose to display results from those that they deem relevant. While all of your site's pages seem relevant to you, the search engines use complex (and secret) formulas for determining “relevancy” in their results.

If you are seeing gaps in search engine coverage of your site, that's likely due to 2 factors:

• Lack of links means Google can't find the pages: Search engine spiders only crawl pages to which they have easy access. So if they are missing some of your pages, it may be because those pages are not linked to enough from other pages, both on your site and from 3rd party sites. This doesn’t sound like the problem if it’s your category pages that are missing (because they are prominent in your linking and file structure) but it’s worth reviewing to be certain.
• Duplicate content: Search engines don’t like to display multiple versions of the same page in their results. So if your category pages are missing from the results, it may be because they contain much of the same info as the product pages to which they link. Check both the visible text copy and also the page headers, titles, and meta tags to see if there might be damaging overlap.

SEO Solution?
The best way to get any particular pages highlighted in the search results is to have lots of links to it. That will “tell” Google and the other search engines that that those web site pages are "important" to the content of your site.

Since it sounds like Google has already indexed most of your pages, I would check the points I list above but then move on. It’s time to stop tweaking the site and instead start working “off” the site using social media and online marketing strategies to spread the word about your new e-business. Get other site owners and directories to start linking to your site. Get the blogs talking about your site (creating more links) and generally start marketing.

SEO is only part of the picture. 
Once you have the basics of SEO in place, you should work on spreading your product presence through a distributed engagement strategy that expands your company's reputation cloud across the web to attract more customers.

There are many other tools available to you in the modern online marketing toolbox. It's time for you to use them!

Best,
Scott Fox