Why pages don't get into Yandex's index, and what to do about it

A situation visible in almost every audit: the site works, pages open in a browser, and search doesn't have them. The causes are almost always technical, and almost never solved by "buying links".

1. Is the page reachable

• The page must return 200 without authentication and without regional restrictions.

• No meta robots noindex — people often forget to remove it after the staging period.

• robots.txt blocks neither the page itself nor the resources needed to render it.

2. A canonical that says "don't index me"

The most underrated cause. If a page points its canonical at the homepage, the search engine takes it literally: "I am a duplicate of the homepage." Such a page won't enter the index no matter how often it's recrawled.

For multilingual sites specifically: language versions must not point their canonical at the Russian version. These are separate documents, each with a canonical to itself, and the relationship between them is expressed through hreflang.

3. A JavaScript-rendered site

If the page is assembled in the browser, the source HTML may contain nothing but an empty container. The reliable options are server-side rendering or prerendering, where text, meta tags and structured data arrive ready-made. API errors, blocked resources and endless loading all hurt indexing, even with a perfect sitemap.

4. robots.txt and the sitemap don't remove pages from the index

A common misconception. These files affect crawling but don't remove URLs the search engine already knows. For a page to drop out of the index you need either noindex on a crawlable page, a 404/410, or the removal tool in the webmaster console.

5. What to do in practice

1. Verify ownership in Yandex Webmaster and upload a current sitemap.

2. Check a specific URL through the page inspection tool: what the crawler sees in the HTML.

3. Use IndexNow to report new and changed URLs without waiting for a scheduled crawl.

4. Test a sample of different page types — homepage, service, article, language version — rather than just one.

What not to do

Changing canonicals and content daily to "shake the crawler awake". It has the opposite effect: a search engine needs to see a stable picture across several crawl cycles before deciding to index.

If pages aren't being indexed and the cause isn't obvious, we can go through the specific site and show where the chain breaks.