Sites & pages

Navigate to Dashboard → Sites and click Add Site. Enter a name and your website’s domain (e.g. docs.example.com).

Once created, the site appears in your list. You can add multiple sites, and each site groups its own set of indexed pages.

The fastest way to index your content. Click Import Sitemap on a site, provide your sitemap URL (e.g. https://example.com/sitemap.xml), then click Fetch pages.

Kanha parses both <urlset> sitemaps (direct page URLs) and <sitemapindex> files (nested sitemaps), recursing up to three levels deep. Nothing is crawled yet at this point. You get the full list of pages it found, so you can choose what to import.

Filter the list by path to keep only the section you care about, for example /docs or /blog. If your site is published in several languages, Kanha groups the pages by language and shows a chip per language with its page count. Click a chip to include or exclude that language in one go, so an English bot is not trained on your German pages. Select all or none applies to whatever the filter is currently showing.

The count line shows how many pages you selected, and for billing admins how many your plan has left this month. When you are happy with the selection, click Import and only those pages are queued for crawling.

No sitemap, or a sitemap that returns 404? Switch the import panel to Paste URLs and paste your page links, one per line. Kanha keeps the https links on your site’s domain, drops blanks and duplicates, and hands the rest to the same selection step.

Quick Start and the site preview on the Try page use the same step. Load the sitemap, confirm the pages you want, and only those pages are read. Nothing is picked for you from a long sitemap.

Don’t have a sitemap? Click Add Page and paste a URL. The page is queued for crawling and will appear in your page list once indexed.

This is useful for adding specific pages that aren’t in your sitemap, or for testing the crawler on a single page before running a full sitemap crawl.

Kanha auto-detects pages that require JavaScript rendering. If the static HTML yields less than 500 characters of content, or the text-to-HTML ratio is below 2%, the crawler automatically retries with a headless browser.

Pages rendered via JS are marked with a JS badge in your page list. You can also manually toggle a page’s render mode if auto-detection doesn’t catch it.

Click any page in your list to preview the extracted content. This shows exactly what the bot will learn from: the cleaned text content, not the raw HTML.

You can also download the content as a text file, or delete pages you don’t want included in training.

Content changes? Recrawl a page to update the index. Both new crawls and recrawls count against your page scrape quota (Free: 50 total, Starter: 500/mo, Pro: 2,500/mo, Business: 10,000/mo). If you expect to need more, contact sales to confirm capacity and pricing.

When a page is recrawled, its QA pairs are automatically regenerated from the new content.

Delete individual pages from the page list. Deleting a page removes it from the index, so it won’t be included in future dataset generation or training.

You can also delete an entire site from the sites list. This removes all of its indexed pages as well.