You want to know whether on-device AI will feel fast enough on your website. There is no honest speed number that answers that for every visitor.

The useful answer is simple: test the exact model on the devices your audience uses. Measure the first visit and a return visit separately. Then choose the smallest model that still gives good answers.

This guide shows what to measure so you can make that decision quickly.

Measure the wait a visitor feels

A single "tokens per second" result misses most of the experience. A token is a small piece of text used by the model. Its generation rate matters, but the visitor also waits for files to download and for the browser to prepare the model.

Record these numbers separately:

Measure	What it answers
Download size and time	How long a first visit waits for model files on that connection
Preparation time	How long the browser takes to make the model ready
Time to first text	How long the visitor waits after asking before the answer begins
Text generation rate	How quickly the rest of the answer appears
Memory use	Whether the model fits comfortably on the device
Repeated-use result	Whether longer chats cause slowdowns or failures

An accurate short answer can feel better than a fast, wandering one. Review answer quality alongside every speed result.

Test the exact model you will publish

"Small model" is not enough information to repeat a test. Record the model name, version, selected size, file format, and chat length. If the model files use compression, record that setting too because it can change file size, memory use, and speed.

Long conversations also take more work than one short question. Use the same questions and chat history when comparing devices or model versions.

Kanha's public install names one model size. It does not read the visitor's memory and automatically choose Small, Medium, or Large. Smaller sizes usually need fewer resources, but you should train and test the size you intend to publish.

Use your visitors' real devices

Two devices from the same year can have different processors, graphics support, memory, drivers, battery settings, and heat limits.

Record the exact device, operating system, and power state. Note whether it is plugged in, in low-power mode, or already busy with other work.

Do not treat one fast laptop as proof for every desktop or phone. A model that feels good on a development machine may run slowly or fail to start elsewhere.

Record the browser version too

The browser's name is only a starting point. Version, graphics support, drivers, and the browser code that runs the model can all change the result.

Save the browser version and Kanha chat-code version with each test. Repeat important tests after a browser or model update instead of assuming an old number still applies.

Kanha takes responsibility for compatibility work when a particular environment needs a different path. That does not mean every device and browser already performs alike. Record a failure to start separately from a slow answer so the problem is clear.

Run one cold test and one warm test

A cold test starts without stored model files. It shows the download, preparation, and first usable answer. Run it on a realistic connection, including a slower mobile connection if that matters to your audience.

A warm test starts with model files already available in the browser. It shows what a return visit may feel like. Browser storage is not permanent. Files can be removed, private browsing may behave differently, and a new model version may require another download.

A successful warm test is not an offline guarantee. The page, chatbot code, model files, or other site resources may still need the network.

Check the first text and the full answer

Time to first text is the delay between sending a question to a ready model and seeing the answer begin. It can grow with the model size, question length, and conversation history.

Run the same question several times and report the usual range, not the best result.

Then measure how quickly the rest of the answer appears. If you use tokens per second, define tokens as small pieces of text and keep the answer length and settings the same in every comparison. Very short answers can make the result noisy.

Keep the test running

A model may start well and struggle later in a conversation. Test repeated questions, growing chat history, and answers close to the length visitors are likely to request.

Watch for severe slowdowns, browser-tab crashes, failures to get more memory, and changes after the device gets warm. Mobile devices may reduce performance to control heat and battery use.

Do not turn one device's heat reading into a broad claim. Record the test length, question sequence, device state, and what actually happened.

Use this repeatable test

Run the same steps on every target device:

Record the device, operating system, browser version, power state, and Kanha chat-code version.
Record the exact model, version, selected size, and chat-length settings.
Clear the site's stored data. Record download size, download time, preparation time, and any failure.
Ask the same short and long questions. Record time to first text, generation speed, answer length, and memory problems.
Repeat with the model files already stored, then repeat with a longer conversation.
Continue long enough to catch slowdowns that appear after several answers.
Read the answers and mark factual mistakes as well as timing results.

Run the test more than once. Report the usual result and the variation. If you trained more than one size, test each one instead of assuming the browser will choose for you.

Make the launch choice

Run your own timings on the device and browser you want to support. They are more useful than a table built from someone else's hardware, and a bot trained on your content can have different files and behavior.

Choose the smallest model that passes your answer-quality checks. Test cold and warm visits. Keep a list of environments that need compatibility work so Kanha can address them.

When you are ready to install the result, follow the website chatbot setup guide. For the full network path, read the on-device AI privacy guide.

On-Device AI Speed: What Makes Browser Chatbots Fast or Slow