How to run an AI chatbot offline on your Android phone

16 September 2026 · about six minutes

You can run a language model on an Android phone with no connection at all. Install an app that carries one, download the model file once over wifi, then turn the connection off. The phone does the work itself, so your questions are never sent anywhere. The catch is size: a model small enough for a phone is far weaker than the ones behind the big cloud chatbots.

Can a phone really run a language model?

Yes, and it has been possible for a couple of years now. The phone in your pocket has more compute than a desktop from ten years ago, and the models have come down to meet it.

Two things made it work. The first is that small models got much better. A model with under a billion numbers in it used to be a toy; the ones published in the last year or so hold a real conversation. The second is quantisation, which is a way of storing each of those numbers in four bits instead of sixteen. It costs a little accuracy and cuts the file to a quarter of its size. A model with 0.8 billion numbers, stored at roughly four bits each, lands near 0.6 GB — a large download, but an ordinary one for a phone.

What you need

• Android 8.0 or newer
• Processor 64-bit ARM (almost anything since 2017)
• Memory 2 GB RAM for a small model, 3 GB for a larger one
• Free storage about 1 GB
• A connection once, for the download

No account, no subscription and no developer tools. If your phone can run a modern game it can run a small model, though an older phone will write its answers more slowly.

How to set it up

1. Install an app that runs the model locally

   The word to look for is on-device or offline. Plenty of apps called "AI chat" are just a thin shell around someone else's server, and those stop working the moment you lose signal. Lite Mind is the one I build, and it is free.

2. Download a model, once

   Expect a few hundred megabytes, so do it on wifi. This is the only step that needs the internet. The file is saved on the phone and reused from then on.

3. Turn the connection off and start chatting

   Aeroplane mode, the underground, a flight, a dead patch of countryside — none of it matters now. The model is on the phone.

How to check it is really offline

Do not take anyone's word for this, mine included. There is a test that takes a minute:

• Download a model and wait for it to finish.
• Switch the phone to aeroplane mode. Wifi and mobile data both off.
• Ask it something.

If a real answer comes back, the model is running on the phone. There is no other explanation. For a stricter check, open Settings → Network & internet → Data usage (the exact path moves around between Android versions) and look at the app's own figure. After the model download it should sit still no matter how much you chat.

What a small model is good at, and what it is not

This is where most articles on the subject oversell, so here it is plainly. The models that fit on a phone are around a thousandth the size of the ones behind the big cloud assistants. That gap is real and you will notice it.

They do well at:

• Everyday questions and explanations of common things
• Rewriting and shortening text you give them
• Drafting a message, a list or a short note
• Simple translation and summarising
• Being available with no signal, which the big ones are not

They are weak at:

• Anything needing deep or obscure knowledge — they invent it instead
• Long chains of reasoning, and arithmetic beyond the basics
• Current events, since a model only knows what it was trained on
• Very long documents, which a phone does not have the memory to hold

So treat it as a capable notebook rather than an oracle. Check anything that matters, and do not use it for medical, legal or money decisions.

Why the download is smaller than you would expect

Two tricks do most of the work, and they are worth knowing if you are choosing between apps.

Quantisation-aware training. Squeezing a model to four bits after it has been trained always costs some accuracy. Some publishers now train the model knowing it will be squeezed, so it holds up better at the same file size. Same download, better answers.

Speculative decoding. A second, much smaller model guesses the next few words and the real model checks the whole guess in one pass instead of writing word by word. When the guess is right, which it often is, the reply arrives noticeably sooner. The text comes out identical either way — the small model only proposes, it never decides. It costs about 0.2 GB of extra download.

Can it take dictation offline too?

Yes, and it is a separate model from the chat one. Speech-to-text that runs on the phone is well established now; a good small voice model is about 57 MB, a fraction of the chat model. Worth checking that an app does this on the device as well, because voice is the part most likely to be quietly sent to a server — it is the easier thing to run remotely, and plenty of apps do.

Other ways to do this

An app is the easy route, but not the only one. Most of these apps, mine included, are built on llama.cpp, an open project that runs language models on ordinary hardware. If you are comfortable at a command line you can install Termux and run llama.cpp directly, which gives you any model you like and a lot more fiddling. The trade-offs are the same whichever route you take: a bigger model gives better answers, needs more memory, and writes more slowly.

Is it actually private?

If the model runs on the phone, the conversation has nowhere to go. That is a stronger guarantee than a privacy policy, because it does not depend on anyone keeping a promise.

But "runs on the phone" and "sends nothing" are not automatically the same thing. An app can run the model locally and still ship analytics, crash reports or an ad SDK. The aeroplane mode test above covers the conversation itself; the data usage screen covers the rest. For what Lite Mind does, which is nothing, there is the privacy policy.

Worth it?

If you want the strongest possible answer to a hard question, no — use a cloud assistant. If you want something that works on a plane, keeps your questions on your own phone, costs nothing and asks you for no account, then a small local model earns its 0.6 GB. I built one because I wanted it on my own phone, and I kept it because I use it.