Gemini on Android works great. It can answer questions, read your screen, and even help you get more out of features you didn’t know existed. But it needs the internet to do any of that. What if you’re on a flight, in a remote area, or just want to keep your conversations private and off someone else’s servers? A local small language model can handle that, and that’s exactly what I tried.

I downloaded PocketPal, loaded an offline small language model, and tested it against Gemini to see how far it could go. For the most part, it did well at quick lookups and basic tasks that didn’t require internet, though it was noticeably slower.

PocketPal brings local AI models to your phone

An open-source app that runs language models without the cloud

Android has multiple ways to run local AI models. MLC Chat and Google AI Edge Gallery are two popular options, but PocketPal makes it the easiest to get started with local AI on your phone. It’s an open-source app for Android and iOS, and it doesn’t need the internet except to download the models initially.

Once a model is on your phone, everything runs locally. Chats stay on your device along with your full chat history, and you can edit messages and retry responses if the output isn’t what you wanted. There’s a voice assistant option as well, but that’s a paid feature, and I’ll get to that later.

PocketPal also has a feature called Pals, which are saved personas. Think of them as system prompts bundled with a specific model and extra settings, so you don’t have to retype instructions every time you start a new chat.

Using PocketPal on Android

How to install, download models, and get chatting

You can start by downloading the app from the Google Play Store. After a short walkthrough, tap Download Model. By default, it shows three models and recommends Balanced, which is Gemma 3 1B at around 1,020MB. You can also pick a smaller model like LFM2.3.1 2B at 700MB or go for the best option, Qwen3.5 3B, at 2.5GB.

I started with the recommended Gemma 3 1B, as it seemed likely to work fine on most smartphones. For reference, I tested these on my Galaxy Z Flip 6 with 12GB of RAM and a Note 20 Ultra with similar memory but an older processor. If your phone has more RAM, feel free to go for the bigger models.

After the download finishes, you need to load the model from the chat menu. Tap the up arrow icon, open the Model tab, and choose your model. You can also do this from the three-dot menu > Models or the Models tab in the hamburger menu.

Chatting with PocketPal

How the local models handled eight basic prompts

Chatting with PocketPal feels like any other chatbot, whether that’s Gemini, Claude, or ChatGPT, except the responses come in slower. That’s not surprising when the processing happens entirely on your phone’s hardware instead of a data center.

I ran a basic test with three models: Gemma 3 1B (1.07GB), Gemma 3 4B (2.36GB), and Gemma 4 E2B (4.03GB). Each model got the same eight prompts in a fresh chat with default settings. The prompts covered general knowledge, math, logic, summarizing, coding, and a deliberate trick question to test for hallucination.

Here’s how they scored:

All three models nailed the basics. They correctly identified Canberra as Australia’s capital, calculated a 25% discount on 800 rupees, solved a transitive logic puzzle, and wrote a working Python function. They even counted the letter “r” in “strawberry” correctly, which is a prompt that trips up some cloud models.

The one prompt that tripped all three was a trap: I asked them to summarize a speech Albert Einstein gave to Congress in 1987. Einstein died in 1955, so no such speech exists. Every model confidently wrote a plausible-sounding summary instead of flagging the false premise. That’s the main reliability risk with local models, and bigger model size didn’t help.

Speed-wise, the 1B model was the fastest and most responsive, which makes it a good default for quick questions. The E2B model uses a “Think” step before answering, which made some short prompts take 30 to 40 seconds despite faster raw token speeds. For casual chat, that’s a real tradeoff.

Customizing PocketPal

Pals, generation settings, and fine-tuning how the model responds

Moving on, we have Pals. A Pal is a saved bundle of a model, a system prompt, and a name, so you don’t have to retype instructions for every chat. By default, PocketPal comes with two pre-installed Pals. First is Pip, a general-purpose assistant that runs on Gemma 3 1B, ideal for short prompts. Second is Lookie, a real-time video assistant that describes your camera feed using SmolVLM-500M, grabbing a frame every 3,000ms.

You can discover more Pals created by the community from palshub.ai, though you need to log in to save them to your own library.

In the settings, you can configure context size, advanced model options, and memory management. There’s even an option to let PocketPal perform an internet search, though it’s turned off by default.

To the right of the chat, tap the three-dot menu > Generation settings. This is where you control how the model writes its reply. You can toggle whether a reasoning model’s thinking text stays in the history, adjust Temperature, Top K, Top P, and Min P to control how much the AI sticks to safe word choices or takes more creative risks, tweak repetition penalties to help it break out of loops, and set N Predict to cap the reply length so a rambling model doesn’t fill your entire context window with irrelevant text.

PocketPal is a solid offline companion, but it’s not replacing Gemini

PocketPal also has a new voice conversation feature that lets you talk hands-free. It’s a Plus-only feature, though, so you’d have to spend at least $5 to unlock it. That said, while optional, it’s a useful addition if you find yourself relying on PocketPal regularly.

Now, PocketPal isn’t a direct rival to Gemini. Gemini can do far more than just answer text prompts: it reads your screen, works across Google apps, automates multi-step tasks, and pulls from real-time internet data. PocketPal can’t do any of that. What it can do is give you a private AI assistant that works without the cloud, runs on hardware you already own, and doesn’t cost a monthly subscription. Also, you are not training an LLM with your data.