Running neural networks on mobile without a server

Running neural networks directly on a mobile device, without a server, sounds straightforward until you actually try it.

I spent a lot of time developing an app on React Native that identifies plant species, detects diseases, and evaluates the age of plants and the number of leaves — all on the device, with just one photo. No backend, no API calls.

Technologies

The technical side turned out to be the most interesting. The source code of the solution is open on GitHub. A 1GB disease detection model based on the Swin Transformer had to be stripped of its text encoder and converted to ONNX, then quantized to float16, bringing it down to 231MB. A 248MB embedding file cannot simply be loaded into JavaScript memory on Android without crashing the app, so I wrote a native Kotlin module that memory-maps the file and runs vector similarity search entirely in native memory, passing only results back to JavaScript. Image preprocessing as decoding, resizing, converting to CHW tensors, also moved to native code for the same reason. The JavaScript layer only ever touches preprocessed data and inference output.

I write articles about the technical solutions underlying my projects, and you can read more technical material about how the project is structured on dev.to.