Getting a model onto a Rockchip NPU is not hard. Getting a quantised model onto the NPU, at the accuracy you signed off, within the latency budget, is where projects lose weeks. These are the six places it usually happens.
Bestom engineering note — written by our own design and BSP engineers from work on Rockchip platforms. Platform parameters quoted here are taken from our published datasheets; see Core Boards & SoM. For translated community notes, see the Tech Notes index.
Model format conversion is a solved mechanical step. The schedule goes on everything around it. These six items are in the order we check them.
Every custom layer or unusual activation is a potential fallback to CPU. The check is not "does it convert", it is which sub-graphs fell back, and what that costs in latency. Run this on the first converted model, not on the last one — a fallback discovered at the end forces a model redesign.
Before it bites: list every operator in the model and mark each one as NPU-native or fallback. Do it on paper.
INT8 quantisation derives its scales from the calibration data. A calibration set drawn from a clean, well-lit dataset will produce scales that fail on the real deployment conditions. This is the single largest source of "it worked in the lab".
Before it bites: the calibration set must be sampled from the field distribution — including the hard cases — not from the training set's easy majority.
Quantisation always costs something. If the accuracy target was agreed as a float-model number, the quantised model will never "pass". The target has to be restated for the deployed precision, with an owner who can accept it.
Before it bites: agree the deployed-precision metric, the acceptance threshold and the sign-off owner before conversion work starts.
NCHW vs NHWC, RGB vs BGR, and the normalisation applied in-graph vs on the host. A channel-order mismatch produces a model that runs correctly and predicts nonsense — often with a plausible-looking confidence.
Before it bites: assert the input tensor's expected layout and colour order as an explicit test, with a known-good reference image and its expected output.
The NPU time is measured; the resize, colour conversion, normalisation, NMS and tracking that surround it are not. On small models the surrounding work frequently exceeds the inference itself.
Before it bites: budget the whole pipeline, end to end, on target hardware — not the inference call in isolation.
A dual-core NPU does not automatically make a single model twice as fast. Utilisation depends on how the model is partitioned and whether there is enough independent work to run concurrently.
Before it bites: decide whether the second core serves one model or a second stream, and measure it — do not assume the headline figure is available to a single graph.
| # | Step | Gate to pass before moving on |
|---|---|---|
| 1 | Freeze the model graph | Operator list written down and classified |
| 2 | Build the calibration set | Sampled from field conditions, hard cases included |
| 3 | Agree the deployed-precision metric | Threshold and sign-off owner named |
| 4 | Convert and quantise | Round-trip test: known input → expected output, on target |
| 5 | Measure the full pipeline | End-to-end latency on the real board, at the real resolution |
| 6 | Integrate pre/post-processing | Pipeline budget still met with everything running |
| 7 | Validate in the enclosure | Accuracy and latency hold at the product's thermal steady state |
Step 7 is the one teams skip. A model validated on a bench with the board at ambient can behave differently in a sealed enclosure at thermal steady state. If the product has a sustained-inference mode, that is the condition the acceptance test must run in.
For the mechanics of the conversion itself, see our annotated walkthroughs in the RKNN / NPU section — the MNIST RKNN quantisation walkthrough and the RK1808 host-side demo.
Model deployment is usually owned jointly: the client owns model and accuracy, we own the board, the BSP, the NPU runtime and the on-target measurements. The split matters more than the tooling.
RK3588 (M88) and RK3576 (M76) both carry a 6 TOPS NPU. The differences that matter for deployment are CPU class beside the NPU and thermal headroom — see the platform comparison.
The annotated MNIST RKNN walkthrough: build → PB → RKNN → INT8.
Read nextWhich of the two 6 TOPS parts fits your pipeline.
Read nextHost-side bring-up for a dedicated NPU accelerator.
Send us the product spec or the constraint list. We design and develop custom hardware on Rockchip platforms — and we will tell you if the platform you have in mind is the wrong one.
Start a project → All Tech Notes