Modern edge System-on-Chips (SoCs) integrate increasingly heterogeneous processing units (PUs) such as CPUs, GPUs, and NPUs, yet current inference stacks still map entire models to a single PU, leaving significant performance and energy efficiency on the table. This limitation is exacerbated by emerging neural architectures such as state-space models (SSMs), Kolmogorov-Arnold networks (KANs), and multi-stage vision-language-action (VLA) pipelines, whose diverse operator characteristics are not uniformly suited to any single PU. We present BIDENT, a unified operator-level orchestration framework for heterogeneous edge inference that maps individual operators to the most suitable PU based on profiled execution characteristics. BIDENT formulates operator-to-PU assignment as a shortest-path problem over a weighted execution graph, enabling efficient and principled scheduling under the profiled cost model for both latency and energy minimization objectives. Unlike prior work that relies on model-specific heuristics or coarse-grained partitioning, BIDENT provides a model-agnostic framework that jointly supports sequential execution, intra-model parallelism across independent operators, and concurrent co-scheduling of two inference requests within a single formulation. Evaluated on an Intel Core Ultra SoC across ten model families and 190 concurrent model pairs, BIDENT achieves up to 1.60x single-model speedup, reduces energy by 9.2% on average, and delivers a 2.28x geometric-mean concurrent speedup over a naive multi-PU placement baseline.

翻译：暂无翻译