Preprint · Under Review · 2026

Scene graphs that store evidence, not decisions.

OpenBelief-Nav is a training-free open-vocabulary scene graph for language-guided robot navigation. Each object node preserves its multi-view phrases, reliability weights, and exact provenance instead of a single committed label — so segmentation, retrieval, and execution-time correction all become task-time readouts over one frozen graph.

0.2742 +71%
ScanNet200 mIoU vs. DualMap 0.1604
76.9% +6.4
SR@1m on HM3D + YCB vs. DualMap 70.5%
1 graph
built once — every task reads, none rewrites
0 training
frozen VLM + segmenter + CLIP only
building
living room
kitchen
bedroom
office
drag to orbit · scroll to zoom · click an object for its evidence
The interactive scene graph couldn't start.
See the pipeline figure below for the full picture.
try a query →
The problem

Maps that make up their mind too early can't take it back

A sofa seen from four viewpoints may be proposed as sofa, chair, bench, or armchair — occlusion, scale, and foundation-model noise guarantee disagreement. ConceptGraphs, HOV-SG, FSR-VLN, and DualMap all resolve that disagreement at construction time, exposing each object as one fused feature, label, or consensus state. Related approaches push on adjacent axes without changing this pattern: ConceptFusion and VLMaps embed features densely rather than per object, and OGScene3D and ThinkGraphs refine tracking and confidence while still exposing a single committed state per object. Three things go wrong.

Commitment destroys evidence

When a majority vote picks one label, the minority phrases are gone. Yet among nodes misclassified by majority vote, the correct category is often still present in the stored phrase set — information the system already paid to acquire, discarded by how it is stored.

store observations, not verdicts

No single readout fits every task

The readout that maximizes benchmark segmentation is not the one that maximizes navigation success — our experiments show full-belief projection wins segmentation while a consensus readout wins retrieval. A map that commits to either loses the other.

choose the readout per task

A target is a hypothesis, not a fact

The top-ranked node can be a distractor, or an instance the arrival observation can't confirm. Retrieval alone can't tell the difference — the robot has to verify the selected binding physically before treating it as ground truth.

verify at arrival, not just at retrieval
Method

Four stages, each pushing commitment later

An object node stores the evidence-preserving state \(\mathcal{B}_i = (P_i, \mathbf{f}_i^v, \mathcal{O}_i, \mathbf{V}_i, \mathbf{p}_i, \mathbf{f}_i^t, \rho_i)\) — point cloud, aggregate visual feature, the raw language observations, concept-level evidence with its smoothed belief, a consensus text feature, and a trust status. The visual and language channels are never destructively fused during mapping; every cross-modal decision is deferred to a readout.

STAGE 1

Evidence acquisition

A frozen VLM proposes phrases; a promptable segmenter grounds each phrase as masks. Every mask is kept as an observation with its phrase, confidence, 3D point cloud, visual feature, and exact frame + mask provenance.

\(o_e = (t_e, j_e, \ell_e, c_e, n_e, P_e, f_e^v)\)
STAGE 2

Geometric–visual association

Observations merge on 3D overlap, with visual similarity only as a rejection cue — never on predicted names. Precision is favored: a false merge is irreversible, a missed one is recoverable at query time.

\(o_e \sim o_{e'} \iff \Omega_r(P_e, P_{e'}) \ge \tau_g \land \langle f_e^v, f_{e'}^v \rangle \ge \tau_v\)
STAGE 3

Belief construction

Observations are weighted by what we actually know — segmentation confidence and 3D support — grouped into synonym-aware concepts \(V_{ik}\), and Dirichlet-smoothed so sparse evidence can't fake certainty.

\(w_e = c_e \cdot \sqrt{n_e} \qquad V_{ik} = \sum_{o_e \in \mathcal{O}_i:\,\ell_e \in \mathcal{C}_{ik}} w_e \qquad p_{ik} = \frac{V_{ik} + \alpha}{\sum_{j=1}^{K_i} V_{ij} + \alpha K_i}\)
STAGE 4

Task-time readouts

Vocabulary projection and free-form retrieval are separate operators over the same frozen evidence — and at deployment, arrival observations physically test the selected hypothesis.

\(\text{readout}(\mathcal{B}_i, \text{task})\) — graph never rewritten
Overview of OpenBelief-Nav
Overview of OpenBelief-Nav. Mapping retains observation-level phrases, reliability cues, frame–mask provenance, and separate aggregate geometric and visual representations. Task-time readouts perform fixed-vocabulary projection or free-form retrieval, while arrival verification enables query-specific candidate correction without modifying the persistent graph.
Interactive

Watch a belief being built — and vote splitting break it

Five views of the same sofa arrive with different phrases. Play the stream, then toggle synonym consolidation off: the lexical variants sofa / couch / loveseat split their votes and a single high-support chair observation wins. Drag α to see Dirichlet smoothing keep sparse evidence from collapsing into false certainty.

Observation stream \(\mathcal{O}_i\)
weight \(w_e = c_e \cdot \sqrt{n_e}\) — confidence \(\times\) spatial support
Concept belief \(\mathbf{p}_i\)
accumulated at the concept level, Dirichlet-smoothed
synonym consolidation
\(\alpha\) 0.8
Press play to stream observations into the node.
One graph, many answers

“What is this object?” depends on who's asking

Because a node stores evidence rather than a decision, the same frozen graph answers differently — and correctly — for each task.

Vocabulary projection

The benchmark vocabulary arrives only at evaluation time. The stored belief \(p_{ik}\) is visually refined into \(\widetilde{p}_{ik}\) within its existing support, then projected onto target class distribution \(\widehat{p}_i(c)\) — secondary hypotheses an early top-1 would have discarded remain recoverable.

\(\widetilde{p}_{ik} = \frac{p_{ik}^{\gamma} \exp(\langle g_{ik}, f_i^v \rangle / T_{\mathrm{vis}})}{\sum_{j=1}^{K_i} p_{ij}^{\gamma} \exp(\langle g_{ij}, f_i^v \rangle / T_{\mathrm{vis}})} \qquad \widehat{p}_i(c) = (1 - \lambda_{\mathrm{vis}}) \sum_{k=1}^{K_i} \widetilde{p}_{ik} W_{ikc} + \lambda_{\mathrm{vis}} v_i(c)\)

Free-form retrieval

Query similarity fuses the visual and language heads \(H_i^{(r)}(q)\) — but the fusion weight \(\lambda_i\) is per-node, conditioned on the trust status \(\rho_i\) produced during belief construction:

no language evidence (\(K_i=0\)) trust vision only \(\lambda_i = 1.0\)
degraded belief (\(\rho_i = \text{degraded}\)) lean visual \(\lambda_i = 0.7\)
trusted belief (\(\rho_i = \text{trusted}\)) trust the text head \(\lambda_i = 0.2\)
\(S_i^{(r)}(q) = \lambda_i \langle f_i^v, f_q \rangle + (1 - \lambda_i) H_i^{(r)}(q) \qquad \text{where } H_i^{(\mathrm{con})}(q) = \langle f_i^t, f_q \rangle\)
Semantic segmentation · ScanNet200
0.2742
best readout: full-belief projection
consensus-only reaches 0.2107 · visual-only 0.1286 · +71% over DualMap
76.9%
best readout: consensus over the same graph
belief-weighted reaches 74.4% · +6.4 pts over DualMap
No single committed readout is optimal for both tasks — the evidence-preserving graph is, because the readout is chosen per task over the same preserved evidence.
Deployment

Navigation is a physical test of the hypothesis

A retrieval result is a hypothesis, not a fact. On arrival, the robot's RGB-D observation runs an ordered verifier — re-observe if the candidate isn't sufficiently visible (absence of view is not evidence of absence), reject on semantic mismatch or on a semantic match that fails to coincide geometrically with the stored node, confirm otherwise. Rejection changes exactly one thing: a per-query exclusion set \(\mathcal{R}_q^{(k+1)} = \mathcal{R}_q^{(k)} \cup \{i^{(k)}\}\). The graph is never modified and the query is never rewritten.

LIVING ROOM KITCHEN BEDROOM OFFICE n17 · “mug” 0.41 n09 · “mug” 0.38
q: “bring me the coffee mug” 🔒 never rewritten
STEP 0 / 5
Two hypotheses in the map
Results

Same graph, strongest result on both benchmarks

Graph construction is performed once per environment; every number below is a readout over that frozen graph, with no re-perception between tasks.

Open-vocabulary 3D semantic segmentation

ScanNet200 (5 scenes) and Replica (8 scenes). Higher is better; bold = best per dataset.

Method mIoU ↑ F-mIoU ↑ mAcc ↑
ScanNet200
ConceptGraphs 0.0882 0.3077 0.3538
HOV-SG 0.1333 0.3381 0.3714
DualMap 0.1604 0.3288 0.3794
OpenBelief-Nav consensus 0.2107 0.4887 0.4580
OpenBelief-Nav full-belief 0.2742 0.5276 0.4685
Replica
ConceptGraphs 0.1501 0.3858 0.3559
HOV-SG 0.2050 0.4846 0.3835
DualMap 0.2538 0.5207 0.4024
OpenBelief-Nav consensus 0.2831 0.4647 0.4774
OpenBelief-Nav full-belief 0.2912 0.5204 0.4822

Language-guided navigation — Habitat-Sim, HM3D + inserted YCB objects

SR@1m (%), 78 trials per method following the DualMap protocol. Internal readouts share one graph.

Method 00829 00848 00880 Avg. SR@1m ↑
ConceptGraphs 69.2 53.8 61.5 61.5
HOV-SG 53.8 46.2 57.7 52.6
DualMap 73.1 69.2 69.2 70.5
OpenBelief-Nav consensus 80.8 76.9 73.1 76.9
OpenBelief-Nav early-commit 84.6 73.1 73.1 76.9
OpenBelief-Nav belief-weighted 76.9 73.1 73.1 74.4

Semantic readout ablation

Same object geometry and observations, scored by four different readouts. Higher is better; bold = best per dataset.

Semantic readout mIoU ↑
ScanNet200
Visual-only 0.1286
Consensus-only 0.2107
Early-commit 0.2393
Full-belief projection 0.2742
Replica
Visual-only 0.2287
Consensus-only 0.2831
Early-commit 0.2701
Full-belief projection 0.2912

Component ablation

Each row disables one mechanism of the belief construction.

Variant ScanNet200 mIoU Nav SR@1m YCB SR@1m
\(w_e = 1\) (no reliability) 0.2010 70.5 55.6
\(w_e = c_e \cdot n_e\) (linear support) 0.2117 74.4 50.0
no synonym consolidation 0.2067 75.6 61.1
no low-evidence regularization 0.2095 70.5 50.0
Default belief construction 0.2107 76.9 61.1

Positioning

“Evidence @ query” = per-observation evidence exposed to task-time readouts.

System Evidence @ query Exec. correction
ConceptGraphs
HOV-SG
FSR-VLN
DualMap
OpenBelief-Nav
Real world

Deployed on a Unitree G1 humanoid

Compute-heavy semantic reasoning runs offboard; an onboard Jetson keeps the motion loop local. The bridge is Zenoh. The robot navigates to a candidate, looks, and lets physics vote.

Embodiment

Unitree G1 + RealSense D435i

Online RGB-D + LiDAR for perception, target grounding, and arrival verification.

Offboard reasoning

RTX 5090 workstation

Semantic mapping, open-vocabulary retrieval, and language-conditioned reasoning.

Onboard navigation

Jetson AGX Orin

ROS 2 Nav2 + CMU navigation stack, interfacing the low-level controller.

Bridge

Zenoh

Decoupled comms: heavy reasoning offboard, motion planning and control onboard.

CORRECT

Valid mapped candidate

The verifier must confirm without a false rejection — correction should cost nothing when the map is right.

CONFUSABLE

Same-category distractor

A look-alike outranks the true target. Arrival evidence rejects it; the query re-grounds to the supported alternative.

Matched real-world target-confirmation results

20 Unitree G1 runs, 10 matched cases (same graph, query, and start pose for both execution modes).

Condition Runs per method Top-1-only SR Full-correction SR Full-correction mean visits
Correct 5 5/5 (100%) 5/5 (100%) 1.00
Confusable 5 1/5 (20%) 3/5 (60%) 1.80
Overall 10 6/10 (60%) 8/10 (80%) 1.40
Unitree G1 platform and hardware configuration Execution-grounded candidate correction on the humanoid
Real-world platform and deployment procedure. Left: the Unitree G1 with RealSense D435i and Livox MID-360. Right: execution-grounded candidate correction — arrival observations confirm, re-observe, or reject the selected candidate, and rejection re-grounds the unchanged query over the remaining mapped candidates without modifying the persistent graph.
Citation

BibTeX

@preprint{2026openbeliefnav,
  title   = {OpenBelief-Nav: Evidence-Preserving Open-Vocabulary Scene Graphs
             for Reliable Language-Guided Robot Navigation},
  
  author  = {anoynimous},
  year    = {2026},
  note    = {Under review}
}