Granular Sim2Sim

Technical Summary

Robotic excavation in granular media depends on material properties that are not directly visible from the initial surface. We study whether a short raw RGB material-probe sequence can be converted into an online material belief that improves subsequent excavation decisions. The controller uses the posterior mean to select from a finite MPM digging action budget. Across matched MPM conditions, estimated posterior control improves target loss over no posterior and wrong posterior, while remaining close to a GT-property finite-action reference. The current claim is intentionally bounded: this is controlled Sim2Sim evidence, not real-camera closed-loop robot excavation. A discussion draft is now available from the top buttons; it should be read as a working paper, not an archival claim.

Results Highlights

Posterior-Conditioned Excavation
            
Same bed, target, and action budget; only the controller belief changes.

Force-Dominant Stress Test
            
A DDBot-core-style target height-field task used only as a scoped supporting check.

Raw-RGB Posterior Control Pipeline

Raw RGB probe to online material posterior, selected action, MPM rollout, and target metrics

The main ablation uses raw RGB frames from a short material-probe interaction to update an online material posterior, then uses its mean to choose a finite-budget excavation motion. No-posterior, wrong-posterior, estimated-posterior, and GT-property beliefs are compared with the same initial bed, target, and action budget.

Task 1: Matched MPM Excavation

A representative rollout under four belief inputs.

Qualitative rollout figure with probe frames, posterior matches, height maps, force traces, and selected action

The evidence figure links posterior evidence, action choice, final height map, and force trace.

Belief input	Target loss	Completion	Force violation	Strict success	GT action
No posterior	2.198	1.19	446 N	2/24	0/24
Wrong posterior	2.629	0.59	2852 N	0/24	0/24
Estimated posterior	2.010	1.03	420 N	5/24	12/24
GT property reference	1.972	0.98	338 N	2/24	24/24

Task 2: Evidence Stack and Failure Modes

Evidence dashboard showing target loss, paired robustness, action changes, and strict success audit

Target loss, paired robustness, action changes, and metric-artifact audit.

Shuffled posterior failure audit showing under-digging artifact

Negative control: scalar loss can improve through conservative under-digging.

Policy regret and action-distance summary

Action-level audit: estimated posterior moves closer to the GT finite action.

Strict depth and force feasibility audit prevents hiding behind one scalar score.

Task 3: Supporting Checks

DDBot-core-style target height-field benchmark with force posterior replanning.

DDBot-core force-posterior stress test summary

A scoped stress test, not full official DDBot superiority.

Modality ablation summary under visually ambiguous inputs

When vision is ambiguous, raw wrench evidence improves the material posterior.

Real-camera soil image bridge examples and predictions

Static real-camera soil image bridge: supporting evidence only.

Evidence Boundary

Item	Evidence in this page	Not claimed
Control	24/64 matched MPM ablations	Real-robot excavation
Vision	Raw procedural RGB video tensors	Real-camera video transfer
Real pixels	Static soil RGB with particle-size labels	Closed-loop robot control
Force	Force/torque data and modality audit	Hidden wrench in the main RGB result
DDBot	Scoped DDBot-core stress test	Full official DDBot superiority

Reproduce

git clone https://github.com/rachy103/Granular_Sim2Sim.git
cd Granular_Sim2Sim
./install.sh --locked

python experiments/raw_rgb_posterior_excavation/run_rgb_posterior_mpm_benchmark.py
python experiments/raw_rgb_posterior_excavation/make_qualitative_rollout_figure.py
python experiments/ddbot_posterior_heightfield_mpc/run_posterior_ablation.py
python scripts/build_project_page_media.py