Federal Capabilities Briefing

Mission Problem

Autonomous vehicle perception systems can unexpectedly fail to detect objects in edge cases—rare, randomly distributed scenarios that are difficult to cost-effectively incorporate into model validation. Mission context: a scalable, comprehensive method was needed to systematically uncover vulnerabilities before field deployment to ensure safety criteria are met.

Technical Approach

• Fine-tuned Stable Diffusion XL using LoRA on KITTI dataset to generate photorealistic road scenes
• Trained LLaMA-11B-Instruct with Direct Preference Optimization (DPO) to produce optimized prompts
• Applied Reinforcement Learning with Human-in-loop Feedback (RLHF) to generate adversarial prompts that induce visually complex edge cases
• Targeted YOLOv8 object detection weaknesses including occlusion, visual ambiguity, and scene artifacts

Validation / Optimization Method

Quantitative evaluation using detection algorithm metrics. RLHF pipeline with seed images achieved 5.7× improvement in iterations needed for edge case generation compared to non-seeded approaches. Generated images successfully induced missed detections in baseline perception models under adverse conditions.

Deliverables

• Validated synthetic dataset for AV perception testing
• Fine-tuned generative models (SDXL + LLaMA) for domain-specific edge case generation
• Documented methodology for repeatable edge case discovery
• Benchmark results against YOLOv8 baseline

Measurable Outcomes / Acceptance Criteria

• Mean iterations for edge case generation: 3.47 ± 1.47 (RLHF with seed image)
• 22% improvement in edge case discovery efficiency using combined DPO/RLHF approach
• Successfully demonstrated missed detections in controlled adversarial scenarios
• Methodology validated for scalable, cost-effective perception system testing

Mission Problem

Large-scale deployment of mission-critical systems (e.g., carbon capture, desalination) requires balancing economies of scale with economies of numbers. Traditional unique-design approaches are expensive and lead to long deployment timelines. Program need: an optimization framework to minimize total lifecycle costs while achieving manufacturing standardization.

Technical Approach

• Developed Mixed-Integer Linear Programming (MILP) formulation for simultaneous platform and process family design
• Incorporated economies of numbers learning curves with smooth discount functions
• Optimized both the number and characteristics of shared unit module designs
• Applied formulation to MEA-based carbon capture systems simulated in Aspen Plus

Validation / Optimization Method

Optimization solved using Gurobi with Pyomo algebraic modeling. Formulation automatically determined optimal platform size (4 absorber designs, 4 regenerator designs out of available candidates) without pre-specification. Results compared against traditional individual variant optimization.

Deliverables

• Optimization formulation capturing economies of numbers trade-offs
• Platform design specifications for standardized unit modules
• Process family assignment matrix mapping variants to shared components
• Techno-economic analysis documentation

Measurable Outcomes / Acceptance Criteria

• Total annualized cost savings: >$2M compared to individual optimization
• 26.8% reduction in capital costs attributed to manufacturing standardization
• Solved in <1 second (1,376 constraints, 646 continuous variables, 896 binary variables)
• Methodology validated for process family design with explicit economies of numbers

Mission Problem

Autonomous/teleop land rovers require reliable real-time tree detection with actionable geometry (position, range) for navigation, safety zones, and terrain analysis. Camera-only detection lacks metric distance; LiDAR provides depth but limited semantics. The system must run on resource-constrained Jetson Nano edge hardware in field conditions.

Technical Approach

• Integrated PercepTreeV1 (Detectron2/Mask R-CNN) for tree detection with optional diameter estimation
• Implemented camera-to-LiDAR sensor fusion using extrinsic calibration for 3D localization
• Deployed YOLO-LiDAR-Fusion pipeline with IoU-based evaluation and tracking modes
• Optimized for Jetson Nano 5W/10W modes with TensorRT/FP16/INT8 precision options

Validation / Optimization Method

Four-bucket metric framework: (A) Detection accuracy—AP50 box/mask, precision/recall, false positives/minute; (B) Real-time performance—FPS, latency P50/P95, dropped frame rate; (C) Robustness—performance splits by lighting, motion, occlusion, weather; (D) Resource usage—CPU/GPU utilization, memory, power, thermals.

Deliverables

• Real-time ROS topics: /tree_detections_2d, /tree_detections_3d, /tree_tracks
• Field-operational metrics logging (JSONL/Parquet)
• Calibrated sensor fusion pipeline with association quality flags
• Performance benchmark documentation across operating conditions

Measurable Outcomes / Acceptance Criteria

• Acceptance: Sustained FPS ≥ target for continuous operation; End-to-end latency P95 ≤ threshold
• Safety-relevant: Recall ≥ R% for trees within [d_min, d_max] meters; False positives ≤ F/minute
• Metric distance: Association rate ≥ A%; Range MAE ≤ E meters on validation set
• Edge stability: No crashes in T hours continuous run; Thermal throttling below K% of runtime