TIER IV has built an end-to-end (E2E) autonomous driving AI model developed autonomously by an agentic AI, using nationwide Japanese driving data collected on Co-MLOps, a data-sharing platform for democratizing autonomous driving AI development, and labels generated by the autolabeling platform CoMET. Dubbed METEOR, the model does not depend on high-definition (HD) maps: from surround-camera video alone, a single neural network performs bird's-eye-view (BEV) perception of the surroundings through to trajectory generation. Its distinguishing point is that it was developed with no additional human labeling, with the train–evaluate–improve loop executed by an agentic AI. Data augmentation with NVIDIA Cosmos (condition transfer and rare-case generation) is combined to cover conditions that real data alone cannot. The source code, trained model and training recipe have been released as open-source software (OSS). Companies participating in the Co-MLOps project can keep improving the model with shared data while extending it with their own data and features.
Autonomous driving AI development is shifting from a stage where model designs are hand-crafted to one where data is collected, models are trained, and evaluation and improvement are repeated. In 2024, TIER IV established Co-MLOps, a data-sharing platform on which participating companies share driving data and grow autonomous driving AI together. Building on the nationwide Japanese driving data on the platform, we have been working to harness that data to create an autonomous driving AI that runs in a real vehicle, through autolabeling and development driven by agentic AI. This article introduces the result: a reference E2E AI model, covering what is inside the model, and how the agentic AI carries the development forward. The model is scheduled to be demonstrated on an in-vehicle computer at Automotive World 2026 in greater Tokyo, September 9-11.
This post focuses on the reference E2E AI model and its development. For an overview of Co-MLOps data collection, autolabeling with the Collaborative Multi-stage Ensemble-based Teacher Model (CoMET) and NVIDIA Cosmos integration, read the TIER IV tech blog “Building a Dataset Foundation for Autonomous Driving with NVIDIA Cosmos” (August 7, 2026).
Co-MLOps collects and shares nationwide Japanese driving data from participating companies' vehicles, advancing the democratization of data. Data is the key asset that determines the performance of autonomous driving AI, but data on its own creates no value. Only with a mechanism for labeling and for model development does data contribute to AI performance.
The reference E2E AI model is one of Co-MLOps' answers to this. It serves as a baseline model that Co-MLOps participants can continuously train, update and improve on the platform using the shared data. The source code, trained model and training recipe are released as OSS, and participants can build on these resources to evaluate on their own data, fine-tune, and add their own features. Because labeling is handled by the CoMET autolabeling platform and model development by an agentic AI, an E2E autonomous driving AI model can be developed efficiently starting from driving data.
Fig. 1 shows the Co-MLOps concept. It has been designed as a shared platform where the benefit to everyone grows as more companies join, with the reference AI continuously updated with large-scale data. Fig. 2 illustrates this cycle from the perspective of the reference E2E AI model. CoMET labels data collected by Co-MLOps participants and an agentic AI develops the model. Following the OSS release, participants extend the model independently. The resulting new data and improvements are shared back to feed the next development cycle.
Fig. 1. Co-MLOps overview
Fig. 2. The loop that produces the reference E2E AI model
The core of the reference E2E AI model is a single neural network that takes only surround in-vehicle camera video as input and does not depend on HD maps. METEOR stands for Multi-task Estimation of Traffic Elements, Objects & Roads, which describes the design policy of estimating traffic elements, objects and roads in a multi-task manner.
Image features from the surround cameras are transformed into the bird's-eye-view (BEV) space with depth, fused over time, and the perception, prediction and trajectory-generation tasks are output simultaneously from a single network. Details of the components are given in Appendix A.
Most of the building blocks are known techniques. Lifting multi-camera image features into BEV space using per-pixel depth distributions follows the lineage that began with Lift, Splat, Shoot [3]; depth supervision from LiDAR [4] and temporal fusion that aligns past BEV features by ego motion [5][6] have both been shown effective in prior work. 3D object detection uses a center-based approach that estimates object centers from a heatmap on the BEV grid [7]. Handling perception and ego-trajectory generation in a single network places the model in the line of recent end-to-end driving research [8][9][10]. What characterizes METEOR is that these are combined using operators that run easily on an in-vehicle SoC, and that the model can be updated continuously on top of the autolabeling and agentic-AI development loop.
Fig. 3. METEOR architecture: surround cameras → depth-aware BEV transform → temporal fusion → simultaneous multi-task output. A rule-based safety check (guardrail) is attached to the trajectory output
Fig. 4 shows inference on public roads not used in training. The top row shows the surround cameras with 2D perception, the middle row camera-only distance estimation, the bottom left a volumetric occupancy grid, and the right a bird's-eye view (lane lines, crosswalks, vehicles; the green line is the E2E driving plan). This frame illustrates the simultaneous outputs of a single network: perception, measurement, analysis and planning.
Fig. 4. Inference on validation data (public roads not used in training): top, surround cameras with 2D perception; middle, camera-only distance estimation; bottom left, volumetric occupancy grid; right, bird's-eye view (green line = E2E driving plan)
In-vehicle implementation is also part of the agentic-AI development. The trained model is exported to ONNX, an INT8 quantized engine [11][12] is built with calibration on real driving frames, and implementation optimizations such as a mixed-precision design based on per-layer quantization sensitivity (layers with large impact stay in FP16) [12][13] and a dedicated CUDA plugin for the BEV transform are applied repeatedly by the agentic AI, guided by latency profiles measured on the hardware. Every engine build is automatically checked for latency, output sanity and the accuracy difference before and after quantization, and only builds that meet the criteria are deployed to the in-vehicle computer. If a build fails, the previous generation is restored automatically.
The performance of physical AI depends on the quality and quantity of data just as much as model design, if not more. We bear this in mind in our data-centric development approach. As a precondition, this requires not only data volume but also quality, diversity and accessibility. Likewise, when accuracy stalls during the improvement cycle of the reference E2E AI model, we look at the data first, identifying weak scenes and addressing the data and labels for those conditions.
Training used nationwide Japanese driving data collected through Co-MLOps. The diversity of regions and conditions provides a level of generalization that is difficult to achieve with data from a single area. These conditions include urban intersections, suburban residential streets, winding mountain roads, nighttime and rain. During development, we observed that adding data from a new region improved perception of that region's characteristic road structures and objects. We believe this data diversity directly expands the range of driving environments the model can handle.
Nationwide collection is carried out by a fleet equipped with the Co-MLOps Data Recording System (DRS) (Fig. 5). Four 120° LiDARs covering front, rear, left and right and eight multi-FoV cameras are recorded in a synchronized, calibration-managed state. The eight-camera input of the reference E2E AI model matches this DRS camera configuration. LiDAR is used only for CoMET label generation (3D bounding boxes and depth); inference uses cameras only. Participants can therefore use DRS data directly for fine-tuning, and with the same vehicle configuration, go straight through to in-vehicle evaluation.
Fig. 5. The Co-MLOps Data Recording System (DRS)
All training labels for the reference E2E AI model are derived from CoMET autolabeling. CoMET is an ensemble teacher model combining twelve large models, and it generates labels on the scale of millions at consistent quality without the throughput limits of human labor. Its construction and robustness are described in the tech blog mentioned above.
What we want to highlight here is that CoMET's labels are general-purpose rather than task-specific. By keeping them in reusable representations, namely 3D bounding boxes, panoptic segmentation and traffic-light recognition, METEOR's training derives all of the following training labels from the same label asset.
Many of the labels for understanding the driving environment, such as BEV road-surface segmentation and 2D segmentation, are derived from panoptic segmentation, and the ability of the HD-map-free reference E2E AI model to read road structure rests on this general-purpose label. Adding a new task requires no re-annotation, only one derivation step from the existing asset. Generate labels once as a representation rather than once per task: that is the essence of autolabeling-centered development (Fig. 6). In this project, ingesting data from a new region, from placing the data to starting training, is done with no human involvement.
Fig. 6. From one label asset to every training label: CoMET generates general-purpose label assets (3D bounding boxes / panoptic segmentation / traffic-light recognition) from camera, LiDAR and GNSS data, and every task's training labels are derived from them
The challenge with autolabels is noise. CoMET and METEOR suppress it with the following mechanisms (Fig. 7).
Fig. 7. Label quality control: Only pixels on which two independently built label sources agree become labels, and the consistency check and the handling of missing labels keep noise out of training.
For the biases of real driving data (scarce rare cases, difficulty of collecting in bad weather), we are using generated data from NVIDIA Cosmos: condition transfer with Cosmos Transfer (converting to rain, night or snow while keeping CoMET labels), rare-case generation with Cosmos Predict, and data-gap discovery with Cosmos Reason. In training the reference E2E AI model, bad-weather and night data converted with Cosmos Transfer are mixed with real data, and we are evaluating the effect on perception performance under adverse conditions. Here too the generated data are labeled by CoMET, so the autolabeling platform is the common foundation for real and generated data.
The core concept of METEOR is an autonomously evolving perception engine. The aim is to run the whole process, from driving-data collection (DRS) through CoMET labeling, training, improvement and in-vehicle implementation, without human intervention (Fig. 8): development that does not presuppose human labeling or human code.
Fig. 8. The METEOR concept: automating the process from collection → labels (CoMET) → training → improvement → in-vehicle implementation
In developing the reference E2E AI model, the agentic AI executes not only training but the development loop itself: evaluation, failure analysis, implementing improvements and retraining. No additional human labeling was done, and the agentic AI also wrote the model and training code. Improvements are made under pre-registered acceptance criteria (for example, automatic rollback if accuracy degrades), and training, evaluation and hardware verification across several machines run continuously. We view this as a practical example of AI developing AI.
In this development approach, the human role centers on directing what to build and reviewing the results. Given a concept or feature request such as "I want a feature that can do X", the agentic AI carries out model development, ingestion of the autolabeled driving data, data conversion, cleaning, implementation optimization (quantization and edge speed-up) and model operations (deployment and auto-recovery), and keeps running as a self-improvement loop (Fig. 9). It is an attempt to replace the constraint of development resources with the constraint of data and compute.
Fig. 9. How development is done: the human directs, the AI develops. From model development through data processing, implementation optimization and operations, agentic AI executes the work
The agentic AI works under the following principles (Fig. 10):
Fig. 10. The agentic-AI development loop: one variable per change, pass criteria registered in advance. Measures that do not help are withdrawn automatically
In-vehicle implementation runs inside the same loop. The standard path from ONNX to TensorRT INT8 is used, with real driving frames for calibration. Optimization is driven by latency profiles on the hardware and by quantization sensitivity analysis [12]: layers with large accuracy impact stay in FP16 and the rest are quantized to INT8. In post-training quantization, the quantization scales are determined from the activation distributions of real driving frames, following the standard procedure for integer-arithmetic inference [11][13]. At every engine build, latency, output sanity and the accuracy difference against FP16 execution are measured automatically, and each optimization is accepted or rejected on the basis of a ledger recording "accuracy contribution per millisecond." Items whose effect does not reproduce are recorded as rejected. See Appendix A for details.
The reference E2E AI model is available as open-source software. This includes the source code, trained model and training recipe (the procedure from data conversion and label derivation through training to in-vehicle implementation). Releasing it as OSS serves two main purposes. First, it presents a camera-only, HD-map-free E2E autonomous driving AI model in a format that can be reproduced, verified and modified. Second, it demonstrates the Co-MLOps development approach (autolabeling plus agentic-AI development) through a working model and code.
Companies participating in the Co-MLOps project can start from this released version and:
The benefit of using the reference E2E AI model is that it reduces the initial setup burden compared to starting E2E development from scratch.
This article introduced our reference E2E AI model, a camera-only, HD-map-free E2E autonomous driving AI model built by combining nationwide Japanese driving data collected through Co-MLOps, CoMET autolabeling, and an agentic-AI development loop.
The performance of E2E AI depends heavily on the volume and diversity of data. The reference E2E AI model is not a finished product but a model that keeps being updated as data grows. As data from new regions and conditions is added to Co-MLOps, the model provided to members improves. Through Co-MLOps and the reference E2E AI model, TIER IV aims to advance autonomous driving AI development together with the community. If you are interested, please explore the public repository and join the discussion. We also welcome inquiries about joining Co-MLOps or sharing data.
Architecture: A single network built around a depth-aware BEV transform
METEOR is a single network built from efficient operators, designed from the outset for quantization and acceleration on an in-vehicle SoC. TensorRT compatibility is treated as a design constraint rather than a porting task after research: the operators are limited to a set that TensorRT executes efficiently, such as Conv, GridSample and Gather, and temporal memory is expressed not as a recurrent structure but as a static graph in which the previous frame's BEV features are passed as ordinary input and output tensors. As a result, the trained model can be converted to a TensorRT engine almost as-is via ONNX, and deployment to NVIDIA Jetson Orin needs no special porting work.
METEOR additionally has a PointPainting-style [15] feature fusion that adds 2D segmentation probabilities to the input features of the BEV transform. A distant pedestrian tends to collapse into a single cell and vanish on the BEV grid, while in the image it remains as a tall, thin shape. Carrying this 2D perception result through the lift into the BEV cells has been confirmed to improve the detection recall of distant pedestrians. The injection is done through a zero-initialized 1×1 projection, and it was verified numerically that the model output is unchanged at the moment of insertion (function-preserving) before training (Fig. 11).
Fig. 11. PointPainting: The probabilities of 2D segmentation are injected before the BEV transform through a zero-initialized 1×1 projection. BEV width, the lift plugin and the distribution layout are unchanged.
How the labels are made
Edge optimization
The projection in the BEV transform is implemented as a dedicated CUDA plugin to secure compute efficiency. In the current configuration, the degradation of BEV segmentation from INT8 quantization stays within the noise band (the accuracy difference before and after quantization is measured following the evaluation procedure of [12]), and latency stays at a practical level on a realistic in-vehicle SoC. We do not regard this as sufficient, however, and plan to keep exploring improvements on both the model-structure and implementation sides. Members deploying to their own Jetson Orin-equipped vehicles can use the same path as-is.
[1] Q. Li et al., “HDMapNet: An Online HD Map Construction and Evaluation Framework,” ICRA 2022.
[2] B. Liao et al., “MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction,” ICLR 2023.
[3] J. Philion, S. Fidler, “Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D,” ECCV 2020.
[4] Y. Li et al., “BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection,” AAAI 2023.
[5] Z. Li et al., “BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers,” ECCV 2022.
[6] A. Hu et al., “FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras,” ICCV 2021.
[7] T. Yin, X. Zhou, P. Krähenbühl, “Center-based 3D Object Detection and Tracking,” CVPR 2021.
[8] Y. Hu et al., “Planning-oriented Autonomous Driving,” CVPR 2023.
[9] B. Jiang et al., “VAD: Vectorized Scene Representation for Efficient Autonomous Driving,” ICCV 2023.
[10] L. Chen et al., “End-to-end Autonomous Driving: Challenges and Frontiers,” IEEE TPAMI, 2024.
[11] B. Jacob et al., “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” CVPR 2018.
[12] H. Wu et al., “Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation,” arXiv:2004.09602, 2020.
[13] M. Nagel et al., “A White Paper on Neural Network Quantization,” arXiv:2106.08295, 2021.
[14] M. Bansal, A. Krizhevsky, A. Ogale, “ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst,” RSS 2019.
[15] S. Vora et al., “PointPainting: Sequential Fusion for 3D Object Detection,” CVPR 2020.
Dan Umeda | Data Scaling Department
Dan joined TIER IV in 2023 and currently serves as a principal AI engineer and team lead, driving the development of Co-MLOps.
TIER IV is always on the lookout for passionate individuals to join our journey. If you share our vision of making autonomous driving accessible to all, get in touch.
We’re currently hiring for the following related positions:
Visit our careers page to view all job openings.
If you’re unsure which roles fit your experience, or if the current job openings don’t quite match your preferences, you can register your interest here. We’ll contact you when a suitable role becomes available and arrange an informal interview.
Inquiries
Social Media
X (Japan/Global) | LinkedIn | Facebook | Instagram | YouTube
More