Client overview
Our client is a leading perception software company headquartered in Korea. They are focused on advancing autonomous vehicle (AV) technology and already work with large amounts of transportation data collected from global sources.
Their core challenge was not in collecting data but in ensuring that massive volumes of raw data could be annotated with precision. Accurate annotation is critical for machine learning models that power AV perception systems. Without high-quality object detection data, these models would struggle to perform in real-world driving scenarios.
What the client needs
‘The client approached us with clear requirements:
Specialized team:
They needed a vendor to build and manage a dedicated annotation team with domain-specific expertise.
Strict QA:
The project required consistent accuracy, with all annotations closely adhering to the specified guidelines.
Scalability:
The annotation workload, involving hundreds of thousands of images, demanded a workforce that could scale seamlessly to meet project demands.
Seamless communication:
To bridge language and cultural gaps, the client wanted a team that could work under their direct guidance.
High security:
Data security was paramount; they required our team to ensure all project information remained confidential to safeguard the partnership.
How we did it
To meet the client’s requirements for accuracy, speed, and secure collaboration, we implemented a step-by-step delivery process.
1. Team setup
We began by building a dedicated classification team based in Vietnam, consisting of 6 annotators and 1 project manager.
To bridge language and cultural gaps, we assigned a Korean-speaking project manager as the main point of contact. This ensured that requirements, feedback, and edge-case discussions were clearly understood and acted on without delay.
The communication and tracking setup included:
– Slack for daily coordination and quick clarifications
– Gmail for formal updates and reporting
– Excel for tracking annotation volume, accuracy metrics, and feedback status
– Client’s proprietary annotation tool for all production work
2. Recruitment and training
Although the project duration was only one month, training played a critical role in ensuring consistency and accuracy from day one. Before moving to full production, each annotator had to demonstrate a consistent understanding of the classification rules. Any gaps were addressed early to avoid downstream rework. This preparation ensured that all team members applied the same logic when reviewing images, even under time pressure.
3. Execution
Once training was completed, the team moved into full-scale execution.
Each image was reviewed and classified based on factors affecting the camera’s view. The defined classes included:
– Frozen: The camera view is impacted by ice or frost
– Sun: Strong sunlight or glare affects visibility
– Blur: Motion blur or focus issues reduce clarity
– Block: The camera view is partially or fully obstructed
Annotators performed image-level classification, with 2D polygon labeling applied where needed to provide additional spatial context for the classification.
4. Validation
Quality benchmarks were defined at the beginning of the project and shared with the entire team. Accuracy was monitored continuously rather than only reviewed at the end. Our quality control approach included 4 QA steps. When accuracy trends showed potential issues, corrective actions were taken immediately through retraining or clarification sessions. This proactive monitoring helped the team maintain a 99% accuracy rate throughout the project.
5. Delivery
Completed annotations were delivered in structured batches according to the agreed schedule.
Throughout the project, we provided:
– Regular progress updates
– Accuracy and volume reports
– Quick responses to client feedback or adjustment requests
By keeping delivery transparent and communication open, the client retained full visibility into progress and quality, even with a short project timeline.
What the results have
Despite the short duration of the project, the results were strong and measurable:
– 50,000 images classified within one month
– 99% accuracy rate, exceeding the client’s expectations
– Consistent quality across all classification categories
– Fast turnaround with minimal rework
