Client overview
Our client is a U.S.-based pioneer in Data-Centric AI, specializing in delivering high-quality data services for advanced frontier AI and agentic systems. The company plays a critical role as a bridge between data platforms and the model development process, enabling more reliable and scalable AI performance.
Objectives
The client is developing a data platform focused on AI models for coding. To support this initiative, they required a trusted vendor capable of:
- Generating large volumes of high-quality, complex coding datasets
- Evaluating data against well-defined, multi-dimensional quality criteria
- Covering a wide range of technical domains
- Delivering within a compressed timeline
Business Challenges
Prior to partnering with LTS GDS, the client faced several key challenges:
- Difficulty finding a vendor capable of mobilizing a large pool of qualified experts to handle massive task volumes
- The need for consistent quality across diverse and complex coding tasks
- Budget constraints while maintaining high standards of data quality
- Limited access to vendors with sufficient scalability to ramp up quickly or handle increasingly sophisticated requirements
Project 1: Design complex coding tasks for AI training
Challenges:
- The need to rapidly onboard a large number of qualified experts within a short timeframe
- High project complexity, requiring thorough training and alignment from the outset to ensure consistent taskquality
Requirements
- Design tasks centered around multi-step workflows (5–10+ commands) executed within live Dockercontainers
- Engineer tasks to meet specific pass-rate targets for model evaluation
- Cover a wide range of technical domains to ensure dataset diversity
Our Human Resources
Solutions
- Implemented GDS’s agile recruitment and training framework to onboard a large and diverse pool of experts within just two weeks
- Leveraged a pre-established expert pool to quickly respond to client demands and ensure scalability at any stage
- Delivered structured onboarding and targeted training to ensure experts could handle complex task requirements from the beginning
Key Results
Achievements
- Successfully delivered 100% of the requested task volume within the required timeline
- Accelerated the model training process, contributing to faster time-to-market for the client’s end customers
- Enabled the client to train a more reliable coding AI assistant capable of supporting thousands of developers globally
- Established a long-term data partnership supporting future AI coding assistance initiatives.
Project 2: Evaluate the AI’s response for AI training
Challenges:
- Difficulty sourcing a large pool of talent with both strong technical expertise and the patience required for AI training tasks, as opposed to traditional software development work
- High task complexity, significantly increasing the time required to complete each evaluation
- Maintaining consistency across thousands of tasks, ensuring all raters apply the same evaluation standards and judgment criteria
Requirements:
- Evaluate and guide AI models in handling real-world coding issues across multiple technical domains
- Perform multi-turn evaluations with multiple quality criteria to ensure high levels of accuracy and reliability
Our Human Resources
Solutions
- Implemented a tiered talent pool model, segmenting experts into specialized groups to improve efficiency and task accuracy.
- Leveraged LTS Group’s broader technology ecosystem to access a diverse talent pool, including developers, testers, and AI engineers across various domains.
- Established tailored evaluation benchmarks and guidelines aligned with the project’s specific requirements, including logic & correctness, naming & clarity, organization & modularity, interface design, error handling & robustness, documentation, and merge readiness, to ensure consistency at scale.
Key Results
Achievements
- Enabled the client to build internal benchmarks for training and validating AI datasets
- Accelerated the model development lifecycle, contributing to faster time-to-market for end clients
- Delivered the entire project efficiently, maintaining high quality while optimizing costs
