Job Description
About the Company
Turing is an AI infrastructure company that works with leading AI research organizations to accelerate model development. Its work includes creating high-quality training data, developing evaluation methods, and connecting organizations with engineering talent to support advanced AI projects.
About the Role
Turing is seeking a Remote Python Engineer to support AI model development through preference data evaluation, rubric creation, and engineering workflow improvements. This role combines strong programming skills with careful technical judgment to help ensure training data and evaluation criteria meet high quality standards.
You will design and refine evaluation rubrics, review data for subtle quality issues, and build the pipelines and infrastructure needed to support data generation, collection, and evaluation. You will also collaborate with researchers and engineers to turn qualitative assessments into consistent,
scalable processes.
The ideal candidate communicates clearly, works confidently in fast-changing environments, and can make sound decisions while moving projects forward.
Key Responsibilities
- Design, refine, and improve rubrics and evaluation criteria for preference data.
- Review and label data while maintaining high quality standards and identifying subtle errors or inconsistencies.
- Build, maintain, and improve pipelines and infrastructure for data generation, collection, and evaluation workflows.
- Analyze findings from data evaluation activities and communicate conclusions through clear written summaries, project updates, and recommendations.
- Collaborate with researchers and engineers to translate qualitative judgments into repeatable, scalable evaluation processes.
- Apply strong technical judgment to assess the quality of examples and explain why particular outputs meet or fail established standards.
- Contribute to engineering workflows that support reliable and efficient AI model development.
Required Qualifications
- Proficiency in Python, particularly for building and maintaining pipelines and infrastructure, or proficiency in TypeScript.
- Hands-on experience with preference data collection, rubric creation, or both.
- Strong written and verbal communication skills, including the ability to summarize complex technical work into concise, actionable information.
- Strong quality judgment and the ability to explain what makes an example effective, accurate, or high quality.
- Ability to work independently in ambiguous, fast-moving environments and prioritize practical delivery.
- Ability to collaborate effectively with engineering and research teams.
Experience contributing to or navigating large codebases, such as Visual Studio Code (VS Code), is a plus for candidates with TypeScript experience.
Salary & Benefits
Compensation is offered at a market-based rate. Applicants are asked to provide a specific compensation expectation rather than a range.
The role offers a fully remote working arrangement and the opportunity to contribute to advanced AI projects with leading large language model (LLM) companies.
This is an independent contractor engagement lasting approximately three months, with an expected start as soon as possible. Medical insurance and paid leave benefits are not included.
The preferred commitment is 40 hours per week, with at least six hours of overlap with Pacific Time. Candidates who cannot commit to 40 hours should provide their available weekly hours and time-zone overlap.
Selection Process
The evaluation process consists of one approximately 30-minute interview covering technical skills and cultural fit.
Need to call your state’s unemployment office? We list every state’s phone lines, as each agency gives them. Find your state →
View Similar Jobs
Similar jobs which you may be interested in. Typically using your existing skillset.