Position: Member of Technical Staff, Coding Research Type: Full-time Compensation: $220K - $500K/yr Location: Remote
Role Responsibilities Design and own evaluation frameworks for coding agents, including benchmark specifications, scoring methodologies, rubrics, and quality standards. Lead end-to-end research initiatives focused on measuring and improving coding model performance across diverse software engineering tasks. Develop high-quality datasets, golden examples, and evaluation protocols that enable reliable assessment of frontier coding systems. Analyze model behavior and failure modes, identifying systematic weaknesses and translating findings into actionable improvements for training and evaluation. Build tooling and infrastructure that support large-scale experimentation, data generation, review workflows, and evaluation pipelines. Establish best practices for coding-agent assessment, ensuring methodological rigor, reproducibility, and measurement quality.
Requirements Have a strong software engineering background with expertise in Python, C++, or comparable programming languages. Have strong relevant experience in software engineering, machine learning, AI research, evaluation, or related technical disciplines. Have experience designing, reviewing, or validating technical assessments, benchmarks, coding tasks, or evaluation methodologies. Have familiarity with large language models, coding agents, reinforcement learning, model evaluation, or related AI systems. Have a proven ability to build tooling, automate workflows, and improve technical processes through systematic experimentation.
Application Process Easy Apply on Linked InCheck email for next steps Participate in resume evaluation & interview stage

Skills: c++, Machine Learning, Data Analysis, Software Engineering