Benchmarking & Evaluation Researcher

About the role

The Benchmarking & Evaluation Researcher will help build standardized tests and benchmarks to assess AI models. This position works within the Core AI Research team to ensure model performance is measured accurately, fairly, and safely.

Key responsibilities

• Develop benchmark datasets that evaluate accuracy, fairness, robustness, and safety of AI models.
• Design evaluation protocols, including metrics, conditions, and processes for model assessment.
• Analyze evaluation results, interpret outcomes, and provide feedback for model improvement.
• Collaborate with external consortiums and organizations to create open benchmarks that promote transparency.

Required profile

• Master's or doctoral degree in statistics, data science, or a related field.
• Proven experience in dataset creation, statistical analysis, and evaluation framework design.

Required skills

• Dataset creation
• Statistical analysis
• Evaluation framework design

What we offer

• Opportunity to contribute to open AI benchmarking initiatives.
• Collaboration with external research consortiums and internal AI teams.
• Volunteer position with potential for future paid appointment pending funding availability.