Research Scientist, Benchmarks & Evals

Job Description:

• Design and ship public benchmarks for image, video and document models
• Build private test sets from real, opt-in data that no model has trained on
• Run data-value experiments by training models with and without Kled data and measuring changes
• Design human evaluation studies and automatic metrics
• Publish leaderboards and reports trusted by researchers at frontier labs
• Identify model shortcomings and advise what data to collect next
• Work directly with founders and lab buyers to turn results into data deals

Requirements:

• 3+ years ML research or research engineering experience
• A benchmark or evaluation you built that other people actually used
• Strong hands-on experience training and fine-tuning image or video models (PyTorch)
• Deep understanding of experimental design and statistics (controls, ablations, human studies)
• Experience running experiments end to end, from GPUs to final report
• Clear writing for both researchers and buyers
• Publications at NeurIPS, ICML, ICLR or CVPR (datasets and benchmarks especially) (bonus)
• Experience on an evaluation or data team at an AI lab (bonus)
• Experience selling or licensing data to AI labs (bonus)
• Experience with data valuation or data attribution research (bonus)
• Python / PyTorch
• Open-weight image and video models
• Cloud GPUs
• PostgreSQL (Supabase)
• S3 storage

Benefits:

• $350,000 – $750,000 equity
• 0.25% – 1% equity