About Mercor

Mercor is training models that predict how well someone will perform on a job better than a human can. We use our platform to source, vet, and onboard expert contractors who help train AI models in a wide variety of domains. Our technology is so effective it’s used by all of the top 5 AI labs.

We scaled from $1-500M in revenue run rate in the last 17 months, making us the fastest growing company in the world. Our growth is accelerating. We averaged 11% week over week growth in July, 18% WoW growth in August, and 19% WoW growth in September. The team is small and we remain profitable because we can’t hire great people as quickly as revenue is growing.

About the Role

Silicon Valley’s top AI companies work with Mercor to find domain experts who can help train and evaluate their models. As a researcher at Mercor, you will be responsible for advancing the frontier of model evaluations to drive model improvements across the industry that create real world economic value.

You will be frequently publishing impactful papers with industry leading collaborators, have ample resources to create high-impact datasets, and access to the frontier of evaluation and training data. You will work closely with Mercors’s Forward Deployed Research, Applied AI, and Operations teams, and have unmatched access to evaluate frontier models

We are looking for an experienced AI researcher. A track record of LLM evaluation publications is preferred but publication experience in evaluation of other types of models or other AI related publications are of interest as well.

Key Responsibilities

• Build benchmarks that measure real world value of AI models.

• Publish LLM evaluation papers in top conferences with the support of the Mercor Applied AI and Operations teams.

• Push the frontier of understanding data ROI in model development including multi-modality, code, tool-use, and more.

• Design and validate novel data collection and annotation offerings for the leading industry labs and big tech companies.

What Are We Looking For?

• PhD or M.S. and 2+ years of work experience in a computer science, electrical engineering, econometrics, or another STEM field that provides a solid understanding of ML and model evaluation.

• Strong publication record in AI research, ideally in LLM evaluation. Dataset and evaluation papers are preferred.

• Strong understanding of LLMs and the data on which they are trained and evaluated against.

• Strong communication skills and ability to present findings clearly and concisely.

• Familiarity with data annotation workflows.

• Good understanding of statistics.

• Willingness to work 6 days a week, with monday-friday in person in San Francisco

Compensation

• Base cash comp from $200K-$300K

• Generous equity grant

• A $20K relocation bonus (if moving to the Bay Area)

• A $10K housing bonus (if you live within 0.5 miles of our office)

• A $1K monthly stipend for meals

• Free Equinox membership

• Health insurance

Compensation Range: $200K - $300K