Overview
This role involves reviewing and evaluating AI-generated responses to U.S. legal hypotheticals to help fine-tune AI models. No prior AI experience is required—your background in U.S. law is the foundation for assessing issue spotting, accuracy, and legal reasoning.
Responsibilities
- Review and evaluate AI-generated responses to U.S. legal hypotheticals across multiple areas of law
- Apply a structured legal rubric to assess:
- Issue spotting
- Accuracy
- Reasoning
- Application of law
- Identify gaps, edge cases, and blind spots in legal reasoning to help define benchmarks for legal AI performance
- Provide detailed, high-quality annotations and written feedback to refine AI models
- Collaborate with researchers and project managers to align evaluation standards and project goals
Qualifications
- Juris Doctor (J.D.) from an accredited U.S. law school
- Bar admission (active/inactive)
- 3+ years of U.S. legal practice or teaching experience
- Strong reasoning skills across core areas of U.S. law
- Excellent written communication and attention to detail
- High ethical standards; interest in AI is a plus
Compensation & Benefits
- Competitive compensation
- Flexible, fully remote work arrangement
Commitment & Duration
- 10–30 hrs/week (flexible)
- Duration: ~1 month, with the possibility of extension based on performance and project needs
Location
- Flexible, fully remote work arrangement