Overview
This role involves shaping the accuracy, reliability, and quality of AI outputs used by hundreds of legal teams in their daily work. You will apply legal experience to design legal evaluations, improve prompts and evaluation processes, validate outputs against real-world legal standards, and translate substantive expertise into product improvements that build user trust.
Responsibilities
- Design and run rigorous evaluations to measure the quality of AI-generated outputs and ensure they meet high standards expected by legal professionals.
- Review and validate product outputs for accuracy, relevance, comprehensiveness, helpfulness, and related quality dimensions.
- Collaborate closely with engineering and GTM teams to identify performance gaps, diagnose root causes, and develop solutions.
- Serve as the legal voice of quality by advocating for precision, consistency, and clarity in AI outputs.
- Translate between legal domain expertise and technical teams by converting legal concepts into product requirements, and product concepts back into legal terms.
- Provide domain-specific legal expertise to guide feature development and product roadmap decisions.
- Generate ideas for new products based on a deep understanding of legal teams’ pain points.
Requirements
- Law degree and license to practice law in a U.S. jurisdiction.
- 5+ years of legal experience, with meaningful in-house experience required.
- Substantive depth in one or more areas of in-house legal work, such as:
- corporate-commercial or transactional law
- contract drafting and negotiation
- product or regulatory counseling
- privacy and compliance
- employment
- fintech
- litigation
- Experience using generative AI for legal work and evaluating generative AI outputs.
- Strong analytical skills, including the ability to design metrics, interpret evaluation results, and turn legal reasoning into logical, repeatable steps.
- Strong cross-functional communication skills, with the ability to work closely with Engineering, Product, and GTM teams.
Preferred Qualifications
- Prior experience using the organization’s AI platform.
- Interest in legal tech, including prompt engineering and quality assurance.
- Strong focus on user experience, particularly how legal professionals interact with and trust AI assistants.
- Experience working closely with software engineers.
- Ability to design and interpret evaluation metrics and abstract legal reasoning and processes into logical, repeatable steps.
Compensation & Benefits
- Compensation range: $150K - $240K
Location
- Distributed role with team members across North America and plans to expand into Europe.
- Up to 10% travel (more for customer-facing roles) for team members in locations without an office.
- Team collaboration schedule:
- Tuesdays, Wednesdays, and Thursdays for team members in the San Francisco Bay Area and Provo, UT (within 50 miles of the office).
- Hiring across the United States and Canada.
Equal Opportunity Employment
The employer is an equal opportunity employer and provides reasonable accommodations to applicants with physical and mental disabilities.
Fraud Notice
To protect applicants from phishing and recruitment fraud, applications are accepted only through official channels. Do not share information if contacted through unofficial or suspicious channels.