Senior Lawyer

Overview

A leading talent connection platform is seeking a Legal AI Evaluator to work full-time or part-time on a contract basis. This role involves evaluating and refining AI-generated legal content to ensure accuracy and quality.

Responsibilities

• Write and refine prompts to guide AI model behavior in legal contexts.
• Evaluate large language model (LLM) responses to legal queries for accuracy, reasoning, clarity, and completeness.
• Conduct fact-checking using trusted public and authoritative sources.
• Annotate model responses by identifying strengths, areas for improvement, and factual or conceptual inaccuracies.
• Assess tone, completeness, and appropriateness of responses for real-world legal scenarios.
• Apply consistent evaluation standards by following established taxonomies, benchmarks, and detailed guidelines.

Requirements

• Minimum of 5 years of professional experience in the legal field.
• Relevant expert degree such as JD, LLB, LLM, or equivalent.
• Experience in one or more legal sub-domains including Corporate, Business & Commercial Law; Employment, Labor & Human Rights; Public, Criminal & International Governance; or Private Law.
• Significant experience using large language models (LLMs).
• Excellent writing skills.
• Ability to evaluate complex arguments and identify subtle reasoning or detail-level issues.

Preferred Qualifications

• Prior experience with reinforcement learning from human feedback (RLHF), model evaluation, or data annotation.
• Experience writing or editing high-quality legal or analytical content.
• Ability to translate legal concepts for non-legal audiences.
• Familiarity with evaluation rubrics, benchmarks, or quality scoring systems.

Compensation & Benefits

• Hourly rate: $105
• Flexible full-time or part-time contract work

Location

• Remote work available for candidates based in the USA, European Union, United Kingdom, or Canada