Film & Television Reviewer (AI Evaluation)

Description

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are looking for an experienced Film & Television Domain Expert to support the evaluation and improvement of advanced Large Language Models.

The role involves creating challenging prompts, comparing AI-generated responses, and assessing factual accuracy, reasoning quality, completeness, and nuance across film, television, streaming, actors, directors, genres, awards, franchises, and entertainment history.

CONTRACT: Contractor assignment, PAY PER TASK, 8 weeks

COMMITMENT: Full-time, 40 hours per week with at least 4 hours of PST overlap

LOCATIONS: Remote - USA

Key Responsibilities

• Create advanced film and television prompts across a range of topics and difficulty levels.
• Evaluate and rank multiple AI-generated responses.
• Verify claims related to films, TV shows, streaming content, actors, directors, awards, genres, and franchises.
• Identify factual inaccuracies, hallucinations, outdated information, logical inconsistencies, and edge cases.
• Explain why responses are correct or incorrect using reliable references.
• Identify ambiguous or weak questions and propose improved prompts.
• Develop benchmark datasets and adversarial test cases.
• Provide clear, objective, and evidence-based feedback.
• Maintain accurate documentation and consistent annotation quality.
• Collaborate with AI researchers and project teams to improve model performance.

Requirements

• Master’s degree or higher in Film Studies, Television Studies, Media Studies, Cinema Studies, Screen Studies, or a closely related discipline.
• At least 3 years of professional, research, or teaching experience related to film, television, media, or entertainment.
• Strong knowledge across film, television, streaming, actors, directors, genres, awards, franchises, or entertainment history.
• Ability to research and verify entertainment-related claims using authoritative sources.
• Strong analytical skills and ability to assess factual and contextual accuracy.
• Excellent written English.
• Strong attention to detail.
• Ability to work independently and provide objective, evidence-backed reviews.