Senior Production Insurance AI Evaluation Specialist  Job Listing at JDA TSG in Columbus, OH (Job ID 1375-Columbus, Ohio, US)

Description

Overview

At JDA TSG, we equip world-class brands with top-tier talent and operational expertise, because we know our people are our true differentiator. As a Senior Production Insurance AI Evaluation Specialist, you will run an expert quality review function for active AI agents performing commercial insurance tasks. We believe AI operational success relies on domain expertise. By auditing live AI outputs against source records, catching discrepancies, and reporting error patterns, your work protects policy accuracy, refines machine learning models, and ensures reliable performance across enterprise insurance workflows.  

In this role, you will establish trusted ground-truth accuracy baselines across two primary Commercial lines workstreams prior to and following General Availability (GA) releases: Submissions Manager (auditing email threads and multi-line submission attachments) and Commercial Lines Autofill / Risk Checker (auditing form schemas and quote extractions)

Title: Senior Production Insurance AI Evaluation Specialist 

Location: Remote, USA 

Salary: $90,000 to $110,000 USD 

Must Have: Active Property & Casualty (P&C) Producer / Agent License 

• Ground-Truth Queue Auditing & Discrepancy Isolation: You will review assigned submissions and AI extractions across multi-file email threads, ACORD forms, quote schedules, and attached policy documents. You will audit AI outputs against target schemas to verify total submission premium, line-of-business (LOB) premium allocations, LOB selections, email thread attribution, extracted schema fields, policy status, and decline reasons, catching both missing and incorrect values across commercial forms and binders. Success is measured by audit completion volume, extraction accuracy rates, and error-catching precision.   

• Disciplined Escalation & Pattern Calibration: You will strictly adhere to client quality standards by flagging genuinely ambiguous edge cases for workstream owner escalation rather than guessing, preserving ground-truth baseline integrity. You will aggregate extraction errors into structured weekly workbooks and participate in peer calibration sessions with workstream owners to align on reference-label standards and agreement targets. Success is measured by baseline data integrity, workbook accuracy, and calibration alignment across workstreams.   

• SLA & Weekly Workbook Execution: Maintain a consistent review pace while meeting established productivity expectations and delivering verified weekly workbooks on schedule. Success is measured by review throughput, SLA compliance, accuracy, and timely workbook delivery.  

• Active P&C License (Mandatory): Active Property & Casualty (P&C) Producer / Agent License in good standing. 
  

• Commercial Lines Expertise: 5+ years of recent operational experience in commercial insurance brokerage, agency, MGA, or carrier operations. Direct experience with Commercial General Liability, Commercial Property, Commercial Auto, Workers' Compensation, and Umbrella/Excess lines is required (personal-lines-only backgrounds will not be considered).   

• Submission & Email Thread Parsing: Proven ability to analyze complex agency email threads, multi-line carrier quotes, ACORD forms, and policy schedules.   

• Analytical Precision & Systems Familiarity: Exceptional visual rigor in comparing unstructured documents against JSON extraction workbooks to isolate missing vs. incorrect values. Practical experience using agency management systems such as Applied Epic, EZLynx, or carrier management portals.  

• Healthcare - Comprehensive coverage for you and your family 

• Employee Assistance Program - Get support when you or your family need it with counseling and coaching 

• 401K with company match 

• Paid time off 

• Paid parental leave 

• Volunteer Day Off 

• Life insurance - Protect your loved ones and their future 

• Business travel accident insurance 

The specific pay rate offered for this role depends on candidate qualifications, relevant experience, technical expertise, and role requirements. Final individual offers are determined through candidate evaluations during the interview process 

We are an equal opportunity employer committed to building an inclusive workplace. To view our Equal Employment Opportunity (EEO) policy, please click here.

• Active P&C License (Mandatory): Active Property & Casualty (P&C) Producer / Agent License in good standing. 
  

• Commercial Lines Expertise: 5+ years of recent operational experience in commercial insurance brokerage, agency, MGA, or carrier operations. Direct experience with Commercial General Liability, Commercial Property, Commercial Auto, Workers' Compensation, and Umbrella/Excess lines is required (personal-lines-only backgrounds will not be considered).   

• Submission & Email Thread Parsing: Proven ability to analyze complex agency email threads, multi-line carrier quotes, ACORD forms, and policy schedules.   

• Analytical Precision & Systems Familiarity: Exceptional visual rigor in comparing unstructured documents against JSON extraction workbooks to isolate missing vs. incorrect values. Practical experience using agency management systems such as Applied Epic, EZLynx, or carrier management portals.  

• Ground-Truth Queue Auditing & Discrepancy Isolation: You will review assigned submissions and AI extractions across multi-file email threads, ACORD forms, quote schedules, and attached policy documents. You will audit AI outputs against target schemas to verify total submission premium, line-of-business (LOB) premium allocations, LOB selections, email thread attribution, extracted schema fields, policy status, and decline reasons, catching both missing and incorrect values across commercial forms and binders. Success is measured by audit completion volume, extraction accuracy rates, and error-catching precision.   

• Disciplined Escalation & Pattern Calibration: You will strictly adhere to client quality standards by flagging genuinely ambiguous edge cases for workstream owner escalation rather than guessing, preserving ground-truth baseline integrity. You will aggregate extraction errors into structured weekly workbooks and participate in peer calibration sessions with workstream owners to align on reference-label standards and agreement targets. Success is measured by baseline data integrity, workbook accuracy, and calibration alignment across workstreams.   

• SLA & Weekly Workbook Execution: Maintain a consistent review pace while meeting established productivity expectations and delivering verified weekly workbooks on schedule. Success is measured by review throughput, SLA compliance, accuracy, and timely workbook delivery.