Lead Specialist, Psychometrician

Lead Specialist, Psychometrician
This role aligns to industry level title of Lead Specialist, Applied AI Measurement & Learning Systems
Measurement & Learning | School Assessment

Role Overview

We are seeking a Lead Specialist, Psychometrician to support the design, validation, and continuous improvement of intelligent measurement systems and AI-assisted assessment and learning workflows.

This role sits within the Measurement & Learning team and operates at the intersection of Measurement, Content Development, Learning Science, Technology, and AI Science, with responsibility for helping ensure AI-assisted systems consistently produce high-quality, valid, scalable, and instructionally meaningful outputs across assessment and learning use cases.

The role primarily supports AI-assisted assessment item development workflows, while also contributing to adjacent applications including AI-generated practice items, learning materials aligned to standards, AI-powered learning and feedback experiences, instructional guidance systems, and emerging approaches to evidencing student skills and competencies.

This role also contributes to the advancement of next-generation assessment approaches, including innovative item types, AI-assisted measurement opportunities, and new ways of assessing reasoning, applied learning, and student competencies.

AI-Assisted Assessment Item Development

Lead and support the design and continuous improvement of AI-assisted assessment item development workflows embedded within broader assessment and learning systems

Translate SME and measurement intent into structured, reusable AI-assisted workflows and agentic frameworks that support scalable, consistent, and auditable content generation and enhancement

Use AI-assisted tools and rapid prototyping approaches to explore, test, and iterate on emerging assessment, content, learning, and instructional workflow innovations

Support assessment and learning content across Mathematics, English Language Arts, Science, Social Studies, K–12 grade levels, TEIs, oral and constructed-response interactions, translation and multilingual content, and alignment to academic content standards

Support fact checking, grammar, style refinement, distractor rationale strengthening, and key validation

Ensure AI-assisted outputs meet expectations for validity, accuracy, fairness, accessibility, and instructional appropriateness

 

Assessment Innovation & Learning Applications 

Lead and support validation approaches for AI-assisted learning and assessment applications beyond core assessment item development workflows including

Generation and evaluation of academically similar items and learning opportunities for student practice

AI-assisted learning materials aligned to standards, learning goals, and instructional intent

Innovative item types and new ways of assessing student skills, competencies, reasoning, communication, and applied learning

Support validation and quality approaches for content and interactions used within AI-assisted online learning guides, instructional supports, and learner-facing experiences

Help ensure AI-assisted assessment, feedback, and learning supports work together in educationally meaningful and instructionally aligned ways

Contribute to the advancement of emerging intelligent measurement and learning system capabilities across assessment and instructional contexts

 

Validation, Monitoring & Quality Frameworks

Design and oversee validation studies evaluating the quality, appropriateness, effectiveness, interpretability, and educational defensibility of AI-assisted systems, outputs, and workflows

Define evaluation criteria in collaboration with psychometricians, learning scientists, and content leaders

Establish processes for ongoing monitoring and continuous improvement

Develop frameworks for ongoing monitoring, drift detection, and continuous improvement of AI-assisted systems over time

Support research and validation frameworks grounded in human expertise, empirical evidence, transparency, and continuous improvement

Support technical documentation and evidence generation related to the validity, quality, effectiveness, interpretability, and performance of AI-assisted systems and outputs

 

Research, Documentation & Thought Leadership

Contribute to internal and external thought leadership related to intelligent measurement systems and AI-assisted learning technologies

Support dissemination of research findings, validation approaches, and innovation frameworks through technical reports, presentations, and publications

Help translate complex AI, learning, and measurement concepts into clear guidance for diverse stakeholder groups

Provide guidance to teams on responsible and effective use of AI-assisted systems within assessment and learning workflows

 

Cross-Functional Collaboration

Partner with Technology teams implementing AI infrastructure and intelligent systems

Partner with AI Science teams building models and systems for scoring, reporting, feedback, guidance, and learning supports

Partner with content teams authoring assessment and learning content

Collaborate with research and measurement teams supporting learner models, recommender systems, and AI-guided instructional experiences

Ensure alignment between AI-assisted content generation, downstream scoring systems, instructional guidance systems, and learner-facing experiences

Support coherence across assessment, learning, scoring, feedback, and instructional systems

 

Required Qualifications

Advanced degree in Educational Measurement, Psychometrics, Learning Sciences, Educational Technology, Artificial Intelligence, Educational Data Mining, Learning Analytics, Cognitive Science, Computer Science, Statistics, or a related field

Experience working with assessment item development, learning systems, or educational content workflows

Experience collaborating across cross-functional environments including content, measurement, technology, product, or AI-related teams

Strong understanding of assessment quality principles, including validity, alignment, fairness, accessibility, and instructional appropriateness

Ability to think systematically about AI-assisted workflows across content generation, scoring, feedback, instructional guidance, and learner-facing experiences

Ability to evaluate where AI-assisted approaches can effectively support assessment and learning workflows and where human expertise remains essential

Strong analytical and problem-solving skills with attention to quality, defensibility, and continuous improvement

Ability to design or support rigorous validation and monitoring approaches for AI-assisted systems and workflows

Comfort exploring, testing, and prototyping AI-assisted approaches using emerging tools and technologies

Strong written and verbal communication skills, including the ability to translate complex AI and measurement concepts for diverse audiences

 

Preferred Qualifications

Experience supporting AI-assisted assessment, scoring, feedback, tutoring, instructional guidance, or content-generation systems

Experience designing or evaluating validation studies for AI-assisted educational systems or workflows

Familiarity with psychometric principles and educational measurement concepts

Experience with innovative item types, technology-enhanced items (TEIs), oral or constructed-response assessment, or competency-based assessment approaches

Experience contributing to technical documentation, dissemination materials, presentations, publications, or thought leadership initiatives

Familiarity with recommender systems, learner models, domain models, or AI-guided instructional experiences

Experience identifying workflow risks, edge cases, drift concerns, or unintended consequences within AI-assisted systems

Experience leveraging AI-assisted tools to accelerate workflow exploration, prototyping, innovation, or analysis

Curiosity about emerging approaches to assessing skills, competencies, reasoning, and applied learning through AI-assisted systems

Experience operating in fast-moving innovation environments where systems and workflows continuously evolve

 

Design Philosophy

Human expertise as a core system foundation

Empirical validation over assumed performance

Reuse, transparency, and system coherence

Ongoing monitoring and improvement of AI-assisted workflows

Innovation grounded in educational usefulness and defensible measurement principles

 

Why This Role Matters

This role is foundational to building intelligent assessment and learning systems that educators, partners, and learners can trust. By anchoring AI-assisted systems in rigorous validation, ongoing monitoring, educational usefulness, and human expertise, this leader will help ensure AI-assisted approaches empower assessment and learning without compromising quality, validity, fairness, or instructional integrity. Compensation at Pearson is influenced by a wide array of factors including but not limited to skill set, level of experience, and specific location.

As required by the California, Colorado, Hawaii, Illinois, Maryland, Minnesota, New Jersey, New York State, New York City, Vermont, Washington State, and Washington DC laws, the pay range for this position is as follows: 
The minimum full-time salary range is between $130,000 - $170,000. This position is eligible to participate in an annual incentive program, and information on benefits offered is here. Applications will be accepted through Friday, May 22nd, 2026.

This window may be extended depending on business needs. 
#LI-CH2
 

Pearson is an Equal Opportunity Employer and a member of E-Verify. Employment decisions are based on qualifications, merit and business need. Qualified applicants will receive consideration for employment without regard to race, ethnicity, color, religion, sex, sexual orientation, gender identity, gender expression, age, national origin, protected veteran status, disability status or any other group protected by law. We actively seek qualified candidates who are protected veterans and individuals with disabilities as defined under VEVRAA and Section 503 of the Rehabilitation Act.

If you are an individual with a disability and are unable or limited in your ability to use or access our career site as a result of your disability, you may request reasonable accommodations by emailing TalentExperienceGlobalTeam@grp.pearson.com.