Psychometrician

About Us 
We’re building an AI-driven hiring platform designed to transform how organizations evaluate talent. Our product delivers real-time candidate insights, identifies skill gaps, and recommends development pathways - while enabling recruiters to efficiently identify top performers through data-driven decision-making. 

About the Role 
We’re looking for a Psychometrician to join us as a part-time, ongoing consultant. This role combines strategic advisory with hands-on work, ensuring our assessment methodologies are scientifically sound, fair, and compliant with professional and legal standards. 
You’ll collaborate closely with product and engineering to validate and improve our AI-driven assessment systems. 

What You’ll Do 
 

• Evaluate and improve candidate assessment frameworks; 
• Design and run validation studies (criterion-related, construct, content validity); 
• Establish and monitor reliability metrics (e.g., internal consistency, test-retest); 
• Ensure AI-driven scoring aligns with psychometric best practices; 
• Audit algorithms for bias, fairness, and adverse impact; 
• Support calibration and continuous improvement of ranking models; 
• Ensure alignment with EEOC, APA, and SIOP standards; 
• Conduct adverse impact analyses and assess subgroup differences;  
• Recommend and implement bias mitigation strategies; 
• Define and track key measurement KPIs (validity, reliability, fairness); 
• Translate technical findings into clear, actionable insights; 
• Prepare documentation for audits, certification, and external validation. 
  
  Success Metrics 
• Assessment validity and reliability clearly established and documented; 
• Fairness and bias benchmarks defined, tracked, and improved over time; 
• Readiness for certification or third-party validation within agreed timelines. 

Core Requirements 
 

• Master’s or PhD in Psychometrics, Industrial-Organizational Psychology, Statistics, or a related field; 
• 5+ years of experience in assessment validation, ideally in hiring or talent assessment; 
• Strong knowledge of Item Response Theory (IRT) and/or Classical Test Theory (CTT); 
• Experience designing and executing validation studies; 
• Hands-on experience with adverse impact and fairness analysis; 
• Strong analytical and problem-solving skills; 
• Ability to work independently in a remote, flexible environment; 
• Clear communication skills, especially with non-technical stakeholders. 

Nice to Have 
 

• Experience with AI/ML-based hiring or assessment systems;  
• Background in HR tech or talent analytics; 
• Experience supporting audits, certification, or compliance processes;  
• Academic or research affiliation. 

Engagement Details 
 

• Part-time, ongoing consulting role; 
• Fully remote with flexible hours; 
• Mix of strategic advisory and hands-on work.