About the role

You will lead the research supporting Coalescent’s overarching aims: our assessments of metacognition in people, AI systems and their interaction. You will own the research programme, shaping which questions we investigate, how we test them and what our results can support. This is a hands-on scientific leadership role. You will design experiments, develop computational methods and analyse results yourself, while leading a team of research engineers.

You will report to our CEO and work closely with Stephen Fleming, our co-founder and Chief Scientific Officer. You will be accountable for scientific validity, and our CTO will be accountable for technical readiness. Together, you will set the shared standards that let research code move reliably into our assessment platform.

Your work will span AI evaluation, computational modelling and human–AI interaction. You should bring depth in at least one of these areas and enough fluency across the others to guide experimental design, review analyses and make sound decisions about the research programme as a whole.

What you’ll do

Depending on the stage of the research programme, your work will include:

• Set and prioritise the research agenda with Stephen and the founding team, balancing scientific questions, validation needs and the requirements of clients and partners.

• Design experiments with human participants, AI systems and people working with AI, measuring confidence, calibration, belief updating, responses to challenge and reliance on advice.

• Develop and refine computational models of metacognition and belief updating.

• Lead the validation of our measures of humans and AI systems, including whether they capture distinct aspects of behaviour, are reliable across repeated assessments, generalise across tasks and populations, and predict decision outcomes.

• Set research standards for the team, including study design, power analysis, preregistered analysis plans where appropriate, parameter- and model-recovery tests, and full review of code and results.

• Analyse data yourself and review the team’s analyses, assessing whether findings reflect the intended construct, differences in underlying performance, sampling variability, task ambiguity, scoring errors or implementation bugs.

• Lead, mentor and help hire research engineers, turning open-ended questions into well-scoped projects and developing each person’s strengths.

• Work with our CTO and software engineers to define how tasks, scoring methods and models are versioned, tested and integrated into the assessment platform.

• Decide what our results can and cannot support, and communicate findings, uncertainty and limitations through research papers, technical reports and discussions with clients, partners and investors.

You may be a good fit if you

• Have a PhD or equivalent research experience in cognitive science, psychology, neuroscience, statistics, machine learning or a related field.

• Have taken a substantial experimental or computational research project from initial question through analysis and interpretation, with evidence of your contribution through publications, technical reports or applied research outcomes.

• Have a strong foundation in probability, statistics and experimental design, and can reason carefully about noisy data, competing explanations and what a result does and does not establish.

• Have substantial hands-on experience in computational modelling, LLM evaluation and/or experiments with human participants, with enough methodological breadth to guide work across the other areas.

• Have strong scientific programming skills in Python and can write, review and extend readable, tested research code.

• Have led research projects or supervised others and can set direction while leaving people room to own their work.

• Can make prioritisation decisions under uncertainty and explain them clearly to colleagues without a research background.

• Use AI tools fluently in your own work, with the judgement to check what they produce.

• Communicate technical ideas clearly, work well with colleagues from different disciplines and are comfortable changing direction when the evidence warrants it.

Strong candidates may also have

We do not expect every candidate to have all of these strengths:

• Measurement. Experience in signal detection theory, metacognition, judgement and decision-making and/or psychometrics.

• Computational modelling. Experience with hierarchical Bayesian models, parameter estimation, identifiability and recovery, model comparison and diagnostic checks, using tools such as Stan, PyMC or NumPyro.

• AI evaluation. Experience designing evaluations of language models, eliciting verbalised confidence or deriving confidence measures from token probabilities where available, and investigating prompt sensitivity, sampling variability and differences across model configurations.

• Human–AI interaction. Experience designing controlled behavioural studies of advice-taking or human–AI decision-making, and running online studies with attention to recruitment, data quality, ethics and data protection.

• Research leadership. A record of setting direction for a team or programme, building research processes from scratch, or turning research into products, standards or applied decisions.

The opportunity

You will build our research function from the ground up, shaping its agenda, standards and team as part of our early leadership.

What we offer

• A competitive salary and meaningful equity through share options.

• Hybrid working from our office in Bloomsbury, central London, with 4 days a week together in person.

• 28 days’ holiday plus bank holidays, and a workplace pension.

• A learning budget and time for conferences.

• The chance to shape a new company and develop a new approach to measuring human and AI metacognition as part of the founding team.