Product Data Scientist – AI Evaluation, Quality

Finom

Product Data Scientist working on AI evaluation and quality in a European tech startup. Join the AI Team to drive AI products and technology at Finom.

Posted 7/3/2026full-timeRemote • 🇩🇪 GermanyMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in Python and SQL for end-to-end analysis, with a solid foundation in statistics and experience in quality analytics for ML systems. Capable of translating data insights into actionable decisions while maintaining a strong analytical mindset.

Highest-signal resume keywords

Python ProgrammingSQL ProficiencyStatistical AnalysisQuality Analytics for ML SystemsLLM Application Evaluation

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data AnalysisSamplingHypothesis TestingVariance AnalysisRegression Analysis

Soft Skills

Analytical MindsetCuriosityCommunication

Tools & Technologies

Quality DashboardsLLM Agents

Industry Keywords

Product ContextMachine LearningEvaluation MethodologyFeedback LoopError Rate

Tech Stack

Tools & technologies

PythonSQL

About the role

Key responsibilities & impact

Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts
Harden methodology: judge stability, non-determinism handling
Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake

Requirements

What you’ll need

3+ years in analyst / data scientist roles, at least one in a product context
Python and SQL — you can build an analysis end-to-end
Solid foundation in statistics — sampling, hypothesis testing, variance
Analytical mindset — you start from the business question, not from the tool
Experience in quality analytics for ML systems — ranking, recommendations, classification, etc. (Nice-to-Haves)
Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges) (Nice-to-Haves)
Experience building LLM agents — side projects, toy builds, personal experiments all count (Nice-to-Haves)
We're looking for analysts who are already curious and fluent with AI coding — or genuinely excited to become fluent fast

Benefits

Comp & perks

Make a genuine impact on the product
Join our upward trajectory, and grow with us
We provide the resources and opportunities for continuous personal and professional development
Enjoy the flexibility of traveling and working remotely or in a hybrid model across Europe
Become a stock options holder
Unlock your inner entrepreneur through our Stock Options Program
Receive unwavering support and care
We offer constant support and care to ensure your Finom experience is successful and fulfilling
Immerse yourself in our exclusive Work & Swim Program
Spend one month in a comfortable corporate apartment in Cyprus
It's the ideal opportunity to strike the perfect work-life balance