Product Data Scientist – AI Evaluation, Quality

Finom

Product Data Scientist focused on AI Evaluation & Quality at Finom, developing AI products and technologies for the evolving financial landscape.

Posted 7/3/2026full-timeRemote • 🇵🇱 PolandMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in Python and SQL for end-to-end analysis, with a solid foundation in statistics and a strong analytical mindset. Proven experience in evaluating LLM applications and building LLM agents, with a focus on translating data into actionable insights.

Highest-signal resume keywords

Python ProgrammingSQL ProficiencyStatistical AnalysisLLM Application EvaluationAnalytical Mindset

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Python ProgrammingSQL ProficiencyStatistical AnalysisSamplingHypothesis TestingVariance AnalysisLLM Application EvaluationData AnalysisRegression AnalysisError Rate Analysis

Soft Skills

CuriosityAnalytical ThinkingClear CommunicationDecision-Making

Tools & Technologies

AI-Assisted CodingQuality DashboardsData Visualization Tools

Industry Keywords

Data ScienceProduct ContextFeedback LoopNon-Determinism HandlingMetrics Evaluation

Tech Stack

Tools & technologies

PythonSQL

About the role

Key responsibilities & impact

Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts
Harden methodology: judge stability, non-determinism handling
Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake

Requirements

What you’ll need

Python and SQL – you can build an analysis end-to-end
Solid foundation in statistics – sampling, hypothesis testing, variance, understanding what a noisy metric is
Analytical mindset – you start from the business question, not from the tool
3+ years in analyst / data scientist roles, at least one in a product context
Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges)
Experience building LLM agents – side projects, toy builds, personal experiments all count
AI-assisted coding is our default authoring environment
Curious and fluent with AI coding, or genuinely excited to become fluent fast
Care about what you ship and how clearly you think

Benefits

Comp & perks

Make a genuine impact on the product
Join our upward trajectory and grow with us
Work in the EU
Enjoy the flexibility of traveling and working remotely or in a hybrid model across Europe
Become a stock options holder
Unlock your inner entrepreneur through our Stock Options Program
Receive unwavering support and care
Constant support and care to ensure your experience is successful and fulfilling
Immerse yourself in our exclusive Work & Swim Program
Spend one month in a comfortable corporate apartment in enchanting Cyprus
Ideal opportunity for work-life balance while enjoying Mediterranean views
Equal Opportunity Employer valuing diversity