Senior Engineer – GenAI Quality Assurance

Johnson & Johnson

Senior Engineer responsible for designing, testing, and improving GenAI applications at Johnson & Johnson Innovative Medicine. Roles include collaboration with data scientists and deployment support.

Posted 6/30/2026full-timeMadrid • 🇪🇸 SpainSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing QA strategies for GenAI systems, with advanced skills in Python and experience in deploying AI/ML products. Proficient in establishing testing standards, conducting code reviews, and collaborating with cross-functional teams to ensure quality and reliability.

Highest-signal resume keywords

Advanced Python SkillsExperience with LLM-Based SystemsProven Ability to Implement Test FrameworksFamiliarity with RAG & Graph FrameworksExperience with Cloud Platforms (AWS, Azure, GCP)

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Software EngineeringQA OwnershipAPI TestingData ValidationData Pipeline TestingTest Framework ImplementationRegression TestingSynthetic Test Data GenerationDebugging PipelinesVersion Control (Git)

Soft Skills

Proactive Problem-SolverStrong Team PlayerComfortable Working IndependentlyAdaptability to Fast-Moving Environments

Tools & Technologies

LLM APIsAWS BedrockAzure FoundryGCP (Gemini)Cloud Platforms

Industry Keywords

GenAI SystemsAI/ML ProductsContinuous IntegrationDeployment WorkflowsSafety and Bias in GenAI

Tech Stack

Tools & technologies

AWSAzureCloudGoogle Cloud PlatformPython

About the role

Key responsibilities & impact

Design and implement automated and manual QA strategies for GenAI systems
Build regression tests, generate synthetic test data, build test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety
Validate prompt logic, model behavior and evaluation metrics
Execute tests and debug pipelines
Support deployment of GenAI products
Lead system design reviews, advocating best practices for architecture and reliability of GenAI applications
Collaborate closely with data scientists and developers to embed testing and quality checkpoints
Establish testing standards and workflows for continuous integration and deployment
Conduct code reviews with a focus on design, testability, and maintainability
Document system design decisions, test cases, and best practices
Close collaboration with domain experts and stakeholders to validate developed solutions

Requirements

What you’ll need

Strong software engineering background with QA ownership
Familiarity with RAG & Graph frameworks
3+ years of experience as a software engineer (preferably 5), or QA engineer in production systems, preferably including at least 1 year experience in GenAI systems
Advanced Python skills
Experience working with LLM-based systems in real environments
Hands-on experience with: LLM APIs, AWS Bedrock (Anthropic & OSS), Azure Foundry (OpenAI), GCP (Gemini), Cloud platforms (AWS, Azure, GCP)
Proven ability to implement test frameworks for GenAI applications
Knowledge of API testing, data validation, and data pipeline testing
Experience with version control (Git)
Experience with building and deploying AI/ML products through the full product lifecycle
Good understanding of scientific computing and ML frameworks
Solid understanding of safety, bias, and failure modes of GenAI systems
Comfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued
Proactive problem-solver who can anticipate needs and find solutions
Comfortable working independently, but also a strong team player.

Benefits

Comp & perks

Inclusive work environment
Opportunities to collaborate with domain experts and stakeholders