Senior Data QA Engineer ETL Testing, Data Validation & Automation

Role: Senior Quality Engineer - ETL Testing, Data Quality Testing, and Test Automation

7+ years

Summary:

We are seeking a highly technical QE Tester to lead quality assurance for our data-driven ecosystem

1. Core Responsibilities

End-to-End Data Validation: Lead testing for complex data pipelines, ensuring integrity from source systems through the Medallion Architecture (Bronze, Silver, Gold layers).Advanced SQL Testing: Write sophisticated queries to validate complex transformations, including advanced joins, window functions, and SCD (SCD1 vs SCD2) logic.LLM & Agentic Evaluation: Develop and execute frameworks to detect LLM hallucinations, bias, and non-deterministic output inconsistencies.Performance & Governance: Conduct stress testing on AI agents, monitor token costs, and validate guardrails to ensure safe and cost-effective AI operations.Collaborative Quality: Partner with Data Engineers and AI Researchers to define "Golden Datasets" and establish business-rule validation criteria.

2. Technical Requirements

Data Engineering Validation:

Advanced SQL: Expertise in complex analytical queries, window functions, and validating Slowly Changing Dimensions (SCD).Warehousing Fundamentals: Deep understanding of Fact vs Dimension validation and Source-to-Target (STT) reconciliation.Pipeline Integrity: Proven experience with Incremental vs Full Load testing, row count reconciliation, and data completeness checks.Data Quality (DQ): Ability to design automated DQ checks, data profiling, and validation of business-specific transformation rules.API Testing: Proficient in validating RESTful APIs and alternative integration patterns.LLM & Generative AI Validation:

Prompt Engineering: Validating system prompts, user prompts, and few-shot learning impacts on model behavior.Hallucination & Bias Detection: Implementing techniques to identify factual inaccuracies and social biases in model responses.Model Parameters: Testing the impact of Temperature, Top-P, and Context Window limits on output quality.Agentic Testing: Validating multi-step reasoning, human-in-the-loop workflows, and stress testing autonomous AI agents.Benchmarking: Evaluating model performance against Golden Datasets using metrics like BLEU, ROUGE, or semantic similarity.

3. Preferred Qualifications

Experience: 6+ years in Data Quality, ETL Testing, or AI Assurance.Tools: Familiarity with dbt, Great Expectations, or LLM evaluation frameworks (LangChain,N8N,).Cloud Environments: Hands-on experience with modern data stacks on Azure.Mindset: A "non-deterministic" testing approachcomfortable working with distributions and probabilistic outputs rather than just True/False outcomes. .