Lead Data Software Engineer

Description

Summary: We are seeking a Lead Data Software Engineer to shape resilient, scalable data solutions, leading design and upkeep of ETL/ELT pipelines and guiding technical quality. Highlights: 1. Lead design and upkeep of ETL/ELT pipelines and batch/streaming platforms. 2. Architect and maintain codebase for data processing systems. 3. Provide technical leadership for delivery and ensure quality standards. We are seeking a **Lead Data Software Engineer** to partner closely with our client and shape resilient, scalable data solutions end to end. You will lead the design and upkeep of ETL/ELT pipelines plus batch and streaming platforms, while guiding delivery practices and technical quality across the lifecycle—apply now. **Responsibilities** * Architect and maintain the codebase for ETL and ELT pipelines, large batch/micro batch processing, and streaming systems * Build the infrastructure needed for efficient data extraction, transformation, and loading from many sources using ADF, Spark, Kafka, or similar technologies * Identify, design, and implement internal process improvements, including automation of manual work, optimization of data delivery, and infrastructure redesign for scalability * Design and implement innovative data services solutions using Spring Boot, ReactJS, NoSQL, or other UI and API related technologies * Ensure governance of processes across delivery management and production in line with the selected delivery model * Own delivery\-related topics for top management and stakeholders, including escalations, upsells, and ramp\-downs * Provide technical leadership for delivery by planning sound, future\-proof architecture and ensuring implementation meets technical quality standards * Write stories and the associated acceptance criteria within an agile/scrum workflow * Coordinate across multiple disciplines and stakeholders * Ensure projects follow client processes and methodologies with a strong focus on agile approaches * Establish a continuous delivery risk management strategy to enable proactive decisions and actions throughout the delivery life cycle * Measure and improve delivery productivity, consult Data Engineers, and perform production support and deployment activities **Requirements** * 5\+ years of experience in SQL, Spark, and Scala * Deep expertise with Databricks * Solid background in designing and delivering ETL/ELT pipelines and large batch/micro batch processing systems * Hands\-on familiarity with data services solutions using Spring Boot, ReactJS, and NoSQL or other UI and API related technologies * Good understanding of agile/scrum workflow and different delivery models * Proven ability to provide technical leadership and act as a consultant to Data Engineers * English proficiency at B2 (Upper\-Intermediate) level or higher for direct communication with the client **Nice to have** * Experience with Spark Streaming and Kafka * Familiarity with Azure **We offer** * International projects with top brands * Work with global teams of highly skilled, diverse peers * Healthcare benefits * Employee financial programs * Paid time off and sick leave * Upskilling, reskilling and certification courses * Unlimited access to the LinkedIn Learning library and 22,000\+ courses * Global career opportunities * Volunteer and community involvement opportunities * EPAM Employee Groups * Award\-winning culture recognized by Glassdoor, Newsweek and LinkedIn