Lead Data Software Engineer

Description

Summary: Seeking a Lead Data Software Engineer to drive scalable data solutions, guide technical delivery with clients, shape robust data pipelines, and set high architecture and quality standards. Highlights: 1. Lead end-to-end technical delivery and shape robust batch/streaming pipelines 2. Develop and maintain codebase for ETL/ELT, batch, and streaming systems 3. Provide technical leadership with a focus on future-proof architecture We are looking for a **Lead Data Software Engineer** to drive scalable data solutions and guide end\-to\-end technical delivery with clients. You will partner directly with stakeholders, shape robust batch and streaming pipelines, and set a high bar for architecture and quality across the program—apply now. **Responsibilities** * Develop and maintain the codebase for ETL and ELT pipelines, large batch/micro batch processing, and streaming systems * Build the infrastructure needed for efficient extraction, transformation, and loading from diverse data sources using ADF, Spark, Kafka, or similar technologies * Identify, design, and implement internal process improvements, including automating manual steps, optimizing data delivery, and reworking infrastructure for better scalability * Design and implement modern data services solutions using Spring Boot, ReactJS, NoSQL, or other UI and API related technologies * Ensure governance of delivery management and production processes according to the selected delivery model * Serve as the single accountable owner for top management and stakeholders on delivery topics, including escalations, upsells, and ramp\-downs * Provide technical leadership across delivery by planning sound, future\-proof architecture and ensuring implementation meets technical quality standards * Write stories and corresponding acceptance criteria to support agile/scrum execution * Coordinate work across multiple disciplines and stakeholders * Ensure projects are delivered in line with defined processes and methodologies, with a strong focus on agile approaches * Establish a continuous delivery risk management strategy that enables proactive decisions and actions throughout the delivery life cycle * Measure and improve delivery productivity, act as a consultant to Data Engineers, and perform production support and deployment activities **Requirements** * 5\+ years of experience in data software engineering * High proficiency with SQL, Spark, and Scala * Deep expertise in Databricks * Solid understanding of ETL/ELT pipelines, batch and streaming data processing * Working knowledge of delivery management, agile methodologies, and technical leadership * Ability to communicate directly with clients in English at B2 (Upper\-Intermediate) level or higher **Nice to have** * Knowledge of Spark Streaming * Familiarity with Kafka * Background in Azure