Lead Data Engineer

Key Responsibilities:

• Lead and mentor a team of data engineers, fostering professional growth and continuous learning
• Design, develop, and optimize scalable data pipelines and architectures on Azure.
• Collaborate with product, data science, and business teams to align data solutions with company objectives.
• Provide technical guidance to solve complex data engineering challenges.
• Develop ETL pipelines to integrate data from multiple sources into Azure Synapse Analytics and Data Lake Storage.
• Ensure data quality, integrity, and compliance with governance best practices.
• Implement automation and monitoring to enhance system reliability and performance.
• Optimize query performance, storage solutions, and processing pipelines for large datasets.
• Stay up to date with emerging trends in data engineering and cloud technologies to drive innovation.

Requirements:

• Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field
• 6+ years of experience in data engineering, with at least 1 year in a leadership or senior role
• Proven experience in building scalable and efficient data pipelines, ETL processes, and cloud-based data solutions
• Strong expertise in data warehousing, data lakes, and cloud-based data platforms, with a focus on Microsoft Azure (Azure Data Lake Storage Gen2, Azure Synapse Analytics, Azure Data Share)
• Advanced proficiency in SQL and Python, with experience in other programming languages (e.g., Java, Scala, Spark) being a plus
• Hands-on experience with big data processing frameworks such as Apache Spark (preferably on Azure Databricks) and streaming solutions like Azure Stream Analytics or Kafka
• Experience with data pipeline orchestration tools such as Azure Data Factory, Apache Airflow, or similar
• Proficiency in version control (Git, Azure DevOps) and CI/CD pipeline automation for data workflows
• Strong understanding of data governance, security, and compliance practices
• Excellent leadership, communication, and collaboration skills
• Strong problem-solving skills with a focus on innovation and practical implementation

Nice to have:

• Relevant Azure certifications (e.g., Azure Data Engineer Associate, Azure Solutions Architect)
• Experience with machine learning data pipelines and integrations with Azure Machine Learning
• Familiarity with containerization and orchestration (e.g., Docker, Kubernetes)
• Knowledge of data visualization tools such as Tableau or Power BI
• Previous experience with data quality frameworks and data testing methodologies

Working at Walter Code:

• Enjoying the Membership
• Rewarding Success
• Good Spokesman
• Having Fun
• Sharing Knowledge
• Prioritizing Health
• Cause We Care