Pesto Tech

Lead Data Engineer

1 June 2024
Full Time
Apply Now
Deadline date:
£45000 - £84000 / year

Job Description

Position: Lead Data EngineerLocation: Gurgaon (Hybrid), Bangalore (Hybrid)Accepted Notice Period: Immediate – 30 daysYears of Experience: 6 – 8 YearsOpportunity Type: Full time
**Note: This position is with one of our hiring partners

Skills Required

  • Extensive experience with Hadoop ecosystem technologies such as Hadoop, Hive, and Pyspark/Spark
  • Proficiency in programming languages like Python, Java, and Unix scripting for data engineering tasks
  • Strong understanding of data engineering principles, best practices, and design patterns
  • Hands-on experience in designing, building, and optimizing large-scale data pipelines and ETL processes
  • Ability to lead and mentor a team of data engineers, providing technical guidance and expertise
  • Knowledge of cloud platforms like AWS/GCP/Azure is highly desirable
  • Excellent problem-solving skills and ability to troubleshoot complex data issues
  • Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams

Responsibilities

  • Lead the design, development, and implementation of data solutions using Hadoop, Hive, and Spark
  • Architect and build scalable and reliable data pipelines to support various data processing and analytics requirements
  • Provide technical leadership and mentorship to the data engineering team, ensuring best practices and standards are followed
  • Collaborate with stakeholders to gather and analyze requirements, and translate them into technical specifications
  • Optimize and tune data workflows for performance, scalability, and efficiency
  • Ensure data quality, integrity, and security across all data processes and systems
  • Stay updated with the latest trends and technologies in the field of data engineering, and drive continuous improvement initiatives
  • Drive innovation and contribute to the development of new data products and services

About Pesto Tech:Pesto is where software developers go to build their career path for the next 5 years. We don’t just offer jobs; we provide unparalleled opportunities for your growth and success in the dynamic landscape of Tech Jobs.

Related Jobs