Ipsos
Sr. Data Engineer
Job Description
This role is for one of the Weekday’s clients
Salary range: Rs 1500000 – Rs 4000000 (ie INR 15-40 LPA)
Min Experience: 5 years
Location: Bangalore
JobType: full-time
We are looking for a Senior Data Engineer to design and implement scalable data management services for data trust, compliance, access, and metadata management. In this role, you will be responsible for developing robust data pipelines, integrating Big Data tools and frameworks, and optimizing ELT processes to ensure seamless data processing and infrastructure scalability.
Requirements
Key Responsibilities:
- Design and implement scalable and configurable data management services.
- Develop ELT processes for efficient data transformation and integration.
- Select and integrate Big Data tools and frameworks to enhance capabilities.
- Monitor and optimize data performance, recommending infrastructure improvements.
- Build data platforms and services for real-time and batch processing of large datasets.
- Take end-to-end ownership of core product functionality and key technical initiatives.
Requirements:
- 5+ years of experience in data engineering within a fast-paced software-driven environment.
- Strong expertise in designing and developing scalable Data Warehouses and Data Pipelines.
- Proficiency in SQL, Java, and Scala.
- Deep understanding of Apache Spark, including Spark tuning, RDDs, and data frames.
- Hands-on experience in distributed environments using Kafka, Spark, Hive, Hadoop, etc.
- Familiarity with NoSQL databases and experience working with file formats like Parquet, Avro, etc.
- Strong understanding of distributed database systems architecture.
- Knowledge of engineering best practices and design principles.
Skills:
- Apache Spark
- Java / Scala
- Data Engineering
- ETL
- Distributed Systems
