Big Data Developer

Job Description:

Unison Consulting Pte Ltd is seeking an experienced Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining big data solutions to enable data-driven decision making and analytics capabilities for our clients. You will work with large volumes of structured and unstructured data using cutting-edge technologies and platforms.

Responsibilities:

• Design, develop, and maintain big data solutions using technologies such as Hadoop, HDFS, Spark, and Hive.
• Ingest, process, and analyze large volumes of structured and unstructured data from diverse sources.
• Develop and optimize ETL processes for efficient data extraction, transformation, and loading.
• Implement data models, schemas, and data governance frameworks to ensure data quality and integrity.
• Perform data exploration, profiling, and validation to identify patterns, trends, and anomalies.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
• Implement and optimize data storage and retrieval strategies, including data partitioning and indexing.
• Build and maintain scalable data pipelines and workflows for data processing.
• Perform performance tuning and optimization of big data applications and queries.
• Implement and maintain data security measures to protect sensitive information.
• Maintain technical documentation of big data solutions, processes, and procedures.

Requirements

Skills and Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Big Data Developer with proficiency in technologies such as Hadoop, Spark, HDFS and Hive.
• Strong programming skills in languages such as Java, Scala, or Python.
• Experience with ETL tools and processes for data extraction, transformation, and loading.
• Good understanding of big data concepts, architectures, and technologies.
• Experience with SQL and NoSQL databases.
• Knowledge of data modeling and schema design.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills to work effectively with cross-functional teams.
• Understanding of data security and privacy principles.
• Ability to adapt to new technologies and learn quickly in a fast-paced environment.