Spark/Scala Developer

Job Title: Spark/Scala Developer

Job Summary:

Key Responsibilities:

• Big Data Expertise: Apply advanced knowledge of big data technologies such as Spark, Hadoop, or Elasticsearch. Demonstrate experience in either implementation or data science.
• Scala and SQL Development: Write efficient code in Scala and advanced SQL queries. Utilize Hive and Parquet for data processing.
• GCP Data Lake: Leverage experience working with Google Cloud Platform (GCP) data lakes.
• Agile Environment: Work effectively within an Agile framework, contributing to iterative development and continuous improvement.
• Coding Skills: Use your strong coding background in Java, Python, or Scala to develop and maintain robust data solutions.
• Testing: Utilize testing libraries such as Scala Test or equivalent for other languages. Understand and explain various test types, including unit and integration tests, and automate testing processes.
• Data Pipelines: Build and maintain data processing pipelines for production environments, including traditional ETL and analytics pipelines.

Desired Skills:

• Expertise in Scala and Spark with practical experience in big data technologies.
• Advanced SQL skills and experience with data formats such as Parquet.
• Experience with GCP data lakes and Agile methodologies.
• Proficiency in at least one major programming language (Java, Python, Scala).
• Familiarity with testing libraries and concepts, including unit and integration testing.
• Experience in developing and managing data pipelines for production environments.

Qualifications:

• Bachelor’s Degree in Computer Science or a related field.