Full-stack Data Engineer (Python, AWS, Airflow)

Xenon7 specializes in integrating Artificial Intelligence (AI) into businesses, offering services that leverage AI to optimize operations, enhance customer experiences, and drive innovation. We emphasize AI as a fundamental tool for business growth and competitive advantage, catering to various industries with a focus on data-driven decision-making and operational efficiency.
We are seeking a versatile Full-Stack Data Engineer with expertise in Python programming, AWS services, DevOps practices, and a solid understanding of both software and data engineering principles. This role is pivotal in building and maintaining scalable data infrastructure, automating data processes, and contributing to the development of software solutions that manage and analyze large data sets.

Responsibilities:

• Design, implement, and manage microservices architecture using AWS cloud services.
• Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG.
• Implement DevOps strategies for continuous integration and continuous deployment (CI/CD) to improve software development processes.
• Write high-quality, maintainable code in Python to support data operations.
• Integrate various internal and external data sources (including SharePoint, One Drive, One Note, etc.) to create a unified data platform.
• Build tools for document ingestion, dynamic file previewing, and tabular data extraction.
• Manage graph database systems and support large text corpus management.
• Implement security controls to maintain data integrity and compliance.
• Use Large Language Models for keyword extraction and data analysis to gain insights.

Requirements

• Proven experience with Python programming and AWS services (EC2, RDS, S3, Lambda, etc.).
• Proficient in managing and orchestrating workflows with Airflow.
• Solid understanding of DevOps principles, including experience with CI/CD tools and practices.
• Demonstrated experience with data engineering and familiarity with graph database management.
• Expertise in data processing, corpus management, and keyword extraction.
• Strong problem-solving abilities, attention to detail, and organizational skills.
• Excellent communication and teamwork skills to collaborate with cross-functional teams.
• Minimum overlap of 5 hours with EST time zone during working hours.