Federal Reserve System
Sr. Data Engineer
Job Description
Company
Federal Reserve Bank of Dallas
We are dedicated to serving the public by promoting a strong financial system and a healthy economy for all. These efforts take a team of dedicated individuals doing many different jobs. Together we’re creating a workplace where talented people can thrive, and we welcome your unique background and perspective to help present the best possible solutions for our partners.
The Data Engineer responsible for designing, developing, and maintaining data pipelines in support of data engineering and data management activities. You must be passionate about data engineering and data quality with a solid background in AWS, Databricks, Python, Trino/Starburst and SQL.
Location: #LI-Hybrid
About the Role:
The Sr. Data Engineer develops data solutions with moderate to high complexity that also have moderate to high business criticality. The Engineer also develops data set processes; works towards improving data, efficiency, and quality; uses large datasets to address business issues; provides full software development lifecycle support; manages design, security, standards, data quality and compliance processes; works independently with guidance only in the most complex situations. The engineer may lead functional teams or projects, has in-depth domain knowledge of the relevant industry. Other responsibilities include requirement analysis, design, code, test, debug, document and maintain data and analytics solutions.
You will report to the Technology Solutions Manager in Dallas who manages an Engineering team consisting of 10 team members with assignments in Dallas and New Yor City. We are a collaborative, passionate team delivering sustainable software and data-driven solutions that meet the needs of our customers across the Federal Reserve System.
You Will:
- Design, develop, monitor, and maintain data pipelines in an AWS Gov Cloud ecosystem with Databricks, Delta Lake and Starburst as the underlying platforms
- Collaborate with cross-functional teams to understand data needs and translate them into effective data pipeline solutions.
- Develop, optimize, and maintain ETL processes to facilitate the smooth and accurate movement of data across systems.
- Establish data quality checks and ensure data integrity and accuracy throughout the data lifecycle.
- Implement and enforce data governance policies and procedures.
- Optimize data processing and query performance for large-scale datasets within AWS and Databricks environments.
- Document data engineering processes, architecture, and configurations.
- Troubleshooting and debugging data-related issues on the AWS Databricks platform.
- Having the knowledge about Optimizing AWS Databricks Streaming jobs.
- Integrating Databricks with other data storage and processing systems.
You Have:
- Bachelor’s or master’s degree in computer science, Information Technology, or a related field.
- Minimum of 5 years of experience in data engineering roles, with a focus on AWS and Databricks.
- Highly proficient with Apache Spark, Starburst/Trino, Python, PySpark and SQL
- Hands-on experience in Gitlab with CI/CD.
- Hands-on experience in AWS Services like S3, RDS, Lambda, SQS, SNS, MSK is required.
- Have a deep understanding of AWS Cloud platforms and data engineering technologies.
- Strong SQL skills to perform data analysis and understanding of source data.
- Familiarity with NoSQL databases and JSON formats.
- Good to have experience with ETL tools.
- Experience with data modeling, schema design, and database optimization.
- Experience with data pipeline orchestration tools.
- Ability to troubleshoot complex data issues and implement effective solutions.
- Strong communication and interpersonal skills.
- Ability to work collaboratively in a team-oriented environment.
- Proactive in staying updated with industry trends and emerging technologies in data engineering.
Our Benefits:
Our total rewards program offers benefits that are the best fit for you at every stage of your career:
- Comprehensive healthcare options (Medical, Dental, and Vision)
- 401K match, and a fully funded pension plan
- Paid vacation, holidays, and volunteer hours; flexible work environment
- Generously subsidized public transportation and free parking
- Annual tuition reimbursement
- Professional development programs, training, and conferences
- And more…
Notes:
This position may be filled at various levels based on candidate’s qualifications as determined by the department.
This position requires access to confidential supervisory information and/or FOMC information, which is limited to “Protected Individuals” as defined in the U.S. federal immigration law. Protected Individuals include, but are not limited to, U.S. citizens, U.S. nationals, and U.S. permanent residents who either are not yet eligible to apply for naturalization or who have applied for naturalization within the requisite timeframe. Candidates who are permanent residents may be eligible for the information access required for this position if they sign a declaration of intent to become a U.S. citizen and pursue a path to citizenship and meet other eligibility requirements. In addition, all candidates must undergo an enhanced background check and comply with all applicable information handling rules.
This role allows for the employee to work in a hybrid manner with some work conducted onsite at the Federal Reserve work location and some work conducted remotely. Specific schedule details will be discussed during the recruitment process.
The Federal Reserve Bank of Dallas is proud to be an Equal Opportunity Employer that believes in the diversity of our people, ideas and experiences, and we are committed to building an inclusive culture that represents the communities we serve.
If you need assistance or an accommodation due to a disability, please notify your Talent Acquisition Consultant.
Full Time / Part Time
Full time
Regular / Temporary
Regular
Job Exempt (Yes / No)
Yes
Job Category
Work Shift
First (United States of America)
The Federal Reserve Banks believe that diversity and inclusion among our employees is critical to our success as an organization, and we seek to recruit, develop and retain the most talented people from a diverse candidate pool. The Federal Reserve Banks are committed to equal employment opportunity for employees and job applicants in compliance with applicable law and to an environment where employees are valued for their differences.
