Unstructured Data Engineering Lead

Job Description:

Unstructured Data Engineering Lead

Collaborate with Innovative 3Mers Around the World

Choosing where to start and grow your career has a major impact on your professional and personal life, so it’s equally important you know that the company that you choose to work at, and its leaders, will support and guide you. With a diversity of people, global locations, technologies and products, 3M is a place where you can collaborate with other curious, creative 3Mers.

This position provides an opportunity to transition from other private, public, government or military experience to a 3M career.

The Impact You’ll Make in this Role
3M is looking for a skilled Unstructured Data Engineering Lead to join our team. As a key member of our organization, you will be responsible for leading the development of pipelines, preprocessing unstructured data, eliminating duplicate data and text noise, chunking data, and generating vector embeddings. In addition to these key capabilities, the candidate should possess strong Python programming skills, expertise in cloud engineering, and experience with open source software to drive innovation and efficiency in handling unstructured data. The ideal candidate will have a strong background in data engineering, particularly in handling unstructured data, and possess the capabilities to drive innovation and efficiency in data preprocessing tasks.

As an Unstructured Data Engineering Lead, you will have the opportunity to tap into your curiosity and collaborate with some of the most innovative and diverse people around the world. Here, you will make an impact by:

• Leading the development of pipelines for preprocessing unstructured data, eliminating duplicate data and text noise, chunking data, and generating vector embeddings.

• Implementing efficient and scalable solutions using Python programming skills and cloud engineering expertise to handle unstructured data effectively.

• Determining the best approaches and techniques for data preprocessing tasks, driving innovation and efficiency in handling unstructured data.

• Supporting the team by providing guidance, mentorship, and technical expertise in data engineering, particularly in the context of unstructured data.

By taking on this role, you will play a crucial part in driving the success of our organization’s unstructured data initiatives and contribute to the advancement of data engineering practices.

Key Responsibilities:

• Lead the design and development of data pipelines for processing and analyzing unstructured data.
• Preprocess unstructured data to eliminate duplicate data, text noise, and other irrelevant information.
• Chunk large volumes of unstructured data into manageable segments for efficient processing.
• Develop and implement algorithms and techniques for generating vector embeddings from unstructured data.
• Collaborate with data scientists, machine learning engineers, and domain experts to define data requirements and objectives.
• Optimize data preprocessing and embedding generation pipelines for scalability and performance.
• Leverage strong Python programming skills to develop efficient and reliable data engineering solutions.
• Utilize cloud engineering expertise to design and implement scalable and cost-effective data processing architectures.
• Explore and leverage open source software and tools to drive innovation and efficiency in handling unstructured data.
• Stay up-to-date with the latest advancements in data engineering and unstructured data processing techniques.
• Mentor and guide junior engineers, fostering a collaborative and innovative team environment.

Your Skills and Expertise 
To set you up for success in this role from day one, 3M requires (at a minimum) the following qualifications:

• Bachelor’s degree or higher (completed and verified prior to start) in Computer Science or Engineering
• Three (3) years of experience in unstructured data engineering at a large manufacturing company in a private, public, government or military environment 
• Three (3) years of experience as a data engineer, with expertise in handling unstructured data.

Additional qualifications that could help you succeed even further in this role include:

• Master’s degree in Computer Science, Engineering, or related field from an accredited institution
• Strong understanding of data engineering concepts and best practices.
• Proficiency in Python programming, with the ability to develop efficient and reliable data engineering solutions.
• Expertise in cloud engineering, with experience in designing and implementing scalable and cost-effective data processing architectures.
• Familiarity with open source software and tools for data engineering and unstructured data processing.
• Experience with data preprocessing techniques, including duplicate elimination, noise removal, and chunking.
• Knowledge of algorithms and methods for generating vector embeddings from unstructured data.
• Knowledge of distributed computing frameworks, such as Apache Spark or Hadoop.
• Strong analytical and problem-solving skills, with the ability to optimize data processing pipelines.
• Excellent communication and collaboration abilities, with the capacity to work effectively in cross-functional teams.
• Ability to adapt to a fast-paced and dynamic environment

Work location:

• Hybrid Eligible (Job Duties allow for some remote work but require travel to Maplewood, MN at least 2 days per week)

#LI-hybrid

Travel: May include up to 10% International

Relocation Assistance: May be authorized

Must be legally authorized to work in country of employment without sponsorship for employment visa status (e.g., H1B status).

Supporting Your Well-being 

3M offers many programs to help you live your best life – both physically and financially. To ensure competitive pay and benefits, 3M regularly benchmarks with other companies that are comparable in size and scope. 

Chat with Max

For assistance with searching through our current job openings or for more information about all things 3M, visit Max, our virtual recruiting assistant on 3M.com/careers.

Applicable to US Applicants Only:The expected compensation range for this position is $177,961 – $217,508, which includes base pay plus variable incentive pay, if eligible. This range represents a good faith estimate for this position. The specific compensation offered to a candidate may vary based on factors including, but not limited to, the candidate’s relevant knowledge, training, skills, work location, and/or experience. In addition, this position may be eligible for a range of benefits (e.g., Medical, Dental & Vision, Health Savings Accounts, Health Care & Dependent Care Flexible Spending Accounts, Disability Benefits, Life Insurance, Voluntary Benefits, Paid Absences and Retirement Benefits, etc.). Additional information is available at: https://www.3m.com/3M/en_US/careers-us/working-at-3m/benefits/.

Learn more about 3M’s creative solutions to the world’s problems at www.3M.com or on Twitter @3M.

Responsibilities of this position include that corporate policies, procedures and security standards are complied with while performing assigned duties.

Our approach to flexibility is called Work Your Way, which puts employees first and drives well-being in ways that enable 3M’s business and performance goals. You have flexibility in where and when work gets done. It all depends on where and when you can do your best work.

Pay & Benefits Overview: https://www.3m.com/3M/en_US/careers-us/working-at-3m/benefits/

3M is an equal opportunity employer.  3M  will not discriminate against any applicant for employment on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, or veteran status.

Please note: your application may not be considered if you do not provide your education and work history, either by: 1) uploading a resume, or 2) entering the information into the application fields directly.

3M Global Terms of Use and Privacy Statement

Carefully read these Terms of Use before using this website. Your access to and use of this website and application for a job at 3M are conditioned on your acceptance and compliance with these terms.

Please access the linked document by clicking here, select the country where you are applying for employment, and review. Before submitting your application you will be asked to confirm your agreement with the terms.