Sr. Data Engineer - Remote

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Core & Main, so some details may not match this job. The employer's own page has the real description.

Job Overview

The Senior Data Engineer at Core & Main will play a pivotal role in designing, developing, and managing robust data pipelines and architectures that ensure data integrity, accessibility, and security. This remote position requires an individual who is proficient in ETL processes, data modeling, and performance tuning to support the analytics and business intelligence teams in delivering insightful solutions. The Senior Data Engineer will work collaboratively with cross-functional teams to optimize data workflows and facilitate the integration of diverse data sources, ultimately enhancing decision-making processes across the organization.

Job Responsibilities

• Design and implement scalable and resilient data architectures and pipelines to support analytics and reporting needs.
• Manage the end-to-end data lifecycle, including data collection, storage, processing, and delivery to end-users.
• Collaborate with data scientists and analysts to understand their data requirements and deliver appropriate solutions.
• Perform data modeling and ETL processes to enhance data transformations, ensuring data quality and accuracy.
• Optimize database performance and troubleshoot issues related to data access, latency, and usability.
• Develop and maintain documentation for data models, architectures, and data processes.
• Stay current with industry trends and advancements in data engineering technologies to recommend improvements and innovations.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field; Master’s degree preferred.
• Minimum of 5 years of experience in data engineering or a related field.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with data processing frameworks like Apache Spark, Apache Hadoop, or similar technologies.
• Solid understanding of SQL and database management systems, including PostgreSQL, MySQL, or similar.
• Expertise in cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Strong analytical and problem-solving skills with attention to detail.

Technologies & Tools

The Senior Data Engineer will engage with a variety of technologies and tools crucial for data engineering tasks. This includes but is not limited to data orchestration tools like Apache Airflow, cloud-based data storage solutions (e.g., Amazon S3, Azure Blob Storage), and relational/non-relational databases. Familiarity with BI tools such as Tableau or Power BI for data visualization will also be essential. The candidate should have hands-on experience with CI/CD practices and tools to enhance DataOps efficiency.

Ideal Candidates

Ideal candidates for the Senior Data Engineer role will possess a passion for data and a strong commitment to delivering high-quality results. They will demonstrate a collaborative spirit, working well within diverse teams and valuing input from various stakeholders. Strong communication skills are vital for effectively translating complex data processes into understandable concepts. Furthermore, candidates should be proactive in identifying and implementing process improvements while maintaining a keen focus on data security and compliance.