Sr Data Engineer (Databricks) -- US

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Zortech Solutions, so some details may not match this job. The employer's own page has the real description.

Job Overview

As a Sr Data Engineer specializing in Databricks at Zortech Solutions, the candidate will play a pivotal role in designing, building, and optimizing data processing pipelines and frameworks that support advanced analytical processes. This position involves collaborating with cross-functional teams to translate data requirements into scalable solutions, ensuring high performance and reliability of data systems. The ideal candidate will leverage their extensive experience with big data technologies to drive the implementation of ETL processes and data transformations, ultimately enhancing our data infrastructure.

Job Responsibilities

• Design and implement data pipelines and architecture using Databricks.
• Collaborate with data scientists to optimize data models for analysis and reporting.
• Develop ETL processes to ingest, cleanse, and transform data from various sources.
• Monitor and troubleshoot data pipeline performance and data accuracy issues.
• Work with stakeholders to gather data requirements and provide technical solutions.
• Document data architectures, processes, and best practices in a clear and concise manner.
• Integrate new data sources and APIs into existing data frameworks.
• Mentor junior data engineers and contribute to knowledge sharing within the team.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• 5+ years of experience in data engineering or related roles, with a focus on big data technologies.
• Proficiency in Apache Spark and Databricks environment.
• Solid understanding of data modeling, ETL concepts, and data warehousing.
• Strong programming skills in Python, SQL, or Scala.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Familiarity with data visualization tools and BI platforms.
• Excellent analytical and problem-solving skills.

Technologies & Tools

The Sr Data Engineer will routinely utilize Databricks for data processing and analysis, along with a suite of Azure or AWS services depending on the project requirements. Familiarity with tools such as Apache Spark, Airflow for workflow management, and various database systems like Kafka, SQL, and NoSQL databases will be crucial to success in this role. Proficiency in data visualization tools like Tableau or Power BI will enhance the quality of insights derived from the data.

Ideal Candidates

The ideal candidates for this Sr Data Engineer position will possess strong analytical skills combined with a passion for data-driven decision-making. They will demonstrate effective communication abilities, enabling them to liaise seamlessly with technical and non-technical stakeholders. Candidates should be proactive, detail-oriented, and adaptable, exhibiting a readiness to embrace new technologies and processes as they evolve. A commitment to mentoring and sharing knowledge within a team environment is also a critical quality for success in this role.