Job Description
This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.
Written by AI from the job title and company, not by Avesta Computer Services, so some details may not match this job. The employer's own page has the real description.
Overview
Avesta Computer Services is seeking a highly skilled Senior Data Engineer with a strong proficiency in Python to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing scalable data pipelines as well as managing and transforming large datasets to ensure our data architecture meets the needs of our business and clients. As a senior member of the engineering team, you will collaborate closely with data scientists, analysts, and various stakeholders to deliver data-driven solutions that enhance operational efficiency and support strategic initiatives.
Job Responsibilities
- Design, implement, and maintain data pipelines that support data processing and analytical needs.
- Develop and optimize ETL processes to ensure efficient data ingestion from varied sources.
- Collaborate with data scientists to understand and define data requirements and provide high-quality datasets.
- Monitor and troubleshoot data pipelines, ensuring optimal performance and reliability.
- Work with large sets of structured and unstructured data while applying best practices for data modeling and architecture.
- Implement and enforce data governance policies, ensuring data integrity and security.
- Stay up to date with industry trends and emerging technologies in data engineering.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.
- A minimum of 5 years of professional experience in data engineering or a similar role.
- Proficient in Python, with actionable experience in libraries such as Pandas, NumPy, and PySpark.
- Strong knowledge of SQL and experience working with relational databases (e.g., PostgreSQL, MySQL).
- Experience with data warehousing solutions and data modeling.
- Familiarity with cloud technologies such as AWS, Azure, or Google Cloud Platform.
- Understanding of big data technologies such as Hadoop, Spark, or Kafka is a plus.
Technologies & Tools
This position primarily involves technologies such as Python for data processing, along with SQL for database management. We utilize cloud platforms like AWS for data storage and processing, and employ tools such as Apache Spark for large-scale data processing and Hadoop for distributed storage. Familiarity with data visualization tools (e.g., Tableau, Power BI) may also be beneficial in this role to effectively communicate data insights.
Ideal Candidates
The ideal candidate for the Senior Data Engineer position at Avesta Computer Services is a proactive problem solver who thrives in a fast-paced environment. They should possess strong analytical and critical thinking skills. A collaborative spirit and excellent communication abilities are essential, allowing for effective work across multidisciplinary teams. The ideal candidate should also be committed to continuous learning and adaptability to emerging technologies in the data engineering field.
South Dakota pays at most $575 a week in unemployment, for up to 26 weeks, its agency says. How to file there →
View Similar Jobs
Similar jobs which you may be interested in. Typically using your existing skillset.