Full Time

Senior Data Engineer

Big Cloud
$100,000 - $150,000* / year (about $48–$72 an hour)

Job Description

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Big Cloud, so some details may not match this job. The employer's own page has the real description.

Job Overview

We are seeking a highly skilled Senior Data Engineer to join Big Cloud's data team. The ideal candidate will play a pivotal role in designing, building, and maintaining robust data pipelines that facilitate data flow and ensure availability for analysis. This position involves collaborating with data scientists and analysts to implement data solutions that enhance business intelligence and support strategic decision-making. The Senior Data Engineer will leverage cutting-edge technologies and best practices to maintain data integrity and optimize query performance in our cloud-based architecture.



Job Responsibilities

  • Design, implement, and maintain scalable data pipelines for both batch and real-time data processing.
  • Collaborate with cross-functional teams to understand data needs and provide data-driven insights.
  • Develop and manage database systems to ensure high availability and performance.
  • Monitor and troubleshoot data flow issues to ensure optimal performance and data integrity.
  • Automate data collection processes and systems to improve efficiency.
  • Document data engineering processes and contribute to best practices within the team.
  • Stay current with industry trends and emerging technologies in data engineering.


Qualifications

  • Bachelor’s degree in Computer Science, Information Technology, or a related field; a Master’s degree is preferred.
  • 5+ years of experience in data engineering, data architecture, or a similar role.
  • Strong proficiency in programming languages such as Python, Java, or Scala.
  • Extensive experience with SQL and NoSQL databases (e.g., PostgreSQL, MongoDB, Cassandra).
  • Knowledge of data warehousing concepts and experience with ETL processes.
  • Experience with cloud platforms (e.g., AWS, Google Cloud Platform, Azure).
  • Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).


Technologies & Tools

The Senior Data Engineer at Big Cloud will utilize various technologies and tools including AWS services (such as AWS Lambda, Redshift, S3), Apache Spark for big data processing, Apache Kafka for real-time data streaming, and Airflow for orchestrating complex workflows. Familiarity with BI tools like Tableau or Looker may also be beneficial for visualizing data and supporting business intelligence efforts.



Ideal Candidates

Ideal candidates for the Senior Data Engineer position will demonstrate analytical thinking, problem-solving capabilities, and a strong attention to detail. They should possess excellent communication skills to articulate complex data concepts to non-technical stakeholders. A collaborative mindset and willingness to learn new technologies are also important, as is the ability to work under tight deadlines in a fast-paced environment. Candidates who have a passion for leveraging data to drive business strategies and outcomes will thrive in this role.

Free tools for your search, no account needed: a LinkedIn headline writer, a resume checker and more. Free tools →

View Similar Jobs

Matches Jobs

Similar jobs which you may be interested in. Typically using your existing skillset.