Software Engineer - Data Infrastructure

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Jobs via Dice, so some details may not match this job. The employer's own page has the real description.

Overview

The Software Engineer - Data Infrastructure is responsible for designing, building, and optimizing data processing pipelines and infrastructure to support the organization’s analytical and operational needs. This role involves collaborating with data scientists, analysts, and other engineers to ensure efficient handling and processing of large datasets. The engineer will work on developing scalable solutions while maintaining data integrity and optimizing performance, all while adhering to industry best practices.

Job Responsibilities

• Design and implement robust data architecture and data processing pipelines.
• Collaborate with cross-functional teams to define data requirements and ensure the successful integration of services.
• Optimize existing data workflows and ensure efficient processing and storage mechanisms.
• Monitor system performance, troubleshoot issues, and implement fixes as necessary.
• Ensure data quality and integrity through automated testing and monitoring solutions.
• Develop documentation for data infrastructure and best practices to facilitate knowledge sharing.
• Stay current with emerging trends and technologies in the data engineering field.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Minimum of 3 years of experience in software engineering with a focus on data infrastructure.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with data management technologies such as SQL, NoSQL, and data warehousing solutions.
• Knowledge of big data frameworks (e.g., Hadoop, Spark) and cloud technologies (AWS, Azure).
• Strong analytical and problem-solving skills.
• Excellent communication skills and ability to work collaboratively in a team environment.

Technologies & Tools

This position utilizes a range of technologies and tools including cloud platforms (AWS, Azure), data processing frameworks (Apache Spark, Apache Hadoop), SQL and NoSQL databases (e.g., MySQL, MongoDB), and data visualization tools (Tableau, Power BI). Familiarity with CI/CD tools and version control systems (Git) is also essential to streamline development processes and enhance collaboration.

Ideal Candidates

Ideal candidates possess a strong passion for data and technology, coupled with a continuous desire to innovate and improve existing systems. They should exhibit critical thinking and adaptability in dynamic environments, demonstrate excellent teamwork abilities, and showcase a commitment to delivering high-quality solutions. Strong communication skills are vital to articulate complex data concepts to both technical and non-technical stakeholders.