Sr. Software Engineer - Spark

This job was posted over 30 days ago, so it has most likely closed. See newer jobs like it.

Written by AI from the job title and company, not by Penn Mutual, so some details may not match this job. The employer's own page has the real description.

Overview

The Sr. Software Engineer - Spark at Penn Mutual will be responsible for designing, developing, and implementing robust data processing solutions using Apache Spark. This role involves collaborating with cross-functional teams to enhance data analytics, improve performance of existing systems, and ensure high standards of code quality and system scalability. The ideal candidate will demonstrate strong technical expertise in big data technologies while maintaining a focus on business objectives and innovative problem-solving.

Job Responsibilities

• Design, develop, and maintain scalable data processing pipelines using Apache Spark.
• Collaborate with data scientists and business analysts to understand data requirements and translate them into technical specifications.
• Optimize and tune Spark applications for maximum performance and efficiency.
• Implement best practices for code development including version control, testing, and documentation.
• Participate in code reviews to ensure adherence to best coding practices and standards.
• Stay updated with emerging trends in big data technologies and apply them to enhance existing processes.
• Mentor junior engineers and support their development and growth within the team.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.
• Minimum of 5 years of experience in software development with a focus on big data technologies.
• Expertise in Apache Spark with a strong understanding of its ecosystem.
• Proficient in programming languages such as Java, Scala, or Python.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Familiarity with cloud platforms (e.g., AWS, Azure) and containerization technologies (e.g., Docker, Kubernetes).
• Knowledge of SQL and database systems (e.g., PostgreSQL, Hive, etc.).
• Strong problem-solving skills and ability to work in a fast-paced environment.

Technologies & Tools

In this role, the Sr. Software Engineer will primarily work with technologies including but not limited to Apache Spark, Hadoop, and various cloud services platforms such as Amazon Web Services (AWS) or Microsoft Azure. Proficiency in Scala, Java, or Python will be essential, along with experience in using tools like Git for version control and Jenkins for continuous integration and development. Familiarity with data storage solutions such as HDFS, NoSQL databases, and SQL-based systems will also be beneficial.

Ideal Candidates

The ideal candidate for this position is a proactive and innovative thinker with a strong passion for technology and data analytics. They should possess exceptional analytical skills and the ability to work collaboratively within a team environment. A successful individual in this role will also demonstrate effective communication skills, a commitment to quality, and a keen attention to detail. A genuine enthusiasm for continuous learning and adaptability to new challenges will further enhance their fit within the Penn Mutual culture.