Senior Data Analyst - Databricks, PySpark & Power BI - HYBRID

Job Description

• Crop to Crop resumes are accepted
  
  

Location Requirement: This is a hybrid position. The manager is unsure how often he will need the contractor onsite, but the maximum requirement would be three days a week on Tuesday, Wednesday, and Thursday.

The ideal candidate will have hands-on expertise with Databricks (PySpark) and visualization tools, including both Databricks native tools and Power BI. In this role, you will be responsible for developing business requirements, supporting change management documentation, and actively collaborating with stakeholders to ensure alignment between data solutions and business needs. You will work closely with a technical lead and participate in design discussions and planning with the broader development team.

Key Responsibilities

Conduct comprehensive data analysis, create data mappings, develop dashboards, and ensure data quality and integrity

• Leverage Spark SQL within Databricks to query and analyze large datasets, writing efficient and complex SQL queries
• Collaborate closely with data engineers to design and deliver high-quality data products
• Develop detailed data mappings to support the integration and transformation of data from diverse sources into structured, analysis-ready formats
• Design and build intuitive, informative dashboards using tools such as Power BI or Databricks' native visualization tools to effectively communicate insights
• Build Gold-layer data products to support custom-built dashboards, APIs, and UI tools
• Create clear and comprehensive documentation for data pipelines, mappings, models, and dashboard functionality
• Perform statistical analysis, data mining, and machine learning to uncover patterns, trends, and actionable insights
  
  

Required Skills

Databricks and PySpark proficiency: hands-on experience working in the Databricks platform and using PySpark for data analysis and transformation.

Advanced Spark SQL: the ability to write efficient, complex queries to analyze large datasets within Databricks.

Data visualization and dashboarding:building clear, intuitive dashboards in Power BI and Databricks’ native visualization tools to communicate insight

Data mapping and modeling: developing detailed mappings that integrate and transform data

Business requirements and stakeholder collaboration: gathering and developing business requirements

Data quality and analysis: conducting thorough data analysis