Databricks Data Consultant

We are seeking an experienced Databricks Data Consultant to join our team supporting the Internal Revenue Service (IRS). The ideal candidate will have a strong background in data engineering, ETL/ELT pipeline development, Databricks, and cloud-based data warehousing, with proven expertise in enterprise-scale data applications. A Databricks Certified Data Engineer Professional certification is required for this role.

Key Responsibilities

Design, develop, and maintain enterprise-scale data applications using Databricks and cloud-native platforms.

Build and optimize ETL/ELT workflows to support reporting, analytics, and decision-making.

Develop software for retrieving, parsing, and processing structured and unstructured data using Python, SQL, or Java.

Implement and manage data pipelines and transformations across distributed systems (Spark, Kafka, AWS EMR, etc.).

Leverage data warehousing technologies such as AWS Redshift, MySQL, PostgreSQL, or Snowflake.

Work with public cloud platforms (AWS, Azure, GCP) to enable scalable, secure data solutions.

Ensure compliance with IRS data security, governance, and audit requirements.

Required Qualifications

6+ years of experience designing, developing, and maintaining enterprise-scale data applications.

3+ years of experience in software development with Python, SQL, or Java.

3+ years of experience building scalable ETL/ELT workflows.

2+ years of direct Databricks experience.

Strong expertise in data warehousing (AWS Redshift, PostgreSQL, MySQL, or Snowflake).

Experience with distributed data and computing tools (Spark, AWS EMR, Kafka).

Hands-on experience with public cloud platforms (AWS, Azure, or GCP).

Databricks Certified Data Engineer Professional certification (mandatory).

Bachelor’s degree in Computer Science, Data Engineering, or related field.

Must be able to obtain and pass an IRS Moderate Risk Background Investigation (MBI).

Nice to Have

Experience with Palantir Foundry.

Hands-on experience with UNIX/Linux, including shell scripting.

Experience working with real-time data and streaming applications.

Advanced academic background (Master’s degree preferred)