Sr. Data Platform Engineer

A World-Class Team

BJ’s Wholesale Club is powered by more than 30,000 team members who make a real impact every day. Whether you're stocking shelves, solving problems or shaping strategy, your work helps families save on what matters most.

We’re a team built on purpose and opportunity. Join us and be part of something meaningful.

Why You’ll Love Working at BJ’s

At BJ’s Wholesale Club, our team members are at the heart of everything we do. That’s why we offer a comprehensive benefits package designed to support your health, well-being and future – both on and off the job. When you grow, we grow.

Here’s just some of what you can look forward to:

Weekly Pay: Get paid every week so that you can manage your money on your terms.

Free BJ’s Memberships: Enjoy a complimentary The Club Card Membership, plus a free Supplemental Membership for someone in your household.*

Generous Paid Time Off: Take the time you need with vacation, personal, sick days, holidays, bereavement, and jury duty leave.*

Flexible and Affordable Health Benefits: Choose from three medical plans, and access optional dental, vision, Health Savings Account (HSA), and flexible spending account options to fit your lifestyle.*

401(k) Retirement Savings Plan: Build your financial future with a company match (available to team members 18 and older).*

Employee Stock Purchase Plan: Accumulate funds through after-tax payroll deductions that can be used to purchase shares of BJ’s common stock at a 15% discount.*

*Eligibility requirements vary by position.

Role Summary

We are seeking a Senior Data Platform Engineer to support the administration, automation, governance, security, and optimization of our enterprise Databricks Lakehouse platform on AWS.

This is a hands-on technical role responsible for maintaining and enhancing the Databricks platform, implementing infrastructure automation, enforcing compute and security standards, supporting cost-governance initiatives, and enabling reliable Data Engineering and Data Science workloads.

The ideal candidate will have strong hands-on experience with Databricks platform administration, AWS cloud services, Terraform-based infrastructure automation, and DevOps practices. The individual will evaluate emerging Databricks capabilities through proof-of-concept implementations and contribute to reusable platform patterns and standards.

This role will collaborate closely with Data Engineering, Data Science, Cloud Platform, Security, and Governance teams to ensure the Databricks environment remains secure, reliable, scalable, and cost-efficient.

Key Responsibilities

Databricks Platform Administration

Support and maintain enterprise Databricks workspace architecture, including networking, security, and compute governance.

Implement and maintain cluster policies, job policies, compute guardrails, and platform governance standards.

Manage and optimize Databricks compute environments including job clusters, interactive clusters, and serverless compute.

Implement Unity Catalog for centralized data governance and access management.

Manage workspace configurations, platform integrations, and operational reliability.

AWS Cloud Architecture for Databricks

Design and maintain secure and scalable Databricks infrastructure leveraging AWS services including:

IAM roles and cross-account access architecture

VPC networking, subnets, and secure connectivity

PrivateLink and secure endpoint configuration

S3 data lake architecture

AWS KMS encryption and key management

CloudWatch monitoring and logging integration

Ensure the platform aligns with enterprise security, compliance, and networking standards.

Infrastructure as Code & DevOps Automation

Build and maintain Databricks infrastructure using Terraform.

Implement infrastructure-as-code frameworks for workspace provisioning, cluster policies, and platform configurations.

Develop automation using:

Databricks CLI

Databricks REST APIs

Python and shell scripting

Design and maintain CI/CD pipelines for Databricks workloads and platform infrastructure.

Integrate deployments with Git-based development workflows (GitHub, GitLab, Azure DevOps).

Enable standardized environment promotion frameworks across Dev, Test, and Production environments.

Platform Cost Governance & Optimization

Monitor and analyze DBU consumption and compute utilization across workloads.

Provide recommendations on cost optimizations

Implement guardrails such as:

Cluster policies

Budget alerts

Resource tagging

Usage monitoring dashboards

Identify and implement cost optimization strategies across:

Compute configurations

Job scheduling patterns

Serverless vs classic compute usage

Storage lifecycle management

Build dashboards for cost transparency.

Data Platform Enablement

Provide technical guidance and platform best practices for data engineering and data science workloads, including:

Data Engineering

Delta Lake architecture and performance optimization

Medallion architecture (Bronze/Silver/Gold)

Batch and streaming data ingestion

Large-scale Spark workloads

Data Science & AI

MLflow experimentation and model lifecycle management

Feature Store usage

Machine learning pipelines and model deployment frameworks.

Innovation & Proof of Concept Development

Evaluate and operationalize new Databricks capabilities by building Proof of Concepts (POCs) and evaluate POCs and support adoption into platform standards.

Areas of focus include:

Databricks Serverless Compute

Delta Live Tables

Auto Loader

Lakehouse AI capabilities

Databricks Feature Store

Vector Search and LLM integrations

Lakehouse monitoring and observability frameworks

Support and maintain reference architectures and reusable design patterns based on POC outcomes.

Required Technical Skills

Databricks Platform

Databricks workspace administration

Cluster and job policy design

Unity Catalog governance

Delta Lake architecture

Databricks REST APIs and CLI automation

Performance optimization and troubleshooting

AWS Cloud Platform

Strong hands-on experience with:

AWS IAM

VPC networking and security

S3 data lake architecture

AWS KMS encryption

CloudWatch monitoring

PrivateLink connectivity

Cross-account access architecture

Enterprise cloud security practices

Infrastructure & Automation

Terraform for Databricks and AWS infrastructure

Infrastructure as Code design patterns

Databricks CLI and API automation

Python / Bash scripting

Platform automation frameworks

DevOps & DataOps

CI/CD pipeline development

Git-based version control workflows

Automated deployment and release processes

Monitoring, logging, and observability frameworks

Incident management and root cause analysis

Data Engineering & Analytics Skills

Apache Spark / PySpark

SQL optimization

Delta Lake performance tuning

Streaming frameworks

Data pipeline orchestration

Data modeling and ETL best practices

Preferred Qualifications

Databricks Certified Data Engineer / Databricks Platform Architect

AWS Certified Solutions Architect

Experience designing enterprise-scale Lakehouse platforms

Experience supporting large-scale ML workloads

Experience implementing data governance frameworks

This is a hybrid role. Tuesday through Thursday are in-office days at BJ's Club Support Center in Marlborough, MA and Monday and Friday are remote days.

In accordance with the Pay Transparency requirements, the following represents a good faith estimate of the compensation range for this position. At BJ’s Wholesale Club, we carefully consider a wide range of non-discriminatory factors when determining salary. Actual salaries will vary depending on factors including but not limited to location, education, experience, and qualifications. The pay range for this position is $100,000.00 - $131,500.00

We recognize the growing role of AI tools, including ChatGPT, and value familiarity with them. That said, we want to hear from your authentic self. Your application should reflect your own skills, experiences, and insights rather than AI-generated responses.