Databricks Administrator Job at Next Ventures, Mountain View, CA

U2JjTWNYTmRGQmcyTUJGVGxhTE1BaFdqdkE9PQ==
  • Next Ventures
  • Mountain View, CA

Job Description

Onsite at client location in Mountain View, CA (4 days per week)

Long term contract with possibility of conversion (~3 years)

Average hourly rate of $80/hour depending on experience and interview performance

Next Venture's financial services consulting client is seeking a Databricks Administrator with subject matter expertise in AWS services and products for a long term contract position out of their end client's office in Mountain View, CA.

The end client is just beginning their Databricks journey and is seeking a seasoned administrator to leadin setting up, managing, and securing the Databricks platform within an AWS environment, ensuring it runs smoothly for the entire team.

Responsibilities:

Key Responsibilities:

Databricks & Cloud Platform Administration:

  • Deploy, configure, and maintain Databricks workspaces on AWS, ensuring high availability and security.
  • Manage cluster configurations, autoscaling, and compute resources for optimal performance.
  • Administer Unity Catalog , enforcing governance policies for structured and unstructured data assets.

Security, Compliance & Data Governance:

  • Implement Unity Catalog for centralized access control , enforcing RBAC (Role-Based Access Control), attribute-based policies, and lineage tracking .
  • Define fine-grained permissions for tables, views, and files stored in AWS S3 .
  • Ensure compliance with banking industry regulations (e.g., SOX, PCI-DSS, GDPR ) through data masking, encryption, and audit logging .
  • Manage IAM roles and authentication mechanisms between AWS and Databricks.

Performance Optimization & Monitoring:

  • Optimize Spark job performance and cluster tuning using best practices.
  • Monitor Databricks workspace health with CloudWatch, Datadog, or Splunk .
  • Automate cost control measures through idle cluster termination, autoscaling policies, and query optimizations .

Integration & Automation:

  • Integrate Databricks with AWS services (S3, Glue, Redshift, Kinesis, IAM) and external databases .
  • Automate workspace provisioning and Unity Catalog policies using Terraform or Infrastructure-as-Code (IaC) .
  • Implement CI/CD pipelines for Databricks notebooks, jobs, and configurations.

Collaboration & Support:

  • Work closely with data engineers, cloud architects, security teams, and compliance officers to align Databricks with banking security policies.
  • Troubleshoot platform issues, user access problems, and data pipeline failures .
  • Develop best practices and documentation for Databricks administration and Unity Catalog implementation.

Qualifications & Experience:

  • 5+ years of experience administering Databricks on AWS in an enterprise environment.
  • Hands-on experience with Unity Catalog , including fine-grained permissions, lineage tracking, and governance policies .
  • Strong knowledge of AWS services (S3, Glue, Redshift, IAM, Lambda, Kinesis).
  • Expertise in Spark cluster management, Databricks Jobs, and SQL analytics .
  • Experience in cloud security frameworks and data compliance for regulated industries.
  • Familiarity with Terraform, CloudFormation, or Infrastructure-as-Code for automation.
  • Understanding of networking & VPC configurations for Databricks on AWS.

Preferred Skills:

  • Experience with Databricks Delta Lake, MLflow, and AI/ML workloads .
  • Proficiency in Python, Bash, or PowerShell for scripting and automation.
  • Knowledge of Airflow, Kafka, or other data orchestration tools .
  • Databricks Certified Associate/Professional Administrator certification is a plus.

Job Tags

Hourly pay, Contract work,

Similar Jobs

Coastal Pools

QC Inspection Superintendent Job at Coastal Pools

 ...throughout their career. Coastal Pools has been recognized as 2025 Best of Annapolis and Eastern Shore and consistently ranked among the top Pool Builders in the area.. Join a Company that Empowers you to Build your Future The Superintendent is a key management... 

Ellaway Blues Consulting

Rigging Engineer III Job at Ellaway Blues Consulting

 ...We are seeking a talented Rigging Engineer III to join our team and support high-profile heavy lifting and construction projects based out of Long Beach, CA. In this role, youll develop engineered lift drawings, perform rigging and lift analyses, design customized lifting... 

William Scott Associates

Financial Operations Controller - CPG Job at William Scott Associates

 ...fast-moving international consumer products business requires a Financial Operations Controller - CPG. Based in Carson, California...  ...risks, and optimize reporting processes. Strategic Planning: Lead business strategy formulation and execution, aligning financial... 

PROCEED,Inc. -National Center for Training, Support and Tech...

Health Communications Coordinator Job at PROCEED,Inc. -National Center for Training, Support and Tech...

 ...JOB TITLE: Bilingual Health Program Coordinator AGENCY/DEPARTMENT: PROCEED Inc. / National Center for Training, Support, and Technical Assistance(NCTSTA) REPORTS TO: Director FLSA Status: Exempt LOCATION: On-site in Elizabeth, New Jersey Interested... 

TotalWellness

RN/LPN for Vaccinations and Wellness Screenings PRN Job at TotalWellness

 ...health professionals who can help deliver this positive message to employees at on-site wellness events. Please remember we do not have full time work available, this is a PRN position - working as needed. Employment Type: Contractor Salary: $ 32.00 45.00 Per Hour