Databricks Administrator Job at Next Ventures, Mountain View, CA

U2JjTWNYTmRGQmcyTUJGVGxhTE1BaFdqdkE9PQ==
  • Next Ventures
  • Mountain View, CA

Job Description

Onsite at client location in Mountain View, CA (4 days per week)

Long term contract with possibility of conversion (~3 years)

Average hourly rate of $80/hour depending on experience and interview performance

Next Venture's financial services consulting client is seeking a Databricks Administrator with subject matter expertise in AWS services and products for a long term contract position out of their end client's office in Mountain View, CA.

The end client is just beginning their Databricks journey and is seeking a seasoned administrator to leadin setting up, managing, and securing the Databricks platform within an AWS environment, ensuring it runs smoothly for the entire team.

Responsibilities:

Key Responsibilities:

Databricks & Cloud Platform Administration:

  • Deploy, configure, and maintain Databricks workspaces on AWS, ensuring high availability and security.
  • Manage cluster configurations, autoscaling, and compute resources for optimal performance.
  • Administer Unity Catalog , enforcing governance policies for structured and unstructured data assets.

Security, Compliance & Data Governance:

  • Implement Unity Catalog for centralized access control , enforcing RBAC (Role-Based Access Control), attribute-based policies, and lineage tracking .
  • Define fine-grained permissions for tables, views, and files stored in AWS S3 .
  • Ensure compliance with banking industry regulations (e.g., SOX, PCI-DSS, GDPR ) through data masking, encryption, and audit logging .
  • Manage IAM roles and authentication mechanisms between AWS and Databricks.

Performance Optimization & Monitoring:

  • Optimize Spark job performance and cluster tuning using best practices.
  • Monitor Databricks workspace health with CloudWatch, Datadog, or Splunk .
  • Automate cost control measures through idle cluster termination, autoscaling policies, and query optimizations .

Integration & Automation:

  • Integrate Databricks with AWS services (S3, Glue, Redshift, Kinesis, IAM) and external databases .
  • Automate workspace provisioning and Unity Catalog policies using Terraform or Infrastructure-as-Code (IaC) .
  • Implement CI/CD pipelines for Databricks notebooks, jobs, and configurations.

Collaboration & Support:

  • Work closely with data engineers, cloud architects, security teams, and compliance officers to align Databricks with banking security policies.
  • Troubleshoot platform issues, user access problems, and data pipeline failures .
  • Develop best practices and documentation for Databricks administration and Unity Catalog implementation.

Qualifications & Experience:

  • 5+ years of experience administering Databricks on AWS in an enterprise environment.
  • Hands-on experience with Unity Catalog , including fine-grained permissions, lineage tracking, and governance policies .
  • Strong knowledge of AWS services (S3, Glue, Redshift, IAM, Lambda, Kinesis).
  • Expertise in Spark cluster management, Databricks Jobs, and SQL analytics .
  • Experience in cloud security frameworks and data compliance for regulated industries.
  • Familiarity with Terraform, CloudFormation, or Infrastructure-as-Code for automation.
  • Understanding of networking & VPC configurations for Databricks on AWS.

Preferred Skills:

  • Experience with Databricks Delta Lake, MLflow, and AI/ML workloads .
  • Proficiency in Python, Bash, or PowerShell for scripting and automation.
  • Knowledge of Airflow, Kafka, or other data orchestration tools .
  • Databricks Certified Associate/Professional Administrator certification is a plus.

Job Tags

Hourly pay, Contract work,

Similar Jobs

JONESWORKS

2025 Summer Internship - NYC & LA Job at JONESWORKS

 ...JONESWORKS , a full-service marketing, public relations, and brand consultancy agency,...  ...Angeles offices. This is an unpaid PR internship in exchange for academic credit only....  ...given the opportunity to gain first-hand experience in the exciting and fast-paced world of... 

Auto-Owners Insurance

Marketing Representative Job at Auto-Owners Insurance

 ...Ohio: Stark, Wayne, Ashland, Holmes, or Richland. Auto-Owners Insurance, a top-rated insurance carrier, is seeking a motivated Marketing Representative to join our team. The position requires the person to: ~ Travel regularly to agencies throughout assigned... 

AGRI-SEARCH

Grain Merchandiser Job at AGRI-SEARCH

 ...Due to an upcoming retirement, a thriving Illinois grain company is seeking an experienced and skilled Grain Merchandiser to lead the charge in driving profitability and keeping grain flowing smoothly at their multiple truck and rail facilities. Responsibilities... 

Ampd

Director of Marketing Job at Ampd

 ...Director of Marketing About the Job *This is a hybrid/in office opportunity open to those located around, and legally eligible to...  ...Ampd.io website to increase organic visibility, authority, and inbound lead volume. Oversee the development of high-converting landing... 

Medical Services of America

Occupational Therapist Home Health Job at Medical Services of America

 ...Health Agency, a proud member of the Medical Services of America, Inc. family, is currently looking to hire an experienced Occupational Therapist Home Health Full-Time to see patients in Durham, Roxboro (Durham, Person) NC. Provides skilled occupational...