Machine Learning Engineer – Unity Catalog (Databricks)
Job Description
Title: Machine Learning Engineer – Unity Catalog (Databricks)
Location: Seattle, WA (Remote for strong candidates, Hybrid preferable with 3 days in-office)
Job Type: Contract
Visa Types: H1B (on our payroll), GC (candidate’s company payroll, no 1099), USC
Overview:
We are looking for an experienced Machine Learning (ML) Engineer with expertise in Unity Catalog and Feature Store in Databricks to help build and maintain a robust foundation for our data and machine learning workflows. You will work on organizing data, managing access, and ensuring ML models operate efficiently in production environments.
Key Responsibilities:
* Set up and manage Unity Catalog in Databricks to organize and secure data access across teams.
* Design, build, and operationalize Feature Stores to support machine learning models in production.
* Build efficient data pipelines to process and serve features to ML workflows.
* Collaborate with teams using Databricks, Azure Cosmos DB, and other Azure tools to integrate data solutions.
* Monitor and optimize the performance of data pipelines and feature stores.
Required Skills & Experience:
* Strong experience with Unity Catalog in Databricks for managing data assets and access control.
* Hands-on experience working with Databricks Feature Store or similar solutions.
* Proven knowledge of building and maintaining scalable ETL pipelines in Databricks.
* Familiarity with Azure tools like Azure Cosmos DB and Azure Container Registry (ACR).
* Understanding of machine learning workflows and the integration of feature stores into pipelines.
* Strong problem-solving skills and a collaborative mindset.
* Proficiency in Python and Spark for data engineering tasks.
* Experience with monitoring tools such as Splunk or Datadog to ensure system reliability.
* Familiarity with Azure Kubernetes Service (AKS) for deploying and managing containers.
Desirable Skills:
* Experience with other Azure tools or services related to machine learning and data engineering.
* Understanding of cloud-native architectures and best practices for data storage and retrieval.
* Knowledge of containerization and deployment practices for machine learning models.
Skill Matrix
Skill
Required
Desired
Proficiency Level
Unity Catalog in Databricks
Yes
No
Expert
Databricks Feature Store
Yes
No
Expert
Python and Spark
Yes
No
Advanced
Azure Cosmos DB
Yes
Yes
Intermediate
Azure Container Registry (ACR)
Yes
Yes
Intermediate
ETL Pipeline Development
Yes
No
Advanced
Monitoring Tools (Splunk, Datadog)
Yes
No
Intermediate
Azure Kubernetes Service (AKS)
Yes
Yes
Intermediate
Machine Learning Workflows
Yes
Yes
Intermediate
Collaboration and Problem Solving
Yes
No
Expert
Thanks & Regards
Sajida@sryven.com
Technical Recruiter
612-515-7155
