Databricks MLOps Stacks v0.5

Major Features

Deprecated & Removed

• Migrated off FeatureStoreClient to the Feature Engineering client (databricks.feature_engineering) — action needed for existing stacks, as the old client is being sunset by Databricks
• Removed non-Unity-Catalog code paths — UC is now the only supported model registry
• Removed classic cluster (new_cluster/job_clusters) definitions in favor of serverless compute
• Removed deprecated MLflow Recipes support
• Flattened project directory structure (removed nested notebooks/ subdirectories)

New Features

• Upgraded to MLflow 3.x across training, validation, and deployment
• All jobs now run on serverless compute by default (environment_key/environments)
• Added GitLab CI/CD support

What's Changed

• Feature eng dev mmt by @hengrumay in #157
• Model validation for FS models by @aliazzzdat in #165
• Fix dead links by @arpitjasa-db in #166
• Update Database name for Feature Engineering notebooks by @arpitjasa-db in #171
• Fix self-referencing filename by @kriscon-db in #177
• Explicitly set data security mode to SINGLE_USER, fix broken tests by @shichengzhou-db in #179
• Adding Gitlab support by @valx in #174
• Update python version from 3.8 to 3.10 in run-tests workflow by @mniels17 in #178
• Add bundle UUID and other metadata by @arpitjasa-db in #185
• Fix deploy pipeline failure ("staging: no such target...") by @ThomasHenckel in #187
• Add conditionals for catalog variable by @arpitjasa-db in #188
• Fix target naming in GitHub Actions files by @arpitjasa-db in #191
• Remove UC + GCP restriction by @arpitjasa-db in #193
• Updates to readme, and batch inference by @ThomasHenckel in #192
• Include notes and reference link on promoting model for stack-customization by @hengrumay in #200
• Pass python-version as string for GitHub Actions by @JaiLaff in #207
• Update MLflow 3.0 deprecated variable "higher_is_better" to "greater_is_better" by @isaacdtaylor in #210
• Simplify project structure — flatten notebooks, remove boilerplate by @veenaramesh in #219
• Remove deprecated MLflow Recipes support by @veenaramesh in #220
• Remove Unity Catalog conditionals — UC is the only model registry by @veenaramesh in #221
• Migrate from FeatureStoreClient to FeatureEngineeringClient by @veenaramesh in #222
• Upgrade to MLflow 3.x and simplify validation by @veenaramesh in #223
• Switch all job compute to serverless by @veenaramesh in #224
• Update docs for renamed products (Git folders, Lakeflow Jobs) by @janniklasrose in #225
• Bump stacks_version to 0.5 by @veenaramesh

New Contributors

• @hengrumay made their first contribution in #157
• @aliazzzdat made their first contribution in #165
• @kriscon-db made their first contribution in #177
• @shichengzhou-db made their first contribution in #179
• @valx made their first contribution in #174
• @mniels17 made their first contribution in #178
• @ThomasHenckel made their first contribution in #187
• @JaiLaff made their first contribution in #207
• @isaacdtaylor made their first contribution in #210
• @veenaramesh made their first contribution in #219
• @janniklasrose made their first contribution in #225

Full Changelog: v0.4...v0.5

Databricks MLOps Stacks v0.4

Major Features

• Monitoring is now integrated into MLOps Stacks! 🎉

What's Changed

• Delete unused images by @arpitjasa-db in #154
• Update the version of matplotlib to avoid security issues by @lbcommer in #155
• Monitoring integration by @s-udhaya,@arpitjasa-db in #156
• Update monitoring resource by @arpitjasa-db in #161
• Make monitoring initially optional to allow creation of inference tables first by @arpitjasa-db in #162
• Bump up Stacks version by @arpitjasa-db in #163

New Contributors

• @lbcommer made their first contribution in #155
• @s-udhaya made their first contribution in #156

Full Changelog: v0.3...v0.4

Databricks MLOps Stacks v0.3

Major Features

• Stacks can now be initialized as Project_Only (recommended for Data Scientists) and CICD_Only (recommended for Machine Learning Engineers or MLEs) on top of the already supported CICD_and_Project option. This will make it easier for everyone to get started with Stacks and provide native support for monorepos
• GCP is now supported on MLOps Stacks 🎉
  NOTE: Models in UC x GCP is not yet supported due to service principal-UC cluster restrictions.

What's Changed

• Make docs clearer by @arpitjasa-db in #109
• Add the minimum CLI version supported by the template by @shreyas-goenka in #116
• Use enum for selection type input parameters by @shreyas-goenka in #110
• Switch example staging and prod workspace URLs based on cloud provider by @shreyas-goenka in #111
• Set welcome message for the stack in the template schema by @shreyas-goenka in #112
• aws aod support by @qili86 in #114
• In the TrainWithFeatureStore notebook, set databricks-uc only when us model registry is used by @mingyu89 in #117
• Move regex validation checks to template schema by @shreyas-goenka in #108
• Update Docs in line with latest changes and BBoMLv2 by @arpitjasa-db in #118
• Update template to work with CLI v0.210.1 by @pietern in #121
• update MLOps Stacks repository landing page and the README template for created projects by @yinxi-db in #120
• Update README.md by @pietern in #126
• Fix Video Rendering on README by @arpitjasa-db in #130
• Add Support for Conditional Initialization by @arpitjasa-db in #132
• Fix the Model Validation by @qili86 in #131
• Fix README for GCP by @arpitjasa-db in #136
• Update ml-pull-request.md.tmpl by @yinxi-db in #139
• Fix Monorepo Setup & Separate by Role by @arpitjasa-db in #133
• Rename assets to resources by @arpitjasa-db in #145
• Fix schema validation by @arpitjasa-db in #146
• Fix README formatting & shorten initialization prompts by @arpitjasa-db in #147
• Remove workflow files to reduce repo size by @arpitjasa-db in #148
• Minor bug Fixes by @aravind-segu in #149
• Increment and standardize Stacks version by @arpitjasa-db in #150
• Fix doc links by @arpitjasa-db in #152
• Gcp workspace by @eric-golinko-db in #153

New Contributors

• @shreyas-goenka made their first contribution in #116
• @qili86 made their first contribution in #114
• @pietern made their first contribution in #121
• @aravind-segu made their first contribution in #149

Full Changelog: v0.2...v0.3

Databricks MLOps Stacks v0.2

Databricks MLOps Stacks Public Preview

MLOps Stacks is now in Public Preview, and the Private Preview sign-up blocker has been removed! 🎉 Anyone can get started with this project now with the latest version (>= 0.208.1) of the Databricks CLI.

Major Features

• Project initialization is now done via Databricks Asset Bundle Templates instead of Cookiecutter
• Models in Unity Catalog is now supported and the recommended option in MLOps Stacks

What's Changed

• Remove an already resolved TODO block by @mingyu89 in #80
• Bump ADO workflow databricks cli version to 0.100.2 by @mingyu89 in #81
• Azure DevOps bundle cli installer update by @niall-turbitt in #82
• Fix link by @mingyu89 in #83
• Fix the path of requirements.txt in doc by @mingyu89 in #85
• Add env prefix to batch inference output table name by @mingyu89 in #87
• Fix inference output table name by @mingyu89 in #88
• Ado feature store by @eric-golinko-db in #91
• Enable azure ADO + feature store test by @mingyu89 in #92
• Replace MLflow Recipes as default option with Delta Tables by @arpitjasa-db in #93
• Migrate Mlops Stacks to use databricks asset templates for project creation by @mingyu89 in #96
• Add explicit link to databricks cli installation doc and releases page by @mingyu89 in #100
• Minor comments and doc improvement by @mingyu89 in #102
• Rename bundle.yml to databricks.yml by @mingyu89 in #101
• Update bundle related CICD to use ubuntu-22.04 by @mingyu89 in #104
• rename environments to targets and use -t instead of -e in bundle commands by @mingyu89 in #103
• Rename databricks-resources folder to resources by @zhe-db in #105
• Models uc update by @niall-turbitt in #106
• Rename Stack to Stacks and Fix Bugs by @arpitjasa-db in #107

New Contributors

• @eric-golinko-db made their first contribution in #91

Full Changelog: v0.1...v0.2

Databricks MLOps Stack v0.1

What's Changed

• Update MLOps Stacks to Use Databricks Asset Bundles by @arpitjasa-db in #77
  • Bundles replace Terraform for resource and workflow deployment

Full Changelog: 0.0.1...v0.1

Databricks MLOps Stack v0.0

What's Changed

• Initial implementation of the Databricks MLOps Stack by @mohamad-arabi in #1
• Resolve final TODO in the code by @mohamad-arabi in #2

New Contributors

• @mohamad-arabi made their first contribution in #1

Full Changelog: https://github.com/databricks/mlops-stack/commits/v0.0