Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Notable Changes

### CLI
* Added the `databricks quickstart` command, a short introduction to the CLI that prints a human-friendly guide interactively and an agent-oriented version when run non-interactively ([#5464](https://github.com/databricks/cli/pull/5464)).

### Bundles
* Set the default `data_security_mode` to `DATA_SECURITY_MODE_AUTO` in bundle templates ([#5452](https://github.com/databricks/cli/pull/5452)).
Expand Down
1 change: 1 addition & 0 deletions acceptance/help/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ Additional Commands:
configure Configure authentication
help Help about any command
labs Manage Databricks Labs installations
quickstart Print an introduction to the Databricks CLI
version Retrieve information about the current version of this CLI

Flags:
Expand Down
3 changes: 3 additions & 0 deletions acceptance/quickstart/out.test.toml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

88 changes: 88 additions & 0 deletions acceptance/quickstart/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Databricks Quickstart

The `databricks` CLI wraps the public Databricks REST API — there's a command for essentially every API endpoint, so almost any Databricks task can be driven from here. If something looks unsupported, it's almost always a command you haven't found yet, not a real gap. The headline workflow: build durable resources as code with **Databricks Asset Bundles (DABs)** and ship them with `databricks bundle deploy` (see Build and deploy below).

## Golden rules

- **Reason, don't refuse.** Before telling the user something can't be done, run `databricks <group> --help` and install the matching skill (below). Almost everything is reachable through the CLI.
- **Build with bundles.** For anything durable — jobs, pipelines, dashboards, apps — define it as code in a Databricks Asset Bundle (DABs) and ship with `databricks bundle deploy`, not one-off API calls.
- **Pick the profile deliberately.** Run `databricks auth profiles`. If only one profile exists, use it. If one is already set as the default (via `databricks auth switch`), assume that one — you can confirm if unsure. Otherwise, show the user the profiles with their workspace URLs and let them choose. Don't invent profile names.
- **Always pass `--profile <name>`.** Each Bash command runs in its own shell, so a separate `export DATABRICKS_CONFIG_PROFILE=...` line does NOT persist. Use the flag, or chain with `&&`.
- **OAuth, never PAT.** Authenticate with `databricks auth login`, not personal access tokens.

## Step 1 — Authenticate

```bash
databricks auth profiles # already set up? lists profiles + validity
databricks auth login --profile <name> # if not — opens login.databricks.com to pick a workspace
```

No host needed: without `--host`, login opens login.databricks.com and the user picks a workspace in the browser. Only pass `--host <url>` if the user already gave you a specific workspace URL. Always pass a descriptive `--profile <name>` (e.g. `dev-aws`, `prod-azure`) so you can target it reliably afterward — ask the user for the name rather than inventing one.

## Step 2 — Verify

```bash
databricks current-user me --profile <name>
```

## Build and deploy with Asset Bundles (DABs)

**DABs are the standard, recommended way to build on Databricks.** A bundle is a project with a `databricks.yml` that declares your resources (jobs, pipelines, dashboards, apps) and its targets (e.g. dev, prod) — one project, one deploy, every resource type:

```bash
databricks bundle init # scaffold a project from a template
databricks bundle validate # check the config
databricks bundle deploy -t dev # deploy everything to a target (dev, prod, ...)
databricks bundle run <resource> -t dev # run a job or pipeline
```

Read the `databricks-dabs` skill for bundle structure, resource schemas, and multi-target setup.

**Databricks Apps** (full-stack TypeScript + React via AppKit) build on bundles too, with a dedicated scaffolder — see `databricks-apps`:

```bash
databricks apps init # scaffold an AppKit app (generates the bundle for you)
databricks apps deploy # deploy it to the workspace
databricks apps run-local # develop locally (or `dev-remote` against the workspace)
```

## Common commands

```bash
databricks jobs list --profile <name>
databricks pipelines list --profile <name>
databricks apps list --profile <name>
databricks warehouses list --profile <name>

# Unity Catalog data uses POSITIONAL args, NOT flags:
databricks catalogs list --profile <name>
databricks schemas list <catalog> --profile <name>
databricks tables list <catalog> <schema> --profile <name>
```

Unsure of a command's shape? Run `databricks <command> --help`.

## Install skills for the task

Skills give you Databricks-specific patterns so you build resources correctly instead of guessing. They install into your coding agent (auto-detected):

```bash
databricks aitools install # stable skills
databricks aitools install --experimental # broader coverage: more areas, lower quality bar
databricks aitools list # available + installed skills — check this for the latest
```

The set evolves, so run `databricks aitools list` rather than relying on a fixed list. Common ones: `databricks-core` (CLI/auth), `databricks-jobs`, `databricks-pipelines`, `databricks-dabs` (deploy as code), `databricks-apps`. Experimental skills reach further (Unity Catalog, vector search, dashboards, AI functions, and more) — worth a look depending on the prompt.

## Explore data

Use the built-in tools instead of guessing SQL or navigating by hand:

```bash
databricks experimental aitools tools discover-schema <catalog>.<schema>.<table> --profile <name>
databricks experimental aitools tools query "SELECT * FROM <table> LIMIT 10" --profile <name>
```

## When you get stuck

Re-read `databricks <group> --help`, install the matching skill, and check https://docs.databricks.com. If you truly can't do something yourself, tell the user how they can — don't tell them Databricks can't.
1 change: 1 addition & 0 deletions acceptance/quickstart/script
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
$CLI quickstart
2 changes: 2 additions & 0 deletions cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"github.com/databricks/cli/cmd/fs"
"github.com/databricks/cli/cmd/labs"
"github.com/databricks/cli/cmd/pipelines"
"github.com/databricks/cli/cmd/quickstart"
"github.com/databricks/cli/cmd/root"
"github.com/databricks/cli/cmd/selftest"
"github.com/databricks/cli/cmd/sync"
Expand Down Expand Up @@ -123,6 +124,7 @@ func New(ctx context.Context) *cobra.Command {
cli.AddCommand(labs.New(ctx))
cli.AddCommand(sync.New())
cli.AddCommand(version.New())
cli.AddCommand(quickstart.New())
cli.AddCommand(selftest.New())
cli.AddCommand(ssh.New())

Expand Down
6 changes: 6 additions & 0 deletions cmd/quickstart/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# These markdown files are embedded into the binary with go:embed and their
# newlines are semantically important: stripFrontmatter matches an LF fence
# ("---\n"), so an extra \r on Windows would leave the frontmatter unstripped.
# Always check them out with \n so the embedded content is identical on every
# platform.
*.md text eol=lf
93 changes: 93 additions & 0 deletions cmd/quickstart/quickstart-agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
name: databricks-quickstart
description: First-contact orientation for the Databricks CLI when no Databricks skill is loaded. Covers OAuth auth, profile selection, command discovery, and installing the right skill for the task. Read this before attempting any Databricks work.
---

# Databricks Quickstart

The `databricks` CLI wraps the public Databricks REST API — there's a command for essentially every API endpoint, so almost any Databricks task can be driven from here. If something looks unsupported, it's almost always a command you haven't found yet, not a real gap. The headline workflow: build durable resources as code with **Databricks Asset Bundles (DABs)** and ship them with `databricks bundle deploy` (see Build and deploy below).

## Golden rules

- **Reason, don't refuse.** Before telling the user something can't be done, run `databricks <group> --help` and install the matching skill (below). Almost everything is reachable through the CLI.
- **Build with bundles.** For anything durable — jobs, pipelines, dashboards, apps — define it as code in a Databricks Asset Bundle (DABs) and ship with `databricks bundle deploy`, not one-off API calls.
- **Pick the profile deliberately.** Run `databricks auth profiles`. If only one profile exists, use it. If one is already set as the default (via `databricks auth switch`), assume that one — you can confirm if unsure. Otherwise, show the user the profiles with their workspace URLs and let them choose. Don't invent profile names.
- **Always pass `--profile <name>`.** Each Bash command runs in its own shell, so a separate `export DATABRICKS_CONFIG_PROFILE=...` line does NOT persist. Use the flag, or chain with `&&`.
- **OAuth, never PAT.** Authenticate with `databricks auth login`, not personal access tokens.

## Step 1 — Authenticate

```bash
databricks auth profiles # already set up? lists profiles + validity
databricks auth login --profile <name> # if not — opens login.databricks.com to pick a workspace
```

No host needed: without `--host`, login opens login.databricks.com and the user picks a workspace in the browser. Only pass `--host <url>` if the user already gave you a specific workspace URL. Always pass a descriptive `--profile <name>` (e.g. `dev-aws`, `prod-azure`) so you can target it reliably afterward — ask the user for the name rather than inventing one.

## Step 2 — Verify

```bash
databricks current-user me --profile <name>
```

## Build and deploy with Asset Bundles (DABs)

**DABs are the standard, recommended way to build on Databricks.** A bundle is a project with a `databricks.yml` that declares your resources (jobs, pipelines, dashboards, apps) and its targets (e.g. dev, prod) — one project, one deploy, every resource type:

```bash
databricks bundle init # scaffold a project from a template
databricks bundle validate # check the config
databricks bundle deploy -t dev # deploy everything to a target (dev, prod, ...)
databricks bundle run <resource> -t dev # run a job or pipeline
```

Read the `databricks-dabs` skill for bundle structure, resource schemas, and multi-target setup.

**Databricks Apps** (full-stack TypeScript + React via AppKit) build on bundles too, with a dedicated scaffolder — see `databricks-apps`:

```bash
databricks apps init # scaffold an AppKit app (generates the bundle for you)
databricks apps deploy # deploy it to the workspace
databricks apps run-local # develop locally (or `dev-remote` against the workspace)
```

## Common commands

```bash
databricks jobs list --profile <name>
databricks pipelines list --profile <name>
databricks apps list --profile <name>
databricks warehouses list --profile <name>

# Unity Catalog data uses POSITIONAL args, NOT flags:
databricks catalogs list --profile <name>
databricks schemas list <catalog> --profile <name>
databricks tables list <catalog> <schema> --profile <name>
```

Unsure of a command's shape? Run `databricks <command> --help`.

## Install skills for the task

Skills give you Databricks-specific patterns so you build resources correctly instead of guessing. They install into your coding agent (auto-detected):

```bash
databricks aitools install # stable skills
databricks aitools install --experimental # broader coverage: more areas, lower quality bar
databricks aitools list # available + installed skills — check this for the latest
```

The set evolves, so run `databricks aitools list` rather than relying on a fixed list. Common ones: `databricks-core` (CLI/auth), `databricks-jobs`, `databricks-pipelines`, `databricks-dabs` (deploy as code), `databricks-apps`. Experimental skills reach further (Unity Catalog, vector search, dashboards, AI functions, and more) — worth a look depending on the prompt.

## Explore data

Use the built-in tools instead of guessing SQL or navigating by hand:

```bash
databricks experimental aitools tools discover-schema <catalog>.<schema>.<table> --profile <name>
databricks experimental aitools tools query "SELECT * FROM <table> LIMIT 10" --profile <name>
```

## When you get stuck

Re-read `databricks <group> --help`, install the matching skill, and check https://docs.databricks.com. If you truly can't do something yourself, tell the user how they can — don't tell them Databricks can't.
67 changes: 67 additions & 0 deletions cmd/quickstart/quickstart-human.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Welcome to Databricks

Databricks is one place for all your data and AI work: store and govern data, run SQL and notebooks, build pipelines and dashboards, train and serve models, and ship apps. No stitching together separate tools.

This guide takes you from zero to your first command.

## What you need

A **workspace** — your Databricks home, at a URL like `https://your-company.cloud.databricks.com`. No access yet? Sign up for a free trial at https://www.databricks.com (Databricks Free Edition is great for learning). You've already got the CLI — it's how you're reading this.

## A few concepts (the 30-second version)

- **Workspace** — the environment you log into. Notebooks, jobs, dashboards, and data all live here.
- **Unity Catalog** — how Databricks governs your data. Tables are named in three parts: `catalog.schema.table` (think folder → subfolder → table).
- **Compute** — where your code runs. *SQL warehouses* run SQL; *clusters* and *serverless* run notebooks and jobs.
- **Profile** — a saved connection to a workspace on your machine, so you don't re-enter credentials each time.

## Step 1 — Connect to your workspace

```bash
databricks auth login
```
This opens **login.databricks.com** in your browser. Sign in, pick the workspace you want, and you're connected — no URL to look up. The CLI saves the connection as a *profile*; if it's your only workspace, press Enter to accept the suggested name and every command will use it automatically. Run `databricks auth login` again any time to add another workspace.

Already know your workspace URL? Go straight to it: `databricks auth login --host https://your-company.cloud.databricks.com --profile my-workspace`.

## Step 2 — Try it out

```bash
databricks current-user me # who am I?
databricks catalogs list # what data can I see?
databricks jobs list # my jobs
```
To see everything a command can do, add `--help` — for example, `databricks jobs --help`.

Got more than one workspace? Name each at login (`databricks auth login --profile prod`), then target it with `--profile`, e.g. `databricks jobs list --profile prod`.

💡 **Tip:** turn on tab-completion so you can `Tab` through commands and flags: `databricks completion install`.

## Step 3 — Build something

The standard way to build on Databricks is **Databricks Asset Bundles (DABs)** — your jobs, pipelines, dashboards, and apps defined as code in one project, shipped with one command:

```bash
databricks bundle init # start from a template
databricks bundle validate # check it
databricks bundle deploy -t dev # ship it to your workspace
databricks bundle run <name> -t dev # run a job or pipeline
```

Want a **full-stack app**? AppKit scaffolds one in TypeScript + React:

```bash
databricks apps init # scaffold the app
databricks apps deploy # deploy it to your workspace
databricks apps run-local # develop locally (or dev-remote)
```

## Where to go next

- **Explore your data** — browse catalogs, schemas, and tables, or query them from the workspace UI.
- **More to build** — Lakeflow Jobs (orchestration), Lakeflow Pipelines (ETL), AI/BI Dashboards, and Model Serving.
- **Using an AI coding assistant?** Install the Databricks skills so it knows these patterns: `databricks aitools install`.

📚 Full docs: https://docs.databricks.com • CLI reference: https://docs.databricks.com/dev-tools/cli

Welcome aboard.
73 changes: 73 additions & 0 deletions cmd/quickstart/quickstart.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
package quickstart

import (
_ "embed"
"fmt"
"strings"

"github.com/databricks/cli/cmd/root"
"github.com/databricks/cli/libs/cmdio"
"github.com/spf13/cobra"
)

// humanQuickstart is the friendly, default introduction shown to people.
//
//go:embed quickstart-human.md
var humanQuickstart string

// agentQuickstart is the denser, agent-oriented version. It is also the
// `databricks-quickstart` skill, so it carries skill frontmatter that
// stripFrontmatter removes before printing.
//
//go:embed quickstart-agent.md
var agentQuickstart string

func New() *cobra.Command {
cmd := &cobra.Command{
Use: "quickstart",
Args: root.NoArgs,
Short: "Print an introduction to the Databricks CLI",
Long: `Print a short introduction to the Databricks CLI: authentication, profiles,
building with Databricks Asset Bundles, and where to go next.

Prints a human-friendly guide by default. When stdout is not an interactive
terminal (for example, when a coding agent runs the command), it prints a
denser, agent-oriented version instead.`,
}

cmd.RunE = func(cmd *cobra.Command, args []string) error {
content := quickstartFor(cmdio.IsPromptSupported(cmd.Context()))
_, err := fmt.Fprintln(cmd.OutOrStdout(), content)
return err
}

return cmd
}

// quickstartFor returns the quickstart text for the caller. Interactive
// terminals (people) get the friendly guide; non-interactive callers (coding
// agents, scripts, CI) get the agent-oriented version. The detection is
// intentionally simple for now and can grow more precise later.
func quickstartFor(interactive bool) string {
if interactive {
return strings.TrimRight(humanQuickstart, "\n")
}
return stripFrontmatter(agentQuickstart)
}

// stripFrontmatter removes a leading YAML frontmatter block ("---\n...\n---\n")
// so the printed output starts at the document heading rather than the skill
// metadata. Input without frontmatter is returned unchanged (trailing newlines
// trimmed). Fprintln re-adds a single trailing newline.
func stripFrontmatter(s string) string {
const fence = "---\n"
if !strings.HasPrefix(s, fence) {
return strings.TrimRight(s, "\n")
}
rest := s[len(fence):]
_, body, found := strings.Cut(rest, "\n"+fence)
if !found {
return strings.TrimRight(s, "\n")
}
return strings.TrimRight(strings.TrimLeft(body, "\n"), "\n")
}
Loading
Loading