StreamsHub Docs Site Source

This repo holds the source code for the StreamsHub site.

Pulling dependant sources

This site aggregates documentation from all the StreamsHub components. Each component's documentation source and the versions which are included are configured via an entry in the sources.json file in the repository root:

{
    "name": "Flink SQL Runner",
    "sourceOwner": "streamshub",
    "sourceRepository": "flink-sql", 
    "developmentBranch": "docs-dev",
    "docsFolderPath": "docs",
    "tags":["0.2.0"] 
}

This file is read by the scripts/docBuilder.java jbang script. You will need to install jbang locally in order to run the documentation build.

The contents of the docsFolderPath in each <sourceOwner>/<sourceRepository> GitHub repository at each supplied reference tag will be pulled and placed in their own folder under content/docs/<name>/<tag>. If a folder already exists for the given tag then it will not be pulled.

The contents of the docsFolderPath folder on the developmentBranch will always be pulled on every build via the .github/workflows/publish.yaml GitHub Action.

A contents file will be generated for each entry in the source.json. This will contain links to the documentation for the development branch and each of the configured tags. You can skip this if you only want to have one version of the docs (just the development branch) by setting the skipContentsPageCreation key to true.

To pull the configured sources locally you will need a GitHub access token with permissions to access all the configured <sourceOwner>/<sourceRepository>:

./scripts/docBuilder.java <github-access-token>

CI/CD Workflows

Caching tagged documentation (cache-tagged-docs.yaml)

Tagged documentation versions (those listed in the tags array of each sources.json entry) are immutable release snapshots that do not change after release. To avoid re-downloading them on every build, this workflow commits them to the repository.

The workflow runs automatically when sources.json is pushed to main (e.g. after adding a new release tag) and can also be triggered manually. It downloads only tagged versions (--tags-only) and removes cached folders for any tags that have been deleted from the configuration (--cleanup).

If new or removed tags are detected, the workflow:

1. Creates a branch (ci/cache-tagged-docs) with the updated cached content
2. Opens a pull request targeting main (or updates the existing one if the branch already has an open PR)

Once the PR is merged, the publish workflow automatically triggers and deploys the updated site.

Publishing the site (publish.yaml)

The publish workflow builds and deploys the site to GitHub Pages. It runs on every push to main, on a daily schedule, and on manual dispatch.

Unlike the caching workflow, it fetches development branch documentation on every run (these are not committed to the repository). Tagged documentation is already present in the repository from the caching workflow, so it does not need to be re-downloaded.

Building the site

Prerequisites

The site uses the hugo static site generator. You will need to install a recent release (the version in your package manager is probably too old) and the PostCSS packages in order to build the source.

You will also need asciidoctor installed to build most of the documentation pages.

Building the site locally

This site uses a hugo theme installed via a git submodule. If you have just cloned the repo then run the following command to pull the theme:

git submodule update --init --recursive

You can build the site by running hugo from the repository root.

hugo build

Or run a live preview server by running:

hugo server --buildDrafts --disableFastRender