Discuss adding a Docker-based workflow for running tests locally

[DimaOshchepkov]

Hi Cloudberry contributors,

I am new to this project and recently tried to run Apache Cloudberry locally to better understand it.

I was able to start the project using the Docker sandbox and connect to the database successfully. That worked well for trying the database manually. After that, I wanted to go one step further and run the project tests
locally.

From what I found, the sandbox containers are mainly useful for running a local Cloudberry cluster, but they do not seem to include the source/build tree or the full test environment needed for commands like:

  make -C src/test/regress installcheck-small

After looking at the GitHub Actions workflow, it seems that CI uses a different approach: a build image, source checkout, configure/build/install steps, creation of a gpAux/gpdemo cluster, and then make installcheck-* targets.

I was able to reproduce a similar workflow locally by using the official build image:
apache/incubator-cloudberry:cbdb-build-rocky9-latest

and mounting the source tree into the container. After building Cloudberry and creating a demo cluster, I was able to run:

PGOPTIONS="-c optimizer=off" make -C src/test/regress installcheck-small

successfully.

This could lower the entry barrier for new contributors who want not only to run Cloudberry locally, but also to validate changes before opening a pull request.
I would like to discuss whether it would be useful to make this workflow easier for new contributors.

Possible options:

1. Add test support to the existing Docker sandbox, if that fits its purpose.
2. Add a separate development/test Docker container or documented workflow, similar to what CI does.
3. Add documentation to src/test/README, devops/sandbox/README.md, or the contributor guide explaining how to run tests locally with Docker.

I understand that the sandbox may not be intended as a full development/test environment, so a separate test-oriented container or documented workflow might be a better fit.

Would the project be open to adding something like this? If so, what direction would maintainers prefer?

[DimaOshchepkov]

We could add something similar to the readme or make the creation more automated. What do you think of this idea?

Local Test Container

You can run tests against a local checkout without installing build
dependencies on the host by using the published Cloudberry build image and a
bind mount. The source tree remains on the host, so edits made on the host or
inside the container affect the same files.

From the repository root:

docker run -dit \
    --privileged \
    --user root \
    --hostname cdw \
    --name cloudberry-devtest \
    --shm-size=2gb \
    --ulimit core=-1 \
    --cgroupns=host \
    -v /sys/fs/cgroup:/sys/fs/cgroup:rw \
    -v "$PWD":/workspace/cloudberry \
    -w /workspace/cloudberry \
    apache/incubator-cloudberry:cbdb-build-rocky9-latest \
    bash -lc "sleep infinity"

Enter the container:

docker exec -it --user gpadmin -w /workspace/cloudberry cloudberry-devtest bash

Prepare the build and demo cluster inside the container:

export SRC_DIR=/workspace/cloudberry
export BUILD_DESTINATION=/usr/local/cloudberry-db

/tmp/init_system.sh

./devops/build/automation/cloudberry/scripts/configure-cloudberry.sh
./devops/build/automation/cloudberry/scripts/build-cloudberry.sh
./devops/build/automation/cloudberry/scripts/create-cloudberry-demo-cluster.sh

After changing code, rebuild the relevant parts or rerun the build script, then
run the tests you need. For example:

source /usr/local/cloudberry-db/cloudberry-env.sh
source /workspace/cloudberry/gpAux/gpdemo/gpdemo-env.sh

PGOPTIONS="-c optimizer=off" make -C src/test/regress installcheck-small
PGOPTIONS="-c optimizer=off" make -C src/test/regress installcheck-tests TESTS="insert"
make -C src/test/isolation2 installcheck-isolation2

Stop or remove the container when it is no longer needed:

docker stop cloudberry-devtest
docker rm -f cloudberry-devtest

[yjhjstz]

welcome to comment on #1692.

[tuhaihe]

Great suggestion!

I think we could add your approach to the /devops/README.md directory at https://github.com/apache/cloudberry/tree/main/devops as well as to the documentation at https://cloudberry.apache.org/docs/deployment/build-based-on-docker. This would make it more convenient for contributors and developers to leverage the existing development image for testing code changes. How about this?

If it sounds good, could you help create the pull request for it? Thanks!