Skip to content

DEVELOPMENT.md

Latest commit

 

History

History
358 lines (257 loc) · 14.5 KB

DEVELOPMENT.md

File metadata and controls

358 lines (257 loc) · 14.5 KB
Raw

Development

In this document you can find information about developing Trino.

Trino organization

Learn about development for all Trino organization projects:

Further information in the development section of the website includes different roles, like contributors, reviewers, and maintainers, related processes, and other aspects.

Trino developer guide

See the Trino developer guide for information about the SPI, implementing connectors and other plugins, the client protocol, writing tests and other lower level details.

Code Style

We recommend you use IntelliJ as your IDE. Code style is managed through airstyle.

To run airstyle and other maven checks before opening a PR: ./mvnw validate

In addition to those you should also adhere to the following:

Readability

The purpose of code style rules is to maintain code readability and developer efficiency when working with the code. All the code style rules explained below are good guidelines to follow but there may be exceptional situations where we purposefully depart from them. When readability and code style rule are at odds, the readability is more important.

Consistency

Keep code consistent with surrounding code where possible.

Alphabetize

Alphabetize sections in the documentation source files (both in the table of contents files and other regular documentation files).

Use streams

When appropriate, use the stream API. However, note that the stream implementation does not perform well so avoid using it in inner loops or otherwise performance sensitive sections.

Categorize errors when throwing exceptions.

Categorize errors when throwing exceptions. For example, TrinoException takes an error code as an argument, TrinoException(HIVE_TOO_MANY_OPEN_PARTITIONS). This categorization lets you generate reports so you can monitor the frequency of various failures.

Add license header

Ensure that all files have the appropriate license header; you can generate the license by running mvn license:format.

Prefer String formatting

Consider using String formatting with the String.formatted method: "Session property %s is invalid: %s".formatted(name, value). Sometimes, if you only need to append something, consider using the + operator. Please avoid formatted() or concatenation in performance critical sections of code.

Avoid ternary operator

Avoid using the ternary operator except for trivial expressions.

Avoid get in method names, unless an object must be a Java bean

In most cases, replace get with a more specific verb that describes what is happening in the method, like find or fetch. If there isn't a more specific verb or the method is a getter, omit get because it isn't helpful to readers and makes method names longer.

Define class API for private inner classes too

It is suggested to declare members in private inner classes as public if they are part of the class API.

Avoid mocks

Do not use mocking libraries. These libraries encourage testing specific call sequences, interactions, and other internal behavior, which we believe leads to fragile tests. They also make it possible to mock complex interfaces or classes, which hides the fact that these classes are not (easily) testable. We prefer to write mocks by hand, which forces code to be written in a certain testable style.

Use AssertJ

Prefer AssertJ for complex assertions.

Use Airlift's Assertions

For thing not easily expressible with AssertJ, use Airlift's Assertions class if there is one that covers your case.

Use var judiciously

Use var only when it improves readability. Prefer it when the type is obvious from the initializer, such as with new expressions, or when the explicit type is long or heavily generic and adds noise without improving clarity.

Avoid var when the inferred type is unclear, surprising, or important to understanding the code.

Prefer Guava immutable collections

Prefer using immutable collections from Guava over unmodifiable collections from JDK. The main motivation behind this is deterministic iteration.

Maintain production quality for test code

Maintain the same quality for production and test code.

Avoid abbreviations

Please avoid abbreviations, slang or inside jokes as this makes harder for non-native english speaker to understand the code. Very well known abbreviations like max or min and ones already very commonly used across the code base like ttl are allowed and encouraged.

Avoid default clause in exhaustive enum-based switch statements

Avoid using the default clause when the switch statement is meant to cover all the enum values. Handling the unknown option case after the switch statement allows static code analysis tools (e.g. Error Prone's MissingCasesInEnumSwitch check) report a problem when the enum definition is updated but the code using it is not.

Vector API

It's safe to assume that the JVM has the Vector API (JEP 508) enabled and available at runtime, but not safe to assume that the Vector API implementation will perform faster than equivalent scalar code on whatever hardware the engine happens to be running on.

Different CPU hardware can exhibit dramatically different performance characteristics, so it's important to use hardware feature detection to determine under which scenarios a vectorized approach will be faster for each implementation. Vectorized code should be tested on AMD, ARM, and Intel CPUs to verify the benefits hold on each of those platforms before deciding to enable a given code path on each of those platforms. Also note that ARM CPUs can exhibit significant differences from between hardware generations as well as between Apple Silicon and datacenter class CPUs.

When adding implementations that use the Vector API, prefer the following approach unless the specifics of the situation dictate otherwise:

  • Provide an equivalent scalar implementation in code, if one does not already exist.
  • Use configuration flags and hardware support detection to ensure that vectorized implementation is only selected when running on hardware where it is expected to perform better than its scalar equivalent.
  • Add tests that ensure the behavior of the vectorized and scalar implementations match.
  • Include micro-benchmarks that demonstrate the performance benefits of the vectorized implementation compared to the scalar equivalent logic. Ensure that the benefits hold for all CPU architectures on which the vectorized implementation is enabled.

Keep pom.xml clean and sorted

There are several plugins in place to keep pom.xml clean. Your build may fail if:

  • dependencies or XML elements are not ordered correctly
  • overall pom.xml structure is not correct

Many such errors may be fixed automatically by running the following: ./mvnw sortpom:sort

Building

The fastest way to build and install the whole project:

./mvnw clean install -T 2C -nsu -DskipTests -Dmaven.javadoc.skip=true -Dair.check.skip-all=true

This builds with two threads per core, skips snapshot update checks, tests, Javadoc, and the airbase checks (checkstyle, modernizer, dependency analysis). Run ./mvnw validate separately before opening a PR to get those checks back.

Branch-scoped local repository

Builds from different branches share ~/.m2/repository and overwrite each other's installed SNAPSHOTs, so a build can silently use jars from another branch. The branch-scoped-local-repository extension in .mvn/extensions.xml keeps installed artifacts separate per branch, so several checkouts or git worktrees can build and install in parallel without interfering.

It is off by default; enable it per build:

./mvnw install -DskipTests -DbranchScopedLocalRepo.enabled=true

Before turning it on:

  • Pass the flag on the command line of every build in that checkout. Builds without it see the unscoped artifacts instead.
  • The first build on a branch must be a full install, and third-party dependencies are downloaded once more.
  • Worktrees on the same branch are still not isolated from each other.
  • Nothing prunes these artifacts, so delete ~/.m2/repository/installed/<branch>/ once the work on a branch is finished.

Additional IDE configuration

When using IntelliJ to develop Trino, we recommend starting with all of the default inspections, with some modifications.

Enable the following inspections:

  • Java | Class structure | Utility class is not 'final',
  • Java | Class structure | Utility class with 'public' constructor,
  • Java | Class structure | Utility class without 'private' constructor,
  • Java | Control flow issues | Redundant 'else' (including Report when there are no more statements after the 'if' statement option),
  • Java | Internationalization | Implicit platform default charset.

Disable the following inspections:

  • Java | Abstraction issues | 'Optional' used as field or parameter type,
  • Java | Code style issues | Local variable or parameter can be 'final',
  • Java | Data flow | Boolean method is always inverted,
  • Java | Performance | Call to 'Arrays.asList()' with too few arguments.

Update the following inspections:

  • Remove com.google.common.annotations.Beta from JVM languages | Unstable API usage.

Enable errorprone (Error Prone Installation#IDEA):

  • Install Error Prone Compiler plugin from marketplace,
  • Check the errorprone-compiler profile in the Maven tab

This should be enough - IDEA should automatically copy the compiler options from the POMs to each module. If that doesn't work, you can do it manually:

  • In Java Compiler tab, select Javac with error-prone as the compiler,
  • Update Additional command line parameters and copy the contents of compilerArgs in the top-level POM (except for -Xplugin:ErrorProne) there
    • Remove the XML comments...
    • ...except the ones which denote checks which fail in IDEA, which you should "unwrap"
  • Remove everything from the list under Override compiler parameters per-module

Note that the version of errorprone used by the IDEA plugin might be older than the one configured in the pom.xml and you might need to disable some checks that are not yet supported by that older version. When in doubt, always check with the full Maven build (./mvnw clean install -DskipTests -Perrorprone-compiler).

Language injection in IDE

In order to enable language injection inside Intellij IDEA, some code elements can be annotated with the @org.intellij.lang.annotations.Language annotation. To make it useful, we recommend:

  • Set the project-wide SQL dialect in Languages & Frameworks | SQL Dialects "Generic SQL" is a decent choice here,
  • Disable inspection SQL | No data source configured,
  • Optionally disable inspection Language injection | Language mismatch.

Even if the IDE does not support language injection this annotation is useful for documenting the API's intent. Considering the above, we recommend annotating with @Language:

  • All API parameters which are expecting to take a String containing an SQL statement (or any other language, like regular expressions),
  • Local variables which otherwise would not be properly recognized by IDE for language injection.

Building docs

Information about writing and building the documentation can be found in the docs module.

Building the Web UI

The Trino Web UI is a React and Vite project located in core/trino-web-ui/src/main/resources/webapp. You must have Bun installed to execute these commands. (Maven builds download Bun automatically, so a local install is only needed to run these commands by hand.) Install dependencies with:

cd core/trino-web-ui/src/main/resources/webapp
bun install

For fast local development, run the WebUiQueryRunner class. This starts a minimal Trino development server configured with the Web UI. Then start the Vite development server:

bun run dev

Open http://localhost:5173/ui in your browser. The Vite development server provides Hot Module Replacement for quick iteration. By default, requests to /ui/auth and /ui/api are proxied to http://127.0.0.1:8080/. To use a different backend, update VITE_BASE_URL in core/trino-web-ui/src/main/resources/webapp/.env.development.

To build the Web UI locally, run:

bun run build

To run frontend checks, run:

bun run check

Maven builds package the Web UI automatically, and Maven verification runs the frontend checks.

Releases

Trino aims for frequent releases, generally once per week. This is a goal but not a guarantee, as critical bugs may lead to a release being pushed back or require an extra emergency release to patch the issue.

At the start of each release cycle, a release notes pull request (PR) is started and maintained throughout the week, tracking all merged PRs to ensure every change is properly documented and noted.

The PR uses the release note template and follows the release notes guidelines to use and improve the proposed release note entries from the merged PRs. When necessary, documentation and clarification for the release notes entries is requested from the merging maintainer and the contributor.

See the release notes for 455 as an example.

Once it is time to release, the release notes PR is merged and the process is kicked off. A code freeze is announced on the Trino Slack in the #releases channel, and then a maintainer utilizes the release scripts to update Trino to the next version.