Skip to content

Commit a0c2790

Browse files
beekldkinyoklion
andauthored
feat: implement DynamoDBDataSource + tests (#534)
## Summary Implements `DynamoDBDataSource`, similar to `RedisDataSource`, using the Proxy Relay schema as per the schema. ```cpp DynamoDBClientOptions options; options.region = "us-east-1"; options.endpoint = "http://localhost:8000"; auto src = DynamoDBDataSource::Create("my-table", "prefix", options); ``` ## Differences from the Redis source - `All()` paginates DynamoDB Query responses via `LastEvaluatedKey`; Redis `hgetall` is single-shot. - AWS SDK `InitAPI`/`ShutdownAPI` are owned by an singleton (`AwsSdkGuard`); Redis++ needs no equivalent. - Config takes a `DynamoDBClientOptions` struct (region/endpoint/credentials) instead of a single URI string. - All reads use `ConsistentRead = true` to avoid DynamoDB's default eventually-consistent staleness (same as Java) - When no prefix is configured, this source uses bare partition-key values (`features`, `$inited`), matching what Relay writes per the spec. The Redis source unconditionally does `prefix + ":" + name`, so an empty Redis prefix would produce leading-colon keys (`:features`) that no Relay deployment writes. That divergence is never exercised — Redis defaults to a non-empty prefix and the Redis tests always pass one, but wanted to note it. ## Test plan - [x] 24/24 tests green against `amazon/dynamodb-local` - [x] Unit-only error tests pass without a backend. - [x] CI Ubuntu job runs against `amazon/dynamodb-local`; macOS / Windows build-only (matches Redis workflow). - [x] No AWS header leakage in public surface. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Adds a new DynamoDB-backed data source implementation plus integration tests and CI wiring, and adjusts CMake/AWS SDK build behavior (static vs shared), which could impact build/link outcomes across platforms. > > **Overview** > Implements a real `DynamoDBDataSource` (replacing the prior scaffold) that reads flags/segments from a Relay-compatible DynamoDB table, including `Get`, `All` (with pagination), and `Initialized` checks, with consistent-read requests and structured error handling for malformed rows. > > Introduces `DynamoDBClientOptions` plus internal helpers (`AwsSdkGuard`, client factory, namespace/prefix utilities) to manage AWS SDK lifecycle and client configuration without leaking AWS headers in the public API. > > Adds a new `tests` target with DynamoDB Local-backed coverage and updates CI/workflows and release scripts to enable DynamoDB builds/tests when requested; also forces the AWS SDK FetchContent build to static archives with cache save/restore to avoid macOS shared-link issues. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c4f97d6. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>
1 parent 64b98e3 commit a0c2790

19 files changed

Lines changed: 1329 additions & 22 deletions

.github/workflows/server-dynamodb.yml

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,22 @@ on:
1414
- cron: '0 8 * * *'
1515

1616
jobs:
17-
build-dynamodb:
17+
build-test-dynamodb:
1818
runs-on: ubuntu-22.04
19+
services:
20+
dynamodb:
21+
image: amazon/dynamodb-local
22+
ports:
23+
- 8000:8000
1924
steps:
2025
# https://github.com/actions/checkout/releases/tag/v4.3.0
2126
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955
2227
- uses: ./.github/actions/ci
2328
with:
2429
cmake_target: launchdarkly-cpp-server-dynamodb-source
25-
# No tests yet; PR 1 is scaffold-only and proves the AWS SDK builds.
26-
run_tests: false
2730
# AWS C++ SDK requires libcurl at link time on Linux/macOS.
2831
install_curl: true
29-
simulate_release: false
32+
simulate_release: true
3033
build-dynamodb-mac:
3134
runs-on: macos-15
3235
steps:
@@ -36,9 +39,9 @@ jobs:
3639
with:
3740
cmake_target: launchdarkly-cpp-server-dynamodb-source
3841
platform_version: 12
39-
run_tests: false
42+
run_tests: false # TODO: figure out how to run dynamodb-local on Mac
4043
install_curl: true
41-
simulate_release: false
44+
simulate_release: true
4245
build-dynamodb-windows:
4346
runs-on: windows-2022
4447
steps:
@@ -55,6 +58,6 @@ jobs:
5558
cmake_target: launchdarkly-cpp-server-dynamodb-source
5659
platform_version: 2022
5760
toolset: msvc
58-
run_tests: false
61+
run_tests: false # TODO: figure out how to run dynamodb-local on Windows
5962
install_curl: true
60-
simulate_windows_release: false
63+
simulate_windows_release: true

cmake/aws-sdk-cpp.cmake

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,42 @@ FetchContent_Declare(aws-sdk-cpp
2323
OVERRIDE_FIND_PACKAGE
2424
)
2525

26+
# Always build the AWS SDK as static archives, even when our own library is
27+
# being built shared (LD_BUILD_SHARED_LIBS=ON). Building aws-sdk-cpp as
28+
# BUILD_SHARED_LIBS=ON produces dylibs on macOS whose dynamodb component
29+
# fails to find AWS Core symbols at link time (the AWS SDK's visibility
30+
# configuration doesn't export them consistently across its FetchContent
31+
# build). Linking aws-sdk-cpp statically into our shared wrapper sidesteps
32+
# the issue.
33+
#
34+
# This needs cache manipulation rather than a function-scoped `set()`:
35+
# aws-sdk-cpp's top-level CMakeLists pins `cmake_policy(SET CMP0077 OLD)`,
36+
# which makes its `option(BUILD_SHARED_LIBS ... ON)` ignore non-cache
37+
# variables and unconditionally write the cache. The only way to stop
38+
# `option()` from setting the cache to ON is to pre-populate the cache
39+
# with OFF before FetchContent_MakeAvailable runs.
40+
#
41+
# The prior cache state is saved before the FORCE-write and restored
42+
# immediately after FetchContent finishes, so other subprojects (e.g.
43+
# libs/server-sdk-otel) see whatever BUILD_SHARED_LIBS value was in the
44+
# cache when this file was first included. An earlier attempt that
45+
# FORCE-wrote OFF without restoring leaked into those subprojects; the
46+
# save/restore here is the fix for that.
47+
if (DEFINED CACHE{BUILD_SHARED_LIBS})
48+
set(_LD_AWS_BSL_PREV "${BUILD_SHARED_LIBS}")
49+
set(_LD_AWS_BSL_WAS_CACHED TRUE)
50+
else ()
51+
set(_LD_AWS_BSL_WAS_CACHED FALSE)
52+
endif ()
53+
54+
set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
55+
2656
FetchContent_MakeAvailable(aws-sdk-cpp)
57+
58+
if (_LD_AWS_BSL_WAS_CACHED)
59+
set(BUILD_SHARED_LIBS "${_LD_AWS_BSL_PREV}" CACHE BOOL "" FORCE)
60+
else ()
61+
unset(BUILD_SHARED_LIBS CACHE)
62+
endif ()
63+
unset(_LD_AWS_BSL_PREV)
64+
unset(_LD_AWS_BSL_WAS_CACHED)

libs/server-sdk-dynamodb-source/CMakeLists.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,3 +27,7 @@ include(FetchContent)
2727
include(${CMAKE_FILES}/aws-sdk-cpp.cmake)
2828

2929
add_subdirectory(src)
30+
31+
if (LD_BUILD_UNIT_TESTS)
32+
add_subdirectory(tests)
33+
endif ()

libs/server-sdk-dynamodb-source/README.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,7 @@ This component will allow the Server-Side SDK to retrieve feature flag configura
1010
from LaunchDarkly.
1111

1212
> [!NOTE]
13-
> This library currently contains only scaffolding. The functional `DynamoDBDataSource` and Big Segments store
14-
> implementation will land in subsequent releases.
13+
> The Big Segments store implementation will land in a subsequent release.
1514
1615
LaunchDarkly overview
1716
-------------------------
Lines changed: 75 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,81 @@
1+
/** @file dynamodb_source.hpp
2+
* @brief Server-Side DynamoDB Source
3+
*/
4+
15
#pragma once
26

7+
#include <launchdarkly/server_side/integrations/data_reader/iserialized_data_reader.hpp>
8+
#include <launchdarkly/server_side/integrations/dynamodb/options.hpp>
9+
10+
#include <tl/expected.hpp>
11+
12+
#include <memory>
13+
#include <string>
14+
15+
namespace Aws::DynamoDB {
16+
class DynamoDBClient;
17+
}
18+
319
namespace launchdarkly::server_side::integrations {
420

5-
// Scaffold-only entry point. The real DynamoDBDataSource class will replace
6-
// this in a subsequent PR; this declaration exists so the smoke .cpp has
7-
// something to define and the library produces a non-empty archive.
8-
void DynamoDBSourceLinkSmoke();
21+
/**
22+
* @brief DynamoDBDataSource represents a data source for the Server-Side SDK
23+
* backed by Amazon DynamoDB. It is meant to be used in place of the standard
24+
* LaunchDarkly Streaming or Polling data sources.
25+
*
26+
* Call DynamoDBDataSource::Create to obtain a new instance. This instance can
27+
* be passed into the SDK's DataSystem configuration via the LazyLoad builder.
28+
*
29+
* The DynamoDB table must already exist and follow the LaunchDarkly schema:
30+
* a String partition key named `namespace` and a String sort key named `key`.
31+
* The LaunchDarkly Relay Proxy populates the table with this schema; this
32+
* class only reads from it.
33+
*
34+
* This implementation is backed by the AWS SDK for C++.
35+
*/
36+
class DynamoDBDataSource final : public ISerializedDataReader {
37+
public:
38+
/**
39+
* @brief Creates a new DynamoDBDataSource, or returns an error if
40+
* construction failed.
41+
*
42+
* @param table_name Name of the DynamoDB table to read from. The table
43+
* must already exist; this class does not create it.
44+
*
45+
* @param prefix Optional namespace prefix. When non-empty, the source
46+
* reads rows whose partition key is `<prefix>:features`,
47+
* `<prefix>:segments`, etc. This allows multiple LaunchDarkly
48+
* environments to share a single table.
49+
*
50+
* @param options Optional AWS DynamoDB client configuration. See
51+
* @ref DynamoDBClientOptions. When defaulted, the AWS SDK resolves
52+
* region, endpoint, and credentials from the standard provider chain
53+
* (environment variables, shared config files, instance metadata).
54+
*
55+
* @return A DynamoDBDataSource, or an error if construction failed.
56+
*/
57+
static tl::expected<std::unique_ptr<DynamoDBDataSource>, std::string>
58+
Create(std::string table_name,
59+
std::string prefix,
60+
DynamoDBClientOptions options = {});
61+
62+
[[nodiscard]] GetResult Get(ISerializedItemKind const& kind,
63+
std::string const& itemKey) const override;
64+
[[nodiscard]] AllResult All(ISerializedItemKind const& kind) const override;
65+
[[nodiscard]] std::string const& Identity() const override;
66+
[[nodiscard]] bool Initialized() const override;
67+
68+
~DynamoDBDataSource() override;
69+
70+
private:
71+
DynamoDBDataSource(std::unique_ptr<Aws::DynamoDB::DynamoDBClient> client,
72+
std::string table_name,
73+
std::string prefix);
74+
75+
std::unique_ptr<Aws::DynamoDB::DynamoDBClient> client_;
76+
std::string const table_name_;
77+
std::string const prefix_;
78+
std::string const inited_namespace_;
79+
};
980

1081
} // namespace launchdarkly::server_side::integrations
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
/** @file options.hpp
2+
* @brief Options for constructing a DynamoDB-backed integration.
3+
*/
4+
5+
#pragma once
6+
7+
#include <optional>
8+
#include <string>
9+
10+
namespace launchdarkly::server_side::integrations {
11+
12+
/**
13+
* @brief Optional knobs for constructing the AWS DynamoDB client used by
14+
* @ref DynamoDBDataSource (and other DynamoDB-backed integrations).
15+
*
16+
* When unset, fields fall through to the AWS SDK's defaults:
17+
*
18+
* - @ref region resolves via the SDK region provider chain (environment,
19+
* shared config file, instance metadata).
20+
* - @ref endpoint defaults to the standard AWS DynamoDB endpoint for the
21+
* resolved region. Set it to point at DynamoDB Local or LocalStack, e.g.
22+
* `http://localhost:8000`.
23+
* - If none of @ref aws_access_key_id / @ref aws_secret_access_key /
24+
* @ref aws_session_token are set, the SDK's default credential provider
25+
* chain is used (environment variables, shared credentials file, EC2/ECS
26+
* roles).
27+
*/
28+
struct DynamoDBClientOptions {
29+
std::optional<std::string> region;
30+
std::optional<std::string> endpoint;
31+
std::optional<std::string> aws_access_key_id;
32+
std::optional<std::string> aws_secret_access_key;
33+
std::optional<std::string> aws_session_token;
34+
};
35+
36+
} // namespace launchdarkly::server_side::integrations

libs/server-sdk-dynamodb-source/src/CMakeLists.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ target_sources(${LIBNAME}
1414
PRIVATE
1515
${HEADER_LIST}
1616
dynamodb_source.cpp
17+
aws_sdk_guard.cpp
18+
client_factory.cpp
1719
)
1820

1921

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#include "aws_sdk_guard.hpp"
2+
3+
namespace launchdarkly::server_side::integrations::detail {
4+
5+
void AwsSdkGuard::Ensure() {
6+
static AwsSdkGuard instance;
7+
(void)instance;
8+
}
9+
10+
AwsSdkGuard::AwsSdkGuard() {
11+
Aws::InitAPI(options_);
12+
}
13+
14+
AwsSdkGuard::~AwsSdkGuard() {
15+
Aws::ShutdownAPI(options_);
16+
}
17+
18+
} // namespace launchdarkly::server_side::integrations::detail
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#pragma once
2+
3+
#include <aws/core/Aws.h>
4+
5+
namespace launchdarkly::server_side::integrations::detail {
6+
7+
// AwsSdkGuard owns the process-wide Aws::InitAPI / Aws::ShutdownAPI lifecycle
8+
// for this library. Multiple DynamoDB-backed integrations within the same
9+
// process share the single static instance; the API is initialized lazily on
10+
// first use and torn down during normal program termination via C++ static
11+
// destruction.
12+
//
13+
// Static-destruction ordering caveat: if a caller stashes a raw AWS SDK
14+
// pointer in their own static and that static is destroyed AFTER this guard,
15+
// AWS SDK calls during that destructor will be undefined. The standard usage
16+
// pattern (holding the data source / store via a unique_ptr or shared_ptr in
17+
// regular program scope, not in another static) is unaffected because those
18+
// smart pointers destruct before the guard.
19+
class AwsSdkGuard {
20+
public:
21+
// Idempotent. First call constructs the singleton, which runs
22+
// Aws::InitAPI in its constructor. Subsequent calls are no-ops. Safe to
23+
// call from any thread.
24+
static void Ensure();
25+
26+
AwsSdkGuard(AwsSdkGuard const&) = delete;
27+
AwsSdkGuard(AwsSdkGuard&&) = delete;
28+
AwsSdkGuard& operator=(AwsSdkGuard const&) = delete;
29+
AwsSdkGuard& operator=(AwsSdkGuard&&) = delete;
30+
31+
private:
32+
AwsSdkGuard();
33+
~AwsSdkGuard();
34+
35+
Aws::SDKOptions options_;
36+
};
37+
38+
} // namespace launchdarkly::server_side::integrations::detail
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
#include "client_factory.hpp"
2+
3+
#include <aws/core/auth/AWSCredentials.h>
4+
#include <aws/core/client/ClientConfiguration.h>
5+
#include <aws/core/http/Scheme.h>
6+
7+
namespace launchdarkly::server_side::integrations::detail {
8+
9+
namespace {
10+
11+
// Verifies that the credential fields in `options` form a valid combination.
12+
// Returns an empty optional on success, or an error string describing what's
13+
// wrong. The valid combinations are:
14+
//
15+
// - none of the three set (fall through to AWS default credential chain)
16+
// - access_key_id + secret_access_key (long-lived IAM keys)
17+
// - access_key_id + secret_access_key + session_token (STS temporary creds)
18+
//
19+
// All other partial combinations would build a misconfigured AWS client that
20+
// fails opaquely at request time; catching them here surfaces the
21+
// misconfiguration up front.
22+
std::optional<std::string> ValidateCredentials(
23+
DynamoDBClientOptions const& options) {
24+
bool const has_key = options.aws_access_key_id.has_value();
25+
bool const has_secret = options.aws_secret_access_key.has_value();
26+
bool const has_token = options.aws_session_token.has_value();
27+
28+
if (has_key != has_secret) {
29+
return "aws_access_key_id and aws_secret_access_key must both be set "
30+
"or both unset";
31+
}
32+
if (has_token && !has_key) {
33+
return "aws_session_token requires aws_access_key_id and "
34+
"aws_secret_access_key";
35+
}
36+
return std::nullopt;
37+
}
38+
39+
Aws::Client::ClientConfiguration BuildConfig(
40+
DynamoDBClientOptions const& options) {
41+
Aws::Client::ClientConfiguration config;
42+
43+
if (options.region) {
44+
config.region = *options.region;
45+
}
46+
47+
if (options.endpoint) {
48+
config.endpointOverride = *options.endpoint;
49+
// Use HTTP if the endpoint starts with "http://"; otherwise default
50+
// to HTTPS. Endpoint overrides are commonly DynamoDB Local or
51+
// LocalStack on plain HTTP for development.
52+
std::string const& ep = *options.endpoint;
53+
if (ep.rfind("http://", 0) == 0) {
54+
config.scheme = Aws::Http::Scheme::HTTP;
55+
config.verifySSL = false;
56+
}
57+
}
58+
59+
return config;
60+
}
61+
62+
} // namespace
63+
64+
tl::expected<std::unique_ptr<Aws::DynamoDB::DynamoDBClient>, std::string>
65+
BuildDynamoDBClient(DynamoDBClientOptions const& options) {
66+
if (auto err = ValidateCredentials(options)) {
67+
return tl::make_unexpected(std::move(*err));
68+
}
69+
70+
auto const config = BuildConfig(options);
71+
72+
if (options.aws_access_key_id) {
73+
Aws::Auth::AWSCredentials credentials{
74+
*options.aws_access_key_id, *options.aws_secret_access_key,
75+
options.aws_session_token.value_or("")};
76+
return std::make_unique<Aws::DynamoDB::DynamoDBClient>(credentials,
77+
config);
78+
}
79+
80+
return std::make_unique<Aws::DynamoDB::DynamoDBClient>(config);
81+
}
82+
83+
} // namespace launchdarkly::server_side::integrations::detail

0 commit comments

Comments
 (0)