Skip to content

[BUG] Athena: GetQueryExecution reports OutputLocation as a directory prefix instead of the result CSV object key #1752

Description

@dhkts1

Service
Athena (result delivery via S3)

AWS API Action
GetQueryExecution (after StartQueryExecution)

Expected behavior
Real AWS Athena writes the query result to a CSV object named after the query execution id and reports that object's key in ResultConfiguration.OutputLocation:

s3://<bucket>/<prefix>/<QueryExecutionId>.csv

s3api get-object on the reported OutputLocation succeeds. SDK-shaped consumers routinely fetch the reported location verbatim to stream results.

Actual behavior
floci reports a directory prefix (trailing slash) and writes the object to results.csv inside it:

"OutputLocation": "s3://probe-results/out/b2e6ffce-5905-468a-baf9-5c5dacdf8b91/"

aws s3 ls s3://probe-results/out/ --recursive shows the object at out/<qid>/results.csv. get-object on the reported key verbatim fails:

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Constructing the AWS-shaped key out/<qid>.csv also returns NoSuchKey — the object only exists at out/<qid>/results.csv, so a client cannot compensate by deriving the conventional key.

So even on a SUCCEEDED query, any client that follows the AWS contract (fetch the reported OutputLocation) breaks with NoSuchKey. The query engine itself works — the query returned correct data; only the reported location diverges from AWS.

Reproduction

export AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_DEFAULT_REGION=us-east-1
EP=--endpoint-url=http://localhost:4566

aws $EP s3 mb s3://probe-data
aws $EP s3 mb s3://probe-results
printf '{"eventname":"login","username":"alice"}\n{"eventname":"logout","username":"bob"}\n' > /tmp/events.json
aws $EP s3 cp /tmp/events.json s3://probe-data/events/events.json

aws $EP glue create-database --database-input Name=probedb
aws $EP glue create-table --database-name probedb --table-input '{
  "Name": "events",
  "StorageDescriptor": {
    "Columns": [{"Name":"eventname","Type":"string"},{"Name":"username","Type":"string"}],
    "Location": "s3://probe-data/events/",
    "SerdeInfo": {"SerializationLibrary": "org.openx.data.jsonserde.JsonSerDe"}
  }
}'

QID=$(aws $EP athena start-query-execution \
  --query-string "SELECT eventname, username FROM events WHERE username='alice'" \
  --query-execution-context Database=probedb \
  --result-configuration OutputLocation=s3://probe-results/out/ \
  --query QueryExecutionId --output text)
# first query on a cold container can take minutes while the engine sidecar starts
until [ "$(aws $EP athena get-query-execution --query-execution-id "$QID" \
  --query 'QueryExecution.Status.State' --output text)" = "SUCCEEDED" ]; do sleep 5; done
aws $EP athena get-query-execution --query-execution-id "$QID" \
  --query 'QueryExecution.{State:Status.State,OutputLocation:ResultConfiguration.OutputLocation}'
aws $EP s3 ls s3://probe-results/out/ --recursive
# AWS contract: fetching the key reported in OutputLocation verbatim must succeed — on floci it returns NoSuchKey
OUT=$(aws $EP athena get-query-execution --query-execution-id "$QID" \
  --query 'QueryExecution.ResultConfiguration.OutputLocation' --output text)
aws $EP s3api get-object --bucket probe-results --key "${OUT#s3://probe-results/}" /tmp/result.csv

Environment

  • Floci version: floci/floci:1.5.30-compat (also reproduces on 1.5.28-compat)
  • Running via: Docker, docker run -p 4566:4566 -v /var/run/docker.sock:/var/run/docker.sock floci/floci:1.5.30-compat (socket mounted so the Athena engine sidecar can start)
  • Client: aws-cli 2.x

Metadata

Metadata

Assignees

No one assigned

    Labels

    athenaAmazon AthenabugSomething isn't workingreleased

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions