[WIP] Add support for using TLS Auth and AppRole auth backends to obtain a Vault token.

[woodrow]

This PR is a work in progress. It is functionally complete and should be correct -- I have tested the provider with these changes -- but I am holding off on adding tests and updating docs until I get feedback from the project maintainer that this is likely to be merged.

---

While Vault allows a number of authentication methods to generate tokens for authentication, the terraform-vault-provider itself only supports tokens directly for authentication. This PR adds support for obtaining Vault tokens using TLS Auth and AppRole auth backends configured in the provider itself, in addition to direct token authentication.

I'm new both to Go and this project and so I'm not sure if the approach I've taken here with the tokenProvider struct is the best, and so I would appreciate feedback from the maintainers on whether there's an approach better aligned with Terraform or Go here.

This should close #351, #428, and #346.

[woodrow]

Hi @tyrannosaurus-becks! Are you the right person to ask for feedback on the approach taken in this PR?

[tyrannosaurus-becks]

Hi @woodrow ! Thanks for working on this!

I like this idea, but I'm hesitant to go forth with it because it opens up a new type of PR into this repository - a specific auth method for each auth plugin that is built for Vault. I'd like to use more generic solutions for this than needing a 1:1 ratio of plugins to custom code in the provider. It'll not only provide a lower level of effort, but if things can be reused, then as new plugins come out they'll be usable day one.

Regarding authentication in particular, would it be possible to use the generic HTTP data source to gain a Vault token, and then to proceed from there?

[tyrannosaurus-becks]

Another way it might be possible to achieve this use case might be to use the AppRole auth agent to obtain a constantly fresh Vault token. I haven't POC'd this, but it may be possible to run it in a script that takes the Vault token it outputs and exports it to the external environment. I'd be curious to hear what you think of that idea, as well.

[woodrow]

Hi @tyrannosaurus-becks thanks for the feedback. I'll address your comments in reverse order:

Another way it might be possible to achieve this use case might be to use the AppRole auth agent to obtain a constantly fresh Vault token. I haven't POC'd this, but it may be possible to run it in a script that takes the Vault token it outputs and exports it to the external environment. I'd be curious to hear what you think of that idea, as well.

I was hoping to use this approach in a Terraform Enterprise environment where I wanted to inject the approle credentials as sensitive environment variables for each workspace. An agent-based approach isn't going to work well under that architecture I'm afraid.

Regarding authentication in particular, would it be possible to use the generic HTTP data source to gain a Vault token, and then to proceed from there?

The HTTP data source doesn't seem like a good fit for this purpose as it is currently implemented. Per the approle login docs /auth/approle/login uses the POST method and requires us to add the credentials in the body of the payload, both of which are not supported by the http data source provider.

I like this idea, but I'm hesitant to go forth with it because it opens up a new type of PR into this repository - a specific auth method for each auth plugin that is built for Vault. I'd like to use more generic solutions for this than needing a 1:1 ratio of plugins to custom code in the provider. It'll not only provide a lower level of effort, but if things can be reused, then as new plugins come out they'll be usable day one.

We should be able to devise a general solution for Vault auth backends that involve POSTing data provided by the user to a Vault HTTP endpoint. For example you could have something like:

provider "vault" {
    address = "..."
    ...
    token_provider {
        path = "/auth/approle/login"
        method = "POST"
        body = "{\"role_id\": \"59d6d1ca-47bb-4e7e-a40b-8be3bc5a0ba8\", \"secret_id\": \"84896a0c-1347-aa90-a4f6-aca8b7558780\"}"
    }
}

This would work well for ~all auth backends, including cert-based auth if the Vault client is set up with the client_auth credentials before the token_provider is invoked.

That said I find the general approach above to fall short from a usability perspective. For many auth backends,vault login does extra work to make it easy to work with the selected backend, and it would be awesome to support a similar approach with the Terraform provider if we could reuse this code where it makes sense. As an example, the aws auth helper does the legwork of automatically checking the usual places for AWS credentials and then generates and signs the sts:getcalleridentity request before passing it through to the Vault API.

If this provider adopted the Vault CLI helper approach, the number of additional PRs that will require your review will grow at a rate less than or equal to the list of helpers in the Vault CLI itself. Based on historic changes, this appears to be about 3 additions/year, and not all helper additions will need to be added to this provider (several of the helpers only facilitate requesting credential data via stdin and are not required in Terraform).

Either way, I would love to try and make progress on this in the short term, even if it using a generic approach as I outlined above, such that I can eventually use approle auth in my own project. Hopefully we can also continue this conversation and see if we can find an approach that works well for both maintainer workload and user experience. Please let me know what you think and I'll consider adjusting my PR accordingly. Thanks!

[tyrannosaurus-becks]

@woodrow thanks for providing your thoughts and more information about your use case. You make excellent points!

There is definitely a usability trade-off with the "something general" versus "something specific" approach. There is also a level of effort trade-off that runs in the exact opposite direction! :-) The more specific and usable, the more work must be done. I don't want to create the expectation in users of this repo that there "should" be a token provider with each auth method, when we don't have the resources to deliver on that expectation.

I do want you to achieve your use case, though, it's certainly valid. I'm totally on board with this approach.

provider "vault" {
    address = "..."
    ...
    token_provider {
        path = "/auth/approle/login"
        method = "POST"
        body = "{\"role_id\": \"59d6d1ca-47bb-4e7e-a40b-8be3bc5a0ba8\", \"secret_id\": \"84896a0c-1347-aa90-a4f6-aca8b7558780\"}"
    }
}

[tyrannosaurus-becks]

Closing this for now since it hasn't moved in a while.