Please use double quotes in the generated CURL command for windows support

[JaneX8]

The generated curl command used single quotes but that doesn't work for Windows users that have shell. Linux supports both single and double quotes. Changing --header='' to --header="" will enable windows users with curl to use it too :).

[darrenjennings]

@ElleshaHackett seems like a good idea. @ahmadnassri do you have any history/justification on the single quotes?

[darrenjennings]

Related: #106, #29

[jgiovaresco]

I plan to work on this issue, but I've found something that borrows me: https://github.com/Kong/httpsnippet/blob/master/src/helpers/shell.js#L6-L9

/**

• Use 'strong quoting' using single quotes so that we only need
• to deal with nested single quote characters.
• http://wiki.bash-hackers.org/syntax/quoting#strong_quoting
  */

This utility function is used for all shell targets. I'm wondering if

• I should rename it for doublequote and update all shell targets 😨
• I should create a doublequote function and only use it for curl target 🧐

Any thoughts? @develohpanda @darrenjennings

[develohpanda]

Hmm, kind of on the fence about this one. Looks like there has been a conscious decision to use strong quoting (single quotes), instead of weak quoting (double quotes), as per #29.

I would be inclined to make the minimum change > only updating the curl target. But, it seems this is an issue with shell targets instead of curl specifically. I'm guessing strong quoting doesn't work at all on Windows? Strong quoting does feel like the safer option, reading the docs here.

Is there something smarter we can do, for example detecting the OS and deciding the type of quote to apply automatically, and allow that to be overridden with a user provided option? This way shell targets would work on Windows automatically, have no impact to Linux/macOS and current behavior, but allow users to override it.

[jgiovaresco]

Deciding the type of quote automatically is a great idea.

This should resolve the issue when using the httpsnippet cli but what about when the lib is used in a web-based environment (I was thinking about apiembeded)

To manage both uses (cli + lib), I would suggest adding an option in convert named platform. We could use the values windows, macos, linux

We would be able to detect the platform in bin/httpsnippet and set the option automatically.
For web-based use, we would let the user set the option if needed.

[develohpanda]

That sounds like a reasonable approach.

What is your opinion on introducing a quotes option with the values single, double, instead of platform? It's a little more explicit, and can automatically be set to the correct value by the CLI bin/httpsnippet.

[dimitropoulos]

@pimterry are you able to comment on what to do here?

[pimterry]

Ok, so just to summarize the above:

• Single quotes in POSIX shells are useful, because they let us ignore all escaping rules except for other single quotes.
• But they make it inconvenient for users to edit the snippet to use variables, which is a common use case (e.g. swap an auth token value for $AUTH_TOKEN - it's annoying to have to switch the quotes out to make this work)
• Apparently Windows's cmd doesn't support single quotes at all - double quotes do broadly what you'd expect, but single quotes are ignored and passed directly to the application. Unfortunately, the escaping behaviour sounds super weird,

I've done a bunch of tests on a Windows 11 cmd shell vs bash on Linux (collapsed here because there's lots)

• No quotes, no spaces: curl -Htest:value example.com
  • Both: Sends the correct test: value header
• Unquoted spaces: curl -Htest: value example.com
  • Both: Sends a request to 'value' (which fails) and a request to example.com, with no 'test' header anywhere (I guess because an empty value was provided?)
• Double-quoted, no spaces: curl -H"test:value" example.com
  • Both: Sends the correct header
• Double-quoted spaces: curl -H"test: value" example.com
  • Both: Sends the correct header
• Single-quoted spaces: curl -H'test: value' example.com
  • Windows: Sends the request, but with no 'test' header, and no request to value? Very odd
  • Linux: works correctly
• Single-quoted, no spaces: curl -H'test:value' example.com
  • Windows: Sends the request with a 'test: value' header (i.e. incorrectly puts the single quote at the start of the header name and the end of the header value)
  • Linux: works correctly
• Double-quoted, containing escaped quotes: curl -H"test: header with \"quoted value\"" example.com
  • Both: Sends the correct header, with the quotes sent literally without the slashes.
• Double-quoted, containing real variables: curl -H"test: %PATH%" example.com ($PATH for linux)
  • Both: Sends a header containing the whole value of the PATH env var
• Double-quoted, not a real variable: curl -H"test: value-%SOMETHINGELSE%" example.com
  • Windows: Sends a header literally containing value-%SOMETHINGELSE%.
  • Linux: Sends a header containing value- (i.e. $SOMETHINGELSE for unrecognized variables disappears)
• Double-quoted, slash-escaped variable: curl -H"test: \%PATH\%" example.com (\$PATH for linux)
  • Windows: Sends a header literally containing \%PATH\% (i.e. including the slashes, so incorrect)
  • Linux: Sends the correct test: $PATH header
• Double-quoted, ^-escaped variable: curl -H"test: ^%PATH^%" example.com
  • Windows: Same thing - sends a header literally containing ^%PATH^%. Weird, because this does work correctly with echo ^%PATH^%
  • Linux: Sends test: ^ + the real contents of the $PATH variable (as expected)
• Double-quoted, selection of unescaped special characters: curl -H"test: $!`" example.com
  • Windows: Sent correctly, all characters preserved
  • Linux: $ and ` both cause errors that make commands totally unrunnable. ! causes errors for Zsh only on my machine (I think this is a bash feature that's optionally enabled though?)
• Double-quoted, escaping various special characters: curl -H"test: \$\!\`" example.com
  • Windows: All sent with the extra slash, i.e. the resulting header is test: \$\!\`.
  • Linux: All sent correctly, without the escaping slash

So:

• Single quotes are indeed totally unusable on Windows, but fine on Linux.
• Double quotes do work fine for both, and we can escape included double quotes with slashes safely everywhere, great.
• Escaping various other characters is totally different between platforms.
• Some things seem to just be unescapable, i.e. real variable names on Windows. As far as I can tell there's nothing we can do about that. Just ignore it I guess 🤷.

That means there is no combination of quote + escaping rules that will ever produce snippets that work correctly everywhere.

I think that means a quotes option is problematic - for a big chunk of users (everybody on Windows) single quotes will always be broken, and that probably won't be obvious to them initially, and for double quotes the escaping rules are very different for the two platforms in complicated ways that will cause weird issues.

A platform option would make this possible OTOH. I think this would be a bit better as shell though, supporting bash & cmd values, because it's totally possible to run bash on Windows (Git on Windows installs a Bash terminal by default, so it's very common), and because that would leave space for supporting shells that have their own unique issues in future.

With that, we could use single quotes in bash (i.e. strong quotes, so no need to escape anything except other single quotes, super easy) and double quotes in cmd. The only leftover case is real %variable% names in snippets, which are always inescapably (!!!) replaced with the real variable value, and AFAICT there's nothing we can do about that. If we ever find a solution it'd be easy to add it though without breaking anything else

A shell option wouldn't help bash users who want double quotes for easier variable usage (#106). That seems like a separate problem tbh - we could add an additional quotes option for that, to switch quote types and add escaping logic for bash special characters (throwing an error if you ever try to use cmd + single-quotes). But I think that's more like a nice-to-have, I'd ignore it for now.

How does that sound?