Failure to query Azure Sphere compiler due to `-B` unhandled and due to shell escaping in custom configuration args

[Colengms]

In the process of looking into an IntelliSense issue the Azure Sphere team reported, I noticed some incorrect include paths in the TU.

Log Diagnostics output

-------- Diagnostics - 3/2/2022, 12:39:40 PM
Version: 1.9.2
Current Configuration:
{
    "name": "Linux",
    "includePath": [
        "${workspaceFolder}/**"
    ],
    "defines": [],
    "compilerPath": "/usr/bin/clang",
    "cStandard": "c11",
    "cppStandard": "c++14",
    "intelliSenseMode": "linux-clang-x64",
    "compilerArgs": [],
    "intelliSenseModeIsExplicit": false,
    "cStandardIsExplicit": false,
    "cppStandardIsExplicit": false,
    "mergeConfigurations": false,
    "compilerPathIsExplicit": false,
    "configurationProvider": "ms-vscode.cmake-tools",
    "browse": {
        "path": [
            "${workspaceFolder}/**"
        ],
        "limitSymbolsToIncludedHeaders": true
    }
}
Custom browse configuration: 
{
    "browsePath": [
        "/home/colen/repos/blink/Blink/HardwareDefinitions/mt3620_rdb/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc/hw",
        "/opt/azurespheresdk/Sysroots/11/usr/include",
        "/home/colen/repos/blink/Blink",
        "/home/colen/repos/blink/Blink/out/ARM-Debug/CMakeFiles",
        "/home/colen/repos/blink/Blink/out/ARM-Debug",
        "/home/colen/repos/blink/Blink/out/ARM-Debug/approotBlink/bin"
    ],
    "compilerPath": "/opt/azurespheresdk/Sysroots/11/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-g++",
    "compilerArgs": []
}
Custom configurations:
[ /home/colen/repos/blink/Blink/main.c ]
{
    "defines": [
        "_POSIX_C_SOURCE"
    ],
    "standard": "c11",
    "includePath": [
        "/home/colen/repos/blink/Blink/HardwareDefinitions/mt3620_rdb/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc",
        "/opt/azurespheresdk/HardwareDefinitions/inc/hw",
        "/opt/azurespheresdk/Sysroots/12/usr/include"
    ],
    "compilerPath": "/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-gcc",
    "compilerArgs": [
        "-B",
        "\"/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi\"",
        "-march=armv7ve",
        "-mthumb",
        "-mfpu=neon-vfpv4",
        "-mfloat-abi=hard",
        "-mcpu=cortex-a7",
        "--sysroot=\"/opt/azurespheresdk/Sysroots/12\"",
        "-g",
        "-std=c11",
        "-Wstrict-prototypes",
        "-Wno-pointer-sign",
        "-ggdb",
        "-O0",
        "-fPIC",
        "-ffunction-sections",
        "-fdata-sections",
        "-fno-strict-aliasing",
        "-fno-omit-frame-pointer",
        "-fno-exceptions",
        "-Wall",
        "-Wswitch",
        "-Wempty-body",
        "-Wconversion",
        "-Wreturn-type",
        "-Wparentheses",
        "-Wno-format",
        "-Wuninitialized",
        "-Wunreachable-code",
        "-Wunused-function",
        "-Wunused-value",
        "-Wunused-variable",
        "-Werror=implicit-function-declaration",
        "-fstack-protector-strong"
    ]
}
Translation Unit Mappings:
[ /home/colen/repos/blink/Blink/main.c ]:
    /home/colen/repos/blink/Blink/main.c
Translation Unit Configurations:
[ /home/colen/repos/blink/Blink/main.c ]:
    Process ID: 13587
    Memory Usage: 30 MB
    Includes:
        /home/colen/repos/blink/Blink/HardwareDefinitions/mt3620_rdb/inc
        /opt/azurespheresdk/HardwareDefinitions/inc
        /opt/azurespheresdk/HardwareDefinitions/inc/hw
        /opt/azurespheresdk/Sysroots/12/usr/include
        /usr/include/x86_64-linux-gnu/c++/9
        /usr/include/c++/9
        /usr/local/include
        /usr/lib/llvm-9/lib/clang/9.0.1/include
        /usr/include/x86_64-linux-gnu
        /usr/include
    Defines:
        _POSIX_C_SOURCE
    Standard Version: c11
    IntelliSense Mode: linux-gcc-x64
Total Memory Usage: 30 MB

------- Workspace parsing diagnostics -------
Number of files discovered (not excluded): 16919

Note that there are "x86_64" related include paths in the TU, despite no such paths in the received custom configuration.

When I query the compiler manually (Using the command line that was logged), I get:

colen@COLENSHOMEPC:~/repos/blink/Blink$ "/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi/arm-poky-linux-musleabi-gcc" -B -march=armv7ve -mthumb -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot="/opt/azurespheresdk/Sysroots/12" -g -std=c11 -Wstrict-prototypes -Wno-pointer-sign -ggdb -O0 -fPIC -ffunction-sections -fdata-sections -fno-strict-aliasing -fno-omit-frame-pointer -fno-exceptions -Wall -Wswitch -Wempty-body -Wconversion -Wreturn-type -Wparentheses -Wno-format -Wuninitialized -Wunreachable-code -Wunused-function -Wunused-value -Wunused-variable -Werror=implicit-function-declaration -fstack-protector-strong  -Wp,-v -E -dM -x c   /dev/null
ignoring nonexistent directory "/opt/azurespheresdk/Sysroots/12/usr/local/include"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/arm-poky-linux-musleabi/include"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/lib/arm-poky-linux-musleabi/gcc/arm-poky-linux-musleabi/9.3.0/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/azurespheresdk/Sysroots/12/usr/include/
End of search list.
#define __SSP_STRONG__ 3
...

The additional paths in the TU appear to be coming from clang, which is the 'default' detected compiler. It's unclear why those paths would be getting merged into this TU, as they are for an entirely different compiler and architecture.

[Colengms]

There are actually 2 bugs here. It looks like that CMake is giving us a non-standard -B arg, which that compiler appears to support. The gcc and clang specs indicate this is a single arg (-Bprefix) not an arg pair (-B prefix). But clang is actually rather lenient and supports many similarly defined options also as arg pairs (not sure about gcc), so we can address this by also supporting it as an arg pair.

The second issue is a CMake Tools issue (if it's breaking up the args into an array based on parsing a command line from CMake. It would instead be a CMake issue if it's providing it already in the form of an array of args). It's providing cpptools with the following compiler args:

"\"/opt/azurespheresdk/Sysroots/12/tools/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-musleabi\"",

"--sysroot=\"/opt/azurespheresdk/Sysroots/12\"",

Because these are being provided in an arg array and not a shell command line, they should not contain shell escaping. The quotes in both of these args are instances of shell escaping. Quotes are used in shell command lines to ensure a space does not get misinterpreted as delimiting multiple arguments. But when args are passed via an API as an array of individual arguments, they should not contain shell escaping. The quotes are instead interpreted literally, as if \" were specified via command line).

(We have our own similar issues with this distinction: #8649 ) We need to be diligent about adding/removing shell escaping between uses of command lines vs. arg arrays. Currently, I believe we also have a known issue in which shell escaping will occur when executing processes on Windows (where we build a command line out of the args) but not on Linux (where we invoke an API using an array of args). That may need to be addressed in addition to the CMake Tools issue, in order for this issue to be addressed on both Windows and Linux.

[Colengms]

This is the result of the query when done programmatically, with extra quotes present due to being present in the custom configuration args:

ignoring nonexistent directory ""/opt/azurespheresdk/Sysroots/12"/usr/local/include"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/arm-poky-linux-musleabi/include"
ignoring nonexistent directory ""/opt/azurespheresdk/Sysroots/12"/usr/include/"
ignoring nonexistent directory "/opt/exp23/3.1.9/sysroots/x86_64-pokysdk-linux/usr/lib/arm-poky-linux-musleabi/gcc/arm-poky-linux-musleabi/9.3.0/include"
#include "..." search starts here:
#include <...> search starts here:
End of search list.
cc1: error: no include path in which to search for stdc-predef.h

[Colengms]

Using this issue to track the mishandling of -B. The quoting issue is being addressed in 1.10 via #8963 .