PowerShell comment recognition is not correct

[alexr00]

From @stefanita-mitran on March 4, 2019 9:50

EG.

.\procedure.ps1 `
   -arg1 $variableforarg1 ` #this syntax highlighting should not resemble a comment
   -arg2 $variableforarg2 ` <#this syntax highlight should not resemble a comment because it's invalid to add a comment here #>
   -arg3 $variableforarg3 <#this syntax higlight should be correct#>`
   -arg4 $variableforarg4

The example above shows that the "arg1" and "arg2" have incorrect comments and should not be categorized as comment.line.powershell

Copied from original issue: microsoft/vscode#69771

[alexr00]

@stefanita-mitran I can reproduce this, but it also occurs in PowerShell ISE. I'll move this issue to the owners of the PowerShell grammar.

[msftrncs]

@stefanita-mitran, (and @alexr00) I do not see how those are not in fact comments. I think your use of the backtick is incorrect. A backtick before a space serves no purpose at the beginning of an unquoted argument, the tokenizer still skips it as its just a 'space'. This might be slightly different than in CMD where ^(space) changes the way the tokenizer handles the next token, CMD treats ^ (space) as a non breaking space, PowerShell does not treat a backtick-space as a non breaking space if it is at the beginning of an unquoted argument but does if it is within the argument. CMD on the other hand doesn't have a notion of arguments, only tokens.

Here is your code rendered with PR #156:

It appears I do have an error in my PR, as its attempting to behave as you suggest, but this is not how PowerShell actually behaves. I tested on Windows PS 5.1 and also on PS 6.2 RC 1 and they both treated the comments as actually comments. This also means that the statement ends after the -arg1 line ends, as there is nothing to continue it to the next line. PR #156 gets that right at least, showing that -arg2 would actually be treated as a command.

[msftrncs]

This is how the code should have appeared with PR 156:

Note it marks the backticks 'invalid' as they will not do anything, but they do not generate an error.

[alexr00]

@msftrncs 👌. Thank you for detailing the correct behavior here. Sounds like this will be fixed with PR #156

[msftrncs]

Clarification, the current grammar is behaving correctly, PR #156 would have caused a regression, and a recent commit has resolved that, as shown in the image in #161 (comment).

[alexr00]

Ah, I thought there was some extra change need to get to the correct behavior in #161 (comment).
Thanks for confirming that it is already correct.