Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Parsing and Quoting articles #8053

Merged
merged 2 commits into from
Sep 7, 2021
Merged

Update Parsing and Quoting articles #8053

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
51b2051
Update Parsing and Quoting articles
sdwheeler Sep 3, 2021
afb4795
disambiguate -> separate
sdwheeler Sep 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 166 additions & 72 deletions reference/5.1/Microsoft.PowerShell.Core/About/about_Parsing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,18 @@
---
description: Describes how PowerShell parses commands.
keywords: powershell,cmdlet
description: Describes how PowerShell parses commands.
Locale: en-US
ms.date: 09/14/2020
ms.date: 09/02/2021
online version: https://docs.microsoft.com/powershell/module/microsoft.powershell.core/about/about_parsing?view=powershell-5.1&WT.mc_id=ps-gethelp
schema: 2.0.0
title: about Parsing
---

# about_Parsing

## SHORT DESCRIPTION
## Short description

Describes how PowerShell parses commands.

## LONG DESCRIPTION
## Long description

When you enter a command at the command prompt, PowerShell breaks the command
text into a series of segments called _tokens_ and then determines how to
Expand Down Expand Up @@ -42,7 +40,7 @@ expression mode and argument mode.
> execute intended cmdlet.
> - `Get-Help` and `Get-Command` don't recognize verb-less names.

### Expression mode
## Expression mode

Expression mode is intended for combining expressions, required for value
manipulation in a scripting language. Expressions are representations of values
Expand All @@ -51,19 +49,24 @@ in PowerShell syntax, and can be simple or composite, for example:
Literal expressions are direct representations of their values:

```powershell
'hello', 32
'hello'
32
```

Variable expressions carry the value of the variable they reference:

```powershell
$x, $script:path
$x
$script:path
```

Operators combine other expressions for evaluation:

```powershell
- 12, -not $Quiet, 3 + 7, $input.Length -gt 1
-12
-not $Quiet
3 + 7
$input.Length -gt 1
```

- _Character string literals_ must be contained in quotation marks.
Expand All @@ -78,93 +81,113 @@ Operators combine other expressions for evaluation:
pasting prefilled parameter sets) is forbidden and causes a parser error.
- Anything else will be treated as a command to be invoked.

### Argument mode
## Argument mode

When parsing, PowerShell first looks to interpret input as an expression. But
when a command invocation is encountered, parsing continues in argument mode.
If you have arguments that contain spaces, such as paths, then you must
enclose those argument values in quotes.

Argument mode is designed for parsing arguments and parameters for commands in
a shell environment. All input is treated as an expandable string unless it
uses one of the following syntaxes:

- Dollar sign (`$`) begins a variable reference (only if it is followed by a
valid variable name, otherwise it is interpreted as part of the expandable
string).
- Quotation marks (`'` and `"`) begin string values
- Parentheses (`(` and `)`) demarcate new expressions.
- Subexpression operator (`$(`…`)`) demarcates embedded expressions.
- Braces (`{` and `}`) demarcate new script blocks.
- Initial at sign (`@`) begins expression syntaxes such as splatting (`@args`),
arrays (`@(1,2,3)`) and hash tables (`@{a=1;b=2}`).
- Dollar sign (`$`) followed by a variable name begins a variable reference,
otherwise it is interpreted as part of the expandable string. The variable
reference can include member access or indexing.
- Additional characters following simple variable references, such as
`$HOME`, are considered part of the same argument. Enclose the variable
name in braces (`{}`) to separate it from subsequent characters. For
example, `${HOME}`.
- When the variable reference include member access, the first of any
additional characters is considered the start of a new argument. For
example `$HOME.Length-more` results in two arguments: the value of
`$HOME.Length` and string literal `-more`.

- Quotation marks (`'` and `"`) begin strings

- Braces (`{}`) demarcate new script blocks

- Commas (`,`) introduce lists passed as arrays, except when the command to be
called is a native application, in which case they are interpreted as part of
the expandable string. Initial, consecutive or trailing commas are not
supported.

Values of embedded expressions are converted to strings.

The following table provides several examples of values processed in expression
mode and argument mode and the evaluation of those values. We assume that the
value of the variable `a` is `4`.
- Parentheses (`()`) demarcate new expressions
- Subexpression operator (`$()`) demarcates embedded expressions
- Initial at sign (`@`) begins expression syntaxes such as splatting (`@args`),
arrays (`@(1,2,3)`), and hash table literals (`@{a=1;b=2}`).
- `()`, `$()`, and `@()` at the start of a token create a new parsing context
that can contain expressions or nested commands.
- When followed by additional characters, the first additional character is
considered the start of a new, separate argument.
- When preceded by an unquoted literal `$()` works like an expandable string,
`()` starts a new argument that is an expression, and `@()` is taken as
literal `@` with `()` starting a new argument that is an expression.

- Everything else is treated as an expandable string, except
metacharacters that still need escaping.
- The argument-mode metacharacters (characters with special syntactic
meaning) are: ``<space> ' " ` , ; ( ) { } | & < > @ #``. Of these, `< > @ #`
are only special at the start of a token.

- The stop-parsing token (`--%`) changes the interpretation of all remaining
arguments. For more information, see the
[stop-parsing token](#the-stop-parsing-token) section below.

### Examples

The following table provides several examples of tokens processed in expression
mode and argument mode and the evaluation of those tokens. For these examples,
the value of the variable `$a` is `4`.

| Example | Mode | Result |
| -------------------- | ---------- | ----------------- |
| `2` | Expression | 2 (integer) |
| `` `2`` | Expression | "2" (command) |
| `echo 2` | Expression | 2 (integer) |
| `Write-Output 2` | Expression | 2 (integer) |
| `2+2` | Expression | 4 (integer) |
| `echo 2+2` | Argument | "2+2" (string) |
| `echo(2+2)` | Expression | 4 (integer) |
| `Write-Output 2+2` | Argument | "2+2" (string) |
| `Write-Output(2+2)` | Expression | 4 (integer) |
| `$a` | Expression | 4 (integer) |
| `echo $a` | Expression | 4 (integer) |
| `Write-Output $a` | Expression | 4 (integer) |
| `$a+2` | Expression | 6 (integer) |
| `echo $a+2` | Argument | 4+2 (string) |
| `Write-Output $a+2` | Argument | "4+2" (string) |
| `$-` | Argument | "$-" (command) |
| `echo $-` | Argument | "$-" (string) |
| `Write-Output $-` | Argument | "$-" (string) |
| `a$a` | Expression | "a$a" (command) |
| `echo a$a` | Argument | "a4" (string) |
| `Write-Output a$a` | Argument | "a4" (string) |
| `a'$a'` | Expression | "a$a" (command) |
| `echo a'$a'` | Argument | "a$a" (string) |
| `Write-Output a'$a'` | Argument | "a$a" (string) |
| `a"$a"` | Expression | "a$a" (command) |
| `echo a"$a"` | Argument | "a4" (string) |
| `Write-Output a"$a"` | Argument | "a4" (string) |
| `a$(2)` | Expression | "a$(2)" (command) |
| `echo a$(2)` | Argument | "a2" (string) |

Every token can be interpreted as some kind of object type, such as Boolean or
string. PowerShell attempts to determine the object type from the expression.
The object type depends on the type of parameter a command expects and on
whether PowerShell knows how to convert the argument to the correct type. The
following table shows several examples of the types assigned to values returned
by the expressions.

| Example | Mode | Result |
| ---------------------- | ---------- | --------------- |
| `Write-Output !1` | argument | "!1" (string) |
| `Write-Output (!1)` | expression | False (Boolean) |
| `Write-Output (2)` | expression | 2 (integer) |
| `Set-Variable AB A,B` | argument | 'A','B' (array) |
| `CMD /CECHO A,B` | argument | 'A,B' (string) |
| `CMD /CECHO $AB` | expression | 'A','B' (array) |
| `CMD /CECHO :$AB` | argument | ':A B' (string) |

The stop-parsing symbol (`--%`), introduced in PowerShell 3.0, directs
PowerShell to refrain from interpreting input as PowerShell commands or
expressions.

When calling an executable program in PowerShell, place the stop-parsing symbol
before the program arguments. This technique is much easier than using escape
characters to prevent misinterpretation.

When it encounters a stop-parsing symbol, PowerShell treats the remaining
characters in the line as a literal. The only interpretation it performs is to
substitute values for environment variables that use standard Windows notation,
such as `%USERPROFILE%`.
| `Write-Output a$(2)` | Argument | "a2" (string) |

The stop-parsing symbol is effective only until the next newline or pipeline
character. You cannot use a continuation character (`` ` ``) to extend its
effect or use a command delimiter (`;`) to terminate its effect.
Every token can be interpreted as some kind of object type, such as **Boolean**
or **String**. PowerShell attempts to determine the object type from the
expression. The object type depends on the type of parameter a command expects
and on whether PowerShell knows how to convert the argument to the correct
type. The following table shows several examples of the types assigned to
values returned by the expressions.

| Example | Mode | Result |
| --------------------- | ---------- | --------------- |
| `Write-Output !1` | argument | "!1" (string) |
| `Write-Output (!1)` | expression | False (Boolean) |
| `Write-Output (2)` | expression | 2 (integer) |
| `Set-Variable AB A,B` | argument | 'A','B' (array) |
| `CMD /CECHO A,B` | argument | 'A,B' (string) |
| `CMD /CECHO $AB` | expression | 'A B' (array) |
| `CMD /CECHO :$AB` | argument | ':A B' (string) |

## Passing arguments to native commands

For example, the following command calls the **Icacls** program.
When running native commands from PowerShell, the arguments are first parsed
by PowerShell. The parsed arguments are then joined into a single string with
each parameter separated by a space.

For example, the following command calls the `icacls.exe` program.

```powershell
icacls X:\VMS /grant Dom\HVAdmin:(CI)(OI)F
Expand All @@ -177,16 +200,87 @@ prevent PowerShell from misinterpreting the parentheses.
icacls X:\VMS /grant Dom\HVAdmin:`(CI`)`(OI`)F
```

Beginning in PowerShell 3.0, you can use the stop-parsing symbol.
### The stop-parsing token

Beginning in PowerShell 3.0, you can use the stop-parsing token (`--%`) to
stop PowerShell from interpreting input asPowerShell commands or expressions.

> [!NOTE]
> The stop-parsing token is only intended for use on Windows platforms.

When calling a native command, place the stop-parsing token before the program
arguments. This technique is much easier than using escape characters to
prevent misinterpretation.

When it encounters a stop-parsing token, PowerShell treats the remaining
characters in the line as a literal. The only interpretation it performs is to
substitute values for environment variables that use standard Windows notation,
such as `%USERPROFILE%`.

```powershell
icacls X:\VMS --% /grant Dom\HVAdmin:(CI)(OI)F
```

PowerShell sends the following command string to the **Icacls** program:
PowerShell sends the following command string to the `icacls.exe` program:

`X:\VMS /grant Dom\HVAdmin:(CI)(OI)F`

## SEE ALSO
The stop-parsing token is effective only until the next newline or pipeline
character. You cannot use a continuation character (`` ` ``) to extend its
effect or use a command delimiter (`;`) to terminate its effect.

Other than `%variable%` environment-variable references, you cannot embed any
other dynamic elements in the command. Escaping a `%` character as `%%`, the
way you can do inside batch files, is not supported. `%<name>%` tokens are
invariably expanded. If `<name>` does not refer to a defined environment
variable the token is passed through as-is.

You cannot use stream redirection (like `>file.txt`) because they are passed
verbatim as arguments to the target command.

### Passing arguments that contain quote characters

Some native commands expect arguments that contain quote characters. Normally,
PowerShell's command line parsing removes the quote character you provided. The
parsed arguments are then joined into a single string with each parameter
separated by a space. This string is then assigned to the **Arguments**
property of a `ProcessStartInfo` object. Quotes within the string must be
escaped using extra quotes or backslash (`\`) characters.

> [!NOTE]
> The backslash (`\`) character is not recognized as an escape character by
> PowerShell. It is the escape character used by the underlying API for
> `ProcessStartInfo.Arguments`.

For more information about the escape requirements, see the documentation for
[ProcessStartInfo.Arguments](/dotnet/api/system.diagnostics.processstartinfo.arguments).

The following examples using the `TestExe.exe` tool. This tool is used by the
Pester tests in the PowerShell source repo. The goal of these examples is to
pass the directory path `"C:\Program Files (x86)\Microsoft\"` to a native
command so that it received the path as a quoted string.

The **echoargs** parameter of `TestExe` displays the values received as
arguments to the executable. You can use this tool to verify that you have
properly escaped the characters in your arguments.

```powershell
TestExe -echoargs """""${env:ProgramFiles(x86)}\Microsoft\\"""""
TestExe -echoargs """""C:\Program Files (x86)\Microsoft\\"""""
TestExe -echoargs --% ""\""C:\Program Files (x86)\Microsoft\\""
TestExe -echoargs --% """C:\Program Files (x86)\Microsoft\\""
TestExe -echoargs --% """%ProgramFiles(x86)%\Microsoft\\""
```

The output is the same for all examples:

```Output
Arg 0 is <"C:\Program Files (x86)\Microsoft\">
```

You can build `TestExe` from the source code. See
[TestExe](https://github.com/PowerShell/PowerShell/blob/master/test/tools/TestExe/TestExe.cs).

## See also

[about_Command_Syntax](about_Command_Syntax.md)
Loading