System.CommandLine: Parsing

[adamsitnik]

Background and motivation

In #68578 (comment), we have introduced a set of symbol types for building a parsable hierarchy.

In this issue, we want to propose a set of APIs that allow for:

• parsing the provided hierarchy
• getting the values of parsed arguments, options and directives
• reporting and getting parse errors
• exposing parse information to symbols: to implement custom parsers and validators

API Proposal

namespace System.CommandLine.Parsing

The types presented in the details below are complete, none of our other proposals is extending them.

namespace System.CommandLine.Parsing

/// <summary>
/// Identifies the type of a <see cref="CliToken"/>.
/// </summary>
public enum TokenType
{
    /// <summary>
    /// An argument token.
    /// </summary>
    /// <see cref="CliArgument"/>
    Argument,

    /// <summary>
    /// A command token.
    /// </summary>
    /// <see cref="CliCommand"/>
    Command,
            
    /// <summary>
    /// An option token.
    /// </summary>
    /// <see cref="CliOption"/>
    Option,
            
    /// <summary>
    /// A double dash (<c>--</c>) token, which changes the meaning of subsequent tokens.
    /// </summary>
    DoubleDash,

    /// <summary>
    /// A directive token.
    /// </summary>
    /// <see cref="CliDirective"/>
    Directive
}

/// <summary>
/// A unit of significant text on the command line.
/// </summary>
public sealed class CliToken : IEquatable<CliToken>
{
    /// <param name="value">The string value of the token.</param>
    /// <param name="type">The type of the token.</param>
    /// <param name="symbol">The symbol represented by the token</param>
    public CliToken(string? value, TokenType type, CliSymbol symbol);

    /// <summary>
    /// The string value of the token.
    /// </summary>
    public string Value { get; }

    /// <summary>
    /// The type of the token.
    /// </summary>
    public TokenType Type { get; }
}

/// <summary>
/// Describes an error that occurs while parsing command line input.
/// </summary>
public sealed class ParseError
{
    internal ParseError(string message, SymbolResult? symbolResult = null);

    /// <summary>
    /// A message to explain the error to a user.
    /// </summary>
    public string Message { get; }

    /// <summary>
    /// The symbol result detailing the symbol that failed to parse and the tokens involved.
    /// </summary>
    public SymbolResult? SymbolResult { get; }
}

/// <summary>
/// A result produced during parsing for a specific symbol.
/// </summary>
public abstract class SymbolResult
{
    // SymbolResultTree is an internal type
    private protected SymbolResult(SymbolResultTree symbolResultTree, SymbolResult? parent);

    /// <summary>
    /// The parent symbol result in the parse tree.
    /// </summary>
    public SymbolResult? Parent { get; }

    /// <summary>
    /// The list of tokens associated with this symbol result during parsing.
    /// </summary>
    public IReadOnlyList<CliToken> Tokens { get; }

    /// <summary>
    /// Adds an error message for this symbol result to it's parse tree.
    /// </summary>
    /// <remarks>Setting an error will cause the parser to indicate an error for the user and prevent invocation of the command line.</remarks>
    public virtual void AddError(string errorMessage);

    /// <summary>
    /// Finds a result for the specific argument anywhere in the parse tree, including parent and child symbol results.
    /// </summary>
    /// <param name="argument">The argument for which to find a result.</param>
    /// <returns>An argument result if the argument was matched by the parser or has a default value; otherwise, <c>null</c>.</returns>
    public ArgumentResult? FindResultFor(CliArgument argument);

    public CommandResult? FindResultFor(CliCommand command);

    public OptionResult? FindResultFor(CliOption option);

    public DirectiveResult? FindResultFor(CliDirective directive);

    public T? GetValue<T>(CliArgument<T> argument);

    public T? GetValue<T>(CliOption<T> option);
}

/// <summary>
/// A result produced when parsing an <see cref="CliArgument"/>.
/// </summary>
public sealed class ArgumentResult : SymbolResult
{
    internal ArgumentResult(CliArgument argument, SymbolResultTree symbolResultTree, SymbolResult? parent);

    /// <summary>
    /// The argument to which the result applies.
    /// </summary>
    public CliArgument Argument { get; }

    /// <summary>
    /// Gets the parsed value or the default value for <see cref="Argument"/>.
    /// </summary>
    /// <returns>The parsed value or the default value for <see cref="Argument"/></returns>
    public T GetValueOrDefault<T>();

    /// <summary>
    /// Specifies the maximum number of tokens to consume for the argument. Remaining tokens are passed on and can be consumed by later arguments, or will otherwise be added to <see cref="ParseResult.UnmatchedTokens"/>
    /// </summary>
    /// <param name="numberOfTokens">The number of tokens to take. The rest are passed on.</param>
    /// <exception cref="ArgumentOutOfRangeException">numberOfTokens - Value must be at least 1.</exception>
    /// <exception cref="InvalidOperationException">Thrown if this method is called more than once.</exception>
    /// <exception cref="NotSupportedException">Thrown if this method is called by Option-owned ArgumentResult.</exception>
    public void OnlyTake(int numberOfTokens);
}

/// <summary>
/// A result produced when parsing an <see cref="CliOption" />.
/// </summary>
public sealed class OptionResult : SymbolResult
{
    internal OptionResult(CliOption option, SymbolResultTree symbolResultTree, Token? token = null, CommandResult? parent = null);

    /// <summary>
    /// The option to which the result applies.
    /// </summary>
    public CliOption Option { get; }

    /// <summary>
    /// Indicates whether the result was created implicitly and not due to the option being specified on the command line.
    /// </summary>
    /// <remarks>Implicit results commonly result from options having a default value.</remarks>
    public bool Implicit { get; }

    /// <summary>
    /// The token that was parsed to specify the option.
    /// </summary>
    public CliToken? IdentifierToken { get; }

    /// <summary>
    /// Gets the parsed value or the default value for <see cref="Option"/>.
    /// </summary>
    /// <returns>The parsed value or the default value for <see cref="Option"/></returns>
    public T? GetValueOrDefault<T>();
}

/// <summary>
/// A result produced when parsing an <see cref="CliDirective"/>.
/// </summary>
public sealed class DirectiveResult : SymbolResult
{
    internal DirectiveResult(CliDirective directive, Token token, SymbolResultTree symbolResultTree);

    /// <summary>
    /// Parsed values of [name:value] directive(s).
    /// </summary>
    /// <remarks>Can be empty for [name] directives.</remarks>
    public IReadOnlyList<string> Values { get; }

    /// <summary>
    /// The directive to which the result applies.
    /// </summary>
    public CliDirective Directive { get; }

    /// <summary>
    /// The token that was parsed to specify the directive.
    /// </summary>
    public CliToken IdentifierToken { get; }
}

/// <summary>
/// A result produced when parsing a <see cref="CliCommand" />.
/// </summary>
public sealed class CommandResult : SymbolResult
{
    internal CommandResult(CliCommand command, Token token, SymbolResultTree symbolResultTree, CommandResult? parent = null);

    /// <summary>
    /// The command to which the result applies.
    /// </summary>
    public CliCommand Command { get; }

    /// <summary>
    /// The token that was parsed to specify the command.
    /// </summary>
    public CliToken IdentifierToken { get; }

    /// <summary>
    /// Child symbol results in the parse tree.
    /// </summary>
    public IEnumerable<SymbolResult> Children { get; }
}

namespace System.CommandLine;

namespace System.CommandLine;

/// <summary>
/// Defines the arity of an option or argument.
/// </summary>
/// <remarks>The arity refers to the number of values that can be passed on the command line.
/// </remarks>
public readonly struct ArgumentArity : IEquatable<ArgumentArity>
{
    /// <summary>
    /// Initializes a new instance of the ArgumentArity class.
    /// </summary>
    /// <param name="minimumNumberOfValues">The minimum number of values required for the argument.</param>
    /// <param name="maximumNumberOfValues">The maximum number of values allowed for the argument.</param>
    /// <exception cref="ArgumentOutOfRangeException">Thrown when <paramref name="minimumNumberOfValues"/> is negative.</exception>
    /// <exception cref="ArgumentException">Thrown when the maximum number is less than the minimum number or the maximum number is greater than MaximumArity.</exception>
    public ArgumentArity(int minimumNumberOfValues, int maximumNumberOfValues);

    /// <summary>
    /// Gets the minimum number of values required for an <see cref="CliArgument">argument</see>.
    /// </summary>
    public int MinimumNumberOfValues { get; }

    /// <summary>
    /// Gets the maximum number of values allowed for an <see cref="CliArgument">argument</see>.
    /// </summary>
    public int MaximumNumberOfValues { get; }

    /// <summary>
    /// An arity that does not allow any values.
    /// </summary>
    public static ArgumentArity Zero => new(0, 0);

    /// <summary>
    /// An arity that may have one value, but no more than one.
    /// </summary>
    public static ArgumentArity ZeroOrOne => new(0, 1);

    /// <summary>
    /// An arity that must have exactly one value.
    /// </summary>
    public static ArgumentArity ExactlyOne => new(1, 1);

    /// <summary>
    /// An arity that may have multiple values.
    /// </summary>
    public static ArgumentArity ZeroOrMore => new(0, MaximumArity);

    /// <summary>
    /// An arity that must have at least one value.
    /// </summary>
    public static ArgumentArity OneOrMore => new(1, MaximumArity);
}

public abstract class CliArgument : CliSymbol
{
    /// <summary>
    /// Gets or sets the arity of the argument.
    /// </summary>
    public ArgumentArity Arity { get; set; }
    
    /// <summary>
    /// Specifies if a default value is defined for the argument.
    /// </summary>
    public abstract bool HasDefaultValue { get; }

    /// <summary>
    /// Gets the default value for the argument.
    /// </summary>
    /// <returns>Returns the default value for the argument, if defined. Null otherwise.</returns>
    public object? GetDefaultValue();
}

public class CliArgument<T> : CliArgument
{
    /// <summary>
    /// A custom argument parser.
    /// </summary>
    /// <remarks>
    /// It's invoked when there was parse input provided for given Argument.
    /// The same instance can be set as <see cref="DefaultValueFactory"/>, in such case
    /// the delegate is also invoked when no input was provided.
    /// </remarks>
    public Func<ArgumentResult, T>? CustomParser { get; set; }

    /// <summary>
    /// The delegate to invoke to create the default value.
    /// </summary>
    /// <remarks>
    /// It's invoked when there was no parse input provided for given Argument.
    /// The same instance can be set as <see cref="CustomParser"/>, in such case
    /// the delegate is also invoked when an input was provided.
    /// </remarks>
    public Func<ArgumentResult, T>? DefaultValueFactory { get; set; }
}

/// <summary>
/// A symbol defining a named parameter and a value for that parameter. 
/// </summary>
public abstract class CliOption : CliSymbol
{
    public ArgumentArity Arity { get; set; }

    public abstract bool HasDefaultValue { get; }

    public object? GetDefaultValue();

    /// <summary>
    /// Gets a value that indicates whether multiple argument tokens are allowed for each option identifier token.
    /// </summary>
    /// <example>
    /// If set to <see langword="true"/>, the following command line is valid for passing multiple arguments:
    /// <code>
    /// > --opt 1 2 3
    /// </code>
    /// The following is equivalent and is always valid:
    /// <code>
    /// > --opt 1 --opt 2 --opt 3
    /// </code>
    /// </example>
    public bool AllowMultipleArgumentsPerToken { get; set; }
}

public class CliOption<T> : CliOption
{
    public Func<ArgumentResult, T>? DefaultValueFactory { get; set; }

    public Func<ArgumentResult, T>? CustomParser { get; set; }
}

public class CliCommand : CliSymbol, IEnumerable<CliSymbol>
{
    /// <summary>
    /// Gets or sets a value that indicates whether unmatched tokens should be treated as errors. For example,
    /// if set to <see langword="true"/> and an extra command or argument is provided, validation will fail.
    /// </summary>
    public bool TreatUnmatchedTokensAsErrors { get; set; } = true;

    /// <summary>
    /// Parses an array strings using the command.
    /// </summary>
    /// <param name="args">The string arguments to parse.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <returns>A parse result describing the outcome of the parse operation.</returns>
    public ParseResult Parse(IReadOnlyList<string> args, CliConfiguration? configuration = null);

    /// <summary>
    /// Parses a command line string value using the command.
    /// </summary>
    /// <remarks>The command line string input will be split into tokens as if it had been passed on the command line.</remarks>
    /// <param name="commandLine">A command line string to parse, which can include spaces and quotes equivalent to what can be entered into a terminal.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <returns>A parse result describing the outcome of the parse operation.</returns>
    public ParseResult Parse(string commandLine, CliConfiguration? configuration = null);
}

/// <summary>
/// Parses command line input.
/// </summary>
public static class CliParser
{
    /// <summary>
    /// Parses a list of arguments.
    /// </summary>
    /// <param name="command">The command to use to parse the command line input.</param>
    /// <param name="args">The string array typically passed to a program's <c>Main</c> method.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <returns>A <see cref="ParseResult"/> providing details about the parse operation.</returns>
    public static ParseResult Parse(CliCommand command, IReadOnlyList<string> args, CliConfiguration? configuration = null);

    /// <summary>
    /// Parses a command line string.
    /// </summary>
    /// <param name="command">The command to use to parse the command line input.</param>
    /// <param name="commandLine">The complete command line input prior to splitting and tokenization. This input is not typically available when the parser is called from <c>Program.Main</c>. It is primarily used when calculating completions via the <c>dotnet-suggest</c> tool.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <remarks>The command line string input will be split into tokens as if it had been passed on the command line.</remarks>
    /// <returns>A <see cref="ParseResult"/> providing details about the parse operation.</returns>
    public static ParseResult Parse(CliCommand command, string commandLine, CliConfiguration? configuration = null);

    /// <summary>
    /// Splits a string into a sequence of strings based on whitespace and quotation marks.
    /// </summary>
    /// <param name="commandLine">A command line input string.</param>
    /// <returns>A sequence of strings.</returns>
    public static IEnumerable<string> SplitCommandLine(string commandLine);
}

/// <summary>
/// Describes the results of parsing a command line input based on a specific parser configuration.
/// </summary>
public sealed class ParseResult
{
    internal ParseResult(CliConfiguration configuration, CommandResult rootCommandResult, CommandResult commandResult,
        List<Token> tokens, IReadOnlyList<Token>? unmatchedTokens, List<ParseError>? errors, string? commandLineText = null, CliAction? action = null);

    /// <summary>
    /// A result indicating the command specified in the command line input.
    /// </summary>
    public CommandResult CommandResult { get; }

    /// <summary>
    /// The configuration used to produce the parse result.
    /// </summary>
    public CliConfiguration Configuration { get; }

    /// <summary>
    /// Gets the root command result.
    /// </summary>
    public CommandResult RootCommandResult { get; }

    /// <summary>
    /// Gets the parse errors found while parsing command line input.
    /// </summary>
    public IReadOnlyList<ParseError> Errors { get; }

    /// <summary>
    /// Gets the tokens identified while parsing command line input.
    /// </summary>
    public IReadOnlyList<CliToken> Tokens { get; }

    /// <summary>
    /// Gets the list of tokens used on the command line that were not matched by the parser.
    /// </summary>
    public string[] UnmatchedTokens { get; }

    /// <summary>
    /// Gets the parsed or default value for the specified argument.
    /// </summary>
    /// <param name="argument">The argument for which to get a value.</param>
    /// <returns>The parsed value or a configured default.</returns>
    public T? GetValue<T>(CliArgument<T> argument);

    /// <summary>
    /// Gets the parsed or default value for the specified option.
    /// </summary>
    /// <param name="option">The option for which to get a value.</param>
    /// <returns>The parsed value or a configured default.</returns>
    public T? GetValue<T>(CliOption<T> option);

    /// <summary>
    /// Gets the parsed or default value for the specified symbol name, in the context of parsed command (not entire symbol tree).
    /// </summary>
    /// <param name="name">The name of the Symbol for which to get a value.</param>
    /// <returns>The parsed value or a configured default.</returns>
    /// <exception cref="InvalidOperationException">Thrown when parsing resulted in parse error(s).</exception>
    /// <exception cref="ArgumentException">Thrown when there was no symbol defined for given name for the parsed command.</exception>
    /// <exception cref="InvalidCastException">Thrown when parsed result can not be casted to <typeparamref name="T"/>.</exception>
    public T? GetValue<T>(string name);
    
    /// <summary>
    /// Gets the result, if any, for the specified argument.
    /// </summary>
    /// <param name="argument">The argument for which to find a result.</param>
    /// <returns>A result for the specified argument, or <see langword="null"/> if it was not provided and no default was configured.</returns>
    public ArgumentResult? FindResultFor(CliArgument argument);

    public CommandResult? FindResultFor(CliCommand command);

    public OptionResult? FindResultFor(CliOption option);

    public DirectiveResult? FindResultFor(CliDirective directive);

    public SymbolResult? FindResultFor(CliSymbol symbol);
}

API Usage

Building dotnet build with S.CL:

// Sample: "dotnet build"
CliCommand dotnetBuild = new CliCommand("build")
{
    new CliArgument<FileInfo?>("file")
    {
        Description = "The project or solution file to operate on. If a file is not specified, the command will search the current directory for one.",
        HelpName = "PROJECT | SOLUTION",
        DefaultValueFactory = (argumentResult) =>
        {
            FileInfo? first = new DirectoryInfo(Directory.GetCurrentDirectory()).EnumerateFiles("*.sln|*.proj").FirstOrDefault();

            if (first is null)
            {
                argumentResult.AddError("Unable to find project or a solution file");
            }
            
            return first;
        }
    },
    new CliOption<bool>("--use-current-runtime", "--ucr")
    {
        Description = "Use current runtime as the target runtime."
    },
    new CliOption<string>("--framework", "-f")
    {
        Description = "The target framework to build for. The target framework must also be specified in the project file.",
        HelpName = "FRAMEWORK",
    },
    new CliOption<Architecture>("--arch", "-a")
    {
        Description = "The target architecture.",
        HelpName = "arch",
        // Enums don't require a custom parser, it's for example purposes
        CustomParser = (argumentResult) => Enum.Parse<Architecture>(argumentResult.Tokens[^1]),
        DefaultValueFactory = (_) => RuntimeInformation.OSArchitecture,
    },
    new CliOption<string>("--os")
    {
        Description = "The target operating system.",
        HelpName = "os",
        CustomParser = GetOsName,
        DefaultValueFactory = GetOsName,
    },
    new CliOption<bool>("--help")
    {
        Description = "Show command line help.",
        // just to show that this syntax is allowed
        Aliases = { "-h", "-?" }
    }
};

// an example of both a parser and a custom value provider
static string GetOsName(ArgumentResult argumentResult)
{
    if (argumentResult.Tokens.Count > 0)
    {
        string provided = argumentResult.Tokens[^1];

        if (provided is "win" or "lin" or "osx")
        {
            return provided;
        }

        argumentResult.AddError($"Unknown Operating System: {provided}");
    }

    return OperatingSystem.IsWindows() ? "win" : OperatingSystem.IsLinux() ? "lin" : "osx";
}

Implementing ping utility:

static void Main(string[] args)
{
    Command command = new("ping")
    {
        new Argument<string>("target"),
        new Option<int>("-n")
        {
            DefaultValueFactory = (_) => 4,
            Description = "Number of echo requests to send."
        }
    };

    ParseResult parseResult = command.Parse(args);

    string target = parseResult.GetValue<string>("target");
    int count = parseResult.GetValue<int>("-n");

    using Ping ping = new();
    for (int i = 0; i < count; i++)
    {
        PingReply pingReply = ping.Send(target);
        Console.WriteLine($"{i}, Status: {pingReply.Status}, RoundtripTime: {pingReply.RoundtripTime}");
    }
}

Please keep in mind that the example used above does not use the concept of action that we are going to present in a separate proposal.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-console
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In #68578 (comment), we have introduced a set of symbol types for building a parsable hierarchy.

In this issue, we want to propose a set of APIs that allow for:

• parsing the provided hierarchy
• getting the values of parsed arguments, options and directives
• reporting and getting parse errors
• exposing parse information to symbols: to implement custom parsers and validators

namespace System.CommandLine.Parsing

namespace System.CommandLine.Parsing

/// <summary>
/// Identifies the type of a <see cref="CliToken"/>.
/// </summary>
public enum TokenType
{
    /// <summary>
    /// An argument token.
    /// </summary>
    /// <see cref="CliArgument"/>
    Argument,

    /// <summary>
    /// A command token.
    /// </summary>
    /// <see cref="CliCommand"/>
    Command,
            
    /// <summary>
    /// An option token.
    /// </summary>
    /// <see cref="CliOption"/>
    Option,
            
    /// <summary>
    /// A double dash (<c>--</c>) token, which changes the meaning of subsequent tokens.
    /// </summary>
    DoubleDash,

    /// <summary>
    /// A directive token.
    /// </summary>
    /// <see cref="CliDirective"/>
    Directive
}

/// <summary>
/// A unit of significant text on the command line.
/// </summary>
public sealed class CliToken : IEquatable<CliToken>
{
    /// <param name="value">The string value of the token.</param>
    /// <param name="type">The type of the token.</param>
    /// <param name="symbol">The symbol represented by the token</param>
    public CliToken(string? value, TokenType type, CliSymbol symbol);

    /// <summary>
    /// The string value of the token.
    /// </summary>
    public string Value { get; }

    /// <summary>
    /// The type of the token.
    /// </summary>
    public TokenType Type { get; }
}

/// <summary>
/// Describes an error that occurs while parsing command line input.
/// </summary>
public sealed class ParseError
{
    internal ParseError(string message, SymbolResult? symbolResult = null);

    /// <summary>
    /// A message to explain the error to a user.
    /// </summary>
    public string Message { get; }

    /// <summary>
    /// The symbol result detailing the symbol that failed to parse and the tokens involved.
    /// </summary>
    public SymbolResult? SymbolResult { get; }
}

/// <summary>
/// A result produced during parsing for a specific symbol.
/// </summary>
public abstract class SymbolResult
{
    // SymbolResultTree is an internal type
    private protected SymbolResult(SymbolResultTree symbolResultTree, SymbolResult? parent);

    /// <summary>
    /// The parent symbol result in the parse tree.
    /// </summary>
    public SymbolResult? Parent { get; }

    /// <summary>
    /// The list of tokens associated with this symbol result during parsing.
    /// </summary>
    public IReadOnlyList<CliToken> Tokens { get; }

    /// <summary>
    /// Adds an error message for this symbol result to it's parse tree.
    /// </summary>
    /// <remarks>Setting an error will cause the parser to indicate an error for the user and prevent invocation of the command line.</remarks>
    public virtual void AddError(string errorMessage);

    /// <summary>
    /// Finds a result for the specific argument anywhere in the parse tree, including parent and child symbol results.
    /// </summary>
    /// <param name="argument">The argument for which to find a result.</param>
    /// <returns>An argument result if the argument was matched by the parser or has a default value; otherwise, <c>null</c>.</returns>
    public ArgumentResult? FindResultFor(CliArgument argument);

    public CommandResult? FindResultFor(CliCommand command);

    public OptionResult? FindResultFor(CliOption option);

    public DirectiveResult? FindResultFor(CliDirective directive);

    public T? GetValue<T>(CliArgument<T> argument);

    public T? GetValue<T>(CliOption<T> option);
}

/// <summary>
/// A result produced when parsing an <see cref="CliArgument"/>.
/// </summary>
public sealed class ArgumentResult : SymbolResult
{
    internal ArgumentResult(CliArgument argument, SymbolResultTree symbolResultTree, SymbolResult? parent);

    /// <summary>
    /// The argument to which the result applies.
    /// </summary>
    public CliArgument Argument { get; }

    /// <summary>
    /// Gets the parsed value or the default value for <see cref="Argument"/>.
    /// </summary>
    /// <returns>The parsed value or the default value for <see cref="Argument"/></returns>
    public T GetValueOrDefault<T>();

    /// <summary>
    /// Specifies the maximum number of tokens to consume for the argument. Remaining tokens are passed on and can be consumed by later arguments, or will otherwise be added to <see cref="ParseResult.UnmatchedTokens"/>
    /// </summary>
    /// <param name="numberOfTokens">The number of tokens to take. The rest are passed on.</param>
    /// <exception cref="ArgumentOutOfRangeException">numberOfTokens - Value must be at least 1.</exception>
    /// <exception cref="InvalidOperationException">Thrown if this method is called more than once.</exception>
    /// <exception cref="NotSupportedException">Thrown if this method is called by Option-owned ArgumentResult.</exception>
    public void OnlyTake(int numberOfTokens);
}

/// <summary>
/// A result produced when parsing an <see cref="CliOption" />.
/// </summary>
public sealed class OptionResult : SymbolResult
{
    internal OptionResult(CliOption option, SymbolResultTree symbolResultTree, Token? token = null, CommandResult? parent = null);

    /// <summary>
    /// The option to which the result applies.
    /// </summary>
    public CliOption Option { get; }

    /// <summary>
    /// Indicates whether the result was created implicitly and not due to the option being specified on the command line.
    /// </summary>
    /// <remarks>Implicit results commonly result from options having a default value.</remarks>
    public bool Implicit { get; }

    /// <summary>
    /// The token that was parsed to specify the option.
    /// </summary>
    public CliToken? IdentifierToken { get; }

    /// <summary>
    /// Gets the parsed value or the default value for <see cref="Option"/>.
    /// </summary>
    /// <returns>The parsed value or the default value for <see cref="Option"/></returns>
    public T? GetValueOrDefault<T>();
}

/// <summary>
/// A result produced when parsing an <see cref="CliDirective"/>.
/// </summary>
public sealed class DirectiveResult : SymbolResult
{
    internal DirectiveResult(CliDirective directive, Token token, SymbolResultTree symbolResultTree);

    /// <summary>
    /// Parsed values of [name:value] directive(s).
    /// </summary>
    /// <remarks>Can be empty for [name] directives.</remarks>
    public IReadOnlyList<string> Values { get; }

    /// <summary>
    /// The directive to which the result applies.
    /// </summary>
    public CliDirective Directive { get; }

    /// <summary>
    /// The token that was parsed to specify the directive.
    /// </summary>
    public CliToken IdentifierToken { get; }
}

/// <summary>
/// A result produced when parsing a <see cref="CliCommand" />.
/// </summary>
public sealed class CommandResult : SymbolResult
{
    internal CommandResult(CliCommand command, Token token, SymbolResultTree symbolResultTree, CommandResult? parent = null);

    /// <summary>
    /// The command to which the result applies.
    /// </summary>
    public CliCommand Command { get; }

    /// <summary>
    /// The token that was parsed to specify the command.
    /// </summary>
    public CliToken IdentifierToken { get; }

    /// <summary>
    /// Child symbol results in the parse tree.
    /// </summary>
    public IEnumerable<SymbolResult> Children { get; }
}

namespace System.CommandLine;

namespace System.CommandLine;

/// <summary>
/// Defines the arity of an option or argument.
/// </summary>
/// <remarks>The arity refers to the number of values that can be passed on the command line.
/// </remarks>
public readonly struct ArgumentArity : IEquatable<ArgumentArity>
{
    /// <summary>
    /// Initializes a new instance of the ArgumentArity class.
    /// </summary>
    /// <param name="minimumNumberOfValues">The minimum number of values required for the argument.</param>
    /// <param name="maximumNumberOfValues">The maximum number of values allowed for the argument.</param>
    /// <exception cref="ArgumentOutOfRangeException">Thrown when <paramref name="minimumNumberOfValues"/> is negative.</exception>
    /// <exception cref="ArgumentException">Thrown when the maximum number is less than the minimum number or the maximum number is greater than MaximumArity.</exception>
    public ArgumentArity(int minimumNumberOfValues, int maximumNumberOfValues);

    /// <summary>
    /// Gets the minimum number of values required for an <see cref="CliArgument">argument</see>.
    /// </summary>
    public int MinimumNumberOfValues { get; }

    /// <summary>
    /// Gets the maximum number of values allowed for an <see cref="CliArgument">argument</see>.
    /// </summary>
    public int MaximumNumberOfValues { get; }

    /// <summary>
    /// An arity that does not allow any values.
    /// </summary>
    public static ArgumentArity Zero => new(0, 0);

    /// <summary>
    /// An arity that may have one value, but no more than one.
    /// </summary>
    public static ArgumentArity ZeroOrOne => new(0, 1);

    /// <summary>
    /// An arity that must have exactly one value.
    /// </summary>
    public static ArgumentArity ExactlyOne => new(1, 1);

    /// <summary>
    /// An arity that may have multiple values.
    /// </summary>
    public static ArgumentArity ZeroOrMore => new(0, MaximumArity);

    /// <summary>
    /// An arity that must have at least one value.
    /// </summary>
    public static ArgumentArity OneOrMore => new(1, MaximumArity);
}

public abstract class CliArgument : CliSymbol
{
    /// <summary>
    /// Gets or sets the arity of the argument.
    /// </summary>
    public ArgumentArity Arity { get; set; }
    
    /// <summary>
    /// Specifies if a default value is defined for the argument.
    /// </summary>
    public abstract bool HasDefaultValue { get; }

    /// <summary>
    /// Gets the default value for the argument.
    /// </summary>
    /// <returns>Returns the default value for the argument, if defined. Null otherwise.</returns>
    public object? GetDefaultValue();
}

public class CliArgument<T> : CliArgument
{
    /// <summary>
    /// A custom argument parser.
    /// </summary>
    /// <remarks>
    /// It's invoked when there was parse input provided for given Argument.
    /// The same instance can be set as <see cref="DefaultValueFactory"/>, in such case
    /// the delegate is also invoked when no input was provided.
    /// </remarks>
    public Func<ArgumentResult, T>? CustomParser { get; set; }

    /// <summary>
    /// The delegate to invoke to create the default value.
    /// </summary>
    /// <remarks>
    /// It's invoked when there was no parse input provided for given Argument.
    /// The same instance can be set as <see cref="CustomParser"/>, in such case
    /// the delegate is also invoked when an input was provided.
    /// </remarks>
    public Func<ArgumentResult, T>? DefaultValueFactory { get; set; }
}

/// <summary>
/// A symbol defining a named parameter and a value for that parameter. 
/// </summary>
public abstract class CliOption : CliSymbol
{
    /// <summary>
    /// Gets or sets the arity of the option.
    /// </summary>
    public ArgumentArity Arity { get; set; }

    /// <summary>
    /// Gets a value that indicates whether multiple argument tokens are allowed for each option identifier token.
    /// </summary>
    /// <example>
    /// If set to <see langword="true"/>, the following command line is valid for passing multiple arguments:
    /// <code>
    /// > --opt 1 2 3
    /// </code>
    /// The following is equivalent and is always valid:
    /// <code>
    /// > --opt 1 --opt 2 --opt 3
    /// </code>
    /// </example>
    public bool AllowMultipleArgumentsPerToken { get; set; }
}

public class CliOption<T> : CliOption
{
    public Func<ArgumentResult, T>? DefaultValueFactory { get; set; }

    public Func<ArgumentResult, T>? CustomParser { get; set; }
}

public class CliCommand : CliSymbol, IEnumerable<CliSymbol>
{
    /// <summary>
    /// Gets or sets a value that indicates whether unmatched tokens should be treated as errors. For example,
    /// if set to <see langword="true"/> and an extra command or argument is provided, validation will fail.
    /// </summary>
    public bool TreatUnmatchedTokensAsErrors { get; set; } = true;

    /// <summary>
    /// Parses an array strings using the command.
    /// </summary>
    /// <param name="args">The string arguments to parse.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <returns>A parse result describing the outcome of the parse operation.</returns>
    public ParseResult Parse(IReadOnlyList<string> args, CliConfiguration? configuration = null);

    /// <summary>
    /// Parses a command line string value using the command.
    /// </summary>
    /// <remarks>The command line string input will be split into tokens as if it had been passed on the command line.</remarks>
    /// <param name="commandLine">A command line string to parse, which can include spaces and quotes equivalent to what can be entered into a terminal.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <returns>A parse result describing the outcome of the parse operation.</returns>
    public ParseResult Parse(string commandLine, CliConfiguration? configuration = null);
}

/// <summary>
/// Parses command line input.
/// </summary>
public static class CliParser
{
    /// <summary>
    /// Parses a list of arguments.
    /// </summary>
    /// <param name="command">The command to use to parse the command line input.</param>
    /// <param name="args">The string array typically passed to a program's <c>Main</c> method.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <returns>A <see cref="ParseResult"/> providing details about the parse operation.</returns>
    public static ParseResult Parse(CliCommand command, IReadOnlyList<string> args, CliConfiguration? configuration = null);

    /// <summary>
    /// Parses a command line string.
    /// </summary>
    /// <param name="command">The command to use to parse the command line input.</param>
    /// <param name="commandLine">The complete command line input prior to splitting and tokenization. This input is not typically available when the parser is called from <c>Program.Main</c>. It is primarily used when calculating completions via the <c>dotnet-suggest</c> tool.</param>
    /// <param name="configuration">The configuration on which the parser's grammar and behaviors are based.</param>
    /// <remarks>The command line string input will be split into tokens as if it had been passed on the command line.</remarks>
    /// <returns>A <see cref="ParseResult"/> providing details about the parse operation.</returns>
    public static ParseResult Parse(CliCommand command, string commandLine, CliConfiguration? configuration = null);

    /// <summary>
    /// Splits a string into a sequence of strings based on whitespace and quotation marks.
    /// </summary>
    /// <param name="commandLine">A command line input string.</param>
    /// <returns>A sequence of strings.</returns>
    public static IEnumerable<string> SplitCommandLine(string commandLine);
}

/// <summary>
/// Describes the results of parsing a command line input based on a specific parser configuration.
/// </summary>
public sealed class ParseResult
{
    internal ParseResult(CliConfiguration configuration, CommandResult rootCommandResult, CommandResult commandResult,
        List<Token> tokens, IReadOnlyList<Token>? unmatchedTokens, List<ParseError>? errors, string? commandLineText = null, CliAction? action = null);

    /// <summary>
    /// A result indicating the command specified in the command line input.
    /// </summary>
    public CommandResult CommandResult { get; }

    /// <summary>
    /// The configuration used to produce the parse result.
    /// </summary>
    public CliConfiguration Configuration { get; }

    /// <summary>
    /// Gets the root command result.
    /// </summary>
    public CommandResult RootCommandResult { get; }

    /// <summary>
    /// Gets the parse errors found while parsing command line input.
    /// </summary>
    public IReadOnlyList<ParseError> Errors { get; }

    /// <summary>
    /// Gets the tokens identified while parsing command line input.
    /// </summary>
    public IReadOnlyList<CliToken> Tokens { get; }

    /// <summary>
    /// Gets the list of tokens used on the command line that were not matched by the parser.
    /// </summary>
    public string[] UnmatchedTokens { get; }

    /// <summary>
    /// Gets the completion context for the parse result.
    /// </summary>
    public CompletionContext GetCompletionContext();

    /// <summary>
    /// Gets the parsed or default value for the specified argument.
    /// </summary>
    /// <param name="argument">The argument for which to get a value.</param>
    /// <returns>The parsed value or a configured default.</returns>
    public T? GetValue<T>(CliArgument<T> argument);

    /// <summary>
    /// Gets the parsed or default value for the specified option.
    /// </summary>
    /// <param name="option">The option for which to get a value.</param>
    /// <returns>The parsed value or a configured default.</returns>
    public T? GetValue<T>(CliOption<T> option);

    /// <summary>
    /// Gets the parsed or default value for the specified symbol name, in the context of parsed command (not entire symbol tree).
    /// </summary>
    /// <param name="name">The name of the Symbol for which to get a value.</param>
    /// <returns>The parsed value or a configured default.</returns>
    /// <exception cref="InvalidOperationException">Thrown when parsing resulted in parse error(s).</exception>
    /// <exception cref="ArgumentException">Thrown when there was no symbol defined for given name for the parsed command.</exception>
    /// <exception cref="InvalidCastException">Thrown when parsed result can not be casted to <typeparamref name="T"/>.</exception>
    public T? GetValue<T>(string name);
    
    /// <summary>
    /// Gets the result, if any, for the specified argument.
    /// </summary>
    /// <param name="argument">The argument for which to find a result.</param>
    /// <returns>A result for the specified argument, or <see langword="null"/> if it was not provided and no default was configured.</returns>
    public ArgumentResult? FindResultFor(CliArgument argument);

    public CommandResult? FindResultFor(CliCommand command);

    public OptionResult? FindResultFor(CliOption option);

    public DirectiveResult? FindResultFor(CliDirective directive);

    public SymbolResult? FindResultFor(CliSymbol symbol);
}

Author:	adamsitnik
Assignees:	-
Labels:	area-System.Console, api-ready-for-review
Milestone:	-

[terrajobst]

Video

• Why is the SymbolResult recursive?
• What is the primary output? SymbolResult or ParseError?
• ParseResult
  • UnmatchedTokens should probably be an IReadOnlyList<string>
• Cli prefix isn't used consistently
  • We probably don't want all types to be prefixed, but we should probably ensure that types that are commonly used together appear consistent
• DefaultValueFactory
  • These are invoked when providing help but the example includes cases where those report errors. Can this "mess up" the help page?
• Virtual/Abstract
  • If we consider them plumging, we shouldn't have public/protected virtuals
  • If we want to support overriding, we should make them more consistent
• Setters should generally not be modified after the first parse but we don't believe we need to enforce that
  • We considered using init but we believe this negatively affects source generation and/or usability of newing types up, also doesn't work for .NET Standard 2.0 consumers
• Can we make CliParser internal?
• FindResultFor should probably be GetResult, so that it matches GetValue

namespace System.CommandLine.Parsing;

public enum TokenType
{
    Argument,
    Command,
    Option,
    DoubleDash,
    Directive
}
public sealed class CliToken : IEquatable<CliToken>
{
    public CliToken(string? value, TokenType type, CliSymbol symbol);
    public string Value { get; }
    public TokenType Type { get; }
}
public sealed class ParseError
{
    public string Message { get; }
    public SymbolResult? SymbolResult { get; }
}
public abstract class SymbolResult
{
    public SymbolResult? Parent { get; }
    public IReadOnlyList<CliToken> Tokens { get; }
    public virtual void AddError(string errorMessage);
    public ArgumentResult? FindResultFor(CliArgument argument);
    public CommandResult? FindResultFor(CliCommand command);
    public OptionResult? FindResultFor(CliOption option);
    public DirectiveResult? FindResultFor(CliDirective directive);
    public T? GetValue<T>(CliArgument<T> argument);
    public T? GetValue<T>(CliOption<T> option);
}
public sealed class ArgumentResult : SymbolResult
{
    public CliArgument Argument { get; }
    public T GetValueOrDefault<T>();
    public void OnlyTake(int numberOfTokens);
}
public sealed class OptionResult : SymbolResult
{
    public CliOption Option { get; }
    public bool Implicit { get; }
    public CliToken? IdentifierToken { get; }
    public T? GetValueOrDefault<T>();
}
public sealed class DirectiveResult : SymbolResult
{
    public IReadOnlyList<string> Values { get; }
    public CliDirective Directive { get; }
    public CliToken IdentifierToken { get; }
}
public sealed class CommandResult : SymbolResult
{
    public CliCommand Command { get; }
    public CliToken IdentifierToken { get; }
    public IEnumerable<SymbolResult> Children { get; }
}

[jeffhandley]

We've continued to work in this space, with efforts continuing into 9.0.0. We're reevaluating the layering of the parser and other concepts that would be wrapped around it. I'm marking this as api-needs-work API needs work before it is approved, it is NOT ready for implementation until we return back with a revised design that enables the layering of completions, validation, invocation, and other concepts without requiring the core parser component to be aware of those concepts.