Merge pull request #185 from epage/tut

docs(tutorial): Integrate a full tutorial

src/_tutorial/chapter_0.rs

@@ -0,0 +1,38 @@
+//! # Chapter 0: Introduction
+//!
+//! This tutorial assumes that you are:
+//! - Already familiar with Rust
+//! - Using `winnow` for the first time
+//!
+//! The focus will be on parsing in-memory strings (`&str`). Once done, you might want to check the
+//! [Special Topics][_topic] for more specialized topics or examples.
+//!
+//! ## About
+//!
+//! `winnow` is a parser-combinator library. In other words, it gives you tools to define:
+//! - "parsers", or functions that takes an input and gives back an output
+//! - "combinators", or functions that take parsers and _combine_ them together!
+//!
+//! While "combinator" might be an unfamiliar word, you are likely using them in your rust code
+//! today, like with the [`Iterator`] trait:
+//! ```rust
+//! let data = vec![1, 2, 3, 4, 5];
+//! let even_count = data.iter()
+//!     .copied()  // combinator
+//!     .filter(|d| d % 2 == 0)  // combinator
+//!     .count();  // combinator
+//! ```
+//!
+//! Parser combinators are great because:
+//!
+//! - The parsers are small and easy to write
+//! - The parsers components are easy to reuse (if they're general enough, please add them to winnow!)
+//! - The parsers components are easy to test separately (unit tests and property-based tests)
+//! - The parser combination code looks close to the grammar you would have written
+//! - You can build partial parsers, specific to the data you need at the moment, and ignore the rest
+
+#![allow(unused_imports)]
+use crate::_topic;
+use std::iter::Iterator;
+
+pub use super::chapter_1 as next;

src/_tutorial/chapter_1.rs

@@ -0,0 +1,89 @@
+//! # Chapter 1: The Winnow Way
+//!
+//! First of all, we need to understand the way that winnow thinks about parsing.
+//! As discussed in the introduction, winnow lets us build simple parsers, and
+//! then combine them (using "combinators").
+//!
+//! Let's discuss what a "parser" actually does. A parser takes an input and returns
+//! a result, where:
+//!  - `Ok` indicates the parser successfully found what it was looking for; or
+//!  - `Err` indicates the parser could not find what it was looking for.
+//!
+//! Parsers do more than just return a binary "success"/"failure" code. If
+//! the parser was successful, then it will return a tuple where the first field
+//! will contain everything the parser did not process. The second will contain
+//! everything the parser processed. The idea is that a parser can happily parse the first
+//! *part* of an input, without being able to parse the whole thing.
+//!
+//! If the parser failed, then there are multiple errors that could be returned.
+//! For simplicity, however, in the next chapters we will leave these unexplored.
+//!
+//! ```text
+//!                                    ┌─► Ok(
+//!                                    │      what the parser didn't touch,
+//!                                    │      what matched the parser
+//!                                    │   )
+//!              ┌─────────┐           │
+//!  my input───►│my parser├──►either──┤
+//!              └─────────┘           └─► Err(...)
+//! ```
+//!
+//!
+//! To represent this model of the world, winnow uses the [`IResult<I, O>`] type.
+//! The `Ok` variant has a tuple of `(remainder: I, output: O)`;
+//! whereas the `Err` variant stores an error.
+//!
+//! You can import that from:
+//!
+//! ```rust
+//! use winnow::IResult;
+//! ```
+//!
+//! You'll note that `I` and `O` are parameterized -- while most of the examples in this book
+//! will be with `&str` (i.e. parsing a string); they do not have to be strings; nor do they
+//! have to be the same type (consider the simple example where `I = &str`, and `O = u64` -- this
+//! parses a string into an unsigned integer.)
+//!
+//! To combine parsers, we need a common way to refer to them which is where the [`Parser`]
+//! trait comes in with [`Parser::parse_next`] being the primary way to drive
+//! parsing forward.
+//!
+//! # Let's write our first parser!
+//!
+//! The simplest parser we can write is one which successfully does nothing.
+//!
+//! To make it easier to implement a [`Parser`], the trait is implemented for
+//! functions of the form `Fn(I) -> IResult<I, O>`.
+//!
+//! This parser function should take in a `&str`:
+//!
+//!  - Since it is supposed to succeed, we know it will return the Ok Variant.
+//!  - Since it does nothing to our input, the remaining input is the same as the input.
+//!  - Since it doesn't parse anything, it also should just return an empty string.
+//!
+//! ```rust
+//! use winnow::IResult;
+//! use winnow::Parser;
+//!
+//! pub fn do_nothing_parser(input: &str) -> IResult<&str, &str> {
+//!     Ok((input, ""))
+//! }
+//!
+//! fn main() {
+//!     let input = "0x1a2b Hello";
+//!
+//!     let (remainder, output) = do_nothing_parser.parse_next(input).unwrap();
+//!     // Same as:
+//!     // let (remainder, output) = do_nothing_parser(input).unwrap();
+//!
+//!     assert_eq!(remainder, "0x1a2b Hello");
+//!     assert_eq!(output, "");
+//! }
+//! ```
+
+#![allow(unused_imports)]
+use crate::IResult;
+use crate::Parser;
+
+pub use super::chapter_0 as previous;
+pub use super::chapter_2 as next;

src/_tutorial/chapter_2.rs

@@ -0,0 +1,155 @@
+//! # Chapter 2: Tokens and Tags
+//!
+//! The simplest *useful* parser you can write is one which matches tokens.
+//!
+//! ## Tokens
+//!
+//! Matching a single token literal so common, `Parser` is implemented for
+//! `char`.
+//!
+//! ```rust
+//! # use winnow::Parser;
+//! # use winnow::IResult;
+//! #
+//! fn parse_prefix(input: &str) -> IResult<&str, char> {
+//!     '0'.parse_next(input)
+//! }
+//!
+//! fn main()  {
+//!     let input = "0x1a2b Hello";
+//!
+//!     let (remainder, output) = parse_prefix.parse_next(input).unwrap();
+//!
+//!     assert_eq!(remainder, "x1a2b Hello");
+//!     assert_eq!(output, '0');
+//!
+//!     assert!(parse_prefix("d").is_err());
+//! }
+//! ```
+//!
+//! ## Tags
+//!
+//! One of the most frequent way of matching a token is when they are combined into a string.
+//! Again, this is common enough that `Parser` is implemented for `&str`:
+//!
+//! ```rust
+//! # use winnow::Parser;
+//! # use winnow::IResult;
+//! #
+//! fn parse_prefix(input: &str) -> IResult<&str, &str> {
+//!     "0x".parse_next(input)
+//! }
+//!
+//! fn main()  {
+//!     let input = "0x1a2b Hello";
+//!
+//!     let (remainder, output) = parse_prefix.parse_next(input).unwrap();
+//!     assert_eq!(remainder, "1a2b Hello");
+//!     assert_eq!(output, "0x");
+//!
+//!     assert!(parse_prefix("0o123").is_err());
+//! }
+//! ```
+//!
+//! In `winnow`, we call this type of parser a [`tag`].
+//!
+//! ## Character Classes
+//!
+//! Selecting a single `char` or a `tag` is fairly limited.  Sometimes, you will want to select one of several
+//! `chars` of a specific class, like digits. For this, we use the [`one_of`] parer:
+//!
+//! ```rust
+//! # use winnow::Parser;
+//! # use winnow::IResult;
+//! use winnow::bytes::one_of;
+//!
+//! fn parse_digits(input: &str) -> IResult<&str, char> {
+//!     one_of("0123456789abcdefgABCDEFG").parse_next(input)
+//! }
+//!
+//! fn main() {
+//!     let input = "1a2b Hello";
+//!
+//!     let (remainder, output) = parse_digits.parse_next(input).unwrap();
+//!     assert_eq!(remainder, "a2b Hello");
+//!     assert_eq!(output, '1');
+//!
+//!     assert!(parse_digits("Z").is_err());
+//! }
+//! ```
+//!
+//! > **Aside:** `one_of` might look straightforward, a function returning a value that implements `Parser`.
+//! > Let's look at it more closely as its used above (resolving all generic parameters):
+//! > ```rust
+//! > # use winnow::IResult;
+//! > pub fn one_of<'i>(
+//! >     list: &'static str
+//! > ) -> impl FnMut(&'i str) -> IResult<&'i str, char> {
+//! >     // ...
+//! > #    winnow::bytes::one_of(list)
+//! > }
+//! > ```
+//! > If you have not programmed in a language where functions are values, the type signature of the
+//! > `one_of` function might be a surprise.
+//! > The function `tag` *returns a function*. The function it returns is a
+//! > `Parser`, taking a `&str` and returning an `IResult`. This is a common pattern in winnow for
+//! > configurable or stateful parsers.
+//!
+//! Some of character classes are common enough that a named parser is provided, like with:
+//! - [`line_ending`][crate::character::line_ending]: Recognizes an end of line (both `\n` and `\r\n`)
+//! - [`newline`][crate::character::newline]: Matches a newline character `\n`
+//! - [`tab`][crate::character::tab]: Matches a tab character `\t`
+//!
+//! You can then capture sequences of these characters with parsers like [`take_while1`].
+//! ```rust
+//! # use winnow::Parser;
+//! # use winnow::IResult;
+//! use winnow::bytes::take_while1;
+//!
+//! fn parse_digits(input: &str) -> IResult<&str, &str> {
+//!     take_while1("0123456789abcdefgABCDEFG").parse_next(input)
+//! }
+//!
+//! fn main() {
+//!     let input = "1a2b Hello";
+//!
+//!     let (remainder, output) = parse_digits.parse_next(input).unwrap();
+//!     assert_eq!(remainder, " Hello");
+//!     assert_eq!(output, "1a2b");
+//!
+//!     assert!(parse_digits("Z").is_err());
+//! }
+//! ```
+//!
+//! We could simplify this further with by using one of the built-in character classes, [`hex_digit1`]:
+//! ```rust
+//! # use winnow::Parser;
+//! # use winnow::IResult;
+//! use winnow::character::hex_digit1;
+//!
+//! fn parse_digits(input: &str) -> IResult<&str, &str> {
+//!     hex_digit1.parse_next(input)
+//! }
+//!
+//! fn main() {
+//!     let input = "1a2b Hello";
+//!
+//!     let (remainder, output) = parse_digits.parse_next(input).unwrap();
+//!     assert_eq!(remainder, " Hello");
+//!     assert_eq!(output, "1a2b");
+//!
+//!     assert!(parse_digits("Z").is_err());
+//! }
+//! ```
+
+#![allow(unused_imports)]
+use crate::bytes::one_of;
+use crate::bytes::tag;
+use crate::bytes::take_while1;
+use crate::character::hex_digit1;
+use crate::stream::ContainsToken;
+use crate::Parser;
+use std::ops::RangeInclusive;
+
+pub use super::chapter_1 as previous;
+pub use super::chapter_3 as next;