Replies: 1 comment 2 replies
-
|
Of course, I'd like everybody's input on this :-) |
Beta Was this translation helpful? Give feedback.
-
|
IMO, it's OK. |
Beta Was this translation helpful? Give feedback.
-
Honestly, I'm not really sure. I want to say that we should think of partial (fragment) parsing as the general behavior, and parsing of a well-formed HTML5 input string as a special case when you have everything available at that moment and you're in a test or in an offline worker or something. This is how I imagined complete and partial parsing working together: // We have the whole input because we're in a test
let input = "<html><body><script>window.n = 1;</script>hi</body></html>";
let doc = Document::shared();
let _ = html5::parse_str(&input, Document::clone(&doc)).unwrap();
println!("{:?}", doc);
mod html5 {
// Used mainly in tests
pub fn parse_str(input: &str, doc: DocumentHandle) -> Result<Progress> {
let mut tokens = TokenStream::from_str(&input);
parse(&mut tokens, doc)
}
// Synchronous convenience function. Not used very often, and not used in the UI rendering thread
pub fn parse(tokens: &mut TokenStream, doc: DocumentHandle) -> Result<Progress> {
let mut context_id = NodeId::root();
loop {
// Modifies the DOM a little with each pass
let progress = parse_fragment(tokens, Document::clone(&doc), context_id)?;
context_id = match progress.status() {
Continue(context_id) => context_id,
Script(script) => javascript::evaluate(script, Document::clone(&doc)), // returns a context_id?
Eof => return Ok(progress),
}
}
unreachable!()
}
// Returns quickly. Stops at a suitable stopping point, e.g., a <script> tag. Tree construction
// happens here.
pub fn parse_fragment(tokens: &mut TokenStream, doc: DocumentHandle, context_id: NodeId) -> Result<Progress> {
let parser = Parser::new();
// ...
Ok(Progress { percent: 20, status: Script(script) })
}
}This is a related code path in WebKit. My own intuition is to more or less copy whatever WebKit (or Chromium) is doing as a starting point, which can later be revisited when we have more experience and knowledge. |
Beta Was this translation helpful? Give feedback.
-
Currently, the parser is able to parse documents, but isn't capable of doing fragment cases.
A fragment case could be a part between "template" tags or other parts where we need to parse html5 without it being a full document. The html5lib tests will test this with the
#document-fragmentkeyword, which indicates that the document to be parsed is not a full document, but a fragment case. Therefor it should a little bit different than the regular parser (not all "modes" are supported).Our html5lib parser currently has a
is_fragment_caseflag that indicates whether or not the parse is in fragment case mode or not. This is not completely implemented but most of the work is checking at the right places to see if this flag is set, and act a little bit differently or jump to another state.The current html5parser works like this:
html5parser::newparse()function with a document handle where the parser will add the nodes into.This current system is not able to deal with fragment cases correctly.
I would propose a small change in setup:
new()without a input streamparse()with a token stream and document(handle) when we want to parse a regular document.parse_fragment(0with a token stream, a document(handle) to a document-fragment, and a context nodeId.@emwalker I think this interferes a bit with your idea's on setting up a pipeline from bytestream to parser? We currently have a tokenizer being initialized in the html5parser, so probably we want to do this outside the parser itself, something like:
I'm not sure this is the way you have envisioned.. I think we can keep the bytestream -> encoding -> tokenizer -> parser pipeline through interfaces/traits, but have the parser tie everything together.
@CharlesChen0823 Would you be ok to change the initial
new()andparse()methods from the parser, and add aparse_fragment()as well.. (and move the input stream toparse()instead of thenew()Beta Was this translation helpful? Give feedback.
All reactions