Add skeleton for layout thread reflow #283

emwalker · 2023-11-25T19:58:37Z

This PR adds the skeleton of a layout thread, following the example of Servo. Because Servo's license is copyleft, I'll be closing this PR. My question is whether it is worth re-doing the PR using WebKit or Chromium (Blink), or whether a different approach would be preferable.

I was looking at what to do with the stylesheets once the CSS3 parser has parsed the intput and produced them. Since it wasn't obvious how they would be used by downstream code, I consulted Servo. This PR is a (greatly simplified) skeleton of what Servo does in the process of handling a reflow event, in which it builds the render tree and sends new graphics data to the compositor to be rendered. The thought with this code is for it to be clay to be gradually shaped into something better suited to what we want to do. So I don't see a reason to stick too closely to what Servo is doing.

Note that there's no render tree in the skeleton code. I'm wondering whether the articles that discuss a render tree are speaking in terms of a high-level abstraction. Instead what is sent to the compositor over IPC is a low-level "display list" of computed values to render. The closest thing to a render tree that I've seen is either the BoxTree struct or the FragmentTree struct. So this PR is unavoidably taking things in a different direction than the recent src/render_tree.rs work. I don't think this PR will step on that other work, but I do think the two lines of action will need to be reconciled at some point once we know more about how the pieces fit / should fit together.

In a later PR I would like to explore using the taffy crate to generate the BoxTree or the FragmentTree (not sure which is applicable, yet), using the CSS3 stylesheets as input.

Leaving this PR in draft because I'm not sure whether it makes sense to have a bunch of unused or barely used structs lying around. I think they can be useful by providing pointers to people to know what code to consult next, so I'm inclined to go with them.

emwalker · 2023-11-25T20:01:59Z

src/layout.rs

+    ) {
+        let (display_list_data, display_list_descriptor) = list.into_data();
+        // TODO: Set up channel somewhere else
+        let (tx, _rx) = channel();


The Servo code uses IPC. I've used an mpsc channel as a placeholder for whatever would eventually be used.

I guess it's not possible, but can you somehow send a pointer over IPC? I think, it's only text. And i also think it's not quite possible to just send a "text-pointer" and interpret it as a pointer on the other side because then the OS goes probably in between

I'm thinking what is serizalized as bytes and sent over IPC is something like

rectangle [x0, y0, x1, y1] rectangle [x0, y0, x1, y1] circle [x, y, radius]

But that's just a wild guess. But I think it's something low-level like that.

EDIT: Looks like those are some of the main things sent over.

emwalker · 2023-11-25T20:15:32Z

src/layout/layout_thread.rs

+    }
+
+    fn build_layout_context(&self) -> LayoutContext {
+        LayoutContext


I think the CSS stylesheets are contained in the LayoutContext, but it might actually be something derived from them.

emwalker · 2023-11-25T20:20:50Z

src/layout/layout_thread.rs

+            .send_display_list(compositor_info, builder.finalize());
+
+        ScriptReflowResult
+    }


This is the main entrypoint for reflow (= rendering).

emwalker · 2023-11-25T20:56:18Z

Seems the Servo code is under an MPL license, which is copyleft and similar in spirit to the GPL. So if the general idea is received well, I'll plan on closing this PR and recreating it using Blink (Chromium) or WebKit as a starting point, since they're under more permissive licenses.

Sharktheone · 2023-11-25T20:57:18Z

So, should i do a basic review of this PR then?

emwalker · 2023-11-25T20:58:11Z

So, should i do a basic review of this PR then?

At this point, I'm looking for a high-level thumbs up or down to the general approach. No detailed review needed for this specific PR since it will be closed soon, one way or another.

Kiyoshika · 2023-11-26T01:41:05Z

I guess my thought of how CSS would be handled is, during parsing the CSSOM tree is built (alongside the DOM) which can be modified during parsing (via JS.) Then after the parsing is finished, those two are combined to compute the layouts (what the goal of "render tree" is, albeit very early stages at the moment.)

I also briefly looked at taffy from Nico and was thinking about how we could incorporate that into what we currently have, since my next major goal with the render tree was to compute proper box layouts.

But granted, you've done a lot more research than I have so maybe that idea isn't sufficient

Sharktheone · 2023-11-26T01:52:12Z

I have no research about this at all, so I might be on a completely wrong path.

But I would think, we need some kind of "styles-merger" by that I mean, something that can merge two "style-sets" (a styles set would be a class, id, the parents inheritable attributes etc). This needs to respect all the different rules. For example selector specificity, the !important thing, and on where the style is placed, this results in the computed styles.

From this, we can calculate it's position and size according to some things: the parent's position, number and size of other siblings, margins, paddings, and other offsets like translate for example.

emwalker · 2023-11-26T02:42:13Z

@Kiyoshika my apologies. Now that I better understand the scope of the work you were thinking of tackling, and that it includes the calculation of the box model and investigating @nicoburns's taffy, I see now that this PR fully overlaps with what you were planning on implementing. During a recent Zulip thread, I thought to set aside the tokenization stuff I was looking at and see how to connect @neuodev's CSS3 parser up with whatever happens after it. I see now that you're already looking at that question.

With @jaytaph looking at the actual CSS3 parser coverage, I think I can resume looking at tokenization.

Kiyoshika · 2023-11-26T02:44:38Z

@Kiyoshika my apologies. Now that I better understand the scope of the work you were thinking tackling, and that it includes the calculation of the box model and investigating @nicoburns's taffy, I see now that this PR fully overlaps with what you were planning on implementing. During a recent Zulip thread, I thought to set aside the tokenization stuff I was looking at and see how to connect Ahmed's CSS3 parser up with whatever happens after it. I see now that you're already looking at that question.

With @jaytaph looking at the actual CSS3 parser coverage, I think I can resume looking at tokenization.

yes, no worries. The current RenderTree I have is very minimal in scope at the moment but that's because I just wanted to draw something on the screen. I am going to put in efforts to move towards incorporating box layouts likely using taffy or whatever I think would be easy to use

nicoburns · 2023-11-26T02:45:25Z

I have no research about this at all, so I might be on a completely wrong path.

But I would think, we need some kind of "styles-merger" by that I mean, something that can merge two "style-sets" (a styles set would be a class, id, the parents inheritable attributes etc). This needs to respect all the different rules. For example selector specificity, the !important thing, and on where the style is placed, this results in the computed styles.

Yep! And that's what the https://docs.rs/selectors crate does :) (well it handles matching style blocks to nodes, I don't think it actually does any merging - we'd need to do that bit ourselves).

Sharktheone · 2023-11-26T12:42:47Z

Yep! And that's what the https://docs.rs/selectors crate does :) (well it handles matching style blocks to nodes, I don't think it actually does any merging - we'd need to do that bit ourselves).

Ah, okay. Interesting. By the term "merging" I meant something that combines the different sources of style attributes and combines it to the computed style by selecting which selector has higher specificity.

nicoburns · 2023-11-26T13:12:45Z

Ah, okay. Interesting. By the term "merging" I meant something that combines the different sources of style attributes and combines it to the computed style by selecting which selector has higher specificity.

I believe selectors will:

Parse selectors
Determine whether a selector matches a given node
Tell you what the specificity of the selector is

But once you have a list of selectors that match a node and there specificities, then you need to do the actual merging yourself. Because selectors doesn't deal with the actual style attributes at all.

Sharktheone · 2023-11-26T14:20:06Z

I think, it's a source we can look at for a rough overview of what to to. But there again is the thing with Licensing, so we can't use it and from the discussion in Zulip we probably also don't want to use it, no matter which license it uses.

nicoburns · 2023-11-26T15:04:08Z

I would guess that MPL is probably fine when used as a library, as it doesn't "infect" the libraries that depend on it. It just can't be used as inspiration for your own library if you don't want to license that library as MPL.

emwalker self-assigned this Nov 25, 2023

emwalker commented Nov 25, 2023

View reviewed changes

emwalker force-pushed the layout-thread branch from f26c938 to 96da2da Compare November 25, 2023 20:05

emwalker requested review from jaytaph, Kiyoshika, neuodev and Sharktheone November 25, 2023 20:09

emwalker commented Nov 25, 2023

View reviewed changes

Add skeleton for layout thread reflow

5477a0d

emwalker force-pushed the layout-thread branch from 96da2da to 5477a0d Compare November 25, 2023 22:29

emwalker closed this Nov 26, 2023

emwalker deleted the layout-thread branch November 26, 2023 02:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add skeleton for layout thread reflow #283

Add skeleton for layout thread reflow #283

emwalker commented Nov 25, 2023 •

edited

Loading

emwalker Nov 25, 2023 •

edited

Loading

Sharktheone Nov 25, 2023

emwalker Nov 25, 2023 •

edited

Loading

emwalker Nov 25, 2023

emwalker Nov 25, 2023

emwalker commented Nov 25, 2023

Sharktheone commented Nov 25, 2023

emwalker commented Nov 25, 2023

Kiyoshika commented Nov 26, 2023 •

edited

Loading

Sharktheone commented Nov 26, 2023 •

edited

Loading

emwalker commented Nov 26, 2023 •

edited

Loading

Kiyoshika commented Nov 26, 2023

nicoburns commented Nov 26, 2023 •

edited

Loading

Sharktheone commented Nov 26, 2023

nicoburns commented Nov 26, 2023

Sharktheone commented Nov 26, 2023

nicoburns commented Nov 26, 2023

Add skeleton for layout thread reflow #283

Add skeleton for layout thread reflow #283

Conversation

emwalker commented Nov 25, 2023 • edited Loading

emwalker Nov 25, 2023 • edited Loading

Choose a reason for hiding this comment

Sharktheone Nov 25, 2023

Choose a reason for hiding this comment

emwalker Nov 25, 2023 • edited Loading

Choose a reason for hiding this comment

emwalker Nov 25, 2023

Choose a reason for hiding this comment

emwalker Nov 25, 2023

Choose a reason for hiding this comment

emwalker commented Nov 25, 2023

Sharktheone commented Nov 25, 2023

emwalker commented Nov 25, 2023

Kiyoshika commented Nov 26, 2023 • edited Loading

Sharktheone commented Nov 26, 2023 • edited Loading

emwalker commented Nov 26, 2023 • edited Loading

Kiyoshika commented Nov 26, 2023

nicoburns commented Nov 26, 2023 • edited Loading

Sharktheone commented Nov 26, 2023

nicoburns commented Nov 26, 2023

Sharktheone commented Nov 26, 2023

nicoburns commented Nov 26, 2023

emwalker commented Nov 25, 2023 •

edited

Loading

emwalker Nov 25, 2023 •

edited

Loading

emwalker Nov 25, 2023 •

edited

Loading

Kiyoshika commented Nov 26, 2023 •

edited

Loading

Sharktheone commented Nov 26, 2023 •

edited

Loading

emwalker commented Nov 26, 2023 •

edited

Loading

nicoburns commented Nov 26, 2023 •

edited

Loading