Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update very outdated README.md #458

Merged
merged 1 commit into from
May 6, 2024
Merged

Update very outdated README.md #458

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 74 additions & 35 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Gosub: Gateway to Optimized Searching and Unlimited Browsing

This repository holds the Gosub browser engine. It will become a standalone library that can be used by other projects but will ultimately be used by the Gosub browser user-agent. See the [About](#about) section for more information.
This repository holds the Gosub browser engine. It will become a standalone library that can be used by other projects
but will ultimately be used by the Gosub browser user-agent. See the [About](#about) section for more information.

Join us at our development [Zulip chat](https://chat.developer.gosub.io)!

For more general information you can also join our [Discord server](https://chat.gosub.io).

If you are interested in contributing to Gosub, please checkout the [contribution guide](CONTRIBUTING.md)!


```
_
| |
Expand All @@ -23,74 +23,113 @@ If you are interested in contributing to Gosub, please checkout the [contributio

## About

This repository is part of the Gosub browser project. This is the main engine that holds at least the following components:
This repository is part of the Gosub browser project. This is the main engine that holds the following components:

- HTML5 tokenizer / parser
- CSS3 tokenizer / parser
- Document tree
- Several APIs for connecting to javascript
- Configuration store
- HTML5 tokenizer / parser
- CSS3 tokenizer / parser
- Document tree
- Several APIs for connecting to javascript
- Configuration store
- Networking stack
- Rendering engine
- JS bridge
- C Bindings

The idea is that this engine will receive some kind of stream of bytes (most likely from a socket or file) and parse this into a valid HTML5 document tree. From that point, it can be fed to a renderer engine that will render the document tree into a window, or it can be fed to a more simplistic engine that will render it in a terminal.
The idea is that this engine will receive some kind of stream of bytes (most likely from a socket or file) and parse
this into a valid HTML5 document tree.
From that point, it can be fed to a renderer engine that will render the document tree into a window, or it can be fed
to a more simplistic engine that will render it in a terminal.
JS can be executed on the document tree and the document tree can be modified by JS.

## Status

> This project is in its infancy. There is no browser you can use yet, but parsing a file into a document tree is possible.
> This project is in its infancy. There is no usable browser yet. However, you can look at simple html pages and parse
> them into a document tree.

We can parse html5 and css3 files into a document tree or the respective css tree.
This tree can be shown in the terminal or be rendered in a very unfinished renderer. Our renderer cannot render
everything yet, but it can render simple html pages.

We already implemented other parts of the engine, for a JS engine, networking stack, a configuration store and other
things however these aren't integrated yet.
You can try these out by running the respective binary.

We can render a part for our own [site](https://gosub.io):

![Gosub.io](resources/images/current_progress.png)

The main goal for the parser is to be able to parse correctly all the tests in the html5lib-tests repository (https://github.com/html5lib/html5lib-tests). This is currently achieved for both the tokenizer tests and the tree-construction tests. There are a few small issues left which are mainly because of handling of UTF-16 characters and test that does dom modification through scripts (there is no javascript engine implemented yet).
Note: the borders are broken because of an issue with taffy (the layout engine we use). This will be fixed in the
future.

From a parsing point of view, the html5 parser and most of the css3 parser is completed for now.
## How to run

## How to build
<details>
<summary> Installing dependencies </summary>

This project uses [cargo](https://doc.rust-lang.org/cargo/) and [rustup](https://www.rust-lang.org/tools/install). First you must install `rustup` at the link provided. After installing `rustup`, run:

This project uses [cargo](https://doc.rust-lang.org/cargo/) and [rustup](https://www.rust-lang.org/tools/install). First
you must install `rustup` at the link provided. After installing `rustup`, run:

```bash
$ rustup toolchain install 1.73
$ rustc --version
rustc 1.73.0 (cc66ad468 2023-10-03)
```

Once Rust is installed, run this command to build the project:
Once Rust is installed, run this command to pre-build the dependencies:

```bash
$ cargo build
$ cargo build --release
```

Doing this will create the following binaries:
</details>


You can run the following binaries:

| File | Type | Description |
|-----------------------------------|------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `target/debug/gosub-parser` | bin | The actual html5 parser/tokenizer that allows you to convert html5 into a document tree. <br/> |
| `target/debug/parser-test` | bin | A test suite for the parser that tests specific tests. This will be removed as soon as the parser is completely finished as this tool is for developement only. |
| `target/debug/html5-parser-tests` | bin | A test suite that tests all html5lib tests for the treebuilding |
| `target/debug/test-user-agent` | bin | A simple placeholder user agent for testing purposes |
| `target/debug/config-store` | bin | A simple test application of the config store for testing purposes |
| Command | Type | Description |
|----------------------------------------|------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `cargo run -r --bin gosub-parser` | bin | The actual html5 parser/tokenizer that allows you to convert html5 into a document tree. <br/> |
| `cargo run -r --bin parser-test` | test | A test suite for the parser that tests specific tests. This will be removed as soon as the parser is completely finished as this tool is for developement only. |
| `cargo run -r --bin html5-parser-test` | test | A test suite that tests all html5lib tests for the treebuilding |
| `cargo run -r --bin test-user-agent` | bin | A simple placeholder user agent for testing purposes |
| `cargo run -r --bin config-store` | bin | A simple test application of the config store for testing purposes |
| `cargo run -r --bin css3-parser` | bin | Show the parsed css tree |
| `cargo run -r --bin renderer` | bin | Render a html page (WIP) |
| `cargo run -r --bin run-js` | bin | Run a JS file (Note: console and event loop are not yet implemented) |
| `cargo run -r --bin style-parser` | bin | Display the html page's text with basic styles in the terminal |

You can then run the binaries like so:

```bash
$ ./target/debug/gosub-parser https://news.ycombinator.com/
cargo run -r --bin renderer file://src/bin/resources/gosub.html
```

To build the release build, run:

To run the tests and benchmark suite, do:

```bash
$ cargo build --release
$ ./target/release/gosub-parser https://news.ycombinator.com/
make test
cargo bench
ls target/criterion/report
index.html
```

To run the tests and benchmark suite, do:
## Wasm

Our engine can also be compiled to WebAssembly. You need to use WasmPack for this. To build the Wasm version, run:

```bash
$ make test
$ cargo bench
$ ls target/criterion/report
index.html
wasm-pack build
```

![Browser in browser](resources/images/browser-in-browser.png)

## Contributing to the project
We welcome contributions to this project but the current status makes that we are spending a lot of time researching, building small proof-of-concepts and figuring out what needs to be done next. Much time of a contributor at this stage of the project will be non-coding.

We do like to hear from you if you are interested in contributing to the project and you can join us currently at our [Zulip chat](https://chat.developer.gosub.io)!
We welcome contributions to this project but the current status makes that we are spending a lot of time researching,
building small proof-of-concepts and figuring out what needs to be done next. Much time of a contributor at this stage
of the project will be non-coding.

We do like to hear from you if you are interested in contributing to the project and you can join us currently at
our [Zulip chat](https://chat.developer.gosub.io)!
Binary file added resources/images/browser-in-browser.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added resources/images/current_progress.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading