Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Fixed patch and frontend charts. #72

Merged
merged 59 commits into from
Nov 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
4a37929
Refactor LogWriter to be an abstract base class
AndPuQing Nov 25, 2023
a3e2a7d
Refactor PackedAction class and NeetActionManager
AndPuQing Nov 25, 2023
e670ac0
patch fix
visualDust Nov 25, 2023
6cc0569
now server forwards neet actions
visualDust Nov 25, 2023
c7b46f8
fixed colored log prefix does not skip ws writer
visualDust Nov 25, 2023
49c2960
now clients only uploads important status via https, ignoring user le…
visualDust Nov 25, 2023
0e5fc70
added description to neet action
visualDust Nov 25, 2023
9b36858
patch fix
visualDust Nov 25, 2023
59fe774
whatever'
visualDust Nov 25, 2023
2871ad4
fixed action query forward direction error
visualDust Nov 25, 2023
b9216f5
added param annotations for action dict sent to frontend
visualDust Nov 25, 2023
5649804
fixed callback not working on actions with blocking=True
visualDust Nov 25, 2023
9ce1de6
fixed unexpected arg event_id in ws_send
visualDust Nov 25, 2023
f228004
feat: frontend
visualDust Nov 23, 2023
5d1945b
fix: frontend works again
visualDust Nov 24, 2023
d6c5b0f
feat(frontend): background data polling
lideming Nov 25, 2023
8733f69
fix(frontend): echarts resize
lideming Nov 25, 2023
3fbcb03
feat(frontend): logs
lideming Nov 25, 2023
253e044
fix: lint
lideming Nov 25, 2023
fc07194
refined interface ProjectStatus
visualDust Nov 25, 2023
0b0cb42
added interface ProjectActions
visualDust Nov 25, 2023
0e3266a
wip
lideming Nov 25, 2023
e48a6cb
wip
lideming Nov 25, 2023
ba44768
feat: action blocking
lideming Nov 25, 2023
ff5ac89
fix: actions
lideming Nov 25, 2023
0fcc8af
now client also returns errors on action execution
visualDust Nov 25, 2023
f8788b6
fix: dynamic actions
lideming Nov 25, 2023
12d64ba
feat: dynamic import echarts
lideming Nov 26, 2023
2eeff54
feat: ws url in prod
lideming Nov 26, 2023
ae8a78e
fixed ws shoud not connect on cli
visualDust Nov 26, 2023
803a964
fixed logger console color missing when using a logger with specific …
visualDust Nov 26, 2023
ab845a3
Merge branch 'dev' of github.com:visualDust/neetbox into dev
visualDust Nov 26, 2023
d59b5f3
Merge branch 'dev' into master
visualDust Nov 26, 2023
364950b
Merge pull request #70 from AndPuQing/master
visualDust Nov 26, 2023
eb9f54a
fix: prevent rerender every log items
lideming Nov 26, 2023
1af0108
feat: auto refresh project list
lideming Nov 26, 2023
af01e30
now server can be launched in cli
visualDust Nov 26, 2023
5e18123
Merge branch 'dev' of github.com:visualDust/neetbox into dev
visualDust Nov 26, 2023
f579b29
fix: display cpu without id
lideming Nov 26, 2023
cbcc66d
integrations now self-launch
visualDust Nov 26, 2023
b9155be
fixed bad imports
visualDust Nov 26, 2023
57aa4c6
added integration auto load option in config
visualDust Nov 26, 2023
6c7295c
chore: fix lint
lideming Nov 26, 2023
8264ae4
chore: add ci for frontend
lideming Nov 26, 2023
4a6db7c
chore: ci yarn cache
lideming Nov 26, 2023
7cfd011
test release
visualDust Nov 27, 2023
a998b7b
fixed frontend build dist missing
visualDust Nov 27, 2023
19b6f6f
patch fix workflow:publish python package
visualDust Nov 27, 2023
ed45e37
patch fix workflow:publish python package
visualDust Nov 27, 2023
e80db85
patch fix workflow:publish python package
visualDust Nov 27, 2023
d64f4b5
patch fix workflow:publish python package
visualDust Nov 27, 2023
9d67633
patch fix workflow:publish python package
visualDust Nov 27, 2023
4cdcfe9
patch fix workflow:publish python package
visualDust Nov 27, 2023
e9c2f9c
patch fix workflow:publish python package
visualDust Nov 27, 2023
e454fd8
fixed pytest run failed
visualDust Nov 27, 2023
38f4e8d
Merge branch 'dev' of github.com:visualDust/neetbox into dev
visualDust Nov 27, 2023
5411306
Update build-frontend.yml and poetry-pytest.yml
AndPuQing Nov 27, 2023
1b8235d
Fix Python version format in workflow
AndPuQing Nov 27, 2023
502fa1f
renamed doc to docs
visualDust Nov 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion .github/workflows/build-and-publish-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ name: Upload Python Package
on:
release:
types: [published]
workflow_dispatch:

permissions:
contents: read
Expand All @@ -20,7 +21,19 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v4
with:
node-version: 18
cache: "yarn"
cache-dependency-path: "neetbox/frontend/yarn.lock"
- name: setup yarn
run: corepack enable
working-directory: neetbox/frontend
- run: yarn install --frozen-lockfile
working-directory: neetbox/frontend
- run: yarn build
working-directory: neetbox/frontend
- name: Build and publish to pypi
uses: JRubics/poetry-publish@v1.16
uses: JRubics/poetry-publish@v1.17
with:
pypi_token: ${{ secrets.PYPI_TOKEN }}
32 changes: 32 additions & 0 deletions .github/workflows/build-frontend.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Frontend Build & Lint

on:
push:
paths:
- 'neetbox/frontend/**'
pull_request:
paths:
- 'neetbox/frontend/**'


jobs:
build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: neetbox/frontend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 18
cache: 'yarn'
cache-dependency-path: 'neetbox/frontend/yarn.lock'
- name: setup yarn
run: corepack enable
- run: yarn install --frozen-lockfile
- run: yarn build
- run: yarn tsc
if: '!cancelled()'
- run: yarn lint
if: '!cancelled()'
39 changes: 39 additions & 0 deletions .github/workflows/maunal-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

name: Manually Build and Publish to TEST.PyPi

on:
workflow_dispatch:

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v4
with:
node-version: 18
cache: "yarn"
cache-dependency-path: "neetbox/frontend/yarn.lock"
- name: setup yarn
run: corepack enable
working-directory: neetbox/frontend
- run: yarn install --frozen-lockfile
working-directory: neetbox/frontend
- run: yarn build
working-directory: neetbox/frontend
- name: Build and publish to pypi
uses: JRubics/[email protected]
with:
pypi_token: ${{ secrets.TEST_PYPI_TOKEN }}
repository_name: "neetbox"
repository_url: "https://test.pypi.org/legacy/"
6 changes: 5 additions & 1 deletion .github/workflows/poetry-pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on: ["pull_request","push"]
jobs:
test:
runs-on: ubuntu-latest
strategy:
max-parallel: 3
matrix:
python-version: ["3.9","3.10","3.11"]
steps:
#----------------------------------------------
# check-out repo and set-up python
Expand All @@ -15,7 +19,7 @@ jobs:
id: setup-python
uses: actions/setup-python@v4
with:
python-version: '3.8'
python-version: ${{ matrix.python-version }}
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.vscode
.pytest_cache
neetbox.toml
__pycache__/
dist/
frontend_dist/
poetry.lock
test/optional
build_and_reinstall_wheel.cmd
Expand Down
1 change: 0 additions & 1 deletion doc/docs/guide/neetcli/workspace.md

This file was deleted.

File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

NEETBOX daemon consists of client side and server side. While client side syncs status of running project and platform information including hardware, server side provides apis for status monitoring and websocket forcasting between client and frontends.

Basically neetbox will also launch a backend on localhost when a project launching configured with daemon server address at localhost. The server will run in background without any output, and you may want to run a server with output for debug purposes.
Basically neetbox will also launch a backend on localhost when a project launching configured with daemon server address at localhost. The server will run in background without any output, and you may want to run a server with output for debug purposes.

## How to test neetbox server

Expand Down
2 changes: 1 addition & 1 deletion doc/docs/develop/index.md → docs/docs/develop/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
sidebar_position: 1
---

# Develop
# Develop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
---
sidebar_position: 2
---
---
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ In the configuration file, there are 4 default feild:
- __logging__ for configuring `neetbox.logging`
- __pipeline__ for configuring `neetbox.pipeline`
- __integrations__ for configuring `neetbox.integrations`
- __daemon__ for configuring `neetbox.daemon`
- __daemon__ for configuring `neetbox.daemon`
4 changes: 2 additions & 2 deletions doc/docs/guide/index.md → docs/docs/guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,12 @@ pip install --index-url https://pypi.org/simple/ neetbox --force-reinstall --no-
- [x] [Dev APIs](/docs/develop/) join us.

:::caution
Another problem here is that since the repository is still under construction, most of the codes do not have related docs. Sorry that some of the codes are massed up without regular comments on them. The docs will appear soon.
Another problem here is that since the repository is still under construction, most of the codes do not have related docs. Sorry that some of the codes are massed up without regular comments on them. The docs will appear soon.
:::

## Useless wordy things

__NEETBox contains useless code snippets for Deep Learning Researchers.__ The repository itself is still under construction and for now it has not much code inside. Nevertheless, we are always adding new code snippets.
__NEETBox contains useless code snippets for Deep Learning Researchers.__ The repository itself is still under construction and for now it has not much code inside. Nevertheless, we are always adding new code snippets.

So here is the thing: I'm new to complex Computer Vision projects and I found myself writing duplicate codes somehow, about convolutional neural networking, figure ploting(for paper writing), visualizing, data processing, etc., in different projects. So I'm going to have code snippets for myself and for other beginners who would like to have one. Certainly there are a lot of extraordinarily frameworks with high performance models integrated in their model hub (or stuff like that). However, that's not what I really want. Personally, I would like to have a collection of standalone code snippets which you can easily plug into your code without importing heavy dependency or doing code migration.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@ sidebar_position: 6
---

# Integrations

Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ sidebar_position: 2

# Advanced logging config

working on the doc...
working on the doc...
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ logger.log("hello", world)
```
output:
```bash
2023-03-18-13:56:03 > test.py > hello world
2023-03-18-13:56:03 > test.py > hello world
```

## Using a decorator
Expand Down Expand Up @@ -76,8 +76,8 @@ a = get_instance_of_A()
```
output:
```bash
2023-03-18-13:57:47 > test.py/get_instance_of_A > Buidling AClass Instance...
2023-03-18-13:57:47 > AClass/__init__ > Class A Ready.
2023-03-18-13:57:47 > test.py/get_instance_of_A > Buidling AClass Instance...
2023-03-18-13:57:47 > AClass/__init__ > Class A Ready.
2023-03-18-13:57:47 > AClass/post_processing > Running PostProcessing...
```

Expand All @@ -89,7 +89,7 @@ logger.log("some message")
```
output:
```bash
2023-03-18-13:58:40 > identity name > some message
2023-03-18-13:58:40 > identity name > some message
```

## Log into files
Expand All @@ -102,13 +102,13 @@ logger.log("this message will be written to stdout only", into_file=False)
```
output:
```bash
[INFO]2023-03-18-14:29:59 > neetbox.logging.logger/Logger/info > Directory ./logdir not found, trying to create.
2023-03-18-14:29:59 > test.py > this message will be written to both stdout and file
2023-03-18-14:29:59 > test.py > this message will be written to stdout only
[INFO]2023-03-18-14:29:59 > neetbox.logging.logger/Logger/info > Directory ./logdir not found, trying to create.
2023-03-18-14:29:59 > test.py > this message will be written to both stdout and file
2023-03-18-14:29:59 > test.py > this message will be written to stdout only
```
in `./logdir/2023-03-18.log`:
```
2023-03-18-14:29:59 | test.py | this message will be written to both stdout and file
2023-03-18-14:29:59 | test.py | this message will be written to both stdout and file

```

Expand All @@ -124,7 +124,7 @@ logger.log("some message", with_datetime=False)
```
output:
```
test.py > some message
test.py > some message
```

The same thing, if you want to log without identifier:
Expand All @@ -133,7 +133,7 @@ logger.log("some message", with_identifier=False)
```
output:
```
2023-03-18-14:44:22 > some message
2023-03-18-14:44:22 > some message
```

Looking for advanced format control? See [advanced logging format](./config-logger.md)
Looking for advanced format control? See [advanced logging format](./config-logger.md)
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class SomeClass:
def __init__(self) -> None:
self.logger = logger(self)
self.logger.log("something")

SomeClass()
```
output:
Expand All @@ -55,8 +55,8 @@ b()
```
output:
```
2023-03-18-22:48:15 > someone/a > a
2023-03-18-22:48:15 > someone/b > b
2023-03-18-22:48:15 > someone/a > a
2023-03-18-22:48:15 > someone/b > b
```
Check the code and run it by yourself. You will find the two lines in the output with different colors. This may be confusing, but it is not a bug. The feature was designed because you still want to distinguish between the two lines since they have different function names, right?

Expand All @@ -75,8 +75,8 @@ logger_another.log("message from", logger_another)
```
output:
```
2023-03-18-22:12:20 > None/ > message from <neetbox.logging.logger.Logger object at 0x00000217D7F60670>
2023-03-18-22:12:20 > None/ > message from <neetbox.logging.logger.Logger object at 0x00000217D7F60670>
2023-03-18-22:12:20 > None/ > message from <neetbox.logging.logger.Logger object at 0x00000217D7F60670>
2023-03-18-22:12:20 > None/ > message from <neetbox.logging.logger.Logger object at 0x00000217D7F60670>
```
As you can see, they are the same object.

Expand All @@ -94,8 +94,8 @@ logger2.log("from", logger2)
```
output:
```
2023-03-18-22:31:33 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000217D8130820>
2023-03-18-22:31:33 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000217D81303D0>
2023-03-18-22:31:33 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000217D8130820>
2023-03-18-22:31:33 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000217D81303D0>
```
As you can see, they are different objects even though they have the same identifier. One was called `Logger object at 0x00000217D8130820` and the other was `Logger object at 0x00000217D81303D0`. But if you are using a console, you will see that they are painted the same color. This is because you did not pass a specific `LogStyle` when constructing the logger, it will use automatically generated style specified by their caller identity.

Expand All @@ -110,19 +110,19 @@ logger2.log("from", logger2)
```
output:
```
[INFO]2023-03-18-22:40:21 > neetbox.logging.logger/Logger/info > Directory ./logdir1 not found, trying to create.
[INFO]2023-03-18-22:40:21 > neetbox.logging.logger/Logger/info > Directory ./logdir2 not found, trying to create.
2023-03-18-22:40:21 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000201B7D116A0>
2023-03-18-22:40:21 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000201B7D11070>
[INFO]2023-03-18-22:40:21 > neetbox.logging.logger/Logger/info > Directory ./logdir1 not found, trying to create.
[INFO]2023-03-18-22:40:21 > neetbox.logging.logger/Logger/info > Directory ./logdir2 not found, trying to create.
2023-03-18-22:40:21 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000201B7D116A0>
2023-03-18-22:40:21 > the_same_identity/ > from <neetbox.logging.logger.Logger object at 0x00000201B7D11070>
```
output in `./logdir1/2023-03-18.log`:
```
2023-03-18-22:40:21 | the_same_identity/ | from <neetbox.logging.logger.Logger object at 0x00000201B7D116A0>
2023-03-18-22:40:21 | the_same_identity/ | from <neetbox.logging.logger.Logger object at 0x00000201B7D116A0>
```
output in `./logdir2/2023-03-18.log`:
```
2023-03-18-22:40:21 | the_same_identity/ | from <neetbox.logging.logger.Logger object at 0x00000201B7D11070>
2023-03-18-22:40:21 | the_same_identity/ | from <neetbox.logging.logger.Logger object at 0x00000201B7D11070>
```
:::

This is not a bug. The feature was designed because someone needs to log different things into different files using the same caller identity.
This is not a bug. The feature was designed because someone needs to log different things into different files using the same caller identity.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sidebar_position: 5

# The NEETBOX cli

The NEETBOX cli helps you to manage your codes, datasets, experiments and more.
The NEETBOX cli helps you to manage your codes, datasets, experiments and more.

## Get started

Expand All @@ -14,9 +14,9 @@ After installing neetbox, at your working directory, try:
neet init
```

NEETBOX then generate a config file namingly "neetbox.toml" as your workspace config.
NEETBOX then generate a config file namingly "neetbox.toml" as your workspace config.

NEETBOX gather information from your project as if you import neetbox or use [@watch](/docs/guide/pipeline/watch_and_listen.md) in your code.
```bash
neet status
```
```
1 change: 1 addition & 0 deletions docs/docs/guide/neetcli/workspace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Manage workspace
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ sidebar_position: 4

Pipeline provides useful tools for building your training or inferencing procedure.

- [x] [@watch and @listen](/docs/guide/pipeline/watch_and_listen.md)
- [x] [@watch and @listen](/docs/guide/pipeline/watch_and_listen.md)
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ sidebar_position: 1

Blocks available for now:

- [x] [Simple ResBlock](./resblock)
- [x] [Simple ResBlock](./resblock)
Loading