Autocompletion and Syntax Highlight. #2171

kareltucek · 2024-02-06T13:31:19Z

Here is some proof of concept of grammar-based autocompletion for Agent.

@ert78gb could you please look at the TODO and either implement it, or advice me on how to do that?

Also, if you have any nitpicks about my code, they are welcome. I am completely unacquainted with typescript customs 🙃.

ert78gb · 2024-02-06T13:44:06Z

Thx for your time. Now worries about typescript :) I could review it only on tomorrow.

kareltucek · 2024-02-06T13:46:44Z

No need to hurry!

mondalaci · 2024-02-06T16:39:08Z

Very fancy, but the default colors could be better:

At least, I'd use a less noticeable color for comments, like light gray. Assuming that there are color scheme presets, we could pick a better one.

kareltucek · 2024-02-06T19:48:20Z

(You realize that this PR is much more about autocomplete than it is about syntax highlight and that the point of adjusting syntax highlight was only to make it stop assigning colours mostly at random, right? Just checking.)

Anyway, what about this?

kareltucek · 2024-02-06T20:07:32Z

Also, this should be the list of built-in themes (may be strongly out of date), although I have no idea on how to activate them except for the 15-minute rebuild. (That can be done on line 36 of agent/packages/uhk-web/src/app/components/macro/action-editor/tab/command/macro-command-editor.component.ts in case you want to toy with it.)

    vs: The default light theme.
    vs-dark: The default dark theme.
    hc-black: A high contrast black theme.
    hc-white: A high contrast white theme.
    kiry-dark: A dark theme with cyan, purple, and dark colors.
    kiry-light: A light theme with cyan, purple, and light colors.
    light: A lighter version of the default light theme.
    dark: A darker version of the default dark theme.

(The above one is custom-coded.)

mondalaci · 2024-02-06T23:51:55Z

You realize that this PR is much more about autocomplete than it is about syntax highlight ...

Now, I do. :)

@ert78gb Please send me the full list of alternative themes and instructions for activating them. (If this issue is relatively time-consuming, the module configuration UI issue takes precedence over this one.)

ert78gb · 2024-02-07T22:06:47Z

@mondalaci we used the vs theme for light theme and custom uhk-dark theme that matches with the Agent dark theme.
When the user changes the Agent theme then automatically changing the monaco editor theme too..
This PR contains a new uhk-light theme and Agent actives it when switching between themes.

I think have to extend the uhk-dark mode to colorise values and keywords.

mondalaci · 2024-02-07T22:11:50Z

@ert78gb Great idea! If there are multiple light/dark Monaco editor theme options, I'd like to try them.

ert78gb · 2024-02-07T22:28:42Z

@kareltucek I started to review PR, but I haven't spent too much time on it, because when I start running the browser version of the Agent then it did not start because the uhk_preset.js file in the naive-autocompletion-parser contains the reference to the node process module. The browser does not support nodejs specific features.

I did a quick insight into the naive-autocompletion-parser package. It downloads file from the firmware repo. Was it not caused an issue when the user does not have internet? Currently the firmware zip contains every resource that needs for the firmware upgrade and smart macro doc. If we liked to provide full customisable firmware maybe we have to read autocomplete and other setups from the firmware zip.

ert78gb · 2024-02-07T22:33:02Z

You can try the built in themes on the monaco editor playground
The available themes listed by Karel in #2171 (comment)

set the theme: 'theme-id' of editor constructor parameters

kareltucek · 2024-02-08T00:53:05Z

I started to review PR, but I haven't spent too much time on it, because when I start running the browser version of the Agent then it did not start because the uhk_preset.js file in the naive-autocompletion-parser contains the reference to the node process module. The browser does not support nodejs specific features.

Ah, I see.

Fixed.

It downloads file from the firmware repo.

Yes. Pulling reference-manual.md should be solved analogous to the way the Agent and firmware handle the smart macro docs. (That's what the todo is about anyways.) I didn't realize that we need to extend the builds for that too though.

kareltucek · 2024-02-08T11:56:34Z

The available themes listed by Karel in #2171 (comment)

Actually, only 'vs' and 'vs-dark' work :-(.

kareltucek · 2024-02-08T12:55:32Z

Hmmm, in a new (non-saved) macro action, completions are listed in many duplicities. In a saved action, everything works right.

This could be caused by duplicit registration of the completion provider, except that I make sure I register the provider only once (via a static variable).

Any ideas?

ert78gb · 2024-02-24T11:21:26Z

@kareltucek

Actually, only 'vs' and 'vs-dark' work :-(.

Do you mean in Agent? Because in Agent maybe yes I have to investigate, but the playground supports all.

This could be caused by duplicit registration of the completion provider, except that I make sure I register the provider only once (via a static variable).

I am investigating it

kareltucek · 2024-02-24T12:52:12Z

Do you mean in Agent? Because in Agent maybe yes I have to investigate, but the playground supports all.

Hm, I believe I tried them in the playground.

kareltucek · 2024-02-26T18:25:27Z

@ert78gb Don't want to press you, but could you give me some pointers on how to:

add the doc-dev/reference-manual.md to the list of files that are pulled from the web or from the firmware release
retrieve the reference-manual.md from the service that takes care of pulling the relevant docs from github or firmware release

(I already added the reference-manual.md to the firmware build.)

ert78gb · 2024-02-26T19:01:06Z

@kareltucek No worries and sorry I don't have enough time to work on it.

add the doc-dev/reference-manual.md to the list of files that are pulled from the web or from the firmware release

retrieve the reference-manual.md from the service that takes care of pulling the relevant docs from github or firmware release

The smart macro document contains the link to the user guide. https://github.com/UltimateHackingKeyboard/firmware/blob/f794a563b2053144c2305c84fa5d3bd70c026f69/doc/dist/index.html#L595C48-L595C80

What is the goal of this md files? Would the same linking enough?

kareltucek · 2024-02-26T19:10:19Z

Ah, The autocompletion parser uses the grammar listed in the reference-manual.md, so I just need to get the file (as a string) and hand it over to the parser. Most handling (like retrieving the grammar from the file) is hidden in the naive-autocompletion-parser package.

Now I am doing it here which leads into the parser code. I guess we consider that to be just a temporary hack and want to replace it with whatever system you are using for the smart macro pane.

(Also, current code does not respect current firmware tag. Which is probably not a big deal at the moment, but given that we have the system in place, it would be nice to be consistent just on general principles.)

ert78gb · 2024-02-26T20:00:47Z

The smart macro doc servered via a webserver. The web server has been started when Agents start on a random port.
Agent has an internal state. The port number of the web server stored in the smart macro substate.

Angular mainly follow OOP, so you have to inject everything.
The HttpClient is a built in Angular http client.
You can inject the HttpClient and the ngrx Store into the CustomCompletionProvider.
With the smartMacroDocState selector you can query the smart macro substate from the store.
Finally you could create any http request to the local web server.

The selectSmartMacroDocUrl selector already build the smart macro doc url. It also handles the specialities of the web and electron build.
I suggest to create a new selector that calculate the base path of the firmware url and the selectSmartMacroDocUrl use this new selector.

I could give more guidance if you need. Or I could finish this PR on this week. I wanted to review it on Sunday but I had a unexpected program.

kareltucek · 2024-02-29T10:08:19Z

I could give more guidance if you need. Or I could finish this PR on this week. I wanted to review it on Sunday but I had a unexpected program.

More guidance would be appreciated. (Maybe just a few links to where these things are defined and used?)

(I find it desirable to learn the ropes when it comes to both Agent codebase in specific, and typescript ways of doing things in general.)

(I am still utterly lost, so somewhere between "give me links to all relevant code sites" and "complete coding-monkey instructions" is the right granularity.)

ert78gb · 2024-02-29T19:08:15Z

I am reviewing this PR on the weekend (Saturday is my current plan). I am trying to write really detailed review that maybe helps to understand Agent. The key framework here is Angular and not really the typescript.
If you would like we could also create a video session after the review. If you will have questions maybe we could talk about it quicker than written.

Agent is a bit different from a simple Angular single page application because it running in electron and browser too.
And we use Angular de facto state management library https://ngrx.io. Worth to review the documentation.

kareltucek · 2024-02-29T21:54:14Z

If you would like we could also create a video session after the review.

I would love to. I will be available in the evenings (sat, sun), if that works for you. Otherwise, I am afraid that written review will have to do.

ert78gb

To be continue tomorrow. Until I submit my recommendations

packages/uhk-web/package.json

packages/uhk-web/package-lock.json

packages/uhk-web/src/app/components/macro/action-editor/tab/command/completion-provider.ts

packages/uhk-web/src/app/components/macro/action-editor/tab/command/highlight-provider.ts

packages/uhk-web/src/app/components/macro/action-editor/tab/command/completion-provider.ts

ert78gb · 2024-03-03T19:37:26Z

...uhk-web/src/app/components/macro/action-editor/tab/command/macro-command-editor.component.ts

+            monaco.languages.register({ id: languageId });
+
+            // Register the custom completion provider
+            const completionProvider = new CustomCompletionProvider(this.logService);


We can use Angular built-in dependency injection. https://angular.io/guide/architecture-services

ert78gb · 2024-03-03T21:26:59Z

...uhk-web/src/app/components/macro/action-editor/tab/command/macro-command-editor.component.ts

+    private setLanguageProperties(editor: MonacoStandaloneCodeEditor) {
+        const languageId = 'uhkScript';
+
+        if (!MacroCommandEditorComponent.completionRegistered) {


This static variable is not really good for manage new language service registration, because if someone upgrade the firmware we have to apply the new language service.

We have to registerCompletionItemProvider and setMonarchTokensProvider.
We could register these new providers as we register the new themes in the MonacoEditorCustomThemeService. The difference is we have to re-register the providers every time when the configurations changed.

We can create a new MonacoEditorUhkLanguageService this service has to subscribe the monacoLoaderService.isMonacoLoaded$ and when the new getLanguageServiceConfig. To combine 2 observers we could use combineLatest function.

How to create the getLanguageServiceConfig observerable?
We can extend the packages/uhk-web/src/app/store/reducers/smart-macro-doc.reducer.ts or create a new reducer. to be continue tomorrow

The Agent has 2 main "process" the electron thread and the browser thread. The 2 world communicate with each other via electron built in inter-process communication

The packages/uhk-common/src/util/ipcEvents.ts file contains all of the ipc events that Agent uses. I tried to organise them. When the main module reply to an event in async mode I used the Reply suffix.

Every time when the user opens the smart macro document panel.

The renderer process dispatches the SmartMacroDoc.downloadDocumentation event.

The main process checks the documentation for the firmware is available or not. if not then download it from the server.

This moment is too late to load the auto-complete md file, because maybe the user start using the smart macro editor before opens the smart macro doc panel.
I think everytime when Agents starts or updates the firmware then have to check the smart macro document is available or not. If it not available, then have to download it and after provide the auto complete markdown file to the renderer process.

If we use the web version of the Agent then when Agents start we have to download the auto complete markdown file from the Agent repo.

To achieve this functionality we have to refactor many small things. I don't know should I able to write every step in high level. My assumption is to provide a really step-by-step guide I have to implement the whole code in my mind. I am afraid if I forget something maybe I create a bigger problem for you than guidance.

I think faster if I finish the PR. Unfortunately, my availability will very limited in the next 3 weeks.
I am happy to help you to learn Agent coding, but I think we have to start with a smaller issue if you don't mind.

I think faster if I finish the PR.

Allright, it is yours then.

Unfortunately, my availability will very limited in the next 3 weeks.

No worries ;)

kareltucek · 2024-05-04T12:13:09Z

@mondalaci @ert78gb

Could this get a bit higher priority now?

If no, then I would vote for merging this as it is now, as I believe that even this version brings a lot of value. Respecting current firmware version is definitely nice to have as well as the clean way of doing things, but at the end of the day, it is a rather minor improvement.

ert78gb · 2024-05-04T13:16:33Z

I think there are 2 issue before it:

usb scripts don't run from none agent directory. It is ts-node and node 20 conflict issue
jump to macro from status buffer error.

I hope both will solve on the next week and I can work on it

kareltucek · 2024-05-04T13:23:31Z

Ok sure!

ert78gb · 2024-05-10T20:36:43Z

superseded by #2270

Integrate autocompletion and a lexer specific to uhk.

491ddec

cla-sign-bot bot added the cla:signed label Feb 6, 2024

kareltucek marked this pull request as draft February 6, 2024 13:32

Remove unused testing code.

69fad64

Try to satisfy linter?

df53a1f

Try to appease CI runner?

9f7e985

kareltucek added 4 commits February 6, 2024 19:10

Try appease web build?

fab9a25

Appease web build.

a807b2d

Override some colors.

b210c3a

Remove unwanted packages.

39a4d9d

Adjust colors.

42853dd

kareltucek added 2 commits February 6, 2024 22:33

Update naive parser.

26049de

Fix package lock.

fd17c3d

kareltucek added 2 commits February 7, 2024 23:48

Update naive-autocompletion-parser. (Huge performance improvement.)

77ee9c6

Update naive autocompletion parser.

75a90a6

Upgrade the parser.

0e9da81

Show nicer icons, and fix some grammar problems.

c598815

kareltucek added 5 commits February 9, 2024 22:24

Order completion items intelligently.

e215106

linter fixes.

4c1f260

Upgrade naive parser.

4ba8c45

Upgrade naive parser - fix problems in prefix unification.

ac7a77d

try ci again?

01c98e7

ert78gb requested changes Mar 3, 2024

View reviewed changes

kareltucek added 3 commits March 9, 2024 10:13

Merge remote-tracking branch 'origin/master' into autocompletion

062515c

Address review findings.

518c9db

Fix linter.

ff78f64

kareltucek changed the title ~~WIP: Autocompletion and Syntax Highlight.~~ Autocompletion and Syntax Highlight. Mar 9, 2024

kareltucek marked this pull request as ready for review March 9, 2024 09:42

ert78gb closed this May 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Autocompletion and Syntax Highlight. #2171

Autocompletion and Syntax Highlight. #2171

kareltucek commented Feb 6, 2024

ert78gb commented Feb 6, 2024

kareltucek commented Feb 6, 2024

mondalaci commented Feb 6, 2024

kareltucek commented Feb 6, 2024 •

edited

Loading

kareltucek commented Feb 6, 2024 •

edited

Loading

mondalaci commented Feb 6, 2024

ert78gb commented Feb 7, 2024

mondalaci commented Feb 7, 2024

ert78gb commented Feb 7, 2024

ert78gb commented Feb 7, 2024

kareltucek commented Feb 8, 2024 •

edited

Loading

kareltucek commented Feb 8, 2024

kareltucek commented Feb 8, 2024 •

edited

Loading

ert78gb commented Feb 24, 2024

kareltucek commented Feb 24, 2024

kareltucek commented Feb 26, 2024

ert78gb commented Feb 26, 2024

kareltucek commented Feb 26, 2024 •

edited

Loading

ert78gb commented Feb 26, 2024

kareltucek commented Feb 29, 2024 •

edited

Loading

ert78gb commented Feb 29, 2024

kareltucek commented Feb 29, 2024

ert78gb left a comment

ert78gb Mar 3, 2024

ert78gb Mar 3, 2024

ert78gb Mar 5, 2024

kareltucek Mar 5, 2024

kareltucek commented May 4, 2024

ert78gb commented May 4, 2024

kareltucek commented May 4, 2024

ert78gb commented May 10, 2024

Autocompletion and Syntax Highlight. #2171

Autocompletion and Syntax Highlight. #2171

Conversation

kareltucek commented Feb 6, 2024

ert78gb commented Feb 6, 2024

kareltucek commented Feb 6, 2024

mondalaci commented Feb 6, 2024

kareltucek commented Feb 6, 2024 • edited Loading

kareltucek commented Feb 6, 2024 • edited Loading

mondalaci commented Feb 6, 2024

ert78gb commented Feb 7, 2024

mondalaci commented Feb 7, 2024

ert78gb commented Feb 7, 2024

ert78gb commented Feb 7, 2024

kareltucek commented Feb 8, 2024 • edited Loading

kareltucek commented Feb 8, 2024

kareltucek commented Feb 8, 2024 • edited Loading

ert78gb commented Feb 24, 2024

kareltucek commented Feb 24, 2024

kareltucek commented Feb 26, 2024

ert78gb commented Feb 26, 2024

kareltucek commented Feb 26, 2024 • edited Loading

ert78gb commented Feb 26, 2024

kareltucek commented Feb 29, 2024 • edited Loading

ert78gb commented Feb 29, 2024

kareltucek commented Feb 29, 2024

ert78gb left a comment

Choose a reason for hiding this comment

ert78gb Mar 3, 2024

Choose a reason for hiding this comment

ert78gb Mar 3, 2024

Choose a reason for hiding this comment

ert78gb Mar 5, 2024

Choose a reason for hiding this comment

kareltucek Mar 5, 2024

Choose a reason for hiding this comment

kareltucek commented May 4, 2024

ert78gb commented May 4, 2024

kareltucek commented May 4, 2024

ert78gb commented May 10, 2024

kareltucek commented Feb 6, 2024 •

edited

Loading

kareltucek commented Feb 6, 2024 •

edited

Loading

kareltucek commented Feb 8, 2024 •

edited

Loading

kareltucek commented Feb 8, 2024 •

edited

Loading

kareltucek commented Feb 26, 2024 •

edited

Loading

kareltucek commented Feb 29, 2024 •

edited

Loading