Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add custom folding regions #1825

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Conversation

danthe1st
Copy link

@danthe1st danthe1st commented Dec 2, 2024

Relevant issues: https://bugs.eclipse.org/bugs/show_bug.cgi?id=173796 (and there are also similar ones like https://bugs.eclipse.org/bugs/show_bug.cgi?id=258965 for CDT).

There is also a Stack Overflow post asking for how to do that with quite a few upvotes so I'd say it seems like many people are interested in such a feature.

What it does

This PR allows to add custom folding regions using comments.

I added new options to the preferences in Java > Editor > Folding (not that I changed this preference page to use Groups):
folding preference dialog including custom folding regions

If a comment containing the start region and a comment containing the end region are present in the same source file, a folding region is added allowing to collapse that section.

non-collapsed custom region
collapsed custom region
hovering over collapsed region indicator showing content

This works even for elements of different levels

Notes / What I'm not sure about

Custom folding regions are disabled if either of the custom texts/preferences are empty or if one contains the other. Should I add some information about that to the preferences? Would it be better to disable it by default (currently, it defaults to #region/#endregion)?

I haven't written any tests so far. Are there existing tests for folding in JDT-UI? What about performance tests (my implementation requires scanning more parts of the source file which could affect performance)?

Regarding performance, I used String#contains to check whether comments contain certain the region begin/end tokens (e.g. #region/#endregion) which requires creating a String for each comment every time the file is edited. Should I change that to use the char[] returned by getCurrentTokenSource() or similar?

I have bumped the minor version of org.eclipse.jdt.ui due to adding new elements in PreferenceConstants. Is this ok that way?

I guess this should be added to the "New and noteworthy" list (assuming this change gets accepted and merged). If so, should I add it and is this the right place?

How to test

  • Create a Java source file
  • Create a comment containing the text #region
  • Create a comment containing the text #endregion
  • Test similar things with the source file
  • After editing the corresponding changes in Java > Editor > Folding, it is necessary to reopen the source file.

Author checklist

@danthe1st danthe1st force-pushed the folding branch 2 times, most recently from 0301e82 to 1828731 Compare December 4, 2024 22:01
@danthe1st danthe1st marked this pull request as ready for review December 4, 2024 22:05
@danthe1st danthe1st force-pushed the folding branch 3 times, most recently from 56d6c84 to a8a21b5 Compare December 7, 2024 15:43
@HannesWell
Copy link
Contributor

@fedejeanne in the dev-call on Thursday you mentioned that you could have a look at this, does this still stand?
Eventueally we need a JDT committer to submit this anyways.

@danthe1st
Copy link
Author

danthe1st commented Dec 16, 2024

@jukzi Sorry to ping you but would you mind taking a look at my PR and giving feedback?

@jukzi jukzi added the enhancement New feature or request label Dec 16, 2024
@jukzi
Copy link
Contributor

jukzi commented Dec 16, 2024

At a rough view the PR's is a legit feature request. The code looks simple enough to be reviewed and tested. There are no obvious flaws. Such a contribution should come with a Junit Test, that tests both the added API and the effect of the change. I am sure that similar tests already exists but i don't know where. You may look in the git history of the changed files or the call hierarchy to find related tests.
Since i am personally not interested in this particular change i can not offer much time for review/testing/adding junit. If someone else (@fedejeanne ?) likes to help with that i can offer to merge once necessary steps done.

@fedejeanne
Copy link
Contributor

Hi. It's on my list but I am currently focusing on issues regarding the Edge browser, which have top priority until the end of the year.
I haven't forgotten about you @danthe1st though, I am just swamped 😓

Copy link
Contributor

@fedejeanne fedejeanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick review and made some comments regarding coding style.

I also tested a bit and this is what I found:

  • It doesn't work if I define a region completely inside the public static void main(String... args) method but it does work on other methods like public boolean equals(Object obj)
  • I can define regions if the comment starts with the custom text e.g. // #region2, if that is intended then the text in the preferences should say so. I would add some extra label explaining how custom regions work, right before the 2 check-boxes
  • If I fold the // #region2 in this snippet, the System.out.println() is folded too:
@Override
public boolean equals(Object obj) {
	// #region1
	// #region2
	if (this == obj)
		return true;
	if (obj == null || getClass() != obj.getClass())
		return false;
	Cat cat = (Cat) obj;
	// #endregion2
	System.out.println();
	return size == cat.size;
	// #endregion1
}
  • Moving regions around works without reopening the source file but creating regions doesn't. I would very much like it to work too, it doesn't feel natural to have to close the editor to add regions.

Your questions

Custom folding regions are disabled if either of the custom texts/preferences are empty or if one contains the other. Should I add some information about that to the preferences? Would it be better to disable it by default (currently, it defaults to #region/#endregion)?

Some info about it is supposed to work is good but I'd keep it short, 2-3 lines in a label. If it gets too long you could consider adding some help page like the Readme example does.
I'd disable (gray out) the group "Custom folding regions" in the preferences if the checkbox "Custom folding regions" is not selected.

I haven't written any tests so far. Are there existing tests for folding in JDT-UI? What about performance tests (my implementation requires scanning more parts of the source file which could affect performance)?

To test performance you'll have to do it manually. Alter this generator (credits to Andrey Loskutov) and let it generate several internal classes with custom folding regions: GenerateJava.zip. Then sample with VisualVM and share the results.

I don't know of any functional tests, maybe someone else does?

Regarding performance, I used String#contains to check whether comments contain certain the region begin/end tokens (e.g. #region/#endregion) which requires creating a String for each comment every time the file is edited. Should I change that to use the char[] returned by getCurrentTokenSource() or similar?

You'll have to sample with VisualVM and see if this is a performance bottle-neck.

I have bumped the minor version of org.eclipse.jdt.ui due to adding new elements in PreferenceConstants. Is this ok that way?

It's fine and this commit did the same so you're safe: d0563dc. Such bumps are no longer necessary since they will be done automatically by GH, as was the case for the mentioned commit.

I guess this should be added to the "New and noteworthy" list (assuming this change gets accepted and merged). If so, should I add it and is this the right place?

Correct, but let's cross that bridge when we get there :-) . Usually a committer adds the "noteworthy" tag to your PR / Issue right after it was merged/solved and that's your queue to do the PR for the N&N entry.

}
case ITerminalSymbols.TokenNameCOMMENT_LINE:
//$FALL-THROUGH$
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd avoid falling through, it hurts readability.

I'd push the if (fCustomFoldingRegionsEnabled) inside checkCustomFolding (make it an if-!-return) and call the method once for each case in this switch.

Comment on lines 755 to 756
private String fCustomFoldingRegionBegin="#region"; //$NON-NLS-1$
private String fCustomFoldingRegionEnd="#endregion"; //$NON-NLS-1$
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are overwritten later anyway.

Suggested change
private String fCustomFoldingRegionBegin="#region"; //$NON-NLS-1$
private String fCustomFoldingRegionEnd="#endregion"; //$NON-NLS-1$
private String fCustomFoldingRegionBegin;
private String fCustomFoldingRegionEnd;

Comment on lines 1218 to 1222
for(int token = scanner.getNextToken(); token != ITerminalSymbols.TokenNameEOF; token=scanner.getNextToken()) {
if(isCommentToken(token)) {
checkCustomFolding(ctx, regions, scanner, scanner.getCurrentTokenStartPosition(), regions.size() - 1);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated code.

Extract to method and call twice.

@danthe1st
Copy link
Author

danthe1st commented Dec 19, 2024

It doesn't work if I define a region completely inside the public static void main(String... args) method but it does work on other methods like public boolean equals(Object obj)

Should be fixed, this was because of it being the only (or last) method.

I can define regions if the comment starts with the custom text e.g. // #region2

It should work whenever a comment includes the text #region so it could be // Hello #region World or anything like that. If you would (intuitively) expect something else, I'd be happy to hear that and possibly change my implementation. Currently, it treats all comments the same way but I'm open to change that if other people prefer it differently.

Moving regions around works without reopening the source file but creating regions doesn't. I would very much like it to work too, it doesn't feel natural to have to close the editor to add regions.

Interestingly, that worked for my by just adding a comment like // #region or whatever. What doesn't work is it being automatically updated when changing the preferences related to custom folding regions.
Maybe you wrote the #region commend but not the #endregion comment?

I'd disable (gray out) the group "Custom folding regions" in the preferences if the checkbox "Custom folding regions" is not selected.

The checkbox is about initially folding/collapsing these elements. If the checkbox is disabled, the elements should still be folded but not collapsed by default.

I don't know of any functional tests, maybe someone else does?

I didn't find any existing tests related to folding so I'll try making my own tests for custom folding regions in a new directory org.eclipse.jdt.ui.tests/ui/org/eclipse/jdt/ui/tests/folding (I currently don't have a good way to access the folding regions/annotations added to the editor and I don't want to add API just for tests but I might find a good way to do it at some point ()). If anyone still finds tests after that, it shouldn't be hard to move them (once I wrote the tests).

My current approach for automated tests is subclassing DefaultJavaFoldingStructureProvider and overriding computeFoldingStructure and getProjectionRanges to access the created IRegion[]s.


Anyway, do you have any opinions on the defaults for the #region/#endregion marker texts? Do you think using a different text would be better? I think one alternatively I have seen in other IDEs was // region where it must be at the start of the comment but I felt it being at any position was more flexible.

@danthe1st
Copy link
Author

danthe1st commented Dec 19, 2024

image

I might still rework that label in the future.

@BeckerWdf
Copy link
Contributor

The texts: "Text marking start of custom folding region" looks a bit unnecessary too long for my
Why not simply saying "Text marking start of region" and "Text marking end of region"
due to the box around you don't need to repeat that it's a "custom folding region".

I am unsure if it's singular "region" or pluar "regions".

@danthe1st
Copy link
Author

danthe1st commented Dec 19, 2024

As discussed in today's devcall, I changed the PR to disallow regions between an element and it's child.

class X{
	// #region
	void a() {
		// #endregion this is not folded with the outer region because it is at a different level
		
		// #region However, this is folded because loops are not considered children elements and I don't think it's a good idea to parse all of that for folding
		for(int i=0; i<10;i++) {
			// #endregion
		}
	}
	// #endregion this will be folded with the outer region
}

Another suggestion during the devcall was to restrict this to comments starting with #region (or whatever is configured) instead of allowing it anywhere (// Hello #region World).
Since the token scanner seems to treat the whole comment as a token (including the prefix like //, /*, /** or ///), I decided to not change this for now. If it is wanted, I can implement something stripping away the prefix of the comment token (skipping up to two slashes/stars and any whitespaces at the beginning) but I am unsure whether this is actually the preferred way of doing this as this might be a maintainance burden if new comment types (or similar) are added to Java.

@danthe1st
Copy link
Author

danthe1st commented Dec 19, 2024

Preliminary profiling results:

I ran the generator mentioned by @fedejeanne with 12_000 inner classes (the default) and added a custom region both outside and inside each of these classes.

I loaded the class in Eclipse, started CPU sampling with VisualVM, added some newlines at the top, bottom and in the middle of the file. I also added a single-line comment in the middle of the file.
This is what VisualVM shows:
image

Similar with CPU profiling:
image

As it seems, the custom folding logic does have an impact on performance.

Loading all the folding annotation takes a bit of time in general (especially with the profiler active) but it does not block the UI.

CPU profiling with similar operations with custom folding regions disabled (loading the folding regions still takes time with in this case):
image

It seems like the biggest issue is that custom folding regions require the scanner to process more tokens (it needs to actually go over method bodies etc which is otherwise not done for methods without any children). I could get rid of the scanner.getCurrentTokenSource() call in every comment but I don't think it would change much here.

I don't know whether this is a problem. The generated file has 156 005 lines. If the performance is a problem, we can disable this feature by default.

@fedejeanne
Copy link
Contributor

@danthe1st can you provide:

  1. The simplified version of the generated file (generate it with 1 inner class)
  2. Sampling/performance analysis for the same scenarios on master (for comparison)?

Keep in mind that generating 12.000 inner classes is an extreme case so performance will always degrade, the question is how much.

@akurtakov
Copy link
Contributor

As a reference for people interested in similar functionality for other editors - Generic editor provides extension point for that https://github.com/eclipse-platform/eclipse.platform.ui/blob/master/bundles/org.eclipse.ui.genericeditor/schema/foldingReconcilers.exsd

@jukzi jukzi added the noteworthy Noteworthy feature label Dec 20, 2024
@danthe1st
Copy link
Author

danthe1st commented Dec 20, 2024

profiling with merge base (not current master but the master I based my PR on):
image

sampling:
image

Generated code with one inner class:

package a.generated;

public class TestClass {

    // #region outside
    public static class InnerClass0 {
        // #region inside
        public int x = 42; // no error
        
        public void helloWorld() {
                System.out.println("Hello World!"); // no error
                System.out.println("In class " + 0 + ", x=" + x); // expected error
        }
        // #endregion inside
    } // end of InnerClass0
    // #endregion

}

(I changed the code to not have any compilation errors - some regions may be missing in that case (probably because of the token scanner behaving differently))

Notably, this doesn't test long comments (the current implementation converts all comment tokens to a String on which it performs the contains() check. As mentioned above, I could change this to only check the beginning of the comment (and use the char[] provided by scanner.getSource() or similar) to only check the first characters (filtering out things like //, ///, /*, /**). If wanted, I can also make a comparison with that.

@danthe1st
Copy link
Author

Other experiment with 6_000 inner classes and each class having a multiline comment with 20 lines each containing the character a 100 times:

// #region outside
    /*
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    * aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
    */
    public static class InnerClass0 {
        // #region inside
        public int x = 42; // no error
        
        public void helloWorld() {
                System.out.println("Hello World!"); // no error
                System.out.println("In class " + 0 + ", x=" + x); // expected error
        }
        // #endregion inside
    } // end of InnerClass0
    // #endregion

Custom regions disabled, sampling:
image

Custom regions disabled, profiling (I had to disable automated collapsing of normal/header comments to be able to do this without Eclipse not responding):
image

Custom regions enabled, sampling:
image

Custom regions enabled, profiling (automated collapsing of comments enabled as well):
image

@danthe1st
Copy link
Author

danthe1st commented Dec 20, 2024

I realized that I can use the information of what the current token is from the scanner to know where I need to start looking for characters (so I don't need to check for things like //). With this change, it is now required that region markers (#region/#endregion) are at the start of the comment so // Hello #region World is now prevented as discussed in the devcall.
I used this chance to change the implementation to use char[]s and not process the entire comment.

Sampling with that change and custom folding regions enabled:
image

Profiling:
image

Note that all VisualVM screenshots I have included so far are about modifying source files after opening them and not about initially loading the source files (this isn't included in any sampling/profiling I've done until now).

@danthe1st
Copy link
Author

danthe1st commented Dec 20, 2024

EDIT: This issue is fixed now.

I noticed a bug in my implementation where the following is not detected:

class Z{
	void a() {
		// #region this is not detected because there is an inner class (considered as a child) in the same scope and there are statements/non-comment tokens between the comment with the region marker and the inner class
		int x;
		
		// #endregion this marker can be detected without issues
		class Inner{
			
		}
	}
}

This seems to only occur with local classes (declared within methods). The "old" folding implementation (master) generally ignores multi-line comments written in positions like that (it only folds comments before classes/methods/etc).

I plan to try fixing it but doing so might require over scanning bigger parts of the source file.

@BeckerWdf
Copy link
Contributor

image

I might still rework that label in the future.

I am not a big fan of big explanatory text directly in the preference page. Can't we simply put this into the help document related to that preference page?

Copy link
Contributor

@BeckerWdf BeckerWdf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the text shortening.

@danthe1st
Copy link
Author

I am not a big fan of big explanatory text directly in the preference page. Can't we simply put this into the help document related to that preference page?

I guess I can remove the label and write a PR changing https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/master/eclipse.platform.common/bundles/org.eclipse.jdt.doc.user/reference/preferences/java/editor/ref-preferences-folding.htm once this is done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request noteworthy Noteworthy feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants