Core

Core

Port of vscode-textmate written in TypeScript to Java

org.eclipse.textmate4e.core is the project which provide TextMate tokenizer which is not linked to Eclipse (you could use with Swing or other UI toolkit).

API

It provides the same API than vscode-textmate.

Registry registry = new Registry();
IGrammar grammar = registry.loadGrammarFromPathSync("JavaScript.tmLanguage",
		Main.class.getResourceAsStream("JavaScript.tmLanguage"));
ITokenizeLineResult lineTokens = grammar.tokenizeLine("function add(a,b) { return a+b; }");
for (int i = 0; i < lineTokens.getTokens().length; i++) {
	IToken token = lineTokens.getTokens()[i];
	System.out.println("Token from " + token.getStartIndex() + " to " + token.getEndIndex() + " with scopes "
			+ token.getScopes());
}

Token from 0 to 8 with scopes [source.js, meta.function.js, storage.type.function.js]
Token from 8 to 9 with scopes [source.js, meta.function.js]
Token from 9 to 12 with scopes [source.js, meta.function.js, entity.name.function.js]
Token from 12 to 13 with scopes [source.js, meta.function.js, meta.function.type.parameter.js, meta.brace.round.js]
Token from 13 to 14 with scopes [source.js, meta.function.js, meta.function.type.parameter.js, parameter.name.js, variable.parameter.js]
Token from 14 to 15 with scopes [source.js, meta.function.js, meta.function.type.parameter.js]
Token from 15 to 16 with scopes [source.js, meta.function.js, meta.function.type.parameter.js, parameter.name.js, variable.parameter.js]
Token from 16 to 17 with scopes [source.js, meta.function.js, meta.function.type.parameter.js, meta.brace.round.js]
Token from 17 to 18 with scopes [source.js, meta.function.js]
Token from 18 to 19 with scopes [source.js, meta.function.js, meta.decl.block.js, meta.brace.curly.js]
Token from 19 to 20 with scopes [source.js, meta.function.js, meta.decl.block.js]
Token from 20 to 26 with scopes [source.js, meta.function.js, meta.decl.block.js, keyword.control.js]
Token from 26 to 28 with scopes [source.js, meta.function.js, meta.decl.block.js]
Token from 28 to 29 with scopes [source.js, meta.function.js, meta.decl.block.js, keyword.operator.arithmetic.js]
Token from 29 to 32 with scopes [source.js, meta.function.js, meta.decl.block.js]
Token from 32 to 33 with scopes [source.js, meta.function.js, meta.decl.block.js, meta.brace.curly.js]