refactor: move the logic to Tokenizer from TokenizerMacros #359

Sign in to view logs

Summary
Jobs
- build-linux
- lint
Run details
- Usage
- Workflow file

Run time

Learn about OS pricing on GitHub Actions

Job	Run time
lint	44s
build-linux	5m 7s
	5m 51s