LuauTokenizer

Basic Luau Tokenizer for script context scanners. Supports basic Luau syntax however its free to edit if you want to support more complex syntax. Has a full fledged error handler, any error that occurs during tokenization will be caught and printed to the console along with the line number on which it occurred. The tokenizer will then skip ahead to the next line and continue tokenizing.