[parley] Limit input text size to 2^30 bytes

[dfrg]

1 GB of text per layout is more than generous. We're likely to see severe performance degradation long before reaching this limit anyway suggesting that applications wanting to support excessively large buffers will need to do some segmentation and caching.

Rationale: this lets us use u32 instead of usize for internal indices which can significantly reduce memory requirements for internal data structures. Using 30 bits instead of 32 (or 31) allows us to occasionally steal a bit for other info (often incredibly useful) while still preserving the use of MAX_SIZE + 1 as the exclusive end boundary.

[DJMcNab]

To be clear, are you wanting MAX_SIZE + 1 to be $$2^{30}$$, or $$2^{30}-1$$? That is, do we have bits 30 and 31 available as "scratch", or just bit 31?

[dfrg]

Yes :) That’s my off by one error. I think we might as well reserve the 2 bits because 1 GB per layout seems to be plenty.