Merge pull request chapel-lang#5748 from ronawho/limit-qt-pool-size

Limit the maximum size of qthreads memory pools to 65MB

Further limit the max pool size for qthreads now that we know there's a stack
pool per worker instead of a global pool.

In chapel-lang#257 (and chapel-lang#1649) we added some code to limit the size of qthreads memory
pools. Qthreads uses memory pools for things like task stacks, thread private
data, and runtime data. qthreads assumed that the item size for things in the
pool is small, however our default stack size is quite large (8MB.)

I did something like `QT_MAX_POOL_ALLOC_SIZE=2 * hwpar * callStackSize` with a
cap of 513MB (68 default-sized task stacks) to limit how much memory we'd use.
At the time I believed that there was a single global pool for task stacks,
so I wanted to make sure the pool would hold at least maxTaskPar (hwpar)
stacks. It turns out that there is actually a pool per worker.

On KNL, this meant we were trying to create 68 pools that each consumed 513MB of
memory. This ended up being ~34GB just for task stacks, which is outrageously
large. For now, just limit the max pool size to 65MB instead of 513MB (room of
8 default-size task stacks)

Note that this _could_ hurt performance for benchmarks that create a lot of
nested parallelism. It doesn't seem to hurt any of our existing benchmarks, and
it shouldn't hurt long lived tests because the allocation cost should be
amortized over the length of the program but it could hurt micro-benchmarks
that test nested parallelism.

runtime/src/tasks/qthreads/tasks-qthreads.c

@@ -646,8 +646,8 @@ static void setupCallStacks(int32_t hwpar) {
         // pages, so we thrown an extra MB.
         if (hwpar > 0) {
             const size_t oneMB = 1024 * 1024;
-            const size_t allocSizeLowerBound =  33 * oneMB;
-            const size_t allocSizeUpperBound = 513 * oneMB;
+            const size_t allocSizeLowerBound = 33 * oneMB;
+            const size_t allocSizeUpperBound = 65 * oneMB;
             size_t maxPoolAllocSize;
             char newenv_alloc[QT_ENV_S];