table cache: use a long-lived ReaderProvider

[RaduBerinde]

An sstable.ReaderProvider is necessary whenever an sstable has value
blocks. Currently, we allocate one every time we create a point
iterator.

This commit makes the tableCacheShardReaderProvider usable in
parallel and maintains a long-lived instance in the tableCacheValue.

[cockroach-teamcity]

This change is 

[sumeerbhola]

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jbowens and @RaduBerinde)

---

table_cache.go line 706 at r1 (raw file):

	// Calling findNode gives us the responsibility of decrementing v's
	// refCount.
	v := rp.c.findNodeInternal(ctx, rp.backingFileNum, rp.dbOpts)

Consider adding a comment that fineNodeInternal can do IO, in order to initialize a Reader, and that we are deliberately holding rp.mu while doing so, in that concurrent calls to GetReader will block until this initialization is done.

[RaduBerinde]

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jbowens and @sumeerbhola)

---

table_cache.go line 706 at r1 (raw file):

Previously, sumeerbhola wrote…

Consider adding a comment that fineNodeInternal can do IO, in order to initialize a Reader, and that we are deliberately holding rp.mu while doing so, in that concurrent calls to GetReader will block until this initialization is done.

Done. TFTR!