Rethink spa_final_dirty_txg() mechanism

[sdimitro]

The mechanism itself is a bit hacky and has caused multiple failures in the past. Most of them looked like #9186 (metaslab_condense_wanted was set in the last TXGs accidentally dirtying the metaslab for subsequent TXGs) and since have been fixed by Paul's change (#9231). Unfortunately these are not the only failures that we've seen historically though related to spa_final_txg.

The variable is set in spa_export_common():

spa_export_common(...)
{
		if (new_state != POOL_STATE_UNINITIALIZED && !hardforce) {
			spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER);
			spa->spa_state = new_state;
			spa->spa_final_txg = spa_last_synced_txg(spa) +       // <---- HERE
			    TXG_DEFER_SIZE + 1;
			vdev_config_dirty(spa->spa_root_vdev);
			spa_config_exit(spa, SCL_ALL, FTAG);
		}
}

and generally checked through this wrapper:

/*
 * Return the last txg where data can be dirtied. The final txgs
 * will be used to just clear out any deferred frees that remain.
 */
uint64_t
spa_final_dirty_txg(spa_t *spa)
{
	return (spa->spa_final_txg - TXG_DEFER_SIZE);
}

The problem is that spa_export_common() will most probably call spa_unload() which itself calls txg_wait_synced() a lot of times through other functions and can easily force the SPA to go beyond spa_final_txg. Functions within spa_unload() that have been added over time that do that are the following: spa_unload_log_sm_flush_all(), vdev_initialize_stop_all(), vdev_trim_stop_all(), vdev_autotrim_stop_all().