[gTest] Add more test coverage for Batchnorm Forward Training

[xinlipn]

More test cases added for better coverage. Here's the summary

1. N C H W: 768 1 14 14
   Covers:
   forward_spatial_single.cpp: variant == 0, variant == 1
   768 1 14 14.txt

2. N C H W: 768 1 23 23
   Covers:
   forward_spatial_single.cpp: variant == 1 and IsApplicable()
   forward_spatial_multiple.cpp: variant == 2 (1st)
   768 1 23 23.txt

3. N C H W: 832 1 14 14
   Covers:
   forward_spatial_single.cpp: variant == 1 and IsApplicable()
   forward_spatial_multiple.cpp: variant == 0 then variant == 2 (2nd)
   832 1 14 14.txt

4. N C H W: 832 1 28 28
   Covers:
   forward_spatial_single.cpp: variant == 1 and IsApplicable()
   forward_spatial_multiple.cpp: variant == 2 (1st) then variant == 2 (2nd)
   832 1 28 28.txt

A few questions/observations:

1. Should NetworkLarge be used for GPU_BN_FWD_Train_Large_FP64?
   

   MIOpen/test/gtest/bn_fwd_train.cpp

   Line 147 in 310192d

   testing::Combine(testing::ValuesIn(NetworkSmall<BNTestCase>()),

2. This blurb in IsApplicable
   

   MIOpen/src/solver/batchnorm/forward_spatial_single.cpp

   Lines 75 to 80 in 310192d

   	if(!(WORKAROUND_SWDEV_253606 == 0 && n < 3) &&
   	!((in_nhw < 33554432 && in_cstride > 1024) ||
   	((n >= 256) && (in_cstride > 60) && bfpmixparm) ||
   	((in_cstride > 512) && bfpmixparm) ||
   	in_cstride <= 512))
   	return false;

   
   is the same as in GetSolution
   

   MIOpen/src/solver/batchnorm/forward_spatial_single.cpp

   Lines 159 to 161 in 310192d

   	if( (in_nhw < 33554432 && in_cstride > 1024) ||
   	((n >= 256) && (in_cstride > 60) && (bfpmixparm || bbfpmixparam)) ||
   	((in_cstride > 512) && (bfpmixparm || bbfpmixparam)))

   
   Which means this snippet will never be run
   

   MIOpen/src/solver/batchnorm/forward_spatial_single.cpp

   Lines 169 to 179 in 310192d

   	else
   	{
   	variant = 2;
   	xlocalsize = 1;
   	ylocalsize = 1024;
   	auto segment = int(std::ceil(double(in_cstride) / double(ylocalsize)));
   	xgridsize = c;
   	ygridsize = segment * ylocalsize;
   	ldsgcn = ylocalsize / 64;
   	ldsnogcn = ylocalsize;
   	}

3. Likewise, these two lines are identical
   

   MIOpen/src/solver/batchnorm/forward_spatial_single.cpp

   Line 83 in 310192d

   if((n > 768) && (in_cstride > 150) && bfp32parm)

   
   

   MIOpen/src/solver/batchnorm/forward_spatial_single.cpp

   Line 182 in 310192d

   if((n > 768) && (in_cstride > 150) && bfp32parm)

Due to 2 and 3 above, variant will never be set to 2 in forward_spatial_single.cpp

4. When this condition is met in forward_spatial_single.cpp, IsApplicable returns false
   

   MIOpen/src/solver/batchnorm/forward_spatial_single.cpp

   Lines 75 to 80 in 310192d

   	if(!(WORKAROUND_SWDEV_253606 == 0 && n < 3) &&
   	!((in_nhw < 33554432 && in_cstride > 1024) ||
   	((n >= 256) && (in_cstride > 60) && bfpmixparm) ||
   	((in_cstride > 512) && bfpmixparm) ||
   	in_cstride <= 512))
   	return false;

   
   And forward_spatial_multiple.cpp will be called. In its GetSolution, this snippet will always be skipped due to the fact it's false as above
   

   MIOpen/src/solver/batchnorm/forward_spatial_multiple.cpp

   Lines 123 to 132 in 310192d

   	if((in_nhw < 33554432 && in_cstride > 1024) ||
   	((n >= 256) && (in_cstride > 60) && bfpmixparm) ||
   	((in_cstride > 512) && bfpmixparm))
   	{
   	variant = 1;
   	}
   	else if(in_cstride <= 512)
   	{
   	variant = 0;
   	}

Therefore, variant will never be 0 or 1 for forward_spatial_multiple.cpp

Please note the cases that do not pass IsApplicable() in forward_spatial_single.cpp will be handled in IsApplicable() in forward_spatial_multiple.cpp

MIOpen/src/solver/batchnorm/forward_spatial_multiple.cpp

Line 53 in 310192d

return !BnFwdTrainingSpatialSingle{}.IsApplicable(context, problem);

[CAHEK7]

Shall it be a new Full instance? Let's keep Smoke tests relatively small, that's the point.

[xinlipn]

Hi @CAHEK7 could you clarify what you mean by "Full" instance? For each data type, there's 4 sets of test data for NetworkSmall and 32 for NetworkLarge.

[bghimireamd]

This is not a new Full instance. Its just there was a typo before. It should be calling NetworkLarge for testing AP2. For AP1 we only test small bn network (4 test).

[CAHEK7]

I mean Full tests and Smoke tests - Smoke is a small and fast subset which checks only a basic functionality and must not have any long-running tests or large configs.
The main point of Smoke tests and a quick sanity check before the long runs.

That's why I'm kind of recommend to move those tests into the Full tests, since it uses large configs.

[bghimireamd]

This is not a new Full instance. Its just there was a typo before. It should be calling NetworkLarge for testing AP2. For AP1 we only test small bn network (4 test).