Cumulative wells implementation

[OliverOverend]

Hi Ben,

Your implementation of cumulative wells sums the depths of all wells.

gym-mdptetris/gym_mdptetris/envs/feature_functions.py

Lines 99 to 145 in 1a47edc

	def get_well_sums(board: board.Board) -> float:
	"""
	Feature 6 in the Dellacherie set. Cumulatively sum well cells.
	A well cell is an empty cell where the left and right cell are both full,
	and the cell above it is empty.
	For example:
	|..........|
	|..........|
	|..........|
	|..........|
	|...X......|
	|...X.XX...|
	|..XX.X....|
	|XXXX.XXXXX|
	|X.X.XXX.X.|
	|X.X.XXX.X.|
	This board has 7 well cells as indicated below with 'o'
	|..........|
	|..........|
	|..........|
	|..........|
	|...X......|
	|...XoXX...|
	|..XXoX....|
	|XXXXoXXXXX|
	|X.X.XXX.X.|
	|XoXoXXXoXo|
	This is calculated using the `np.roll()` function to shift the array
	one column left, one column right, and one row down. The result of
	these are then AND-ed with the complement of the cell to indicate
	if the cell is a well, before summing.
	To indicate that board walls count for wells extra columns are added either
	side, and a row of false is added at the bottom to indicate well cells
	at the top row count.
	:param board: The current board state
	:return: The sum total of well cells on the board.
	"""
	temp = np.ones((board.height + 1, board.width + 2), dtype='bool')
	temp[:1] = False
	temp[1:,1:-1] = board.board[:board.height,]
	return (np.roll(temp, 1, axis=1) & np.roll(temp, -1, axis=1) & ~temp & ~np.roll(temp, -1, axis=0)).sum()

However, I reckon that you should sum i from i=1 to d(w) for all wells w, where d(w) is the depth of w, then add the sums. You can find a more precise definition here.

I believe that my implementation is correct, although there is probably a faster way using NumPy, which you could help me out with here?

Olly :)

[Benjscho]

Hi Olly,

I believe you're right! My version will need a reimplementation to account for the cumulative depth summation.

I'll look into a reimplementation of this once the dissertation marking has passed.

[OliverOverend]

Excellent!

As a result, I expect your agent's mean performance to increase.

[OliverOverend]

Hi Ben,

Apologies if this comes across as critical.

I was looking at your implementation for inspiration and realised that it's incorrectly treating some of the squares as part of a well when they should not be — "A square that is part of a well is an empty square that can be reached from above with full squares on the sides." (Link).

I wonder if a solution could be obtained by excluding the squares that are below the highest full squares in their respective columns and using numpy.cumsum to account for the cumulative depth summation.

[OliverOverend]

I have written a solution and the tests in my test suite still pass. Feel free to utilise/critique it.

Apologies that I haven't created a PR — I wasn't able to set up a copy of your codebase for testing.