Caching improvements #3989
Caching improvements #3989
Conversation
* Disk cache generated code * Only generate loopy code on one rank and broadcast to others
|
|
…haviour of SPMD strict mode to raise an error when called non-collectively
Needs to be collective. Therefore the SPMD_STRICT partition does not make much sense.
firedrake/tsfc_interface.py
Outdated
| def _make_interface_key(interface): | ||
| if interface: | ||
| # Passing interface here is a small hack done in patch.py. What | ||
| # really matters for caching is what is used in the 'dont_split' kwarg. | ||
| assert isinstance(interface, functools.partial) | ||
| assert interface.func is KernelBuilder | ||
| return tuple(map(hash_expr, interface.keywords["dont_split"])) | ||
| else: | ||
| return None |
There was a problem hiding this comment.
Calling hash_expr element by element will not produce useful cache key here. If we want to hash dont_split, we should compute the indices of the elements of dont_split in (sub)form.coefficients().
There was a problem hiding this comment.
Sorry I don't really understand what you mean. Why isn't it useful? It might be overkill but I don't think that it is necessarily wrong.
There was a problem hiding this comment.
Given a form with form.coefficients() = (f, g, h, l), say, what distinguishes dont_split = (f,) and dont_split = (g,) is the index of f and g in (f, g, h, l); i.e. (0,) and (1,), so we can not get useful cache key by just looking at the individual coefficients.
For instance, if we run:
mesh = UnitIntervalMesh(1)
V = FunctionSpace(mesh, "CG", 1)
f = Function(V, count=1000)
g = Function(V, count=2000)
print(hash_expr(f))
print(hash_expr(g))
we get:
877b3c5df999b747710667a087dece19123b8045b21ba174c593b75c3c87ea1168b2fec9f8ce0634834ac3a3aea52b25ad9038474b9ef8319deb63b73f4de2d0
877b3c5df999b747710667a087dece19123b8045b21ba174c593b75c3c87ea1168b2fec9f8ce0634834ac3a3aea52b25ad9038474b9ef8319deb63b73f4de2d0
``
There was a problem hiding this comment.
Ah you're quite right. I will fix that tomorrow.
There was a problem hiding this comment.
I have fixed this.
| @@ -87,6 +88,7 @@ jobs: | |||
| --install defcon \ | |||
| --install gadopt \ | |||
| --install asQ \ | |||
| --package-branch fiat connorjward/fix-quadrature-hash \ | |||
There was a problem hiding this comment.
| --package-branch fiat connorjward/fix-quadrature-hash \ |
Co-authored-by: Josh Hope-Collins <[email protected]>
Needs firedrakeproject/fiat#132
PYOP2_SPMD_STRICTin CI, which includes some extra checksThanks to @pbrubeck for flagging this. Apparently these changes make a big difference.