src: gpu: intel: jit: conv: add reorder-based precomputed zero points

[hidefromkgb]

Description

This is a novel approach to calculating zero points ahead of time, appending the precalc results to the convolutions weights buffers when they are either uploaded to the GPU or reordered on the GPU to fit the layout requested by the conv primitive.

Perf results on YOLO-v8n:

	FP16, ms/iter	INT8+ZP, ms/iter	INT8 no ZP, ms/iter
ATSM/DG2	3.3633792	2.6763831	2.4270376
BMG	1.46378	1.42166	1.2961

Addresses MFDNN-12548.

General

• Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
• Have you formatted the code using clang-format?

[dzarukin]

What if we don't compute this partial summation on the library side but instead provide an API to request compensations and accept them as a per-channel array?

[hidefromkgb]

What if we don't compute this partial summation on the library side but instead provide an API to request compensations and accept them as a per-channel array?

An important remark: this is not just a per-channel array but a full-fledged hidden buffer with DHW spatials.
Still, it's definitely possible, in theory, to expose ZP precompute as a part of oneDNN API — but maybe not right now.

In fact, this would be more appropriate from the architectural standpoint to expose rather than hide, as the current approach is all about tackling a graph-level problem with primitive-level tools.
Furthermore it'd lift the scalar-ZP-only requirement — a serious limitation the proposed PR has.

On the downside, though, the users would then become responsible for allocating network nodes to handle the ahead-of-time ZP precalculation.

@karturov @dzarukin What do you think of this?

[echeresh]

In fact, this would be more appropriate from the architectural standpoint to expose rather than hide, as the current approach is all about tackling a graph-level problem with primitive-level tools.
Furthermore it'd lift the scalar-ZP-only requirement — a serious limitation the proposed PR has.

On the downside, though, the users would then become responsible for allocating network nodes to handle the ahead-of-time ZP precalculation.

@hidefromkgb What do you mean exactly by "expose"? Exposing the structure of the memory_extra_desc_t and its contents? If so, what is the potential benefit?

I assume we would still provide some API (as e.g. reorder) to compute a compensation buffer in this form. It can't be easily/efficiently pre-computed by the user as it's not really trivial - so we have to expose it from the library in some form, say now it's done under reorder. Then I would say - it's not really important for the users to have any details on the buffer contents as the flow is 1) allocate an extra buffer 2) call reorder (or whatever) primitive 3) pass this buffer to convolution. There is no need to expose more details.

Furthermore it'd lift the scalar-ZP-only requirement — a serious limitation the proposed PR has.

This can be implemented under the current design, right? The user can pass vector zero-points as an additional reorder argument - then the resulting compensation buffer would be computed accordingly. This should require only minor API changes at worst - something like an extra attribute for the convolution for the user to pass to opt in for this feature.

[hidefromkgb]

make test
disable benchdnn_all
enable benchdnn_nightly
enable benchdnn_conv
enable benchdnn_deconv
enable benchdnn_pool
enable benchdnn_reorder
enable benchdnn_rnn

[hidefromkgb]

make test perf-gpu
set primitive=conv