Measurement/inference error (3): hidden_states

[Katehuuh]

(venv) C:\exllamav2>python convert.py -i "C:\Llama-3-8B-abliterated-v3-4Chan-simpo" -o "C:\Llama-3-8B-abliterated-v3-4Chan-simpo-8.0bpw-h8-exl2" -cf "C:\Llama-3-8B-abliterated-v3-4Chan-simpo-8.0bpw-h8-exl2" -b 8.0 -hb 8 --no_resume
 -- Beginning new job
 !! Warning: Output directory is not empty: C:\Llama-3-8B-abliterated-v3-4Chan-simpo-8.0bpw-h8-exl2
 !! Cleaning output directory: C:\Llama-3-8B-abliterated-v3-4Chan-simpo-8.0bpw-h8-exl2
 -- Input: C:\Llama-3-8B-abliterated-v3-4Chan-simpo
 -- Output: C:\Llama-3-8B-abliterated-v3-4Chan-simpo-8.0bpw-h8-exl2
 -- Using default calibration dataset
 -- Target bits per weight: 8.0 (decoder), 8 (head)
 -- Max shard size: 8192 MB
 -- Full model will be compiled to: C:\Llama-3-8B-abliterated-v3-4Chan-simpo-8.0bpw-h8-exl2
 -- Tokenizing samples (measurement)...
 -- Token embeddings (measurement)...
 -- Measuring quantization impact...
 -- Layer: model.layers.0 (Attention)
 -- model.layers.0.self_attn.q_proj                    0.05:3b_64g/0.95:2b_64g s4                         2.13 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:3b_64g/0.9:2b_64g s4                           2.18 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:4b_128g/0.9:3b_128g s4                         3.16 bpw
 -- model.layers.0.self_attn.q_proj                    1:4b_128g s4                                       4.04 bpw
 -- model.layers.0.self_attn.q_proj                    1:4b_64g s4                                        4.07 bpw
 -- model.layers.0.self_attn.q_proj                    1:4b_32g s4                                        4.13 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:5b_128g/0.9:4b_128g s4                         4.16 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:5b_64g/0.9:4b_64g s4                           4.18 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:5b_32g/0.9:4b_32g s4                           4.23 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:6b_128g/0.9:5b_128g s4                         5.16 bpw
 -- model.layers.0.self_attn.q_proj                    0.1:6b_32g/0.9:5b_32g s4                           5.23 bpw
 -- model.layers.0.self_attn.q_proj                    1:6b_128g s4                                       6.04 bpw
 -- model.layers.0.self_attn.q_proj                    1:6b_32g s4                                        6.13 bpw
 -- model.layers.0.self_attn.q_proj                    1:8b_128g s4                                       8.04 bpw
 -- model.layers.0.self_attn.k_proj                    0.05:3b_64g/0.95:2b_64g s4                         2.16 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:3b_64g/0.9:2b_64g s4                           2.20 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:4b_128g/0.9:3b_128g s4                         3.19 bpw
 -- model.layers.0.self_attn.k_proj                    1:4b_128g s4                                       4.06 bpw
 -- model.layers.0.self_attn.k_proj                    1:4b_64g s4                                        4.09 bpw
 -- model.layers.0.self_attn.k_proj                    1:4b_32g s4                                        4.16 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:5b_128g/0.9:4b_128g s4                         4.19 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:5b_64g/0.9:4b_64g s4                           4.20 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:5b_32g/0.9:4b_32g s4                           4.26 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:6b_128g/0.9:5b_128g s4                         5.19 bpw
 -- model.layers.0.self_attn.k_proj                    0.1:6b_32g/0.9:5b_32g s4                           5.26 bpw
 -- model.layers.0.self_attn.k_proj                    1:6b_128g s4                                       6.06 bpw
 -- model.layers.0.self_attn.k_proj                    1:6b_32g s4                                        6.16 bpw
 -- model.layers.0.self_attn.k_proj                    1:8b_128g s4                                       8.06 bpw
 -- model.layers.0.self_attn.v_proj                    0.05:3b_64g/0.95:2b_64g s4                         2.16 bpw
 -- model.layers.0.self_attn.v_proj                    0.25:3b_64g/0.75:2b_64g s4                         2.34 bpw
 -- model.layers.0.self_attn.v_proj                    0.1:4b_128g/0.9:3b_128g s4                         3.19 bpw
 -- model.layers.0.self_attn.v_proj                    0.1:4b_64g/0.9:3b_64g s4                           3.20 bpw
 -- model.layers.0.self_attn.v_proj                    1:4b_128g s4                                       4.06 bpw
 -- model.layers.0.self_attn.v_proj                    1:4b_64g s4                                        4.09 bpw
 -- model.layers.0.self_attn.v_proj                    1:4b_32g s4                                        4.16 bpw
 -- model.layers.0.self_attn.v_proj                    0.1:5b_64g/0.9:4b_64g s4                           4.20 bpw
 -- model.layers.0.self_attn.v_proj                    0.1:5b_32g/0.9:4b_32g s4                           4.26 bpw
 -- model.layers.0.self_attn.v_proj                    1:5b_64g s4                                        5.09 bpw
 -- model.layers.0.self_attn.v_proj                    1:5b_32g s4                                        5.16 bpw
 -- model.layers.0.self_attn.v_proj                    1:6b_128g s4                                       6.06 bpw
 -- model.layers.0.self_attn.v_proj                    1:6b_32g s4                                        6.16 bpw
 -- model.layers.0.self_attn.v_proj                    1:8b_32g s4                                        8.16 bpw
 -- model.layers.0.self_attn.v_proj                    1:8b_128g s4                                       8.06 bpw
 -- model.layers.0.self_attn.o_proj                    0.05:3b_64g/0.95:2b_64g s4                         2.13 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:3b_64g/0.9:2b_64g s4                           2.18 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:4b_128g/0.9:3b_128g s4                         3.16 bpw
 -- model.layers.0.self_attn.o_proj                    1:4b_128g s4                                       4.04 bpw
 -- model.layers.0.self_attn.o_proj                    1:4b_64g s4                                        4.07 bpw
 -- model.layers.0.self_attn.o_proj                    1:4b_32g s4                                        4.13 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:5b_128g/0.9:4b_128g s4                         4.16 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:5b_64g/0.9:4b_64g s4                           4.18 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:5b_32g/0.9:4b_32g s4                           4.23 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:6b_128g/0.9:5b_128g s4                         5.16 bpw
 -- model.layers.0.self_attn.o_proj                    0.1:6b_32g/0.9:5b_32g s4                           5.23 bpw
 -- model.layers.0.self_attn.o_proj                    1:6b_128g s4                                       6.04 bpw
 -- model.layers.0.self_attn.o_proj                    1:6b_32g s4                                        6.13 bpw
 -- model.layers.0.self_attn.o_proj                    1:8b_128g s4                                       8.04 bpw
 -- 2.1378 bpw  accuracy: 0.88402815
 -- 2.1987 bpw  accuracy: 0.89086039
 -- 2.2831 bpw  accuracy: 0.93022617
 -- 2.6768 bpw  accuracy: 0.93031679
 -- 3.1689 bpw  accuracy: 0.94701780
 -- 3.1705 bpw  accuracy: 0.95090065
 -- 4.0439 bpw  accuracy: 0.97438885
 -- 4.0471 bpw  accuracy: 0.97676111
 -- 4.0816 bpw  accuracy: 0.97956295
 -- 4.1381 bpw  accuracy: 0.97920668
 -- 4.1705 bpw  accuracy: 0.97990224
 -- 4.1902 bpw  accuracy: 0.98221093
 -- 4.2737 bpw  accuracy: 0.99013564
 -- 4.3295 bpw  accuracy: 0.99131829
 -- 5.2564 bpw  accuracy: 0.99549627
 -- 5.3295 bpw  accuracy: 0.99606395
 -- 6.0439 bpw  accuracy: 0.99567138
 -- 6.3381 bpw  accuracy: 0.99796946
 -- 8.0439 bpw  accuracy: 0.99922012
-------------------------------------------
| Measured: model.layers.0 (Attention)    |
| Duration: 6.87 seconds                  |
| Completed step: 1/67                    |
| Avg time / step (rolling): 6.87 seconds |
| Estimated remaining time: 7min 33sec    |
| Last checkpoint layer: None             |
-------------------------------------------
 -- Layer: model.layers.0 (MLP)
 ## Measurement/inference error (3): hidden_states

I've used base failspy/Meta-Llama-3-8B-Instruct-abliterated-v3, Any related arg such as --skip_missing_layers required?