Evaluation code of Table 3

[ofzlo]

I attempted to reproduce the T2I-CompBench scores from Table 3 in your paper, but observed discrepancies in 'shape' and 'texture'. Could you share your evaluation code or inform if there's a plan to release it? If code sharing is challenging, information about these image generation settings would be helpful: 1) random seed, 2) inference steps, 3) dataset used, and 4) image generation code, etc.

[Gus-Guo]

I attempted to reproduce the T2I-CompBench scores from Table 3 in your paper, but observed discrepancies in 'shape' and 'texture'. Could you share your evaluation code or inform if there's a plan to release it? If code sharing is challenging, information about these image generation settings would be helpful: 1) random seed, 2) inference steps, 3) dataset used, and 4) image generation code, etc.

@ofzlo hi,except the discrepancies of 'shape' and 'texture', did you reproduce results of other attributes? e.g. color, I use 512x512 ckpt and only got nearly 0.40 on color.