Predictions tend to zero

[AlexMili]

Hi,

I tested your script with default parameters and no specific configuration but the predicted scanpaths all tend to 0.

Here is the generated file for 1 observer:

Idx, longitude, latitude, start timestamp
1, 0.34878289699554443, 0.9589308500289917, 0.0,
2, 0.29669225215911865, 0.9125264286994934, 247.83013916015625,
3, 0.2256612777709961, 0.8126437067985535, 495.3912048339844,
4, 0.144346684217453, 0.6298840641975403, 742.7520141601562,
5, 0.07867268472909927, 0.38607722520828247, 989.9747924804688,
6, 0.03853226825594902, 0.18494655191898346, 1237.1138916015625,
7, 0.01780589111149311, 0.07793145626783371, 1484.2125244140625,
8, 0.00803335476666689, 0.03312600776553154, 1731.3065185546875,
9, 0.003687845775857568, 0.015189948491752148, 1978.4261474609375,
10, 0.0017942111007869244, 0.007524003740400076, 2225.59765625,
11, 0.0009365906589664519, 0.003964598756283522, 2472.840576171875,
12, 0.0005228217341937125, 0.0022032433189451694, 2720.16796875,
13, 0.00031133301672525704, 0.001299730152823031, 2967.58544921875,
14, 0.00019891020201612264, 0.0008188027422875166, 3215.09423828125,
15, 0.0001359538728138432, 0.0005491344491019845, 3462.68994140625,
16, 9.847166802501306e-05, 0.0003882487362716347, 3710.36669921875,
17, 7.488742267014459e-05, 0.0002860023523680866, 3958.119873046875,
18, 5.927786696702242e-05, 0.00021779075905214995, 4205.9453125,
19, 4.824537973036058e-05, 0.0001705095055513084, 4453.83837890625,
20, 4.0030274249147624e-05, 0.00013643488637171686, 4701.7958984375,
21, 3.3662123314570636e-05, 0.00011114001972600818, 4949.8134765625,
22, 2.8576745535247028e-05, 9.18341102078557e-05, 5197.88818359375,
23, 2.4390950784436427e-05, 7.665887824259698e-05, 5446.0166015625,
24, 2.0838222553720698e-05, 6.43888270133175e-05, 5694.19677734375,
25, 1.7792901417124085e-05, 5.4311683925334364e-05, 5942.42578125,
26, 1.5233459635055624e-05, 4.610263567883521e-05, 6190.7001953125,
27, 1.3146351193427108e-05, 3.955434658564627e-05, 6439.0166015625,
28, 1.1480399734864477e-05, 3.4381653676973656e-05, 6687.37109375,
29, 1.0147859939024784e-05, 3.0248620532802306e-05, 6935.76171875,
30, 9.053159374161623e-06, 2.6849664209294133e-05, 7184.185546875,
31, 8.118025107251015e-06, 2.393318573012948e-05, 7432.642578125,
32, 7.294305760296993e-06, 2.1363421183195896e-05, 7681.1328125,
33, 6.554642368428176e-06, 1.906968464027159e-05, 7929.65673828125,
34, 5.892342869628919e-06, 1.7035014025168493e-05, 8178.21484375,
35, 5.310011147230398e-06, 1.5266794434865005e-05, 8426.80859375,
36, 4.8133933887584135e-06, 1.3778957509202883e-05, 8675.435546875,
37, 4.4048711060895585e-06, 1.257122858078219e-05, 8924.09375,
38, 4.07878951591556e-06, 1.1619880751823075e-05, 9172.7802734375,
39, 3.821476639132015e-06, 1.0880620720854495e-05, 9421.4912109375,
40, 3.617363745433977e-06, 1.0304410352546256e-05, 9670.2236328125,
41, 3.4524412058090093e-06, 9.847855835687369e-06, 9918.974609375,
42, 3.3158203223138116e-06, 9.479034815740306e-06, 10167.7421875,
43, 3.200215132892481e-06, 9.172943464363925e-06, 10416.5244140625,
44, 3.101444463027292e-06, 8.911775694286916e-06, 10665.3203125,
45, 3.0155501917761285e-06, 8.686445653438568e-06, 10914.1279296875,
46, 2.940721515187761e-06, 8.490957043250091e-06, 11162.9482421875,
47, 2.8757906420651125e-06, 8.32066416478483e-06, 11411.7783203125,
48, 2.8200927317811875e-06, 8.172157322405837e-06, 11660.6171875,
49, 2.7725955078494735e-06, 8.041871296882164e-06, 11909.462890625,
50, 2.731704398684087e-06, 7.926501893962268e-06, 12158.3154296875,
51, 2.6963793970935512e-06, 7.821890903869644e-06, 12407.1708984375,
52, 2.6656191494112136e-06, 7.724168426648248e-06, 12656.0283203125,
53, 2.638428213685984e-06, 7.630750587850343e-06, 12904.88671875,
54, 2.614998948047287e-06, 7.539902071584947e-06, 13153.744140625,
55, 2.5955225737561705e-06, 7.449644272128353e-06, 13402.6005859375,
56, 2.5807157726376317e-06, 7.3622218224045355e-06, 13651.453125,
57, 2.5714023195178015e-06, 7.272059519891627e-06, 13900.302734375,
58, 2.569549224062939e-06, 7.173614903877024e-06, 14149.1494140625,
59, 2.576446831881185e-06, 7.05991396898753e-06, 14397.994140625,
60, 2.592764076325693e-06, 6.920993655512575e-06, 14646.8369140625,
61, 2.618595090098097e-06, 6.753362868039403e-06, 14895.6787109375,
62, 2.6515297122386983e-06, 6.568572644027881e-06, 15144.5205078125,
63, 2.6875611638388364e-06, 6.394213869498344e-06, 15393.365234375,

I am using:

Keras: 2.2.4
Tensorflow: 1.12.0

Do you have any idea what could possibly go wrong ?

[Cogito2012]

@AlexMili The output locations of fixations are normalized to [0, 1], as stated in the paper. However, the authors @massens neither gave the training codes nor provided the normalization parameters in the paper. maybe there's no way to decipher the predicted results?

[Tianlong-Chen]

@AlexMili @Cogito2012
Yes, I meet the same problem as yours.

Could the authors share the training codes or fix this issue with a more reliable pretrained model file? @massens

Thank you so much!

[LvJC]

@AlexMili @Cogito2012
Yes, I meet the same problem as yours.

Could the authors share the training codes or fix this issue with a more reliable pretrained model file? @massens

Thank you so much!

OMG...I thought I was wrong... This pretrained model could not even output a reasonable scanpath for its own training image. What's wrong with this project???

[sendjasni]

Hi @AlexMili,
Did you figure it out? I'm trying to use this model as part of my project and I'm also facing the same issue as you.

[AlexMili]

Hello @sendjasni, unfortunately I gave up. The only way I see this is to try to retrain the model from scratch since the issue is open for more than one year now.

[sendjasni]

Hi @AlexMili
Thanks for the reply. Do you have an alternative, I really need a scan-path predictor that works on 360° images/videos?

[AlexMili]

I suggest you to check the challenge this model was proposed for: Salient360. There are multiple models for video and images but I don't know if any is available on Github.

[sendjasni]

Thanks @AlexMili
I've checked them and I don't think there are any available.

[saitejamalyala]

@AlexMili @sendjasni @Cogito2012 @Tianlong-Chen @LvJC @massens @amaiasalvador @xavigiro
Do you have the iSUN dataset that they have used to train the model initially ?? Also, how is EOS value added to ground truth fixations?

Trying to retrain from scratch, inputs will be appreciated. Thanks in advance

[sendjasni]

@AlexMili @sendjasni @Cogito2012 @Tianlong-Chen @LvJC @massens @amaiasalvador @xavigiro
Do you have the iSUN dataset that they have used to train the model initially ?? Also, how is EOS value added to ground truth fixations?

Trying to retrain from scratch, inputs will be appreciated. Thanks in advance

Try this :
https://salient360.ls2n.fr/datasets/