Threshold in arcface loss

[mohamedamara7]

In arcface loss a certain threshlod is used: theta_valid = tf.where(y_pred_vals > self.threshold, theta, self.theta_min - theta). I didn't understand why we should use this threshold, how to determine its value and finally if we violet the threshold we set the value to 'self.theta_min - theta' based on what this value is chosen? Thanks in advance.

[leondgarse]

That threshold is actually added by me, and it just controls the adjusted y_pred_vals will not go beyond its original value. We can plot to see how arcface is modifying the y_pred_vals.

• Without the threshold check, that there will be some value on the tail go larger than its original value.

  xx = np.arange(-1, 1, 0.01)
  
  for margin in np.arange(0.1, 1.5, 0.2):
      margin_cos, margin_sin = tf.cos(margin), tf.sin(margin)
      threshold = tf.cos(np.pi - margin)
      theta_min = -2
  
      theta = xx * margin_cos - tf.sqrt(1 - tf.pow(xx, 2)) * margin_sin
      # theta = tf.where(xx > threshold, theta, theta_min - theta)
      plt.plot(xx, theta, label="margin: {}".format(margin))
  plt.grid(True)
  plt.legend()

  
• With the threshold check, and its just a simple adjust on the value, that turning the up down. The threshold value is actually where the line gradiend == 0.

  xx = np.arange(-1, 1, 0.01)
  
  for margin in np.arange(0.1, 1.5, 0.2):
      margin_cos, margin_sin = tf.cos(margin), tf.sin(margin)
      threshold = tf.cos(np.pi - margin)
      theta_min = -2
  
      theta = xx * margin_cos - tf.sqrt(1 - tf.pow(xx, 2)) * margin_sin
      theta = tf.where(xx > threshold, theta, theta_min - theta)
      plt.plot(xx, theta, label="margin: {}".format(margin))
  plt.grid(True)
  plt.legend()

  
• Anyway, it's not an important change, as mostly the value is too small even it truns up. You can also modify above plotting function to check your thoughts.

[mohamedamara7]

I really appreciate your effort but when I calculate cos(theta+margin) manually, I found that the first plot is correct and I think mathematically, the second plot is wrong as the cos of any angle should be within [-1, 1] but it exceeds -1.

[leondgarse]

Ya, if feed back into cos, it's not correct, Just this value here will multiply with scale and then go into categorical_crossentropy with from_logits=self.from_logits in next step, the value range doesn't matter. You can also use something like theta_valid = tf.where(y_pred_vals > self.threshold, theta, -1) or just comment off it.