You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Perform initial experiments with the contextual log line embeddings.
Our current embedding is based on aggregating (averaging) of per-token fastText embeddings. Contextual embeddings are expected to improve the performance of the downstream task similarly to NLP.
MLM(only content) - eval_loss: 0.7880746566936903 Masked Language Modeling
CLM(only content) - eval_loss: 3.7285281655768676e-07 Next word prediction(Causal Language Modeling)
Perform initial experiments with the contextual log line embeddings.
Our current embedding is based on aggregating (averaging) of per-token fastText embeddings. Contextual embeddings are expected to improve the performance of the downstream task similarly to NLP.
The text was updated successfully, but these errors were encountered: