You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For large, full-results (not just summaries) storing the evaluation results in the CR's status is not practical.
An option should be available to persist full evaluation results to PVC.
The text was updated successfully, but these errors were encountered:
Here is my thought on this feature, suggestions/comments are welcome
provide an option to save the outputs, including stdout, stderr, results, and sampling log, to external storage
for external storage, we can support different types of storage, i.e. PVC, COS, etc. Let's add the PVC for now.
For the argument's sake, the new option is named Outputs and users can specify a full data struct of the PersistentVolumeClaim . LM-Eval controller would create the PVC and mount it on the /opt/app-root/src/output and all the outputs would store to the PVC. The data struct for the Outputs could be:
One thing worth mentioning is If users want to create a PVC by themselves and then use it in an LM-Eval job, they can do that already. Here is an example and the name of the PVC that has been created is mypvc:
If we want to make the case of using an existing PVC easier, the Outputs data struct could be:
typeOutputsstruct {
// Use an existing PVC to store the outputs// +optionalPersistentVolumeClaimName*string`json:"pvcName,omitempty"`// Create a PVC and use it to store the outputs// +optionalPersistentVolumeClaim*corev1.PersistentVolumeClaim`json:"pvc,omitempty"`
}
When the pvcName is specified, the LM-Eval controller will create the corresponding volumes and volumeMOunts for the job pod and store the outputs in the PVC. Basically, it makes the LMEvalJob example in previous comment become this:
For large, full-results (not just summaries) storing the evaluation results in the CR's status is not practical.
An option should be available to persist full evaluation results to PVC.
The text was updated successfully, but these errors were encountered: