Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Improvments to amount of data displayed by minari list remote and minari list local cli commands #100

Open
1 task
balisujohn opened this issue Jun 26, 2023 · 3 comments

Comments

@balisujohn
Copy link
Collaborator

balisujohn commented Jun 26, 2023

Proposal

Add gymasium env id, file size and dataset group to the displayed table when running command minari list remote or minari list local.

image

So this would mean adding new columns named something like "env_id", "size on disk" and something like "dataset group."

Right now, the datasets do not have a dataset_group value, so for backwards compatibility, the PR should check for a dataset_group attribute, and if there is none, it should use the string "Unknown" as a placeholder value.

This should be a useful hint for getting started with getting the file size for remote datasets: https://stackoverflow.com/questions/50875461/google-cloud-storage-get-object-size-api

To get started with this, it would be useful to look at the code in cli.py local.py and hosting.py

The doc will also need to be updated to reflect the existence of the new field dataset_group. Definitely on this page, https://minari.farama.org/main/content/dataset_standards/, and probably also on the individual dataset pages.

Motivation

This is partially to address #79, and also it's useful to know how large each dataset is to get an idea of how long it will take to download or process a particular dataset.

Checklist

  • I have checked that there is no similar issue in the repo (required)
@balisujohn
Copy link
Collaborator Author

@Aaisha-Rani Let me know if this task sounds good, and if so, I will assign it to you. Feel free to ask questions in the minari channel on the discord if you have any.

@Aaisha-Rani
Copy link

@balisujohn Thank you very much. i will do my best.

@Aaisha-Rani
Copy link

@balisujohn . I have done the additions and have also fixed one bug in point_maze_dataset.py. let me know of any mistakes. And thank you for your help and guidance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants