New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

咨询一下xxxConstraint里使用的dataloader_cfg在训练过程中resample事宜 #1095

Open

pecanjk opened this issue Mar 5, 2025 · 2 comments

pecanjk commented Mar 5, 2025

请提出你的问题 Please ask your question

请问一下，xxxConstraint里使用的dataloader_cfg里常用的IterableNamedArrayDataset, ContinuousNamedArrayDataset有什么区别，在训练过程中，他们会有resample的设置么，是训练开始前就只采样一次数据点以后都固定了，还是会每次训练都重新采样训练数据点。还有训练数据点batch_size设置和Iteration_per_epoch设置时是怎么分配训练数据点的。

如果想训练过程中不断的resample数据点，如何做，以及训练过程中不断的增加采样的数据点，能如何做，谢谢

Collaborator

HydrogenSulfate commented Mar 5, 2025

NamedArrayDataset是最常见的，用于大量固定点分batch训练，IterableNamedArrayDataset是用于固定点全量训练，ContinuousNamedArrayDataset接受的是一个数据和标签生成函数，用于随机点训练

Collaborator

HydrogenSulfate commented Mar 5, 2025

不断resample数据点，请参考：examples/allen_cahn/allen_cahn_piratenet.py，增加数据点的话你直接在gen_input_batch里添加你的策略，不断增加就行了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment