Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Converter to tf.record improvements #118

Open
iislucas opened this issue Jul 1, 2018 · 1 comment
Open

Converter to tf.record improvements #118

iislucas opened this issue Jul 1, 2018 · 1 comment

Comments

@iislucas
Copy link
Contributor

iislucas commented Jul 1, 2018

  • The current CSV to tf.record converter has hard-wired field selectors. These should be specified on the command line.
  • CSV is a bad format: there are multiple usually incompatible and badly supported 'standards'. If people use CSV, print out a warning, that CSVs are fragile, and the user should consider a more robust format like jsonlines, or json.
  • Support jsonlines, and json input, and when we do that, maybe rename script appropriately.
  • Consider: make our dataset class natively support and convert examples inline instead of requiring pre-processing.
@iislucas iislucas self-assigned this Jul 2, 2018
@iislucas
Copy link
Contributor Author

jsonlines support landed in #181

ipavlopoulos pushed a commit to ipavlopoulos/conversationai-models that referenced this issue Mar 2, 2019
@vegetable68
Merge pull request conversationai#118 from conversationai/yiqing-viz
d283f22 
Yiqing viz
@iislucas iislucas removed their assignment Jun 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@iislucas