Question: So what about the neural network?

[ryanpeach]

So this seems to be an implementation of the tree as described in the paper, however, how then do we distill the knowledge from a neural network into the tree?

Thanks

[kimhc6028]

Hello,
are you asking that "how neural network formulates into tree, and how to train that?"
If so, a neural network outputs probability of choosing left or right node (which are also same neural network). Following this chain of probability, the last node outputs deterministic probability of class expectation.
When you back propagate this, probability links between nodes so that you can differentiate chain of nodes which are chosen.
Thank you.

[Airconaaron]

@kimhc6028 I think the issue @ryanpeach bought up was that the paper was mainly concerned about the distillation aspect.

This project currently trains a novel decision tree that uses a convolutional filter to make its decision. The labels you use are based off of the MNIST data rather than the outputs of a NN.

Happy to submit a PR next week to rectify this issue and add some argument parsing in a bit.

[jnclayiii]

If it's not too late, I would be interested in seeing such a PR.

[benediktwagner]

Would be interested too.

[lmartak]

@jnclayiii @benbetze if still relevant, see my recent implementation that includes distillation as well as visualization of learned parameters and follows the paper exactly in every detail: https://github.com/lmartak/distill-nn-tree