Estimated limitations of the Decision tree method (in a sort of best case scenario, where neural net is relatively small/piece wise):
1. Reasonably strictly applicable with exact solutions to non-linear functions that are piecewise
2. For non toy-problems, the vapnik chervonenkis dimension for these trees can apparently become extremely gigantic.
So, given a relatively small piecewise based neural network, an equivalent tree of depth roughly 16, one would need to plot a tree at 2^n=16 nodes!!!! 😧😧
⚫Ironically, apparently reasoning becomes quite difficult, for equivalent decision trees wrt relatively small neural networks.
3. Unlike neural networks decision trees may require optimal statistics, where as neural networks have been demonstrated to make progress in cases where input statistics are not optimal.
Intriguing albeit, can reasonably help to understand relatively small neural networks, for apparently simple/fixed input slices like tabular data, under piece-wise constraints
