We empirically investigate the effect of class manifold entanglement and the intrinsic and extrinsic dimensionality of the data distribution on the sample complexity of supervised classification with deep ReLU networks. We separate the effect of entanglement and intrinsic dimensionality and show statistically for artificial and real-world image datasets that the intrinsic dimensionality and the entanglement have an interdependent effect on the sample complexity. Low levels of entanglement lead to low increases of the sample complexity when the intrinsic dimensionality is increased, while for high levels of entanglement the impact of the intrinsic dimensionality increases as well. Further, we show that in general the sample complexity is primarily due to the entanglement and only secondarily due to the intrinsic dimensionality of the data distribution.
|Title of host publication||Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence|
|Publication status||Accepted/In press - 1 Dec 2021|
|Event||36th AAAI Conference on Artificial Intelligence 2022 - Vancouver, Canada|
Duration: 22 Feb 2022 → 1 Mar 2022
|Conference||36th AAAI Conference on Artificial Intelligence 2022|
|Period||22/02/22 → 1/03/22|