Andras, PE (2018) High-Dimensional Function Approximation with Neural Networks for Large Volumes of Data. IEEE Transactions on Neural Networks and Learning Systems, 29 (2). pp. 500-508. ISSN 2162-237X

[thumbnail of TNNLS-2016-P-6183 - Final.pdf]
TNNLS-2016-P-6183 - Final.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (762kB) | Preview


Approximation of high-dimensional functions is a challenge for neural networks due to the curse of dimensionality. Often the data for which the approximated function is defined resides on a low-dimensional manifold and in principle the approximation of the function over this manifold should improve the approximation performance. It has been show that projecting the data manifold into a lower dimensional space, followed by the neural network approximation of the function over this space, provides a more precise approximation of the function than the approximation of the function with neural networks in the original data space. However, if the data volume is very large, the projection into the low-dimensional space has to be based on a limited sample of the data. Here, we investigate the nature of the approximation error of neural networks trained over the projection space. We show that such neural networks should have better approximation performance than neural networks trained on high-dimensional data even if the projection is based on a relatively sparse sample of the data manifold. We also find that it is preferable to use a uniformly distributed sparse sample of the data for the purpose of the generation of the low-dimensional projection. We illustrate these results considering the practical neural network approximation of a set of functions defined on high-dimensional data including real world data as well.

Item Type: Article
Additional Information: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Uncontrolled Keywords: Big data, function approximation, high-dimensional data, manifold mapping, neural networks
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Natural Sciences > School of Computing and Mathematics
Depositing User: Symplectic
Date Deposited: 11 Jan 2017 11:59
Last Modified: 09 Apr 2021 10:21

Actions (login required)

View Item
View Item