Entropy learning and relevance criteria for neural network pruning

Entropy learning and relevance criteria for neural network pruning

In this paper, entropy is a term used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence an entropy approach is proposed to dampen the early creation of such nodes by using...

Full description

Bibliographic Details
Main Authors: Geok, See Ng, Abdul Rahman, Abdul Wahab, Shi, Daming
Format: Article
Language:English
Published: World Scientific Publishing Company 2003
Subjects:
T Technology (General)
Online Access:http://irep.iium.edu.my/38198/
http://irep.iium.edu.my/38198/
http://irep.iium.edu.my/38198/1/Entropy_learning_and_relevance_criteria_for_neural_network_pruning.pdf
id iium-38198
recordtype eprints
spelling iium-381982014-09-12T01:37:35Z http://irep.iium.edu.my/38198/ Entropy learning and relevance criteria for neural network pruning Geok, See Ng Abdul Rahman, Abdul Wahab Shi, Daming T Technology (General) In this paper, entropy is a term used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence an entropy approach is proposed to dampen the early creation of such nodes by using a new computation called entropy cycle. Entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be pruned to reduce the memory requirements of the neural network. World Scientific Publishing Company 2003-10 Article PeerReviewed application/pdf en http://irep.iium.edu.my/38198/1/Entropy_learning_and_relevance_criteria_for_neural_network_pruning.pdf Geok, See Ng and Abdul Rahman, Abdul Wahab and Shi, Daming (2003) Entropy learning and relevance criteria for neural network pruning. Internation Journal of Neural Systems, 13 (5). pp. 291-305. ISSN 0129-0657 (P), 1793-6462 (O) http://www.worldscientific.com/doi/abs/10.1142/S0129065703001637
repository_type Digital Repository
institution_category Local University
institution International Islamic University Malaysia
building IIUM Repository
collection Online Access
language English
topic T Technology (General)
spellingShingle T Technology (General)
Geok, See Ng
Abdul Rahman, Abdul Wahab
Shi, Daming
Entropy learning and relevance criteria for neural network pruning
description In this paper, entropy is a term used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence an entropy approach is proposed to dampen the early creation of such nodes by using a new computation called entropy cycle. Entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be pruned to reduce the memory requirements of the neural network.
format Article
author Geok, See Ng
Abdul Rahman, Abdul Wahab
Shi, Daming
author_facet Geok, See Ng
Abdul Rahman, Abdul Wahab
Shi, Daming
author_sort Geok, See Ng
title Entropy learning and relevance criteria for neural network pruning
title_short Entropy learning and relevance criteria for neural network pruning
title_full Entropy learning and relevance criteria for neural network pruning
title_fullStr Entropy learning and relevance criteria for neural network pruning
title_full_unstemmed Entropy learning and relevance criteria for neural network pruning
title_sort entropy learning and relevance criteria for neural network pruning
publisher World Scientific Publishing Company
publishDate 2003
url http://irep.iium.edu.my/38198/
http://irep.iium.edu.my/38198/
http://irep.iium.edu.my/38198/1/Entropy_learning_and_relevance_criteria_for_neural_network_pruning.pdf
first_indexed 2023-09-18T20:54:51Z
last_indexed 2023-09-18T20:54:51Z
_version_ 1777410235434008576

Similar Items