Variance-Based Feature Importance in Neural Networks

Cláudio Rebelo de Sá*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

18 Citations (Scopus)
13 Downloads (Pure)


This paper proposes a new method to measure the relative importance of features in Artificial Neural Networks (ANN) models. Its underlying principle assumes that the more important a feature is, the more the weights, connected to the respective input neuron, will change during the training of the model. To capture this behavior, a running variance of every weight connected to the input layer is measured during training. For that, an adaptation of Welford’s online algorithm for computing the online variance is proposed. When the training is finished, for each input, the variances of the weights are combined with the final weights to obtain the measure of relative importance for each feature. This method was tested with shallow and deep neural network architectures on several well-known classification and regression problems. The results obtained confirm that this approach is making meaningful measurements. Moreover, results showed that the importance scores are highly correlated with the variable importance method from Random Forests (RF).

Original languageEnglish
Title of host publicationDiscovery Science
Subtitle of host publication22nd International Conference, DS 2019, Split, Croatia, October 28-30, 2019. Proceedings
EditorsPetra Kralj Novak, Sašo Džeroski, Tomislav Šmuc
Place of PublicationCham
Number of pages10
ISBN (Electronic)978-3-030-33778-0
ISBN (Print)978-3-030-33777-3
Publication statusPublished - 16 Oct 2019
Event22nd International Conference on Discovery Science, DS 2019 - Radisson Blu Resort and Spa, Split, Croatia
Duration: 28 Oct 201930 Oct 2019

Publication series

NameLecture Notes in Artificial Intelligence; subseries of Lecture Notes in Computer Science
Volume11828 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference22nd International Conference on Discovery Science, DS 2019
Abbreviated titleDS2019
Internet address


Dive into the research topics of 'Variance-Based Feature Importance in Neural Networks'. Together they form a unique fingerprint.

Cite this