On The Discriminability of Samples Using Binarized ReLU Activations

Michal Lewandowski; Werner Zellinger; Hamid Eghbal-Zadeh; Natalia Shepeleva; Bernhard A. Moser

On The Discriminability of Samples Using Binarized ReLU Activations

Michal Lewandowski
, Werner Zellinger
, Hamid Eghbal-Zadeh
, Natalia Shepeleva
, Bernhard A. Moser

Institute of Computational Perception

Research output: Chapter in Book/Report/Conference proceeding › Conference proceedings › peer-review

Abstract

Binarized ReLU activations are considered as a metric space equipped with the Hamming distance. While for two-layer ReLU networks with random Gaussian weights it can be shown theoretically that local metric properties are approximately preserved, we experimentally study the discrimination capability in this Hamming space for deeper ReLU networks and look also at the non-local behaviour. It turns out that the discrimination capability is approximately preserved as expected.

Original language	German (Austria)
Title of host publication	Proceedings of the 3rd International Conference on Data Science, Machine Learning and Applications (ICDSMLA 2021), 2021
Number of pages	8
Publication status	Published - 2021

Fields of science

202002 Audiovisual media
102 Computer Sciences
102001 Artificial intelligence
102003 Image processing
102015 Information systems

JKU Focus areas

Digital Transformation

Cite this

@inproceedings{9b0c92cab198459ca0e2e931d54e80f8,

title = "On The Discriminability of Samples Using Binarized ReLU Activations",

abstract = "Binarized ReLU activations are considered as a metric space equipped with the Hamming distance. While for two-layer ReLU networks with random Gaussian weights it can be shown theoretically that local metric properties are approximately preserved, we experimentally study the discrimination capability in this Hamming space for deeper ReLU networks and look also at the non-local behaviour. It turns out that the discrimination capability is approximately preserved as expected.",

author = "Michal Lewandowski and Werner Zellinger and Hamid Eghbal-Zadeh and Natalia Shepeleva and Moser, \{Bernhard A.\}",

year = "2021",

language = "Deutsch ({\"O}sterreich)",

booktitle = "Proceedings of the 3rd International Conference on Data Science, Machine Learning and Applications (ICDSMLA 2021), 2021",

}

TY - GEN

T1 - On The Discriminability of Samples Using Binarized ReLU Activations

AU - Lewandowski, Michal

AU - Zellinger, Werner

AU - Eghbal-Zadeh, Hamid

AU - Shepeleva, Natalia

AU - Moser, Bernhard A.

PY - 2021

Y1 - 2021

N2 - Binarized ReLU activations are considered as a metric space equipped with the Hamming distance. While for two-layer ReLU networks with random Gaussian weights it can be shown theoretically that local metric properties are approximately preserved, we experimentally study the discrimination capability in this Hamming space for deeper ReLU networks and look also at the non-local behaviour. It turns out that the discrimination capability is approximately preserved as expected.

AB - Binarized ReLU activations are considered as a metric space equipped with the Hamming distance. While for two-layer ReLU networks with random Gaussian weights it can be shown theoretically that local metric properties are approximately preserved, we experimentally study the discrimination capability in this Hamming space for deeper ReLU networks and look also at the non-local behaviour. It turns out that the discrimination capability is approximately preserved as expected.

M3 - Konferenzbeitrag

BT - Proceedings of the 3rd International Conference on Data Science, Machine Learning and Applications (ICDSMLA 2021), 2021

ER -