Speaker
Description
We study the effect of adding intra-layer connections in restricted Boltzmann machines (RBM), in the hidden layer, in the visible layer, or in both layers at the same time. The improvement obtained with these new connections is evaluated with the negative log-likelihood in the MNIST dataset. We have also implemented different ways to calculate the connection updates, some more precise (and more computationally expensive) than others. In all cases we have found improvements that, although considerable in some cases, are not an overwhelming change. We find that, on the light of these results, new training methods like RAPID, which we introduced in [1] and which is motivated by the control of the spin-glass properties of the Ising model, are very useful. We are currently testing RAPID and some variations of it on deeper Boltzmann machines architectures using the "mean field" theory of condensed matter physics
[1] A. Pozas-Kerstjens, G. Muñoz-Gil, E. Piñol, M. A. García-March, A. Acín, M. Lewenstein, and P. R. Grzybowsky, Mach. Learn.: Sci. Technol. 2, 025026 (2021).
