Code to reproduce the results of the paper "Expectation propagation on the diluted classifier" by Alfredo Braunstein, Thomas Gueudré, Andrea Pagnani and Mirko Pieropan [1]. This package uses the implementation of Gaussian EP available at [2].

Installation

After dowloading the repository, open Julia, change your current directory to that of the repository and type ] to enter the Pkg REPL. From there, run activate . and then instantiate in order to install all required dependencies. These instructions can be executed automatically at Julia startup by including the following lines of code in the ~/.julia/config/startup.jl file (create the ~/.julia/config folder and the file if not already present):

using Pkg 
if isfile("Project.toml") && isfile("Manifest.toml") 
    Pkg.activate(".") 
end

Usage

The routines needed to perform the EP inference can be found in SparsePerceptron.jl and in run_instances.jl. The generative models for the weights of the teacher perceptron and for the training set of patterns are implemented in generate_instances.jl.

Tests

Some simple tests can be run by including the following scripts in Julia from the Julia REPL:

• test_gaussian_iid_noiseless.jl: sparse perceptron learning from a set of correctly labeled i.i.d. Gaussian patterns;
• test_gaussian_cor_noiseless.jl: sparse perceptron training using a set of correctly labeled patterns sampled from a multivariate Gaussian distribution;
• test_noisy_case.jl: the script runs an example of student perceptron learning from a set of i.i.d. Gaussian patterns and another one from correlated Gaussian patterns. In both cases, a small fraction of the examples are mislabeled.
• test_corr_binary_patterns_noiseless.jl: sparse perceptron learning from a set of correctly labeled binary patterns generated by a network of perceptrons according to an asynchronous Glauber dynamics at zero temperature.

Each script runs the EP based inference on a randomly generated instance and prints the teacher-student overlap and the related mean squared error in dB (both as the estimation is performed and at the end of the inference procedure).

References

[1] Alfredo Braunstein, Thomas Gueudré, Andrea Pagnani, and Mirko Pieropan. Expectation propagation on the diluted Bayesian classifier. Phys. Rev. E 103 043301 (Apr. 2021).

[2] GaussianEP package: https://github.com/abraunst/GaussianEP