Deep Models

DCCA

class cca_zoo.deepmodels.dcca.DCCA(latent_dims, objective=<class 'cca_zoo.deepmodels.objectives.MCCA'>, encoders=None, r=0, eps=1e-05, **kwargs)[source]

A class used to fit a DCCA model.

Citation

Andrew, Galen, et al. “Deep canonical correlation analysis.” International conference on machine learning. PMLR, 2013.

Constructor class for DCCA

Parameters

  • latent_dims (int) – # latent dimensions

  • objective – # CCA objective: normal tracenorm CCA by default

  • encoders – list of encoder networks

  • r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller

  • eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

forward(*args, **kwargs)[source]

We use the forward model to define the transformation of views to the latent space

Parameters

args – batches for each view separated by commas

loss(*args)[source]

Define the loss function for the model. This is used by the DeepWrapper class

Parameters

args –

Returns

post_transform(z_list, train=False)[source]

Some models require a final linear CCA after model training.

Parameters

  • z_list – a list of all of the latent space embeddings for each view

  • train – if the train flag is True this fits a new post transformation

on_train_epoch_end(unused=None)[source]

Called in the training loop at the very end of the epoch.

To access all batch outputs at the end of the epoch, either:

  1. Implement training_epoch_end in the LightningModule OR

  2. Cache data across steps on the attribute(s) of the LightningModule and access them in this hook

on_validation_epoch_end(unused=None)[source]

Called in the validation loop at the very end of the epoch.

pairwise_correlations(loader, train=False)[source]
Parameters

  • loader (DataLoader) – a dataloader that matches the structure of that used for training

  • train – whether to fit final linear transformation

score(loader, train=False)[source]
Parameters

  • loader (DataLoader) – a dataloader that matches the structure of that used for training

  • train – whether to fit final linear transformation

Returns

by default returns the average pairwise correlation in each dimension (for 2 views just the correlation)

plot_latent_label(loader)[source]
training: bool

DCCA by Non-Linear Orthogonal Iterations

class cca_zoo.deepmodels.dcca_noi.DCCA_NOI(latent_dims, N, encoders=None, r=0, rho=0.2, eps=1e-09, shared_target=False, **kwargs)[source]

A class used to fit a DCCA model by non-linear orthogonal iterations

Citation

Wang, Weiran, et al. “Stochastic optimization for deep CCA via nonlinear orthogonal iterations.” 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2015.

Constructor class for DCCA

Parameters

  • latent_dims (int) – # latent dimensions

  • N (int) – # samples used to estimate covariance

  • encoders – list of encoder networks

  • r (float) – regularisation parameter of tracenorm CCA like ridge CCA

  • rho (float) – covariance memory like DCCA non-linear orthogonal iterations paper

  • eps (float) – epsilon used throughout

  • shared_target (bool) – not used

forward(*args, **kwargs)[source]

We use the forward model to define the transformation of views to the latent space

Parameters

args – batches for each view separated by commas

loss(*args)[source]

Define the loss function for the model. This is used by the DeepWrapper class

Parameters

args –

Returns

training: bool
use_amp: bool
precision: int
prepare_data_per_node: bool
allow_zero_length_dataloader_with_multiple_devices: bool

Deep Canonically Correlated Autoencoders

class cca_zoo.deepmodels.dccae.DCCAE(latent_dims, objective=<class 'cca_zoo.deepmodels.objectives.MCCA'>, encoders=None, decoders=None, r=0, eps=1e-05, lam=0.5, **kwargs)[source]

A class used to fit a DCCAE model.

Citation

Wang, Weiran, et al. “On deep multi-view representation learning.” International conference on machine learning. PMLR, 2015.

Parameters

  • latent_dims (int) – # latent dimensions

  • objective – # CCA objective: normal tracenorm CCA by default

  • encoders – list of encoder networks

  • decoders – list of decoder networks

  • r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller

  • eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

  • lam – weight of reconstruction loss (1 minus weight of correlation loss)

forward(*args, **kwargs)[source]

We use the forward model to define the transformation of views to the latent space

Parameters

args – batches for each view separated by commas

loss(*args)[source]

Define the loss function for the model. This is used by the DeepWrapper class

Parameters

args –

Returns

recon_loss(x, recon)[source]
training: bool
use_amp: bool
precision: int
prepare_data_per_node: bool
allow_zero_length_dataloader_with_multiple_devices: bool

Deep Tensor CCA

class cca_zoo.deepmodels.dtcca.DTCCA(latent_dims, encoders=None, r=0, eps=1e-05, **kwargs)[source]

A class used to fit a DTCCA model.

Is just a thin wrapper round DCCA with the DTCCA objective and a TCCA post-processing

Citation

Wong, Hok Shing, et al. “Deep Tensor CCA for Multi-view Learning.” IEEE Transactions on Big Data (2021).

Parameters

  • latent_dims (int) – # latent dimensions

  • encoders – list of encoder networks

  • r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller

  • eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

post_transform(z_list, train=False)[source]

Some models require a final linear CCA after model training.

Parameters

  • z_list – a list of all of the latent space embeddings for each view

  • train – if the train flag is True this fits a new post transformation

Return type

Iterable[ndarray]

training: bool
use_amp: bool
precision: int
prepare_data_per_node: bool
allow_zero_length_dataloader_with_multiple_devices: bool

Deep Variational CCA

class cca_zoo.deepmodels.dvcca.DVCCA(latent_dims, encoders=None, decoders=None, private_encoders=None, latent_dropout=0, log_images=True, img_dim=(1, 28, 28), **kwargs)[source]

A class used to fit a DVCCA model.

Citation

Wang, Weiran, et al. “Deep variational canonical correlation analysis.” arXiv preprint arXiv:1610.03454 (2016).

https: // arxiv.org / pdf / 1610.03454.pdf

https: // github.com / pytorch / examples / blob / master / vae / main.py

Parameters

  • latent_dims (int) – # latent dimensions

  • encoders – list of encoder networks

  • decoders – list of decoder networks

  • private_encoders (Optional[Iterable[BaseEncoder]]) – list of private (view specific) encoder networks

forward(*args, mle=True, **kwargs)[source]
Parameters

  • args –

  • mle –

Returns

loss(*args)[source]
Parameters

args –

Returns

static kl_loss(mu, logvar)[source]
recon_loss(x, recon)[source]
on_validation_epoch_end()[source]

Called in the validation loop at the very end of the epoch.

Return type

None

plot_latent_label(loader)[source]
training: bool

Split Autoencoders

class cca_zoo.deepmodels.splitae.SplitAE(latent_dims, encoder=<class 'cca_zoo.deepmodels.architectures.Encoder'>, decoders=None, latent_dropout=0, **kwargs)[source]

A class used to fit a Split Autoencoder model.

Citation

Ngiam, Jiquan, et al. “Multimodal deep learning.” ICML. 2011.

Parameters

  • latent_dims (int) – # latent dimensions

  • encoder (BaseEncoder) – list of encoder networks

  • decoders – list of decoder networks

forward(*args, **kwargs)[source]

We use the forward model to define the transformation of views to the latent space

Parameters

args – batches for each view separated by commas

loss(*args)[source]

Required when using the LightningTrainer

static recon_loss(x, recon)[source]
training: bool

Deep Objectives

class cca_zoo.deepmodels.objectives.CCA(latent_dims, r=0, eps=0.001)[source]

Differentiable CCA Loss. Loss() method takes the outputs of each view’s network and solves the CCA problem as in Andrew’s original paper

Parameters

  • latent_dims (int) – the number of latent dimensions

  • r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions

  • eps (float) – an epsilon parameter used in some operations

class cca_zoo.deepmodels.objectives.MCCA(latent_dims, r=0, eps=0.001)[source]

Differentiable MCCA Loss. Loss() method takes the outputs of each view’s network and solves the multiset eigenvalue problem as in e.g. https://arxiv.org/pdf/2005.11914.pdf

Parameters

  • latent_dims (int) – the number of latent dimensions

  • r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to

the number of latent dimensions :type eps: float :param eps: an epsilon parameter used in some operations

class cca_zoo.deepmodels.objectives.GCCA(latent_dims, r=0, eps=0.001)[source]

Differentiable GCCA Loss. Loss() method takes the outputs of each view’s network and solves the generalized CCA eigenproblem as in https://arxiv.org/pdf/2005.11914.pdf

Parameters

  • latent_dims (int) – the number of latent dimensions

  • r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to

the number of latent dimensions :type eps: float :param eps: an epsilon parameter used in some operations

class cca_zoo.deepmodels.objectives.TCCA(latent_dims, r=0, eps=0.0001)[source]

Differentiable TCCA Loss.

Parameters

  • latent_dims (int) – the number of latent dimensions

  • r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions

  • eps (float) – an epsilon parameter used in some operations

Model Architectures