Deep Models

DCCA

class cca_zoo.deepmodels._dcca.DCCA(latent_dims, objective=<class 'cca_zoo.deepmodels._objectives.MCCA'>, encoders=None, r=0, eps=1e-05, **kwargs)[source]

A class used to fit a DCCA model.

Citation

Andrew, Galen, et al. “Deep canonical correlation analysis.” International conference on machine learning. PMLR, 2013.

Constructor class for DCCA

Parameters

latent_dims (int) – # latent dimensions
objective – # CCA objective: normal tracenorm CCA by default
encoders – list of encoder networks
r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller
eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

forward(views, **kwargs)[source]: We use the forward model to define the transformation of views to the latent space :param views: batches for each view separated by commas

loss(views, **kwargs)[source]

Define the loss function for the model. This is used by the DeepWrapper class

Parameters: views –
Returns

post_transform(z, train=False)[source]: Some models require a final linear CCA after model training. :param z: a list of all of the latent space embeddings for each view :param train: if the train flag is True this fits a new post transformation

batch_correlation(loader, train=False)[source]

Parameters

loader (DataLoader) – a dataloader that matches the structure of that used for training
train – whether to fit final linear transformation

Returns

by default returns the average pairwise correlation in each dimension (for 2 views just the correlation)

configure_callbacks()[source]

Configure model-specific callbacks. When the model gets attached, e.g., when .fit() or .test() gets called, the list or a callback returned here will be merged with the list of callbacks passed to the Trainer’s callbacks argument. If a callback returned here has the same type as one or several callbacks already present in the Trainer’s callbacks list, it will take priority and replace them. In addition, Lightning will make sure ModelCheckpoint callbacks run last.

Returns: A callback or a list of callbacks which will extend the list of callbacks in the Trainer.

Example:

def configure_callbacks(self):
    early_stop = EarlyStopping(monitor="val_acc", mode="max")
    checkpoint = ModelCheckpoint(monitor="val_loss")
    return [early_stop, checkpoint]

Note

Certain callback methods like on_init_start() will never be invoked on the new callbacks returned here.

training: bool

DCCA by Non-Linear Orthogonal Iterations

class cca_zoo.deepmodels._dcca_noi.DCCA_NOI(latent_dims, N, encoders=None, r=0, rho=0.2, eps=1e-09, shared_target=False, **kwargs)[source]

A class used to fit a DCCA model by non-linear orthogonal iterations

Citation

Wang, Weiran, et al. “Stochastic optimization for deep CCA via nonlinear orthogonal iterations.” 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2015.

Constructor class for DCCA

Parameters

latent_dims (int) – # latent dimensions
N (int) – # samples used to estimate covariance
encoders – list of encoder networks
r (float) – regularisation parameter of tracenorm CCA like ridge CCA
rho (float) – covariance memory like DCCA non-linear orthogonal iterations paper
eps (float) – epsilon used throughout
shared_target (bool) – not used

forward(views, **kwargs)[source]: We use the forward model to define the transformation of views to the latent space :param views: batches for each view separated by commas

loss(views, **kwargs)[source]

Define the loss function for the model. This is used by the DeepWrapper class

Parameters: views –
Returns

training: bool

trainer: Optional['pl.Trainer']

precision: int

prepare_data_per_node: bool

allow_zero_length_dataloader_with_multiple_devices: bool

Deep Canonically Correlated Autoencoders

class cca_zoo.deepmodels._dccae.DCCAE(latent_dims, objective=<class 'cca_zoo.deepmodels._objectives.MCCA'>, encoders=None, decoders=None, r=0, eps=1e-05, lam=0.5, latent_dropout=0, img_dim=None, recon_loss_type='mse', **kwargs)[source]

A class used to fit a DCCAE model.

Citation

Wang, Weiran, et al. “On deep multi-view representation learning.” International conference on machine learning. PMLR, 2015.

Parameters

latent_dims (int) – # latent dimensions
objective – # CCA objective: normal tracenorm CCA by default
encoders – list of encoder networks
decoders – list of decoder networks
r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller
eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller
lam – weight of reconstruction loss (1 minus weight of correlation loss)

forward(views, **kwargs)[source]: We use the forward model to define the transformation of views to the latent space :param views: batches for each view separated by commas

loss(views, **kwargs)[source]

Define the loss function for the model. This is used by the DeepWrapper class

Parameters: views –
Returns

configure_callbacks()[source]

Configure model-specific callbacks. When the model gets attached, e.g., when .fit() or .test() gets called, the list or a callback returned here will be merged with the list of callbacks passed to the Trainer’s callbacks argument. If a callback returned here has the same type as one or several callbacks already present in the Trainer’s callbacks list, it will take priority and replace them. In addition, Lightning will make sure ModelCheckpoint callbacks run last.

Returns: A callback or a list of callbacks which will extend the list of callbacks in the Trainer.

Example:

def configure_callbacks(self):
    early_stop = EarlyStopping(monitor="val_acc", mode="max")
    checkpoint = ModelCheckpoint(monitor="val_loss")
    return [early_stop, checkpoint]

Note

Certain callback methods like on_init_start() will never be invoked on the new callbacks returned here.

training: bool

trainer: Optional['pl.Trainer']

precision: int

prepare_data_per_node: bool

allow_zero_length_dataloader_with_multiple_devices: bool

Deep Tensor CCA

class cca_zoo.deepmodels._dtcca.DTCCA(latent_dims, encoders=None, r=0, eps=1e-05, **kwargs)[source]

A class used to fit a DTCCA model.

Is just a thin wrapper round DCCA with the DTCCA objective and a TCCA post-processing

Citation

Wong, Hok Shing, et al. “Deep Tensor CCA for Multi-view Learning.” IEEE Transactions on Big Data (2021).

Parameters

latent_dims (int) – # latent dimensions
encoders – list of encoder networks
r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller
eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

post_transform(z, train=False)[source]

Some models require a final linear CCA after model training. :param z: a list of all of the latent space embeddings for each view :param train: if the train flag is True this fits a new post transformation

Return type: Iterable[ndarray]

training: bool

trainer: Optional['pl.Trainer']

precision: int

prepare_data_per_node: bool

allow_zero_length_dataloader_with_multiple_devices: bool

Deep Variational CCA

class cca_zoo.deepmodels._dvcca.DVCCA(latent_dims, encoders=None, decoders=None, private_encoders=None, latent_dropout=0, img_dim=None, recon_loss_type='mse', **kwargs)[source]

A class used to fit a DVCCA model.

Citation

Wang, Weiran, et al. ‘Deep variational canonical correlation analysis.’ arXiv preprint arXiv:1610.03454 (2016).

https: // arxiv.org / pdf / 1610.03454.pdf

https: // github.com / pytorch / examples / blob / master / vae / main.py

Parameters

latent_dims (int) – # latent dimensions
encoders – list of encoder networks
decoders – list of decoder networks
private_encoders (Optional[Iterable[BaseEncoder]]) – list of private (view specific) encoder networks

forward(views, mle=True, **kwargs)[source]

Parameters

views –
mle –

Returns

loss(views, **kwargs)[source]

Parameters: args –
Returns

recon_uncertainty(views, **kwargs)[source]

configure_callbacks()[source]

Configure model-specific callbacks. When the model gets attached, e.g., when .fit() or .test() gets called, the list or a callback returned here will be merged with the list of callbacks passed to the Trainer’s callbacks argument. If a callback returned here has the same type as one or several callbacks already present in the Trainer’s callbacks list, it will take priority and replace them. In addition, Lightning will make sure ModelCheckpoint callbacks run last.

Returns: A callback or a list of callbacks which will extend the list of callbacks in the Trainer.

Example:

def configure_callbacks(self):
    early_stop = EarlyStopping(monitor="val_acc", mode="max")
    checkpoint = ModelCheckpoint(monitor="val_loss")
    return [early_stop, checkpoint]

Note

Certain callback methods like on_init_start() will never be invoked on the new callbacks returned here.

training: bool

Split Autoencoders

class cca_zoo.deepmodels._splitae.SplitAE(latent_dims, encoder=<class 'cca_zoo.deepmodels._architectures.Encoder'>, decoders=None, latent_dropout=0, recon_loss_type='mse', img_dim=None, **kwargs)[source]

A class used to fit a Split Autoencoder model.

Citation

Ngiam, Jiquan, et al. “Multimodal deep learning.” ICML. 2011.

Parameters

latent_dims (int) – # latent dimensions
encoder (BaseEncoder) – list of encoder networks
decoders – list of decoder networks

forward(views, **kwargs)[source]: We use the forward model to define the transformation of views to the latent space :param views: batches for each view separated by commas

loss(views, **kwargs)[source]: Required when using the LightningTrainer

configure_callbacks()[source]

Configure model-specific callbacks. When the model gets attached, e.g., when .fit() or .test() gets called, the list or a callback returned here will be merged with the list of callbacks passed to the Trainer’s callbacks argument. If a callback returned here has the same type as one or several callbacks already present in the Trainer’s callbacks list, it will take priority and replace them. In addition, Lightning will make sure ModelCheckpoint callbacks run last.

Returns: A callback or a list of callbacks which will extend the list of callbacks in the Trainer.

Example:

def configure_callbacks(self):
    early_stop = EarlyStopping(monitor="val_acc", mode="max")
    checkpoint = ModelCheckpoint(monitor="val_loss")
    return [early_stop, checkpoint]

Note

Certain callback methods like on_init_start() will never be invoked on the new callbacks returned here.

training: bool

Deep Objectives

class cca_zoo.deepmodels._objectives.CCA(latent_dims, r=0, eps=0.001)[source]

Differentiable CCA Loss. Loss() method takes the outputs of each view’s network and solves the CCA problem as in Andrew’s original paper

Parameters

latent_dims (int) – the number of latent dimensions
r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions
eps (float) – an epsilon parameter used in some operations

class cca_zoo.deepmodels._objectives.MCCA(latent_dims, r=0, eps=0.001)[source]

Differentiable MCCA Loss. Loss() method takes the outputs of each view’s network and solves the multiset eigenvalue problem as in e.g. https://arxiv.org/pdf/2005.11914.pdf

Parameters

latent_dims (int) – the number of latent dimensions
r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to

the number of latent dimensions :type eps: float :param eps: an epsilon parameter used in some operations

class cca_zoo.deepmodels._objectives.GCCA(latent_dims, r=0, eps=0.001)[source]

Differentiable GCCA Loss. Loss() method takes the outputs of each view’s network and solves the generalized CCA eigenproblem as in https://arxiv.org/pdf/2005.11914.pdf

Parameters

latent_dims (int) – the number of latent dimensions
r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to

the number of latent dimensions :type eps: float :param eps: an epsilon parameter used in some operations

class cca_zoo.deepmodels._objectives.TCCA(latent_dims, r=0, eps=0.0001)[source]

Differentiable TCCA Loss.

Parameters

latent_dims (int) – the number of latent dimensions
r (float) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions
eps (float) – an epsilon parameter used in some operations

Deep Models

DCCA

DCCA by Non-Linear Orthogonal Iterations

Deep Canonically Correlated Autoencoders

Deep Tensor CCA

Deep Variational CCA

Split Autoencoders

Deep Objectives

Model Architectures