Deep Models¶

DCCA¶

class cca_zoo.deepmodels.dcca.DCCA(latent_dims, objective=<class 'cca_zoo.deepmodels.objectives.MCCA'>, encoders=None, r=0, eps=0.001)[source]¶

Bases: cca_zoo.deepmodels._dcca_base._DCCA_base

A class used to fit a DCCA model.

Citation:

Andrew, Galen, et al. “Deep canonical correlation analysis.” International conference on machine learning. PMLR, 2013.

Constructor class for DCCA

Parameters:

latent_dims (int) – # latent dimensions
objective – # CCA objective: normal tracenorm CCA by default
encoders – list of encoder networks
r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller
eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

forward(*args)[source]¶

We use the forward model to define the transformation of views to the latent space

Parameters:	args – batches for each view separated by commas

loss(*args)[source]¶

Define the loss function for the model. This is used by the DeepWrapper class

Parameters:	args –
Returns:

post_transform(z_list, train=False)[source]¶

Some models require a final linear CCA after model training.

Parameters:	z_list – a list of all of the latent space embeddings for each view train – if the train flag is True this fits a new post transformation

DCCA by Non-Linear Orthogonal Iterations¶

class cca_zoo.deepmodels.dcca_noi.DCCA_NOI(latent_dims, N, encoders=None, r=0, rho=0.2, eps=1e-09, shared_target=False)[source]¶

Bases: cca_zoo.deepmodels.dcca.DCCA

A class used to fit a DCCA model by non-linear orthogonal iterations

Citation:

Wang, Weiran, et al. “Stochastic optimization for deep CCA via nonlinear orthogonal iterations.” 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2015.

Constructor class for DCCA

Parameters:	latent_dims (`int`) – # latent dimensions N (`int`) – # samples used to estimate covariance encoders – list of encoder networks r (`float`) – regularisation parameter of tracenorm CCA like ridge CCA rho (`float`) – covariance memory like DCCA non-linear orthogonal iterations paper eps (`float`) – epsilon used throughout shared_target (`bool`) – not used

forward(*args)[source]¶

We use the forward model to define the transformation of views to the latent space

Parameters:	args – batches for each view separated by commas

loss(*args)[source]¶

Define the loss function for the model. This is used by the DeepWrapper class

Parameters:	args –
Returns:

Deep Canonically Correlated Autoencoders¶

class cca_zoo.deepmodels.dccae.DCCAE(latent_dims, objective=<class 'cca_zoo.deepmodels.objectives.MCCA'>, encoders=None, decoders=None, r=0, eps=0.001, lam=0.5)[source]¶

Bases: cca_zoo.deepmodels._dcca_base._DCCA_base

A class used to fit a DCCAE model.

Citation:

Wang, Weiran, et al. “On deep multi-view representation learning.” International conference on machine learning. PMLR, 2015.

Parameters:

latent_dims (int) – # latent dimensions
objective – # CCA objective: normal tracenorm CCA by default
encoders – list of encoder networks
decoders – list of decoder networks
r (float) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller
eps (float) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller
lam – weight of reconstruction loss (1 minus weight of correlation loss)

forward(*args)[source]¶

We use the forward model to define the transformation of views to the latent space

Parameters:	args – batches for each view separated by commas

recon(*args)[source]¶

Parameters:	args –
Returns:

loss(*args)[source]¶: Required when using the LightningTrainer

Deep Tensor CCA¶

class cca_zoo.deepmodels.dtcca.DTCCA(latent_dims, encoders=None, r=0, eps=0.001)[source]¶

Bases: cca_zoo.deepmodels.dcca.DCCA

A class used to fit a DTCCA model.

Is just a thin wrapper round DCCA with the DTCCA objective and a TCCA post-processing

Citation:

Wong, Hok Shing, et al. “Deep Tensor CCA for Multi-view Learning.” IEEE Transactions on Big Data (2021).

Parameters:	latent_dims (`int`) – # latent dimensions encoders – list of encoder networks r (`float`) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller eps (`float`) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller

post_transform(z_list, train=False)[source]¶

Some models require a final linear CCA after model training.

Parameters:	z_list – a list of all of the latent space embeddings for each view train – if the train flag is True this fits a new post transformation
Return type:	`Iterable`[`ndarray`]

Deep Variational CCA¶

class cca_zoo.deepmodels.dvcca.DVCCA(latent_dims, encoders=None, decoders=None, private_encoders=None)[source]¶

Bases: cca_zoo.deepmodels._dcca_base._DCCA_base

A class used to fit a DVCCA model.

Citation:

Wang, Weiran, et al. “Deep variational canonical correlation analysis.” arXiv preprint arXiv:1610.03454 (2016).

https: // arxiv.org / pdf / 1610.03454.pdf

https: // github.com / pytorch / examples / blob / master / vae / main.py

Parameters:	latent_dims (`int`) – # latent dimensions encoders – list of encoder networks decoders – list of decoder networks private_encoders (`Optional`[`Iterable`[`BaseEncoder`]]) – list of private (view specific) encoder networks

forward(*args, mle=True)[source]¶

Parameters:	args – mle –
Returns:

recon(*args)[source]¶

Parameters:	args –
Returns:

loss(*args)[source]¶

Parameters:	args –
Returns:

vcca_loss(*args, mu, logvar)[source]¶

Parameters:	args – mu – logvar –
Returns:

vcca_private_loss(*args, mu, logvar, mu_p, logvar_p)[source]¶

Parameters:	args – mu – logvar –
Returns:

Split Autoencoders¶

class cca_zoo.deepmodels.splitae.SplitAE(latent_dims, encoder=<class 'cca_zoo.deepmodels.architectures.Encoder'>, decoders=None)[source]¶

Bases: cca_zoo.deepmodels._dcca_base._DCCA_base

A class used to fit a Split Autoencoder model.

Citation:

Ngiam, Jiquan, et al. “Multimodal deep learning.” ICML. 2011.

Parameters:	latent_dims (`int`) – # latent dimensions encoder (`BaseEncoder`) – list of encoder networks decoders – list of decoder networks

forward(*args)[source]¶

We use the forward model to define the transformation of views to the latent space

Parameters:	args – batches for each view separated by commas

recon(*args)[source]¶

Parameters:	args –
Returns:

loss(*args)[source]¶: Required when using the LightningTrainer

static recon_loss(x, recon)[source]¶

CCALightning Module for training with pytorch-lightning¶

Hooks to be used in LightningModule.

cca_zoo.deepmodels.CCALightning.automatic_optimization¶

If set to False you are responsible for calling .backward(), .step(), .zero_grad().

Return type:	`bool`

cca_zoo.deepmodels.CCALightning.current_epoch¶

The current epoch in the Trainer.

If no Trainer is attached, this propery is 0.

Return type:	`int`

cca_zoo.deepmodels.CCALightning.example_input_array¶

The example input array is a specification of what the module can consume in the forward() method. The return type is interpreted as follows:

Single tensor: It is assumed the model takes a single argument, i.e., model.forward(model.example_input_array)
Tuple: The input array should be interpreted as a sequence of positional arguments, i.e., model.forward(*model.example_input_array)
Dict: The input array represents named keyword arguments, i.e., model.forward(**model.example_input_array)

Return type:	`Any`

cca_zoo.deepmodels.CCALightning.global_rank¶

The index of the current process across all nodes and devices.

Return type:	`int`

cca_zoo.deepmodels.CCALightning.global_step¶

Total training batches seen across all epochs.

If no Trainer is attached, this propery is 0.

Return type:	`int`

cca_zoo.deepmodels.CCALightning.hparams¶

The collection of hyperparameters saved with save_hyperparameters(). It is mutable by the user. For the frozen set of initial hyperparameters, use hparams_initial.

Returns:	mutable hyperparameters dicionary
Return type:	Union[AttributeDict, dict, Namespace]
Return type:	`Union`[`AttributeDict`, `dict`, `Namespace`]

cca_zoo.deepmodels.CCALightning.hparams_initial¶

The collection of hyperparameters saved with save_hyperparameters(). These contents are read-only. Manual updates to the saved hyperparameters can instead be performed through hparams.

Returns:	immutable initial hyperparameters
Return type:	AttributeDict
Return type:	`AttributeDict`

cca_zoo.deepmodels.CCALightning.local_rank¶

The index of the current process within a single node.

Return type:	`int`

cca_zoo.deepmodels.CCALightning.logger¶: Reference to the logger object in the Trainer.

cca_zoo.deepmodels.CCALightning.model_size¶

Returns the model size in MegaBytes (MB)

Note

This property will not return correct value for Deepspeed (stage 3) and fully-sharded training.

Return type:	`float`

cca_zoo.deepmodels.CCALightning.on_gpu¶

Returns True if this model is currently located on a GPU.

Useful to set flags around the LightningModule for different CPU vs GPU behavior.

cca_zoo.deepmodels.CCALightning.truncated_bptt_steps¶

Enables Truncated Backpropagation Through Time in the Trainer when set to a positive integer.

It represents the number of times training_step() gets called before backpropagation. If this is > 0, the training_step() receives an additional argument hiddens and is expected to return a hidden state.

Return type:	`int`

Deep Objectives¶

class cca_zoo.deepmodels.objectives.CCA(latent_dims, r=0, eps=0.001)[source]¶

Bases: object

Differentiable CCA Loss. Loss() method takes the outputs of each view’s network and solves the CCA problem as in Andrew’s original paper

Parameters:	latent_dims (`int`) – the number of latent dimensions r (`float`) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions eps (`float`) – an epsilon parameter used in some operations

class cca_zoo.deepmodels.objectives.MCCA(latent_dims, r=0, eps=0.001)[source]¶

Bases: object

Differentiable MCCA Loss. Loss() method takes the outputs of each view’s network and solves the multiset eigenvalue problem as in e.g. https://arxiv.org/pdf/2005.11914.pdf

Parameters:	latent_dims (`int`) – the number of latent dimensions r (`float`) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to

the number of latent dimensions :type eps: float :param eps: an epsilon parameter used in some operations

class cca_zoo.deepmodels.objectives.GCCA(latent_dims, r=0, eps=0.001)[source]¶

Bases: object

Differentiable GCCA Loss. Loss() method takes the outputs of each view’s network and solves the generalized CCA eigenproblem as in https://arxiv.org/pdf/2005.11914.pdf

Parameters:	latent_dims (`int`) – the number of latent dimensions r (`float`) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to

the number of latent dimensions :type eps: float :param eps: an epsilon parameter used in some operations

class cca_zoo.deepmodels.objectives.TCCA(latent_dims, r=0, eps=0.001)[source]¶

Bases: object

Differentiable TCCA Loss.

Parameters:	latent_dims (`int`) – the number of latent dimensions r (`float`) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions eps (`float`) – an epsilon parameter used in some operations

Model Architectures¶

class cca_zoo.deepmodels.architectures.BaseEncoder(latent_dims, variational=False)[source]¶

Bases: torch.nn.modules.module.Module

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.BaseDecoder(latent_dims)[source]¶

Bases: torch.nn.modules.module.Module

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.Encoder(latent_dims, variational=False, feature_size=784, layer_sizes=None)[source]¶

Bases: cca_zoo.deepmodels.architectures.BaseEncoder

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.Decoder(latent_dims, feature_size=784, layer_sizes=None, norm_output=False)[source]¶

Bases: cca_zoo.deepmodels.architectures.BaseDecoder

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.CNNEncoder(latent_dims, variational=False, feature_size=(28, 28), channels=None, kernel_sizes=None, stride=None, padding=None)[source]¶

Bases: cca_zoo.deepmodels.architectures.BaseEncoder

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.CNNDecoder(latent_dims, feature_size=(28, 28), channels=None, kernel_sizes=None, strides=None, paddings=None, norm_output=False)[source]¶

Bases: cca_zoo.deepmodels.architectures.BaseDecoder

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.LinearEncoder(latent_dims, feature_size, variational=False)[source]¶

Bases: cca_zoo.deepmodels.architectures.BaseEncoder

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class cca_zoo.deepmodels.architectures.LinearDecoder(latent_dims, feature_size)[source]¶

Bases: cca_zoo.deepmodels.architectures.BaseDecoder

forward(x)[source]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.