Deep Models¶
DCCA¶
-
class
cca_zoo.deepmodels.dcca.
DCCA
(latent_dims, objective=<class 'cca_zoo.deepmodels.objectives.MCCA'>, encoders=None, r=0, eps=0.001)[source]¶ Bases:
cca_zoo.deepmodels._dcca_base._DCCA_base
A class used to fit a DCCA model.
Citation: Andrew, Galen, et al. “Deep canonical correlation analysis.” International conference on machine learning. PMLR, 2013.
Constructor class for DCCA
Parameters: - latent_dims (
int
) – # latent dimensions - objective – # CCA objective: normal tracenorm CCA by default
- encoders – list of encoder networks
- r (
float
) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller - eps (
float
) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller
-
forward
(*args)[source]¶ We use the forward model to define the transformation of views to the latent space
Parameters: args – batches for each view separated by commas
- latent_dims (
DCCA by Non-Linear Orthogonal Iterations¶
-
class
cca_zoo.deepmodels.dcca_noi.
DCCA_NOI
(latent_dims, N, encoders=None, r=0, rho=0.2, eps=1e-09, shared_target=False)[source]¶ Bases:
cca_zoo.deepmodels.dcca.DCCA
A class used to fit a DCCA model by non-linear orthogonal iterations
Citation: Wang, Weiran, et al. “Stochastic optimization for deep CCA via nonlinear orthogonal iterations.” 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2015.
Constructor class for DCCA
Parameters: - latent_dims (
int
) – # latent dimensions - N (
int
) – # samples used to estimate covariance - encoders – list of encoder networks
- r (
float
) – regularisation parameter of tracenorm CCA like ridge CCA - rho (
float
) – covariance memory like DCCA non-linear orthogonal iterations paper - eps (
float
) – epsilon used throughout - shared_target (
bool
) – not used
- latent_dims (
Deep Canonically Correlated Autoencoders¶
-
class
cca_zoo.deepmodels.dccae.
DCCAE
(latent_dims, objective=<class 'cca_zoo.deepmodels.objectives.MCCA'>, encoders=None, decoders=None, r=0, eps=0.001, lam=0.5)[source]¶ Bases:
cca_zoo.deepmodels._dcca_base._DCCA_base
A class used to fit a DCCAE model.
Citation: Wang, Weiran, et al. “On deep multi-view representation learning.” International conference on machine learning. PMLR, 2015.
Parameters: - latent_dims (
int
) – # latent dimensions - objective – # CCA objective: normal tracenorm CCA by default
- encoders – list of encoder networks
- decoders – list of decoder networks
- r (
float
) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller - eps (
float
) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller - lam – weight of reconstruction loss (1 minus weight of correlation loss)
- latent_dims (
Deep Tensor CCA¶
-
class
cca_zoo.deepmodels.dtcca.
DTCCA
(latent_dims, encoders=None, r=0, eps=0.001)[source]¶ Bases:
cca_zoo.deepmodels.dcca.DCCA
A class used to fit a DTCCA model.
Is just a thin wrapper round DCCA with the DTCCA objective and a TCCA post-processing
Citation: Wong, Hok Shing, et al. “Deep Tensor CCA for Multi-view Learning.” IEEE Transactions on Big Data (2021).
Parameters: - latent_dims (
int
) – # latent dimensions - encoders – list of encoder networks
- r (
float
) – regularisation parameter of tracenorm CCA like ridge CCA. Needs to be VERY SMALL. If you get errors make this smaller - eps (
float
) – epsilon used throughout. Needs to be VERY SMALL. If you get errors make this smaller
- latent_dims (
Deep Variational CCA¶
-
class
cca_zoo.deepmodels.dvcca.
DVCCA
(latent_dims, encoders=None, decoders=None, private_encoders=None)[source]¶ Bases:
cca_zoo.deepmodels._dcca_base._DCCA_base
A class used to fit a DVCCA model.
Citation: Wang, Weiran, et al. “Deep variational canonical correlation analysis.” arXiv preprint arXiv:1610.03454 (2016).
https: // arxiv.org / pdf / 1610.03454.pdf
https: // github.com / pytorch / examples / blob / master / vae / main.py
Parameters: - latent_dims (
int
) – # latent dimensions - encoders – list of encoder networks
- decoders – list of decoder networks
- private_encoders (
Optional
[Iterable
[BaseEncoder
]]) – list of private (view specific) encoder networks
- latent_dims (
Split Autoencoders¶
-
class
cca_zoo.deepmodels.splitae.
SplitAE
(latent_dims, encoder=<class 'cca_zoo.deepmodels.architectures.Encoder'>, decoders=None)[source]¶ Bases:
cca_zoo.deepmodels._dcca_base._DCCA_base
A class used to fit a Split Autoencoder model.
Citation: Ngiam, Jiquan, et al. “Multimodal deep learning.” ICML. 2011.
Parameters: - latent_dims (
int
) – # latent dimensions - encoder (
BaseEncoder
) – list of encoder networks - decoders – list of decoder networks
- latent_dims (
CCALightning Module for training with pytorch-lightning¶
Hooks to be used in LightningModule.
-
cca_zoo.deepmodels.CCALightning.
automatic_optimization
¶ If set to
False
you are responsible for calling.backward()
,.step()
,.zero_grad()
.Return type: bool
-
cca_zoo.deepmodels.CCALightning.
current_epoch
¶ The current epoch in the Trainer.
If no Trainer is attached, this propery is 0.
Return type: int
-
cca_zoo.deepmodels.CCALightning.
example_input_array
¶ The example input array is a specification of what the module can consume in the
forward()
method. The return type is interpreted as follows:- Single tensor: It is assumed the model takes a single argument, i.e.,
model.forward(model.example_input_array)
- Tuple: The input array should be interpreted as a sequence of positional arguments, i.e.,
model.forward(*model.example_input_array)
- Dict: The input array represents named keyword arguments, i.e.,
model.forward(**model.example_input_array)
Return type: Any
- Single tensor: It is assumed the model takes a single argument, i.e.,
-
cca_zoo.deepmodels.CCALightning.
global_rank
¶ The index of the current process across all nodes and devices.
Return type: int
-
cca_zoo.deepmodels.CCALightning.
global_step
¶ Total training batches seen across all epochs.
If no Trainer is attached, this propery is 0.
Return type: int
-
cca_zoo.deepmodels.CCALightning.
hparams
¶ The collection of hyperparameters saved with
save_hyperparameters()
. It is mutable by the user. For the frozen set of initial hyperparameters, usehparams_initial
.Returns: mutable hyperparameters dicionary Return type: Union[AttributeDict, dict, Namespace] Return type: Union
[AttributeDict
,dict
,Namespace
]
-
cca_zoo.deepmodels.CCALightning.
hparams_initial
¶ The collection of hyperparameters saved with
save_hyperparameters()
. These contents are read-only. Manual updates to the saved hyperparameters can instead be performed throughhparams
.Returns: immutable initial hyperparameters Return type: AttributeDict Return type: AttributeDict
-
cca_zoo.deepmodels.CCALightning.
local_rank
¶ The index of the current process within a single node.
Return type: int
-
cca_zoo.deepmodels.CCALightning.
logger
¶ Reference to the logger object in the Trainer.
-
cca_zoo.deepmodels.CCALightning.
model_size
¶ Returns the model size in MegaBytes (MB)
Note
This property will not return correct value for Deepspeed (stage 3) and fully-sharded training.
Return type: float
-
cca_zoo.deepmodels.CCALightning.
on_gpu
¶ Returns
True
if this model is currently located on a GPU.Useful to set flags around the LightningModule for different CPU vs GPU behavior.
-
cca_zoo.deepmodels.CCALightning.
truncated_bptt_steps
¶ Enables Truncated Backpropagation Through Time in the Trainer when set to a positive integer.
It represents the number of times
training_step()
gets called before backpropagation. If this is > 0, thetraining_step()
receives an additional argumenthiddens
and is expected to return a hidden state.Return type: int
Deep Objectives¶
-
class
cca_zoo.deepmodels.objectives.
CCA
(latent_dims, r=0, eps=0.001)[source]¶ Bases:
object
Differentiable CCA Loss. Loss() method takes the outputs of each view’s network and solves the CCA problem as in Andrew’s original paper
Parameters: - latent_dims (
int
) – the number of latent dimensions - r (
float
) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions - eps (
float
) – an epsilon parameter used in some operations
- latent_dims (
-
class
cca_zoo.deepmodels.objectives.
MCCA
(latent_dims, r=0, eps=0.001)[source]¶ Bases:
object
Differentiable MCCA Loss. Loss() method takes the outputs of each view’s network and solves the multiset eigenvalue problem as in e.g. https://arxiv.org/pdf/2005.11914.pdf
Parameters: - latent_dims (
int
) – the number of latent dimensions - r (
float
) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to
the number of latent dimensions :type eps:
float
:param eps: an epsilon parameter used in some operations- latent_dims (
-
class
cca_zoo.deepmodels.objectives.
GCCA
(latent_dims, r=0, eps=0.001)[source]¶ Bases:
object
Differentiable GCCA Loss. Loss() method takes the outputs of each view’s network and solves the generalized CCA eigenproblem as in https://arxiv.org/pdf/2005.11914.pdf
Parameters: - latent_dims (
int
) – the number of latent dimensions - r (
float
) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to
the number of latent dimensions :type eps:
float
:param eps: an epsilon parameter used in some operations- latent_dims (
-
class
cca_zoo.deepmodels.objectives.
TCCA
(latent_dims, r=0, eps=0.001)[source]¶ Bases:
object
Differentiable TCCA Loss.
Parameters: - latent_dims (
int
) – the number of latent dimensions - r (
float
) – regularisation as in regularized CCA. Makes the problem well posed when batch size is similar to the number of latent dimensions - eps (
float
) – an epsilon parameter used in some operations
- latent_dims (
Model Architectures¶
-
class
cca_zoo.deepmodels.architectures.
BaseEncoder
(latent_dims, variational=False)[source]¶ Bases:
torch.nn.modules.module.Module
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
BaseDecoder
(latent_dims)[source]¶ Bases:
torch.nn.modules.module.Module
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
Encoder
(latent_dims, variational=False, feature_size=784, layer_sizes=None)[source]¶ Bases:
cca_zoo.deepmodels.architectures.BaseEncoder
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
Decoder
(latent_dims, feature_size=784, layer_sizes=None, norm_output=False)[source]¶ Bases:
cca_zoo.deepmodels.architectures.BaseDecoder
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
CNNEncoder
(latent_dims, variational=False, feature_size=(28, 28), channels=None, kernel_sizes=None, stride=None, padding=None)[source]¶ Bases:
cca_zoo.deepmodels.architectures.BaseEncoder
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
CNNDecoder
(latent_dims, feature_size=(28, 28), channels=None, kernel_sizes=None, strides=None, paddings=None, norm_output=False)[source]¶ Bases:
cca_zoo.deepmodels.architectures.BaseDecoder
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
LinearEncoder
(latent_dims, feature_size, variational=False)[source]¶ Bases:
cca_zoo.deepmodels.architectures.BaseEncoder
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
cca_zoo.deepmodels.architectures.
LinearDecoder
(latent_dims, feature_size)[source]¶ Bases:
cca_zoo.deepmodels.architectures.BaseDecoder
-
forward
(x)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-