Torch Mist Documentation

Documentation for the torch_mist mutual information estimation package. Code and additional information can be found in the Github repository.

The documentation includes a quickstart guide, detailed estimator tutorials showcasing the estimators, and practical use-case examples.

Quickstart

Getting Started

• Installation
• Usage
  • Basic
  • Command Line Interface
  • Advanced
    • Estimators
    • Training
    • Evaluation

Estimator tutorials

Estimators

• Discriminative Estimators
  • Estimating the log-partition function
  • Energy parametrizations
  • Critic Architectures
    • Number of Negative Samples
  • Alternative Training objectives
• Generative Estimators
  • Approximating Marginal Entropy
    • Variational Approach
    • Marginal Approximation Methods
  • Estimating Conditional Entropy
    • Conditional Variational Approach
    • Joint Variational Approach
  • Summary
• Transformed Estimators
• Quantized Estimators
  • Binned Mutual Information Estimation
    • Predictive Quantization
• Hybrid Mutual Information Estimators
  • Generative Estimators: High Bias
  • Discriminative Estimators: High Variance
  • Hybrid mutual information estimation
    • Hybrid: Difference of Entropies + MINE
    • Hybrid: Predictive Quantization and hard-negatives.

Use-case Examples

Examples

• Training InfoMax representations
  • Data loading
  • Instantiating the Encoder
  • Defining the Mutual Information Estimator
  • Training the estimator
  • Results and visualizations
• Feature Selection
  • Estimating the maximum information content
  • Finding the most informative feature
  • Determining the least informative feature
• Estimating and maximizing information with respect to multiple targets in Deep Representations
  • Loading the CelebA dataset
  • Multi-Mutual Information Estimation
    • Defining the Encoders
    • One estimator for each pair
    • Aggregating estimators
    • Applying transformations
  • Training the estimators
  • Results and visualizations
• K-Fold Mutual Information Estimation
• Dimensionality reduction methods
  • Maximally Informative Dimensions
  • Information Bottleneck
• Information on Time series
  • Time-lagged InfoMax
  • Time-lagged Information Bottleneck
• API Reference
  • torch_mist
    • Subpackages
    • Submodules
    • Package Contents