Distributed machine learning training

Here you can find a collection of tutorials for distributing PyTorch and Tensorflow based workflows.

Distributed ML with PyTorch

• 1. Introduction to distributed ML with PyTorch
• 2. Distributed training on MNIST dataset
• 3. Using the itwinai TorchTrainer Class
• 4. GAN tutorial with PyTorch
• 5. PyTorch scaling test
• 6. itwinai and containers (Docker and Singularity)
• 7. Tutorial on Kubeflow and TorchTrainer class
• 8. Distributed Machine Learning on HPC from k8s using KubeRay operator and interLink

Distributed ML with TensorFlow

• 1. Introduction on distributed training with TensorFlow
• 2. Tensorflow ImageNet example
• 3. Tensorflow scaling test

Machine Learning Workflows

Here you can find a collection of tutorials for various complexity ML workflows.

• Simple Workflow and Pipeline Components
• Using Configuration Files in itwinai
• DAG workflows
• Integrating configuration with command line arguments

Hyperparameter Optimization

This tutorial provides an overview of Hyperparameter Optimization (HPO) workflows.

• Hyperparameter Optimization with TorchTrainer on MNIST

Code Profiling and Optimization

Here you can find our tutorials on how to do profiling with itwinai:

• Profiling Overview
• Profiling using the py-spy Profiler
• Profiling Lattice QCD with the py-spy Profiler—an Example