Guide

From zero to training in 15 minutes. Coming from PyTorch, start with the porting guide; it routes you to the Rust primer if you want the language first, and the migration reference is there to look things up in.

Coming from PyTorch

Coming from PyTorch

Porting from PyTorch

Module mapping, FlowBuilder patterns, training-loop translation. AI-assisted via /port.

 Coming from PyTorch

Rust for PyTorch Users

10 Rust patterns in 15 minutes. Everything you need to write training scripts.

 Coming from PyTorch

PyTorch Migration Reference

Side-by-side PyTorch vs floDl for every concept, operation, and pattern.

Tutorials

1

Tensors

Creation, operations, memory management, device transfer.

 2

Autograd

Variables, gradient tracking, backward pass, no_grad.

 3

Modules

Linear, Conv2d, normalization, dropout, RNN cells, activations.

 4

Training

Losses, optimizers, the Trainer tiers, mixed precision, the full training loop.

 5

The Graph Builder

The fluent API: chains, residuals, parallel branches, mapping.

 6

Advanced Graphs

Forward references, loops, gates, switches, recurrent state.

 7

Visualization

DOT/SVG output, reading diagrams, profiling overlay.

 8

Utilities

Gradient clipping, checkpoints, initialization, LR scheduling, reproducibility, trend control.

 9

Training Monitor

ETA, resource tracking, live web dashboard, HTML export.

 10

Graph Tree

Hierarchical composition, freeze/thaw subtrees, subgraph checkpoints, cross-boundary observation.

 11

Multi-GPU Training

Trainer::run / builder, process-per-rank auto-promote, ElChe, DataLoader integration.

 12

Heterogeneous & Multi-Host DDP

ElChe cadence, process-per-rank clusters, A/B-testable backends.

 13

Data Loading

DataLoader, resident/streaming modes, VRAM-aware prefetch, DDP integration.

 14

HuggingFace Integration

Load BERT, RoBERTa, DistilBERT, ALBERT, XLM-R or DeBERTa-v2; classify, NER, QA, fill-mask; fine-tune and round-trip back.

DDP Reference

DDP Reference

Configuration Reference

Cadence modes, average backends, every TrainerConfig and ElCheConfig knob, callbacks, resume, elastic membership.

 DDP Reference

Multi-Host Cluster Guide

Deploying a real cluster: the join window, fdl.cluster.yml, sshd guardrails, per-case libtorch, probe and status.

 DDP Reference

Internals and Expert APIs

ElChe phase machine, outer optimizer, A/B testing, Ddp::wrap, NCCL and CUDA primitives.

 DDP Reference

Troubleshooting

NCCL init failures, version skew, CUDA context corruption, OOM, CPU-averaging timeouts, cluster hangs.

CLI

CLI

Install and Global Flags

Installing fdl, the global flags every command accepts, and update checks.

 CLI

Project and libtorch Commands

fdl setup, libtorch build/download, init, add, diagnose - plus the libtorch directory layout.

 CLI

Cluster and Hardware Commands

fdl probe, status, start, join, nccl, --gpus, and @cluster multi-host fan-out.

 CLI

Introspection and Tooling

fdl api-ref for the generated API surface, skill install for AI assistants, shell completions.

 CLI

The fdl.yml Manifest

Command kinds, declaring flags in Rust with FdlArgs, environment overlays, presets, schema caching.

 CLI

Working in the Source Checkout

The contributor loop: tests, CUDA suites, benchmarks, the DDP validation suite, HF gates.

Internals

Internals

Distributed Architecture

Seven views of the distributed pattern: role topology, training lifecycle, reduce backends, ElChe scheduling, message catalog, failure handling.

Platform & Help

Platform & Help

Mac / Apple Silicon

Running flodl on Apple Silicon through the Docker CPU path.

 Platform & Help

Troubleshooting

Real error messages and how to fix them. Build, runtime, training.