Announcing dataplan: An Industrial-Grade, Air-Gapped Synthetic Data Generation Engine for PyTorch Fine-Tuning
PyTorch Forums
Announcing dataplan: An Industrial-Grade, Air-Gapped Synthetic Data Generation Engine for PyTorch Fine-Tuning
Hi everyone in the PyTorch community! I’m excited to share an open-source library I’ve been building called dataplan. When building and fine-tuning models, the biggest bottleneck isn’t usually the training loop or compute—it’s data. Existing synthetic data scripts often suffer from API rate limits dropping dataset sizes below target, mode collapse, or security risks when handling sensitive code and domains. I built dataplan to solve this by turning synthetic data generation into a determinist...
0 comments
No comments yet.