servo/deepseek-infra-index
1
0
Fork 0
mirror of https://github.com/deepseek-ai/open-infra-index.git synced 2025-03-31 07:46:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

add 3fs

Browse source
This commit is contained in:
Huang Panpan 2025-02-28 09:02:00 +08:00 • committed by GitHub
parent 3fc5e3a484
commit 4673c8d06c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 13 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
README.md
View file

@ -67,7 +67,19 @@ Introducing **DeepGEMM** - an FP8 GEMM library that supports both dense and MoE
📊 Analyze computation-communication overlap in V3/R1.
🔗 [**GitHub Repo**](https://github.com/deepseek-ai/profile-data)
### Ongoing Releases...
### Day 5 - 3FS, Thruster for All DeepSeek Data Access
Fire-Flyer File System (3FS) - a parallel file system that utilizes the full bandwidth of modern SSDs and RDMA networks.
⚡ 6.6 TiB/s aggregate read throughput in a 180-node cluster
⚡ 3.66 TiB/min throughput on GraySort benchmark in a 25-node cluster
⚡ 40+ GiB/s peak throughput per client node for KVCache lookup
🧬 Disaggregated architecture with strong consistency semantics
✅ Training data preprocessing, dataset loading, checkpoint saving/reloading, embedding vector search & KVCache lookups for inference in V3/R1
📥 3FS → https://github.com/deepseek-ai/3FS
⛲ Smallpond - data processing framework on 3FS → https://github.com/deepseek-ai/smallpond
## 2024 AI Infrastructure Paper (SC24)
### Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning