servo/deepseek-infra-index
1
0
Fork 0
mirror of https://github.com/deepseek-ai/open-infra-index.git synced 2025-03-31 07:46:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

day6: one more thing

Browse source
This commit is contained in:
Huang Panpan 2025-03-01 12:02:32 +08:00 • committed by pp
parent cbb9c3be27
commit 9673391d9b
6 changed files with 16 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
202502OpenSourceWeek/figures/Communication-Computation Overlapping during Decoding Phase.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 157 KiB

BIN
202502OpenSourceWeek/figures/Communication-Computation Overlapping during Prefilling Phase.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 134 KiB

BIN
202502OpenSourceWeek/figures/Cost And Theoretical Income.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 182 KiB

BIN
202502OpenSourceWeek/figures/Diagram of DeepSeek's Online Inference System.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 297 KiB

BIN
202502OpenSourceWeek/figures/H800 Node Count For Inference Service.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 82 KiB

16
README.md
View file

@ -81,6 +81,22 @@ Fire-Flyer File System (3FS) - a parallel file system that utilizes the full ban
⛲ Smallpond - data processing framework on 3FS → https://github.com/deepseek-ai/smallpond
### Day 6 - One More Thing: DeepSeek-V3/R1 Inference System Overview
Optimized throughput and latency via:
🔧 Cross-node EP-powered batch scaling
🔄 Computation-communication overlap
⚖️ Load balancing
Production data of V3/R1 online services:
⚡ 73.7k/14.8k input/output tokens per second per H800 node
🚀 Cost profit margin 545%
![Cost And Theoretical Income.jpg](202502OpenSourceWeek/figures/Cost%20And%20Theoretical%20Income.jpg)
💡 We hope this week's insights offer value to the community and contribute to our shared AGI goals.
📖 Deep Dive: 🔗[Day 6 - One More Thing: DeepSeek-V3/R1 Inference System Overview](202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md)
## 2024 AI Infrastructure Paper (SC24)
### Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning