Reading Time: 12 minutes
Nice write up! Took me a bit to understand the moe and routing relevance on vram, plus the context size. This confirms the low end of the numbers I was calculating. Appreciate the math in public!😁
Nice write up! Took me a bit to understand the moe and routing relevance on vram, plus the context size. This confirms the low end of the numbers I was calculating. Appreciate the math in public!😁