Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe

[tatef]

Article URL: GitHub - t8/hypura: Run models too big for your Mac's memory

Comments URL: Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe | Hacker News

Points: 97

# Comments: 47

Continue reading...