1200TB SSD modules are in the pipeline thanks to Pure Storage — but you definitely won't be able to plug one in your workstation PC and it will be shockingly expensive
Meta sheds more light on how it is evolving Llama 3 training — it relies for now on almost 50,000 Nvidia H100 GPU, but how long before Meta switches to its own AI chip?