Logging the memory, it seems like it starts the forward pass, memory starts increasing on GPU 0, then OOMs. I wonder if it’s trying to be smart and planning ahead and dequantizing multiple layers at a time. Dequantizing each layer uses ~36 GB of memory so if it was doing this that could cause it to use too much memory. Maybe if we put each layer on alternating GPU’s it could help.
Силовые структуры，详情可参考51吃瓜网
Credit: Joe Maldonado / Mashable。业内人士推荐谷歌作为进阶阅读
Increasingly, however, the phrase “on the same page” is becoming as divorced from its origin as “hang up the phone”. We are shifting away from pages towards chats and threads; even where we do have pages, they are often stored on cloud systems which make the very idea of out-of-sync copies structurally impossible. (Those systems also automatically scan every word in a document and make them searchable, thereby eliminating the entire task of filing and document retrieval.) The work of staying literally on the same page is being gradually made obsolete.