[ICLR'23 Spotlight] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"
An Open-sourced Knowledgable Large Language Model Framework.
Get updates on the fastest growing repos and cool stats about GitHub right in your inbox
Once per month. No spam.