Meta AI OPT-66B
Leading developer of large-scale language models based on Transformers architecture, renowned for generating human-like text through Opt-66b.
Meta AI OPT-66B is a pioneering company specializing in large-scale language models using Transformers architecture, with their flagship Opt-66b model generating human-like text from prompts. Trained on 175 billion parameters across 992 GPUs, the model offers open-source access through the Transformers library and has been downloaded thousands of times.
About Meta AI OPT-66B
Meta AI OPT-66B is a leading developer of large-scale language models based on the Transformers architecture. Their flagship model, Opt-66b, is a causal language model that utilizes AutoModelForCausalLM and AutoTokenizer for processing and generation. This model is renowned for its ability to generate human-like text based on given prompts. The preprocessing of texts involves the use of Byte Pair Encoding (BPE) with a vocabulary size of 50272, tokenizing sequences into 2048 consecutive tokens. Opt-66b was trained on approximately 175 billion model parameters across 992 80GB A100 GPUs over roughly 33 days of continuous training. The model is accessible through the Transformers library and has been downloaded over 15,000 times in the last month. OPT is open-source and available under a non-restrictive license, with resources including a model card, extensive documentation, and an active community of users for collaboration and learning.
