Falcon LLM is TII's flagship series of large language models, built from scratch using a custom data pipeline and distributed training library. Papers coming soon 😊.
To promote collaborations and drive innovation, we have open-sourced a number of artefacts:

The Falcon-7/40B pretrained and instruct models, under the Apache 2.0 software license . Falcon-7B/40B models are state-of-the-art for their size, outperforming most other models on NLP benchmarks.
The RefinedWeb dataset, a massive web dataset with stringent filtering and large-scale deduplication, enabling models trained on web data alone to match or outperform models trained on curated corpora. RefinedWeb is licensed under Apache 2.0.

https://archive.is/Ip5pI#selection-512.0-539.228
tiiuae (Technology Innovation Institute)

--
[Cookie]
[Received]
[Client]