OCR, layout analysis, reading order, line detection in 90+ languages

Mix of Minimal Optimal Sets (MMOS) of dataset has two advantages for two aspects, higher performance and lower construction costs on math reasoning.

Training LLMs with QLoRA + FSDP

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.

PyTorch Volume Models for 3D data

AICI: Prompts as (Wasm) Programs

Fast and flexible image augmentation library. Paper about the library: https://www.mdpi.com/2078-2489/11/2/125

1st Place Solution for LLM - Detect AI Generated Text Kaggle Competition

📘 The experiment tracker for foundation model training

A series of large language models trained from scratch by developers @01-ai

KenLM: Faster and Smaller Language Model Queries

Bounding Box is a library to plot pretty bounding boxes with a simple Python API.

Implementation of Nougat Neural Optical Understanding for Academic Documents

1st place solution to the Google - American Sign Language Fingerspelling Recognition competition

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Official implementation of Character Region Awareness for Text Detection (CRAFT)

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

Github action to upload datasets to kaggle

GLM-130B: An Open Bilingual Pre-Trained Model (ICLR 2023)

Breast cancer segmentation on mammograms

Winning solution for the Kaggle Feedback Prize Challenge.

Various transformers for FSDP research