[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"

[Under Review] Official PyTorch implementation code for realizing the technical part of Phantom of Latent representing equipped with enlarged hidden dimension to build super frontier vision languag…

Qwen2-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

Official implementation of the Law of Vision Representation in MLLMs

Critique-out-Loud Reward Models

A Survey on Benchmarks of Multimodal Large Language Models

A flexible and efficient codebase for training visually-conditioned language models (VLMs)

🌍 Repository for "AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agent", ACL'24 Best Resource Paper.

SimPO: Simple Preference Optimization with a Reference-Free Reward

[ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score ru…

[NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333

😸 💬 A module to compute textual lexical richness (aka lexical diversity).

Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)

Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

This is the official repository of our paper "What If We Recaption Billions of Web Images with LLaMA-3 ?"

A benchmark dataset and simple code examples for measuring the perception and reasoning of multi-sensor Vision Language models.

🌋👵🏻 Yo'LLaVA: Your Personalized Language and Vision Assistant

Python Library to evaluate VLM models' robustness across diverse benchmarks

cuML - RAPIDS Machine Learning Library

Experiments and data for the paper "When and why vision-language models behave like bags-of-words, and what to do about it?" Oral @ ICLR 2023

A Home Assistant integration & Model to control your smart home using a Local LLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use th…

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

🐥 Code and Dataset for our EMNLP 2022 paper - "ProsocialDialog: A Prosocial Backbone for Conversational Agents"

[ACL 2024] LangBridge: Multilingual Reasoning Without Multilingual Supervision

👻 Code and benchmark for our EMNLP 2023 paper - "FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions"

Official repository for "Intriguing Properties of Vision Transformers" (NeurIPS 2021--Spotlight)

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]