Discover the Best Open-Source Multimodal Projects Sorted by Stars

🔮 Multimodal AI services & pipelines with cloud-native stack: gRPC, Kubernetes, Docker, OpenTelemetry, Prometheus, Jaeger, etc.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

LAVIS - A One-stop Library for Language-Vision Intelligence

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

Visual Instruction Tuning: Large Language-and-Vision Assistant built towards multimodal GPT-4 level capabilities.

Reading list for research topics in multimodal machine learning

Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022

:sparkles::sparkles:Latest Papers and Datasets on Multimodal Large Language Models, and Their Evaluation.

🪩 Create Disco Diffusion artworks in one line

notes for software engineers getting up to speed on new AI developments. Serves as datastore for https://latent.space writing, and product brainstorming, but has cleaned up canonical references under the /Resources folder.

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

An open-source framework for training large multimodal models.

OpenMMLab Pre-training Toolbox and Benchmark

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

🧬 Represent, send, and store multimodal data · Neural Search · Vector Search · Document Store

Foundation Architecture for (M)LLMs