OmniParse
Last updated
Last updated
OmniParse is a platform that ingests/parses any unstructured data into structured, actionable data optimized for GenAI (LLM) applcaitons. Whether working with documents, tables, images, videos, audio files, or web pages, OmniParse prepares your data to be clean, structured and ready for AI applications, such as RAG , fine-tuning and more.
✅ Completely local, no external APIs ✅ Fits in a T4 GPU ✅ Supports 10+ file types ✅ Convert documents, multimedia, and web pages to high-quality structured markdown ✅ Table extraction, image extraction/captioning, audio/video transcription, web page crawling ✅ Easily deployable using Docker and Skypilot ✅ Colab friendly
It's challenging to process data as it comes in different shapes and sizes. OmniParse aims to be an ingestion/parsing platform where you can ingest any type of data, such as documents, images, audio, video, and web content, and get the most structured and actionable output that is GenAI (LLM) friendly.
Original PDF | OmniParse | PyPDF |
---|---|---|
Note: The server only works on Linux-based systems. This is due to certain dependencies and system-specific configurations that are not compatible with Windows or macOS. To install OmniParse, you can use
pip
:
Create a Virtual Environment:
Install Dependencies:
To use OmniParse with Docker, execute the following commands:
Pull the OmniParse API Docker image from Docker Hub:
Run the Docker container, exposing port 8000: 👉🏼Docker Image
Alternatively, if you prefer to build the Docker image locally: Then, run the Docker container as follows:
Run the Server:
--documents
: Load in all the models that help you parse and ingest documents (Surya OCR series of models and Florence-2).
--media
: Load in Whisper model to transcribe audio and video files.
--web
: Set up selenium crawler.
To start the API server, run the following command:
Arguments:
--host
: Host IP address (default: 0.0.0.0)
--port
: Port number (default: 8000)
Type | Supported Extensions |
---|---|
Documents | .doc, .docx, .odt, .pdf, .ppt, .pptx |
Images | .png, .jpg, .jpeg, .tiff, .bmp, .heic |
Video | .mp4, .mkv, .avi, .mov |
Audio | .mp3, .wav, .aac |
Web | dynamic webpages, http://.com |
🦙 LlamaIndex | Langchain | Haystack integrations coming soon 📚 Batch processing data ⭐ Dynamic chunking and structured data extraction based on specified Schema 🛠️ One magic API: just feed in your file prompt what you want, and we will take care of the rest 🔧 Dynamic model selection and support for external APIs 📄 Batch processing for handling multiple files at once 📦 New open-source model to replace Surya OCR and Marker
Final goal: replace all the different models currently being used with a single MultiModel Model to parse any type of data and get the data you need.
OmniParse is licensed under the GPL-3.0 license. See LICENSE
for more information.
This project builds upon the remarkable Marker project created by Vik Paruchuri. We express our gratitude for the inspiration and foundation provided by this project. Special thanks to Surya-OCR and Texify for the OCR models extensively used in this project, and to Crawl4AI for their contributions.
Models being used:
Surya OCR, Detect, Layout, Order, and Texify
Florence-2 base
Whisper Small
Thank you to the authors for their contributions to these models.