Text Generation Web UI

The Text Generation Web UI is a Gradio-based interface for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA. It provides a user-friendly interface to interact with these models and generate text, with features such as model switching, notebook mode, chat mode, and more. The project aims to become the go-to web UI for text generation and is similar to AUTOMATIC1111/stable-diffusion-webui in terms of functionality.

Features

• Dropdown menu for switching between models
• Notebook mode that resembles OpenAI's playground
• Chat mode for conversation and role-playing
• Instruct mode compatible with various formats, including Alpaca, Vicuna, Open Assistant, Dolly, Koala, ChatGLM, and MOSS
• Nice HTML output for GPT-4chan
• Markdown output for GALACTICA, including LaTeX rendering
• Custom chat characters
• Advanced chat features (send images, get audio responses with TTS)
• Efficient text streaming
• Parameter presets
• Layers splitting across GPU(s), CPU, and disk
• CPU mode
• and much more!

Installation

There are different installation methods available, including one-click installers for Windows, Linux, and macOS, as well as manual installation using Conda. Detailed installation instructions can be found in the Text Generation Web UI repository.

Downloading Models

Models should be placed inside the models folder. You can download models from Hugging Face, such as Pythia, OPT, GALACTICA, and GPT-J 6B. Use the download-model.py script to automatically download a model from Hugging Face.

Starting the Web UI

After installing the necessary dependencies and downloading the models, you can start the web UI by running the server.py script. The web UI can be accessed at http://localhost:7860/?__theme=dark. You can customize the interface and behavior using various command-line flags.

System Requirements

Check the wiki for examples of VRAM and RAM usage in both GPU and CPU mode.

Contributing

Pull requests, suggestions, and issue reports are welcome. Before reporting a bug, make sure you have followed the installation instructions provided and searched for existing issues.