EAGLE Inference 1.0.0 for Linux + NVIDIA

Fast like an eagle.
On your own GPU.

One download installs vLLM with EAGLE-3 speculative decoding. You get the same answers your model would give, generated in fewer steps, behind an OpenAI-compatible API.

Download for Linux Other downloads

Free download · .deb and .rpm · x86_64 and ARM64

EAGLE Inference

    ███████╗ █████╗  ██████╗ ██╗     ███████╗
    ██╔════╝██╔══██╗██╔════╝ ██║     ██╔════╝
    █████╗  ███████║██║  ███╗██║     █████╗
    ██╔══╝  ██╔══██║██║   ██║██║     ██╔══╝
    ███████╗██║  ██║╚██████╔╝███████╗███████╗
    ╚══════╝╚═╝  ╚═╝ ╚═════╝ ╚══════╝╚══════╝

    Fast like an eagle · AI at blazing speed

  ✔ Runtime ready (vLLM 0.31.0)
  › Target model  Qwen/Qwen3-8B
  › Draft head    RedHatAI/Qwen3-8B-speculator.eagle3
  ✔ Server ready on http://127.0.0.1:8000/v1

  EAGLE chat · Qwen/Qwen3-8B · /exit to quit

you › Explain speculative decoding in one line.
eagle › A small draft model guesses the next few
tokens and the big model checks them all at once,
so you keep its exact output and skip the waiting.▍

Download

Get EAGLE Inference

Pick your Linux distribution. Open the file and your system's software installer does the rest.

eagle-inference_1.0.0-1_amd64.deb

Opens in App Center or Software Install on Ubuntu, Debian, Mint and Pop!_OS.

Version: 1.0.0
Download: 22 MB
First-run setup: ~8 GB

sha256

Download .deb

Installing on a server over SSH?

sudo apt install ./eagle-inference_1.0.0-1_amd64.deb

Run it from the folder you downloaded to. All checksums are in SHA256SUMS.

What you need

An NVIDIA GPU
24 GB of VRAM for the default model, like an RTX 4090, L4, A10 or better
NVIDIA driver installed
EAGLE Inference picks the matching CUDA build for you
About 8 GB of disk, plus models
The default model is about 16 GB
A modern Linux
Ubuntu 22.04+, Debian 12+, Fedora 40+ or RHEL / Rocky 9+

How it works

Three steps, no commands to type

Download and open

Double-click the file. Your software center shows EAGLE Inference and installs it in seconds.

.deb or .rpm~22 MB

The package adds the EAGLE Inference app, the eagle command and an optional background service. Uninstall it the same way you'd remove any app.

The engine sets itself up

In the background, EAGLE downloads vLLM and the PyTorch build that matches your GPU driver. It only happens once.

~8 GB, one timeIsolated in /opt/eagle

Your system Python and packages aren't touched. If the download is interrupted, it picks up again the next time you open the app.

Open EAGLE Inference

Launch it from your app menu. It starts the server, downloads the model on first use, and opens a chat.

ChatOpenAI-compatible API

Point any OpenAI SDK, IDE plugin or agent framework at http://localhost:8000/v1 and it gets EAGLE speed automatically.

Why it's fast

Guess ahead, verify at once

A large model normally writes one token per forward pass. With EAGLE, a tiny draft head proposes several tokens. The full model then checks them all in a single pass and keeps every one it agrees with.

Every token you get is one the full model accepts, so the output is the same as running it on its own.

IllustrationStep 1

Draft head

Output

Drafted Verified Rejected From the full model

0tokens written

0full-model passes

0passes without EAGLE

No accuracy loss

Verification keeps the full model's output distribution. You get speed without changing what your model says.

Drop-in API

An OpenAI-compatible server on localhost. Existing apps and SDKs work by changing one URL.

Runs on your hardware

Models run locally on your GPU. Prompts and outputs stay on your machine, and usage telemetry is off.

Models

Ready-to-run EAGLE-3 presets

Each preset pairs an open model with a trained EAGLE-3 draft head. Switch with eagle start <preset>, or set a default in /etc/eagle/eagle.env.

Preset	Model	EAGLE-3 draft head	GPU memory
`qwen3-8b`Default	Qwen3 8BQwen/Qwen3-8B	RedHatAI/Qwen3-8B-speculator.eagle3	24 GB
`gpt-oss-20b`	gpt-oss 20Bopenai/gpt-oss-20b	RedHatAI/gpt-oss-20b-speculator.eagle3	24 GB
`llama3.1-8b`	Llama 3.1 8B Instructmeta-llama/Llama-3.1-8B-Instruct · needs Hugging Face access	RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3	24 GB
`qwen3-14b`	Qwen3 14BQwen/Qwen3-14B	RedHatAI/Qwen3-14B-speculator.eagle3	40 GB
`qwen3-32b`	Qwen3 32BQwen/Qwen3-32B	RedHatAI/Qwen3-32B-speculator.eagle3	80 GB
`qwen3-30b-a3b`	Qwen3 30B-A3B InstructQwen/Qwen3-30B-A3B-Instruct-2507	RedHatAI/Qwen3-30B-A3B-Instruct-2507-speculator.eagle3	80 GB

Bring your own pair: eagle serve --model <model> --draft <eagle-head>. Model weights download from Hugging Face on first use.

For developers

Your apps, at EAGLE speed

EAGLE Inference speaks the OpenAI API. Change the base URL and keep your code.

eagle startRun the server in the background

eagle chatChat in the terminal with live tokens/sec

eagle statsDraft acceptance rate and tokens per step

eagle modelsList presets

eagle doctorCheck the GPU, driver and runtime

On a server, run it as a service: sudo systemctl enable --now eagle. Settings are in /etc/eagle/eagle.env.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="eagle",  # any value works locally
)

reply = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "Hello, EAGLE!"}],
)
print(reply.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8000/v1",
  apiKey: "eagle", // any value works locally
});

const reply = await client.chat.completions.create({
  model: "Qwen/Qwen3-8B",
  messages: [{ role: "user", content: "Hello, EAGLE!" }],
});
console.log(reply.choices[0].message.content);

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-8B",
    "messages": [{"role": "user", "content": "Hello, EAGLE!"}]
  }'

FAQ

Questions

Does it work on Mac or Windows?

Not yet. EAGLE Inference needs an NVIDIA GPU and runs on Linux, which covers most GPU workstations, servers and cloud instances. On Windows, you can install Ubuntu with WSL2, install the .deb inside it, then run sudo eagle setup once.

Which GPU do I need?

Any recent NVIDIA GPU with enough memory for the model you pick: 24 GB for the default 8B models (RTX 3090/4090/5090, L4, A10, A100, H100 and so on) and up to 80 GB for the largest presets. Run eagle doctor to check your setup.

Is it free?

Yes. EAGLE Inference is a free download. The models it serves are open-weight and come under their own licenses, shown on each model's Hugging Face page.

What gets installed, and how do I remove it?

The app and the eagle command go in /opt/eagle, with settings in /etc/eagle. The engine (vLLM and PyTorch) downloads into /opt/eagle/runtime. Model weights go to the standard Hugging Face cache.

Remove it from your software center, or with sudo apt remove eagle-inference or sudo dnf remove eagle-inference. That also deletes the downloaded engine.

Does anything leave my machine?

Your prompts and outputs don't. EAGLE Inference only goes online to download the engine (from PyPI) and model weights (from Hugging Face). vLLM's anonymous usage statistics are turned off by default.

What's inside?

EAGLE Inference is built on open source: the vLLM engine (Apache-2.0), PyTorch, uv and open EAGLE-3 draft heads (Apache-2.0). Each one keeps its own license. vLLM is a community project and isn't affiliated with EAGLE.

This release installs vLLM 0.31.0.

Ready when you are

One file. A few minutes of setup. Then every model you serve runs with EAGLE.

Download EAGLE Inference

Fast like an eagle.On your own GPU.