Receber por email
← Voltar para o catálogo

dspy-vllm

Use vLLM for high-throughput production serving of self-hosted models with DSPy. Use when you want production LLM serving, tensor parallelism, multi-GPU inference, batch processing, or high-concurrency self-hosted models. Also used for vllm, vLLM, production serving, high throughput LLM, tensor parallelism, self-hosted production, PagedAttention, local production server, GPU serving, batch inferen

6estrelas
Atualizado há 4 meses

Ver no GitHub ↗

Como adicionar

/plugin marketplace add lebsral/DSPy-Programming-not-prompting-LMs-skills

O comando exato pode variar conforme o repositório. Confira o README no GitHub.

Para o autor da skill

Cole no README do seu repo

Mostra que sua skill está catalogada na Skillteca, gera backlink e tráfego rastreável.

[![Skillteca](https://www.skillteca.com.br/api/badge/dspy-vllm/svg)](https://www.skillteca.com.br/skills/dspy-vllm)

Skills relacionadas

Ver todas de Desenvolvimento →

Alerta por categoria

Receba novas skills de Desenvolvimento toda segunda

1 email curto, só com as skills novas de Desenvolvimento. 4 minutos de leitura, sem spam, cancela com 1 clique.

Você confirma o email no primeiro envio. Sem spam. Cancela com 1 clique.

Ler descrição completa

vLLM — High-Throughput Production Serving for DSPy

Guide the user through serving self-hosted models with vLLM for production DSPy deployments. High concurrency, multi-GPU, OpenAI-compatible API.

Step 1: Understand the setup

Before generating vLLM configuration, clarify:

  1. What GPU hardware? — Model (A100, H100, RTX 4090), count, and VRAM per GPU. This determines tensor parallelism and quantization needs.
  2. Which model? — Model name and size (7B, 13B, 70B). Determines VRAM req

[Description truncada. Veja o README completo no GitHub.]

Compartilhar X LinkedIn