ClaudeSkillz

ComfyUI Workflow Helper Skill

Expert guidance for ComfyUI workflow creation, node configuration, and optimization.

Capabilities

This skill provides expert assistance with:

1. Workflow Setup & Management
2. Creating custom workflows from scratch
3. Importing and modifying existing workflows
4. Workflow organization and structure
5. Node connection best practices

1. Node Configuration
2. Model loaders (Checkpoints, LoRAs, VAEs)
3. Samplers and schedulers
4. Conditioning nodes (prompts)
5. Image processing nodes
6. ControlNet integration

1. Model Management
2. Installing models (checkpoints, VAEs, text encoders, diffusion models)
3. Model organization in directories
4. Model format compatibility
5. OmniGen2 setup

1. Performance Optimization
2. VRAM management strategies
3. Batch processing optimization
4. Workflow efficiency improvements
5. Node caching strategies

1. Troubleshooting
2. Model loading errors
3. Node connection issues
4. OOM (Out of Memory) errors
5. Workflow execution failures

Key Knowledge Base

ComfyUI Directory Structure

/srv/comfyui/
├── models/
│   ├── checkpoints/          # SD models (.safetensors, .ckpt)
│   ├── vae/                  # VAE models
│   ├── loras/                # LoRA models
│   ├── text_encoders/        # Text encoder models
│   ├── diffusion_models/     # Diffusion models
│   ├── controlnet/           # ControlNet models
│   ├── upscale_models/       # Upscaler models
│   └── embeddings/           # Textual inversion embeddings
├── input/                    # Input images
├── output/                   # Generated images
└── custom_nodes/             # Custom node extensions

Essential Nodes

Loading Nodes:

• Load Checkpoint - Load SD models
• Load VAE - Load VAE models
• Load LoRA - Load LoRA models

Conditioning Nodes:

• CLIP Text Encode (Prompt) - Positive and negative prompts
• Conditioning Combine - Merge multiple conditioning

Sampling Nodes:

• KSampler - Main sampler node
• KSampler Advanced - Advanced sampling control

Image Nodes:

• VAE Decode - Convert latent to image
• VAE Encode - Convert image to latent
• Save Image - Save output

Model Nodes:

• Model Sampling Discrete - Sampling configuration
• Model Sampling SD3 - SD3-specific sampling

OmniGen2 Setup

Required Models:

1. VAE: ae.safetensors → /srv/comfyui/models/vae/
2. Text Encoder: qwen_2.5_vl_fp16.safetensors → /srv/comfyui/models/text_encoders/
3. Diffusion Model: omnigen2_fp16.safetensors → /srv/comfyui/models/diffusion_models/

Download Script:

# Use the provided download script
/root/homelab/scripts/download-comfyui-omnigen2-models.sh

# Or manual download:
mkdir -p /srv/comfyui/models/{vae,text_encoders,diffusion_models}
wget -O /srv/comfyui/models/vae/ae.safetensors \
  "https://huggingface.co/Comfy-Org/Lumina_Image_2.0_Repackaged/resolve/main/split_files/vae/ae.safetensors"
# ... (similar for other models)

Common Workflows

Basic txt2img Workflow

Nodes Required:

1. Load Checkpoint
2. CLIP Text Encode (Prompt) x2 (positive & negative)
3. Empty Latent Image
4. KSampler
5. VAE Decode
6. Save Image

Connection Flow:

Load Checkpoint → CLIP (positive/negative) → KSampler
Empty Latent → KSampler → VAE Decode → Save Image
Load Checkpoint → VAE Decode

img2img Workflow

Additional Nodes:

1. Load Image
2. VAE Encode

Connection Flow:

Load Image → VAE Encode → KSampler
(Rest similar to txt2img)

ControlNet Workflow

Additional Nodes:

1. Load ControlNet Model
2. Apply ControlNet
3. Preprocessor nodes (depends on ControlNet type)

Usage Examples

Example 1: Setting Up OmniGen2

User: "Help me set up OmniGen2 in ComfyUI"

Assistant Response: I'll guide you through setting up OmniGen2 in ComfyUI.

Step 1: Download Models

Run the download script:

bash /root/homelab/scripts/download-comfyui-omnigen2-models.sh

Or download manually:

• VAE: ae.safetensors → /srv/comfyui/models/vae/
• Text Encoder: qwen_2.5_vl_fp16.safetensors → /srv/comfyui/models/text_encoders/
• Diffusion Model: omnigen2_fp16.safetensors → /srv/comfyui/models/diffusion_models/

Step 2: Restart ComfyUI

# If using Docker:
docker restart comfyui-container

# If using systemd:
sudo systemctl restart comfyui

# If running manually:
# Stop and restart the ComfyUI process

Step 3: Verify in WebUI

1. Open ComfyUI web interface
2. Check that models appear in dropdowns:
3. VAE loader should show "ae.safetensors"
4. Text encoder nodes should show "qwen_2.5_vl_fp16.safetensors"
5. Diffusion model loader should show "omnigen2_fp16.safetensors"

Step 4: Create Workflow

Use OmniGen2-specific nodes:

• Load the OmniGen2 diffusion model
• Load Qwen text encoder
• Load Lumina VAE
• Configure sampler for OmniGen2

Example 2: Creating a Basic Workflow

User: "How do I create a simple txt2img workflow in ComfyUI?"

Assistant Response: I'll help you create a basic txt2img workflow.

Nodes to Add:

1. Right-click → Add Node → loaders → Load Checkpoint
2. Select your SD model

1. Right-click → Add Node → conditioning → CLIP Text Encode (Prompt) × 2
2. One for positive prompt
3. One for negative prompt

1. Right-click → Add Node → latent → Empty Latent Image
2. Set width: 512
3. Set height: 512
4. Batch size: 1

1. Right-click → Add Node → sampling → KSampler
2. Seed: -1 (random)
3. Steps: 20-40
4. CFG: 4.5 (for photorealism)
5. Sampler: dpmpp_2m_karras
6. Scheduler: karras

1. Right-click → Add Node → latent → VAE Decode

1. Right-click → Add Node → image → Save Image

Connections:

Load Checkpoint [MODEL] → KSampler [model]
Load Checkpoint [CLIP] → CLIP Text Encode (pos) [clip]
Load Checkpoint [CLIP] → CLIP Text Encode (neg) [clip]
Load Checkpoint [VAE] → VAE Decode [vae]

CLIP Text Encode (pos) [CONDITIONING] → KSampler [positive]
CLIP Text Encode (neg) [CONDITIONING] → KSampler [negative]

Empty Latent Image [LATENT] → KSampler [latent_image]

KSampler [LATENT] → VAE Decode [samples]
VAE Decode [IMAGE] → Save Image [images]

Usage:

1. Enter your prompt in positive CLIP Text Encode
2. Enter negative prompt in negative CLIP Text Encode
3. Click "Queue Prompt" to generate

Example 3: Troubleshooting OOM Errors

User: "ComfyUI keeps crashing with out of memory errors. How do I fix this?"

Assistant Response: OOM errors in ComfyUI can be resolved with these strategies:

1. Reduce Resolution

• Change Empty Latent Image dimensions:
• From 768×768 → 512×512
• From 1024×1024 → 768×768
• Lower resolution = less VRAM usage

2. Reduce Batch Size

• In Empty Latent Image node:
• Set batch_size to 1
• Generate images one at a time

3. Use VRAM-Efficient Models

• Use pruned models (smaller file size)
• Use fp16 models instead of fp32
• Example: v1-5-pruned-emaonly.safetensors vs full versions

4. ComfyUI Launch Args Add these flags when starting ComfyUI:

--lowvram          # Aggressive VRAM optimization
--medvram          # Moderate VRAM optimization
--normalvram       # Normal VRAM usage
--cpu              # Use CPU instead of GPU (slow but no VRAM)

5. Clear Cache

• Restart ComfyUI between generations
• Clear browser cache
• Delete temporary files

6. Check Node Efficiency

• Remove unnecessary nodes
• Avoid multiple simultaneous samplers
• Use single VAE decode instead of multiple

For 2GB VRAM:

• Stick to 512×512 resolution
• Use --lowvram flag
• Generate single images (batch=1)
• Use pruned/optimized models

Best Practices

1. Workflow Organization
2. Group related nodes together
3. Use reroute nodes for clean connections
4. Add note nodes to document workflow sections
5. Save workflows with descriptive names

1. Node Connections
2. Always connect correct output to