Out-of-core DataFrame for billion-row data via lazy evaluation and memory-mapped files. Use when data exceeds RAM (10 GB–TB) for fast aggregation, filtering, virtual columns, and visualization without loading. Supports HDF5, Arrow, Parquet, CSV with cloud (S3, GCS, Azure). Built-in ML transformers (scaling, PCA, K-means). In-memory: polars; distributed: dask.
Receba novas skills de Dados e Análise toda segunda
1 email curto, só com as skills novas de Dados e Análise. 4 minutos de leitura, sem spam, cancela com 1 clique.
Você confirma o email no primeiro envio. Sem spam. Cancela com 1 clique.
Ler descrição completaRecolher descrição
Vaex DataFrames
Overview
Vaex is a high-performance Python library for lazy, out-of-core DataFrame operations on datasets too large to fit in RAM. It processes over a billion rows per second using memory-mapped files and lazy evaluation, enabling interactive exploration and analysis without loading data into memory.
When to Use
Processing tabular datasets larger than available RAM (10 GB to terabytes)
Fast statistical aggregations on massive datasets (mean, std, quantiles at bill
[Description truncada. Veja o README completo no GitHub.]
