cuplyr
cuplyr runs dplyr code on an NVIDIA GPU. It is a dplyr backend, like dbplyr, built on RAPIDS cuDF: you upload a data frame with tbl_gpu(), use the dplyr verbs you already know, and collect() the result back into R.
library(cuplyr)
library(dplyr)
set.seed(1)
n <- 1e5
sales <- data.frame(
year = sample(2018:2024, n, replace = TRUE),
region = sample(c("north", "south", "east", "west"), n, replace = TRUE),
quarter = sample(1:4, n, replace = TRUE),
amount = rnorm(n, mean = 50, sd = 30),
price = runif(n, 1, 10)
)
regions <- data.frame(
region = c("north", "south", "east", "west"),
manager = c("Ana", "Ben", "Chi", "Dev")
)
tbl_gpu(sales) |>
filter(year >= 2020, amount > 0) |>
mutate(revenue = amount * price) |>
group_by(region, quarter) |>
summarise(total = sum(revenue), .groups = "drop") |>
inner_join(tbl_gpu(regions), by = "region") |>
arrange(desc(total)) |>
collect()
No GPU? The Colab notebook installs cuplyr on a free T4 and runs a few pipelines.
cuplyr is experimental. Expect breaking changes before 1.0.
What runs on the GPU
The goal is every dplyr verb, with dplyr's results. A result that differs from dplyr's is a bug, and the test suite checks cuplyr against dplyr itself, including dplyr's own tests.
These run on the GPU:
filter(), mutate(), transmute(), select(), rename(), relocate(), arrange(), distinct(), pull()group_by(), summarise(), reframe(), count(), tally(), rowwise()slice() and its variants, including grouped slice_min() and slice_max()- every join: mutating, filtering and cross joins, and
join_by() with inequality, between(), within(), overlaps() and closest() conditions
bind_rows(), bind_cols(), the set operations and the rows_*() verbsacross(), if_any() and if_all()- inside expressions: arithmetic and comparisons,
if_else(), case_when(), case_match(), coalesce(), na_if(), between(), the window functions (lag(), lead(), the ranking functions, cumsum() and the other cumulative functions), first(), last(), nth(), n_distinct(), median(), quantile(), sd(), var(), base string functions (paste(), substr(), nchar(), grepl(), sub(), gsub(), toupper() and others) and date parts (year(), month(), fixed-unit floor_date() and round_date())
- rolling windows with
roll_mean(), roll_sum(), roll_median(), roll_min(), roll_max() and roll_sd()
- Parquet files, read and written on the GPU with
read_parquet_gpu() and write_parquet_gpu()
Anything else falls back to the CPU: cuplyr collects the data, runs dplyr and uploads the result. Your code still works, but it copies data across. The fallback is silent by default. Set options(cuplyr.fallback = "warn") to see each one, with the expression and the reason, or "error" to forbid them.
Column types:
| R | GPU |
|---|
| logical | BOOL8 |
| integer | INT32 |
| double | FLOAT64 |
| character | STRING |
| factor | DICTIONARY32, levels kept |
| Date | TIMESTAMP_DAYS |
| POSIXct | TIMESTAMP_MICROSECONDS, time zone kept |
| integer64 | FLOAT64, with a warning: values above 2^53 lose precision |
Not supported: list columns, other column classes (difftime, hms, ts and the like raise an error at upload) and more than one GPU. The open issues track what is planned.
Lazy by default
tbl_gpu() is lazy. Each verb checks its arguments and records a step, so a typo or a type error still fails at the verb that caused it. Nothing runs until you collect() the result into R or compute() it into a table that stays on the GPU. cuplyr then optimizes the whole plan, runs it and frees every intermediate column.
Printing a lazy table runs the plan to show the first rows, and keeps nothing. Print it twice and it runs twice, so compute() a table you want to look at often.
explain() shows the recorded steps and the plan the optimizer will run:
tbl_gpu(mtcars) |>
filter(mpg > 20) |>
mutate(kpl = mpg * 0.425) |>
select(cyl, kpl) |>
explain()
The optimizer drops columns the result never reads, fuses consecutive mutate() calls, pushes filters below mutates, selects and some joins, and merges consecutive filters into one. vignette("query-optimization") describes each pass.
On a single pipeline lazy is level with eager up to 1e6 rows and under a millisecond slower at 1e7. In exchange it holds no GPU memory between calls. An eager pipeline keeps its intermediate results on the GPU until R's next gc(), and in a loop they fill the card. Five eager runs of a pipeline over 5e7 rows, back to back, took 68 ms each. Lazy took 18 ms.
Eager mode runs each verb as soon as you call it, which is easier when you step through code by hand:
tbl_gpu(mtcars, lazy = FALSE) # one table
options(cuplyr.exec_mode = "eager") # every table created afterwards
cuplyr_options() lists the other options.
Performance
A five-verb pipeline (filter, mutate, group_by, summarise, arrange) on an RTX 5070, median ms per call:
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
