Quantcast
Channel: User alistaire - Stack Overflow
Viewing all articles
Browse latest Browse all 40

Answer by alistaire for dplyr syntax for arrow to sum columns specified in a variable

$
0
0

I'm not sure why, but if you store the recursive code in a function (named or anonymous), it will let you run recursive code (or more simply written with Reduce):

library(arrow) library(dplyr)example_data = InMemoryDataset$create(data.frame(a1 = c(1,2,3), b2=c(4,5,6), c3=c(7,8,9)))cols_to_sum = c('a1','b2','c3')f <- function(...) Reduce(`+`, list(...))example_data %>%  mutate(computed_sum = f(!!!syms(cols_to_sum))) %>%  collect()#>   a1 b2 c3 computed_sum#> 1  1  4  7           12#> 2  2  5  8           15#> 3  3  6  9           18# calling directly errors outexample_data %>% mutate(computed_sum = Reduce(`+`, syms(cols_to_sum)))#> Error: Expression Reduce(`+`, syms(cols_to_sum)) not supported in Arrow#> Call collect() first to pull data into R.# anonymous functions do workexample_data %>% mutate(computed_sum = (function(...) Reduce(`+`, list(...)))(!!!syms(cols_to_sum)))#> InMemoryDataset (query)#> a1: double#> b2: double#> c3: double#> computed_sum: double (add_checked(add_checked(a1, b2), c3))#> #> See $.data for the source Arrow object

Viewing all articles
Browse latest Browse all 40

Latest Images

Trending Articles





Latest Images