Overview
Kuber is a small toolkit that leverages docker, Google’s cloud SDK and kubernetes to help you with massively parallel tasks.
Installation
For now Kuber can only be installed from Github:
# install.packages("devtools")
devtools::install_github("curso-r/kuber")Before installing, make sure you have a GCP project to deploy your scripts.
Example
A simple example of how you could use Kuber to help you deploy a parallel task via expansion is reproduced below. For a more complete guide, see the “Getting started” and “Toy example” vignettes.
library(kuber)
kub_create_cluster("my-cluster")
#> ✔ Creating cluster
kub_create_task("~/my-dir", "my-cluster", "my-bucket", "my-image", "~/my-key.json")
#> ✔ Fetching cluster information
#> ✔ Fetching bucket information
#> ✔ Creating bucket
#> ● Edit `~/my-dir/exec.R'`
#> ● Create `~/my-dir/list.rds` with usable objects
#> ● Run `kub_push_task("~/my-dir")`
file.edit("~/my-dir/exec.R")
file.copy("~/my-list.rds", "~/my-dir/list.rds", TRUE)
kub_push_task("~/my-dir")
#> ✔ Building image
#> ✔ Authenticating
#> ✔ Pushing image
#> ✔ Removing old jobs
#> ✔ Creating new jobs
kub_run_task("~/my-dir")
#> ✔ Authenticating
#> ✔ Setting cluster context
#> ✔ Creating jobs
#> ● Run `kub_list_pods()` to follow up on the pods
kub_list_pods("~/my-dir")
#> ✔ Setting cluster context
#> ✔ Fetching pods
#> NAME READY STATUS RESTARTS AGE
#> 1 process-ykwgkf-item-1-zxdbx 1/1 Running 0 2h
#> 2 process-ykwgkf-item-2-zkt4t 1/1 Running 0 2h
#> 3 process-ykwgkf-item-3-74nxj 1/1 Running 0 2h