Basic Provisioner Configuration

Select your deployment type to see provisioner configuration instructions.

Static Provisioner

The most basic provisioner configuration simply matches nodes based on hostnames. To add a single node, obtain the hostname of the node that you want to add and run the following commands on your server node:

# COMPUTE_HOSTNAME="" # populate with the hostname for your compute node.

# cat >provisioner.yaml<<EOF
definitions:
  - id: compute1
    provisioner: static
    provisionerSpec:
      condition: |-
        hostname() matches "${COMPUTE_HOSTNAME}"
EOF

# fuzzball admin config set provisioner.yaml

The hostname() matches field supports pattern matching, so you can add more than one node with patterns that match the hostnames.

Once you've run the set command, you can restart Substrate on the compute node like so:

# systemctl restart fuzzball-substrate.service

The AWS deployment method includes a shortcut allowing you to quickly and easily configure AWS EC2 compute node instance types (launched from the AMIs used by the deployment). During deployment, the CloudFormation template specifies 2 default node types for smaller and larger jobs. After deployment, you can run the following command to create the appropriate provisioner configuration to take advantage of these nodes:

$ fuzzball admin provisioner resource-defs setup

In the default scenario where t3a.2xlarge and p3.2xlarge nodes are specified in the CloudFormation template during deployment, this command produces and applies the following provisioner configuration.

definitions:
  - id: t3a.2xlarge
    provisioner: aws

    provisionerSpec:
      instanceType: t3a.2xlarge


  - id: p3.2xlarge
    provisioner: aws

    provisionerSpec:
      instanceType: p3.2xlarge

You can also copy this text into a file (or a modified version as applicable) and apply it with the following command:

$ fuzzball admin config set aws-provisioner.yaml

The GCP deployment method includes a shortcut allowing you to quickly and easily configure some basic GCP compute node instance types if you used --instance-types during deployment. After deployment, you can run the following command to create the appropriate provisioner configuration to take advantage of these nodes:

$ fuzzball provisioner resource-defs setup

For each machine type you specified with --instance-types, the setup command automatically creates provisioner definitions. This may include variations of smaller and larger instance types in the same family. For example, if you specified --instance-types=n2-standard-8,g2-standard-8 during deployment, the command may create the following definition:

$ fuzzball cluster get-config

definitions:
  - id: n2-standard-4
    provisioner: gcp
    provisionerSpec:
      machineType: n2-standard-4
  - id: n2-standard-8
    provisioner: gcp
    provisionerSpec:
      machineType: n2-standard-8
  - id: n2-standard-16
    provisioner: gcp
    provisionerSpec:
      machineType: n2-standard-16
  - id: g2-standard-4
    provisioner: gcp
    provisionerSpec:
      machineType: g2-standard-4
  - id: g2-standard-8
    provisioner: gcp
    provisionerSpec:
      machineType: g2-standard-8
  - id: g2-standard-16
    provisioner: gcp
    provisionerSpec:
      machineType: g2-standard-16

You can save the configuration above and edit it to apply custom settings such as adding new instance types or applying policies. You can then apply this configuration with the following command:

$ fuzzball cluster set-config custom-gcp.yaml

CoreWeave Provisioning

CoreWeave supports dynamic and static provisioning:

1. Dynamic Provisioning: Fuzzball creates and destroys nodes on-demand based on workflow requirements.
2. Static Provisioning: Use pre-existing CoreWeave NodePool resources after installing the coreweave-substrate-static Helm chart (see Static Provisioning Guide)

Dynamic Provisioning Configuration

Dynamic provisioning is the default approach where Fuzzball automatically manages the CoreWeave NodePool lifecycle.

Define CoreWeave Instance Types

Create provisioner definitions specifying which CoreWeave instance types your cluster can use, and you want to make available to workflows:

# coreweave-provisioner-definitions.yaml
definitions:
  - id: cpu-gp-genoa
    provisioner: coreweave
    provisionerSpec:
      instanceType: "cd-gp-a192-genoa"
      costPerHour: 7.78
  - id: gpu-h100
    provisioner: coreweave
    provisionerSpec:
      instanceType: "gd-8xh100ib-i128"
      costPerHour: 49.24

Apply the configuration:

$ fuzzball admin config set coreweave-provisioner-definitions.yaml

Initialize Provisioner Definitions

Before workflows can use a CoreWeave instance type, you must create at least one instance of that type to initialize and discover its available resources:

$ fuzzball admin provisioner instance create cpu-gp-genoa
$ fuzzball admin provisioner instance create gpu-h100

Monitor node provisioning:

$ fuzzball admin scheduler node list

Static Provisioning Configuration

For static provisioning with pre-existing NodePool resources, you define provisioner resources with provisioner: static and deploy Substrate using the coreweave-substrate-static Helm chart.

Define Static Provisioner Definitions

Create provisioner definitions for your existing CoreWeave NodePools:

# static-provisioner-definitions.yaml
definitions:
  - id: coreweave-cpu-small
    provisioner: static
    provisionerSpec:
      condition: "true"
      costPerHour: 7.78
  - id: coreweave-gpu-large
    provisioner: static
    provisionerSpec:
      condition: "true"
      costPerHour: 49.24

Apply the definitions:

$ fuzzball admin config set static-provisioner-definitions.yaml

Create Values File

Create a values file mapping your targets to node pools:

# static-provisioner-values.yaml
namespace: fuzzball

# Must match your Fuzzball version
imageTag: "<Version />"

# Image pull secret name
imagePullSecretName: "registry-image-credentials"

# Automatically patch CoreWeave NodePools with substrate taint
autopatchNodePools: true

# Define targets - one DaemonSet per target
targets:
  - name: fuzzball-cpu
    definitionId: "coreweave-cpu-small"
    nodeSelector:
      compute.coreweave.com/node-pool: "fuzzball-cpu-node-pool"

  - name: fuzzball-gpu
    definitionId: "coreweave-gpu-large"
    nodeSelector:
      compute.coreweave.com/node-pool: "fuzzball-gpu-node-pool"
    tolerations:
      - key: nvidia.com/gpu
        operator: Exists
        effect: NoSchedule

Install Static Provisioner Helm Chart

Install the coreweave-substrate-static Helm chart to deploy substrate as DaemonSets on your NodePools:

$ helm upgrade --install fuzzball-substrate-static \
  oci://depot.ciq.com/fuzzball/fuzzball-images/helm/coreweave-substrate-static \
  --namespace fuzzball \
  --version <Version /> \
  --values static-provisioner-values.yaml