Host Networking for Multinode Services

Overview

Multinode services always run with host networking: every rank runs directly on its node's network stack instead of in an isolated network namespace. Fuzzball enables this automatically for any service that declares a multinode block, so setting network.host: true explicitly is allowed but not required. A multinode service cannot opt out: network.host: false is overridden.

Host networking is what lets a distributed service work across nodes: ranks reach each other at node addresses on dynamically negotiated ports, and the workload gets direct access to high-speed interconnects like InfiniBand or RoCE for low-latency, high-bandwidth communication.

Warning

Host networking means the service shares the node's network namespace. This can expose additional network interfaces and may have security implications.

Configuring a Multinode Service in YAML

Declare the multinode block on the service; host networking is applied automatically. The network.host field may still be set explicitly for clarity:


services:
  distributed-inference:
    image:
      uri: docker://myregistry/mpi-inference:latest
    script: |
      #!/bin/bash
      python -m inference_server --model /models/llm
    multinode:
      nodes: 4
      implementation: openmpi
    resource:
      cpu:
        cores: 16
        affinity: NUMA
      memory:
        size: 64GB
      devices:
        nvidia.com/gpu: 2
    network:
      host: true  # optional: implied for multinode services
      ports:
        - name: http
          port: 8080
      endpoints:
        - name: inference
          port-name: http
          protocol: https
          type: subdomain
          scope: group

Key YAML Fields

network.host: Boolean field to enable host networking (implied true for multinode services; default false otherwise)
multinode.nodes: Number of compute nodes to allocate
multinode.implementation: Coordination framework (ompi, openmpi, mpich, gasnet, or generic)
resource: Specifies resources per node (total resources = nodes × per-node resources)
persist: Keeps the service running when nothing depends on it. Without it a service that has no dependents is stopped as soon as it reports ready

Understanding Multinode Network Endpoints

In a multinode service:

Rank 0 is the first node (node 0) and serves as the primary endpoint
Ranks 1, 2, 3, ... are worker nodes that participate in distributed computation
All network ports and endpoints are exposed only on rank 0
Clients connect to rank 0, which coordinates with other ranks internally

For example, in a 4-node distributed inference service:

Rank 0 runs the API server and accepts HTTP requests (rank 0 can also be a worker)
Ranks 1-3 run workers that process inference requests coordinated by rank 0
Clients send requests to rank 0's HTTPS endpoint

Example: vLLM distributed cluster with Host Networking


version: v1

volumes:
  models:
    reference: volume://user/persistent/vllm-cache

services:
  vllm:
    cwd: /home/vllm
    env:
      - MULTINODE_WRAPPER_FORWARD_STREAMS=1
    image:
      uri: >-
        docker://public.ecr.aws/deep-learning-containers/vllm:0.20.0-gpu-py312-cu130-ubuntu22.04-ec2-v1.1-2026-04-29-18-08-36-soci
    mounts:
      data:
        location: /home/vllm
    script: |
      #!/bin/sh
      HOSTNAME=$(hostname)

      vllm serve "openai/gpt-oss-20b" \
      --dtype auto \
      --tool-call-parser openai \
      --reasoning-parser openai_gptoss \
      --enable-auto-tool-choice \
      --tensor-parallel-size 2 \
      --nnodes 2 --node-rank 0 \
      --master-addr ${MULTINODE_NODE_IP} \
      --gpu-memory-utilization 0.9 \
      --kv-cache-dtype auto \
      --max-num-batched-tokens 2048 \
      --max-model-len 131072 \
      --host 0.0.0.0 \
      --port 8080 &

      pid=$!

      IFS=','

      for HOST in ${MULTINODE_HOSTLIST_NOSLOTS}; do
        if [ "$HOSTNAME" != "$HOST" ]; then
          $MULTINODE_SSH_WRAPPER $HOST vllm serve "openai/gpt-oss-20b" \
                                      --dtype auto \
                                      --tool-call-parser openai \
                                      --reasoning-parser openai_gptoss \
                                      --enable-auto-tool-choice \
                                      --tensor-parallel-size 2 \
                                      --nnodes 2 --node-rank 1 \
                                      --master-addr ${MULTINODE_NODE_IP} --headless \
                                      --gpu-memory-utilization 0.9 \
                                      --kv-cache-dtype auto \
                                      --max-num-batched-tokens 2048 \
                                      --max-model-len 131072 \
                                      --host 0.0.0.0 \
                                      --port 8080
        fi
      done

      wait $pid
    resource:
      cpu:
        cores: 16
        affinity: NUMA
      memory:
        size: 120GB
      devices:
        nvidia.com/gpu: 1
      annotations:
        nvidia.com/gpu.model: NVIDIA L4
    multinode:
      nodes: 2
      implementation: generic
    network:
      host: true        # optional: implied for multinode services
      ports:
        - name: openai-api
          port: 8080
          protocol: tcp
      endpoints:
        - name: openai-vllm
          type: subdomain
          scope: public
          protocol: http
          port-name: openai-api
    readiness-probe:
      tcp-socket:
        port: 8080
      period-seconds: 30
      failure-threshold: 60
      success-threshold: 1
      initial-delay-seconds: 60
    persist: true

This example demonstrates:

A 2-node multinode service with 1 GPUs per node (2 GPUs total)
Host networking (automatic for multinode services) providing high-speed node-to-node communication
Rank 0 serves the vLLM API on port 8080
HTTPS endpoint exposed for organization-wide access

Tip

The IFS loop and $MULTINODE_SSH_WRAPPER invocation in the script above can be replaced with the generic-helper functions exec_rank0, exec_rankN, and wait_rank0, which handle these details automatically. See Generic Helper Script for details.