Proxmox VE Cluster Stability Improvement

We run a Proxmox VE 7.4 cluster with approximately 10 production nodes hosting client VMs. The Problem: Two nodes have been intermittently disconnecting and reconnecting from the cluster (flapping). Each time a node rejoins, it resets the global cluster MTU to 1397 (instead of 1500), which triggers a cascade of failures: cpg_send_message failures Token timeouts (Token has not been received in ~30,000ms) pmxcfs leaving the CPG group Full cluster becoming unresponsive (UI freezes, VMs unreachable from management plane) We have isolated the two problematic nodes as a temporary fix, but we need a permanent solution. Root cause (suspected): Network instability on bond0 interface causing a network loop, confirmed by: vmbr0: received packet on bond0 with own address as source address What we need: A Proxmox/Corosync configuration that prevents one flapping node from destabilizing the entire cluster Recommendations on [login to view URL] tuning (token, MTU handling, etc.) Best practices for bond0 configuration to prevent network loops Ideally a monitoring/alerting script that detects MTU changes and auto-isolates the offending node Important notes: -No direct server access will be provided -Full logs will be shared (journalctl, dmesg, corosync logs) -We can run any diagnostic commands you specify and share output -We can test configuration changes during a maintenance window Skills needed: Proxmox VE, Corosync, Linux networking, bonding/bridging, cluster administration Budget: Open to offers from experienced Proxmox admins only. Key observation (important for diagnosis): When we manually stop and mask corosync on the affected nodes: bashsystemctl stop corosync systemctl mask corosync The entire cluster recovers immediately — UI returns, all other nodes go green, and cpg_send_message errors stop completely. This confirms the issue is isolated to these two nodes and their network behavior, not a general cluster misconfiguration. The moment they are removed from the cluster ring, everything stabilizes. The #1 goal: We need a solution where if any single node becomes unstable or starts flapping, it gets automatically isolated, and go down — WITHOUT affecting the rest of the cluster. Currently, one bad node can bring down all 10 nodes. This is unacceptable in a production environment.

Project ID: 40418430