Installing Lustre Server

Lustre is a high-performance, open-source parallel filesystem designed for large-scale computing environments, such as high-performance computing (HPC) clusters. It separates metadata and data storage for efficiency. This guide focuses on installing Lustre servers for Management Server (MGS), Metadata Server (MDS), and Object Storage Server (OSS) nodes using Lustre version 2.17.0.

Prerequisites: Before starting, ensure you have root access. Lustre requires a compatible kernel, matching Lustre RPMs, and e2fsprogs (an extended filesystem utilities package). Disable SELinux and firewalls to avoid interference (or configure them appropriately). Install DKMS (Dynamic Kernel Module Support) and kernel development packages for building modules. Set up RAID or other storage configurations for your targets (e.g., MGT for management target, MDT for metadata, OST for object storage). Always back up any important data on disks before formatting, as these operations are destructive.

Warnings:

Best Practices:

RHEL 9.7 / 10.1 (and Clones)

This section covers installation on Red Hat Enterprise Linux (RHEL) 9.7 or 10.1 and compatible clones. We'll disable security features, install required repositories and packages, update the kernel, and provide examples for formatting and mounting. Each step includes explanations for beginners.

# Disable firewall (or open port 988 for connection) and SELinux
# Explanation: Firewalld is the default firewall; disabling it prevents blocking Lustre communications. SELinux enforces security policies that may interfere with Lustre modules.
systemctl disable --now firewalld
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
reboot  # Reboot to apply SELinux changes.

# Install EPEL and DKMS
# Explanation: EPEL provides extra packages; DKMS allows dynamic kernel module building for Lustre.
dnf install epel-release dkms kernel-devel-$(uname -r)

# Add e2fsprogs repo
# Explanation: e2fsprogs is needed for filesystem tools like mkfs.lustre. This repo provides the latest version compatible with Lustre.
cat <<EOF > /etc/yum.repos.d/e2fsprogs.repo
[e2fsprogs]
name=e2fsprogs
baseurl=https://downloads.whamcloud.com/public/e2fsprogs/latest/el9/
enabled=1
gpgcheck=0
EOF

# Install e2fsprogs
# Explanation: Installs the extended filesystem utilities.
dnf install e2fsprogs

# Add Lustre server repo (adjust for el10.1)
# Explanation: This repo contains Lustre-specific packages. For RHEL 10.1, change 'el9.7' to 'el10.1' if needed.
cat <<EOF > /etc/yum.repos.d/lustre-server.repo
[lustre-server]
name=Lustre Server
baseurl=https://downloads.whamcloud.com/public/lustre/lustre-2.17.0/el9.7/server/
enabled=1
gpgcheck=0
EOF

# Install server packages and patched kernel
# Explanation: Updates the system and installs Lustre-patched kernel and modules. The patched kernel includes Lustre support.
dnf update
dnf install kernel-*_lustre lustre-server kmod-lustre-osd-ldiskfs lustre-osd-ldiskfs-mount
grubby --set-default /boot/vmlinuz-*_lustre*  # Sets the Lustre kernel as default boot option.
reboot  # Reboot to load the new kernel.

# Load module
# Explanation: Loads the Lustre kernel module to enable functionality.
modprobe lustre

# Format and mount (example for MGT)
# Warning: This will erase all data on /dev/sdX! Replace /dev/sdX with your actual device.
# Explanation: mkfs.lustre formats the device for Lustre use (--mgs for management target). mount.lustre mounts it.
mkfs.lustre --mgs /dev/sdX
mount.lustre /dev/sdX /mnt/mgt

After these steps, verify the installation by running lustre_version or checking dmesg | grep Lustre for load confirmation. If issues arise, check repository URLs for updates or consult Lustre forums.

SLES 15 SP7

This section is for SUSE Linux Enterprise Server (SLES) 15 Service Pack 7. Steps are similar but use zypper instead of dnf. Explanations are provided for each command.

# Disable firewall
# Explanation: SuSEfirewall2 is the default; disabling prevents port blocks.
systemctl disable --now SuSEfirewall2

# Install DKMS/kernel-devel (adapt as needed)
# Explanation: DKMS for module building; kernel-devel for headers matching your kernel.
zypper install dkms kernel-devel

# Add e2fsprogs repo
# Explanation: Provides compatible e2fsprogs for Lustre filesystems.
cat <<EOF > /etc/zypp/repos.d/e2fsprogs.repo
[e2fsprogs]
name=e2fsprogs
baseurl=https://downloads.whamcloud.com/public/e2fsprogs/latest/sles15sp7/
enabled=1
gpgcheck=0
EOF

# Install e2fsprogs
# Explanation: Installs filesystem utilities.
zypper install e2fsprogs

# Add Lustre server repo
# Explanation: Source for Lustre packages. Use 'latest-feature-release' if needed for updates.
cat <<EOF > /etc/zypp/repos.d/lustre-server.repo
[lustre-server]
name=Lustre Server
baseurl=https://downloads.whamcloud.com/public/lustre/latest-feature-release/sles15sp7/server/
enabled=1
gpgcheck=0
EOF

# Install server packages and patched kernel
# Explanation: Refreshes repos and installs Lustre components.
zypper refresh
zypper install kernel-*_lustre lustre-server kmod-lustre-osd-ldiskfs lustre-osd-ldiskfs-mount
# Set default kernel (use update-bootloader or similar)
# Explanation: Use 'update-bootloader --refresh' or manually edit /etc/default/grub and run grub2-mkconfig.
reboot  # Reboot to apply changes.

# Load module
# Explanation: Enables Lustre in the kernel.
modprobe lustre

# Format and mount (example for MGT)
# Warning: This erases data on /dev/sdX! Use the correct device.
# Explanation: Formats and mounts the management target.
mkfs.lustre --mgs /dev/sdX
mount.lustre /dev/sdX /mnt/mgt

Post-installation, configure your MGS, MDT, and OST as detailed in the Lustre Operations Manual. Use lctl list_nids to verify network interfaces. For troubleshooting, enable debug logging with lctl set_param debug=+trace.

Additional Tips

For multi-node setups, ensure consistent versions across nodes. Consider using ZFS backend for OSTs if ldiskfs isn't sufficient (requires additional modules). Join the Lustre community mailing list for support. Always test failover and performance tuning (e.g., stripe sizes) before production use.