Self-Extending Layout (SEL)
Self-Extending Layout (SEL) is an extension of Progressive File Layout (PFL) introduced in Lustre 2.13 that allows the Metadata Server (MDS) to dynamically modify and extend the defined PFL layout as files grow, handling scenarios like OST out-of-space or tiered storage spillover. It enables automatic repetition or spillover of the last component, making it ideal for files with unknown final sizes or in hybrid storage environments (e.g., SSD to HDD). This guide is based on the Lustre Operations Manual (updated 2025) for Lustre 2.17.0 (January 2026). For more, see Lustre Manual.
Core Concepts
| Concept | Description |
|---|
| Dynamic Extension | MDS extends the layout by repeating the last component or spilling over to a new tier/pool when the file grows beyond defined extents or encounters space issues. |
| Extension Size (-z) | Specifies the increment size for each extension (e.g., 64M). Multiple of stripe size; min 64K. |
| Spillover | Automatically moves to a slower tier (e.g., HDD) when fast tier (SSD) is full, using different pools per component. |
| Repeating Components | Last component repeats indefinitely if -E eof and -z set, or until a max size. |
| Instantiation | Components instantiate on write; SEL triggers extension via Layout Intent RPC. |
| Integration with FLR | Interoperates with File Level Redundancy; extensions apply to all mirrors. |
| Backend Support | Requires ldiskfs with ea_inode or ZFS. ZFS uses dynamic inodes for large stripes. |
| Client Compatibility | Clients ≥2.13 required for SEL creation/use; older clients may access but not extend. |
Commands
| Command | Purpose | Key Options |
|---|
| lfs setstripe | Set SEL with extension size; integrated with PFL syntax. | -z <size> (extension size), -E <end> (component end), combined with -c, -S, -p |
| lfs getstripe | View SEL details, including extension sizes and repeated components. | -v (verbose) |
| lfs migrate | Migrate files to/from SEL layouts. | Same as setstripe with -z |
| lfs mirror create | Create mirrored SEL files. | -N, combined with -z per mirror |
| lfs mirror resync | Resync after SEL extensions. | --only <id> |
| lctl set_param lod.*.sel_enable=1 | Enable SEL globally on MDS. | Toggle for dynamic extension |
| lctl lfsck_start -t layout | Repair SEL inconsistencies. | -A, -o |
Examples
# Basic SEL with Repeating Last Component
lfs setstripe -E 1G -c 1 -z 64M -E -1 -c 4 -z 256M /mnt/lustre/sel_file
# Tiered Spillover (SSD to HDD)
lfs setstripe -E 4G -c 2 -p ssd_pool -z 1G -E -1 -c -1 -p hdd_pool -z 4G /mnt/lustre/tiered_file
# Directory Default SEL
lfs setstripe -E 256M -c 1 -z 128M -E eof -c -1 -z 1G /mnt/lustre/sel_dir
# Mirrored SEL
lfs mirror create -N -E 1G -c 1 -z 64M -E -1 -c 4 -z 256M -N -c -1 -p archive /mnt/lustre/mirrored_sel
# View SEL
lfs getstripe -v /mnt/lustre/sel_file
Best Practices
- Set Extension Size: Choose -z based on growth patterns (small for frequent extensions, large for efficiency).
- Tiered Storage: Use pools for fast/slow tiers; ensure spillover components have wider striping.
- Enable Globally: Set lod.*.sel_enable=1 on MDS for automatic handling.
- Monitor Extensions: Use lfs getstripe to check instantiated components post-write.
- Combine with FLR: For redundancy in dynamic layouts.
- Avoid Fixed Ends: Use -E -1 for the last component to enable indefinite extension.
- LFSCK Regularly: After OST additions/failures to repair layouts.
- Test Writes: Trigger extensions with appends to verify behavior.
Limitations
- Forward-Only: Extends only; cannot shrink or revert without migration.
- Write-Triggered: Extensions happen on write; pre-allocation not supported.
- Max Components: Limited to 2000 (ea_inode); impacts long extensions.
- Client Version: ≥2.13 required; older clients cannot extend SEL files.
- Backend: Full support on ldiskfs + ea_inode; ZFS has partial (dynamic inodes but no full SEL).
- Mirrors: Extensions require resync; no auto-extension across mirrors.
- Space Issues: If no space for extension, write fails with ENOSPC.
- Directory: SEL on dirs propagates but extensions are file-specific.
Recent Changes (Up to Lustre 2.17.0)
- 2.13: SEL introduced: dynamic extension with -z, spillover for tiering.
- 2.14: Improved diagnostics in lfs getstripe; pool quotas integration.
- 2.15: Enhanced error handling for out-of-space during extension.
- 2.16: Better interoperability with FLR resync post-extension.
- 2.17: Minor optimizations; no major SEL changes.
Related Tools & Diagnostics
- lfs getstripe: Inspect SEL extensions, sizes, and status.
- lctl get_param lod.*.sel_enable: Check/toggle SEL.
- lctl lfsck_start -t layout: Repair after extensions or failures.
- Procfs: mdd.*.lfsck_layout for scan progress.
- lfs df: Monitor space before/after extensions.