Hi everyone!
My name is Viktor Pashaiev. I’m a systems engineer, and like many of you here, I genuinely love this damn Linux 
In my spare time I enjoy writing down technical postmortems and real-world troubleshooting notes. Hopefully this write-up is useful or interesting to someone. 
I think this write-up will be interesting to anyone who wants to see in practice how the difference between slab allocation and page alignment triggers a BUG_ON() - plus a few other practical things along the way
A few highlights from the investigation:
-
Root cause: kmemdup() allocated resident data from the slab cache at offset 512, causing a 3800-byte write to cross into the next physical page and trip iomap’s BUG_ON() boundary assertion.
-
Backporting trap: upstream purged the legacy NTFS driver and renamed fs/ntfs3/ to fs/ntfs/, breaking automated git cherry-picks.
-
The fix: allocating a dedicated page with alloc_page() to guarantee a zero page offset, and replacing kfree() with put_page().
-
Verification: clean rebuild against 7.0.0-30-generic headers, tested with KASAN and memory leak checks.
When ntfs3 met iomap:
debugging a kernel panic in Ubuntu 26.04
A technical postmortem on LP #2165844: resident NTFS attributes, page boundary invariants, and backporting across kernel driver renames.
A few days ago I was looking through the Ubuntu bug tracker and saw LP #2165844. The setup was Ubuntu 26.04 development kernel (7.0.0-30-generic), an external NTFS drive, and a regular file copy. Then, kernel panic:
kernel BUG at fs/iomap/buffered-io.c:1061!
Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP
CPU: 2 UID: 1000 PID: 3412 Comm: dd Not tainted 7.0.0-30-generic #31-Ubuntu
Call Trace:
iomap_write_end+0x1ac/0x1c0
iomap_file_buffered_write+0x148/0x250
ntfs_file_write_iter+0x1c8/0x410 [ntfs3]
vfs_write+0x240/0x3a0
ksys_write+0x68/0xf0
What caught my eye in the report was the write size: Artem was syncing a collection of small files with rsync, and the panic fired on a write of only a few hundred bytes. An assertion about exceeding PAGE_SIZE on a sub-page write did not make sense at first glance.
1. Reproducing the crash and the first false lead
My initial thought was disk alignment or MFT fragmentation. External NTFS drives formatted under Windows often have odd cluster offsets or fragmented attribute lists that confuse Linux drivers. I pulled the MFT record for the test file with ntfsinfo -f -m /dev/sdb1 and checked the data runs. Nothing unusual - the file was completely resident, single record, no fragmentation. The on-disk layout was completely clean. That meant the bug lived purely in kernel memory.
To isolate the crash cleanly, I set up a minimal test on a scratch partition using dd:
- Enable crash dumps: Verify kdump is ready: sudo apt install linux-crashdump && sudo kdump-config show.
- Prepare test filesystem: sudo mkfs.ntfs -Q /dev/sdb1 && sudo mount -t ntfs3 /dev/sdb1 /mnt/ntfs
- Trigger the panic: Create a small resident file, then rewrite it near the page boundary using conv=notrunc (in-place write):
dd if=/dev/urandom of=/mnt/ntfs/testfile bs=700 count=1
sync
dd if=/dev/urandom of=/mnt/ntfs/testfile bs=3800 count=1 conv=notrunc
Watching dmesg -w in a second terminal shows the panic immediately at fs/iomap/buffered-io.c:1061. Any resident write where offset + length crosses 4096 bytes triggers the assertion.
2. Tracing the failure into ntfs3
Each file in NTFS has a Master File Table (MFT) record, typically 1024 bytes (or up to 4096 bytes depending on cluster configuration). Within that record, metadata and file data are stored as attributes.
For small files, NTFS uses a resident attribute: the file content is stored directly inside the MFT record under the $DATA attribute (like a letter kept inside an envelope instead of renting an external storage locker). Only when the file grows beyond the MFT record capacity does NTFS allocate external disk clusters and convert the attribute to non-resident.
You can verify this state on disk using ntfsprogs:
$ stat -c %i /mnt/ntfs/testfile
42
$ sudo ntfsinfo -f 42 -m /dev/sdb1
Dumping Attribute $DATA (0x80)...
Resident: Yes
Allocated size: 3800 bytes
ntfs3 handles file writes through the VFS iomap subsystem (iomap_file_buffered_write). In Ubuntu’s 7.0 kernel, function attr_data_get_block_locked() in fs/ntfs3/attrib.c prepared resident data using kmemdup():
/* fs/ntfs3/attrib.c:1044 (Ubuntu 7.0 kernel - buggy) */
if (res && data_size) {
*res = kmemdup(resident_data(attr_b), data_size, GFP_KERNEL);
if (!*res)
err = -ENOMEM;
}
Then in fs/ntfs3/inode.c, ntfs_iomap_begin() handed this buffer directly to iomap:
/* fs/ntfs3/inode.c:856 */
iomap->private = iomap->inline_data = res;
iomap->type = IOMAP_INLINE;
iomap->offset = 0;
iomap->length = clen; /* resident size in bytes */
3. Why the buffer crosses the page boundary
Looking at line 1061 of fs/iomap/buffered-io.c in iomap_write_end():
/* fs/iomap/buffered-io.c:1061 */
BUG_ON(offset_in_page(iomap->inline_data) + length > PAGE_SIZE);
That’s the bug, right there.
The assertion is protecting an iomap buffer boundary invariant. When handling inline data (IOMAP_INLINE), iomap requires that all bytes of the write fit within a single 4096-byte memory page. If an inline write crosses into the next physical page, buffered I/O writes past the buffer and corrupts kernel memory.
kmemdup() allocated this buffer from the slab allocator (kmalloc-4k or kmalloc-8k slab caches). Think of a 4096-byte page as a 4-meter garage, and our resident write as 3800 bytes (a 3.8-meter car). The slab returned an address that was 512 bytes into its underlying page:
Offset in page: 512 bytes
Write length: 3800 bytes
Total: 512 + 3800 = 4312 bytes
Limit: PAGE_SIZE (4096 bytes)
Result: 4312 > 4096 -> BUG_ON triggered!
The buffer length was 3800 bytes, but because it started at offset 512, it crossed the page boundary by 216 bytes. iomap caught the violation and halted the kernel.
4. The cherry-pick failure
I ran git cherry-pick 36ee1313199b to pull the upstream maintainer’s fix from kernel 7.1. Git immediately threw an error: the file fs/ntfs/attrib.c did not exist in our 7.0 tree. Upstream had purged the old read-only NTFS driver and renamed fs/ntfs3/ to fs/ntfs/. Because of the directory reorganization, automatic cherry-picking was out. I pulled the commit diff directly with git show 36ee1313199b:fs/ntfs/attrib.c and adapted it for 7.0 by hand.
5. Backporting the fix
To satisfy iomap’s invariant, resident data must be placed in a dedicated, page-aligned buffer. Instead of calling kmemdup(), we allocate a full zeroed page with alloc_page():
--- a/fs/ntfs3/attrib.c
+++ b/fs/ntfs3/attrib.c
@@ -1042,10 +1042,14 @@ int attr_data_get_block_locked(struct ntfs_inode *ni, CLST vcn, CLST clen,
*lcn = RESIDENT_LCN;
*len = data_size;
if (res && data_size) {
- *res = kmemdup(resident_data(attr_b), data_size,
- GFP_KERNEL);
- if (!*res)
+ struct page *page = alloc_page(GFP_NOFS | __GFP_ZERO);
+ if (page) {
+ memcpy(page_address(page), resident_data(attr_b),
+ data_size);
+ *res = page;
+ } else {
err = -ENOMEM;
+ }
}
goto out;
}
In fs/ntfs3/inode.c, we store the struct page * in iomap->private and pass page_address() to iomap->inline_data. Notice that we also must replace kfree() with put_page() in both the error exit and in ntfs_iomap_end():
--- a/fs/ntfs3/inode.c
+++ b/fs/ntfs3/inode.c
@@ -845,7 +845,7 @@ static int ntfs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
if (lcn == RESIDENT_LCN) {
if (offset >= clen) {
- kfree(res);
+ put_page((struct page *)res);
if (flags & IOMAP_REPORT) {
/* special code for report. */
return -ENOENT;
@@ -853,7 +853,8 @@ static int ntfs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
return -EFAULT;
}
- iomap->private = iomap->inline_data = res;
+ iomap->private = res;
+ iomap->inline_data = page_address((struct page *)res);
iomap->type = IOMAP_INLINE;
iomap->offset = 0;
iomap->length = clen; /* resident size in bytes. */
@@ -965,7 +966,7 @@ static int ntfs_iomap_end(struct inode *inode, loff_t pos, loff_t length,
out:
if (iomap->type == IOMAP_INLINE) {
- kfree(iomap->private);
+ put_page((struct page *)iomap->private);
iomap->private = NULL;
}
The fix solves the offset issue directly:
Because page_address(page) always points to the start of a physical page, offset_in_page(iomap->inline_data) is always 0. With offset 0, 0 + 3800 = 3800 <= 4096, and the assertion holds.
6. Testing and a couple of hiccups
Testing had one initial hiccup. My first dd test ran without conv=notrunc and truncated the file to 0 bytes before writing. Because the file was completely recreated, it stayed small and hid the bug. Only when I used conv=notrunc to force an in-place overwrite near the 4096-byte boundary did the bug reproduce reliably.
Also, my first test kernel build crashed during boot because of an incompatible debug config combination with KASAN outline instrumentation on this test container. Once I switched to inline KASAN and rebuilt against the clean 7.0.0-30-generic headers, the test run went smoothly:
-
Module compilation: Built ntfs3.ko against 7.0.0-30-generic headers. Clean build with vermagic matching the running kernel.
-
Reproduction test: Reran the exact crash trigger (3800 bytes conv=notrunc). Write completed with 0 errors in dmesg.
-
Write size sweep: Tested across sizes: 500, 1024, 2048, 3000, 3500, 3800, 3900 bytes. All writes completed without warnings.
-
Memory leak check: Monitored MemFree in /proc/meminfo before and after 50,000 resident writes. Dropped caches with echo 3 > /proc/sys/vm/drop_caches. Free memory returned to baseline, confirming put_page() frees every allocation.
-
Unmount verification: umount /mnt/ntfs returned 0 cleanly without hung tasks or delayed buffer warnings.
-
KASAN testing: Verified with CONFIG_KASAN=y to confirm zero out-of-bounds reads or writes during page copy.
If I had checked git log with rename detection five minutes earlier, I would have saved myself an hour of staring at a failing cherry-pick. Mixing slab-allocated buffers with subsystems that assume full page alignment is an easy trap to fall into in file systems, especially on legacy code paths that handle inline data.
The backported patch has been reviewed, accepted, and committed to the Ubuntu 26.04 Resolute kernel tree (commit 18c7ba7a1e6d). The status on LP #2165844 is now Fix Committed, and the fix has been verified on a clean test VM.
Viktor Pashaiev (aka steelf)
