MarkBase FUSE Phase 2 Implementation Status

Current Status (2026-05-17)

Completed

FUSE-T Installation ✅
- Binary: /Library/Application Support/fuse-t/bin/go-nfsv4-1.2.6 (20MB)
- Backend support: nfs, fskit, smb
- Installation verified via sudo installer
FileSystem Trait Implementation ✅
- File: src/fuse/markbase_fs.rs (370 lines)
- Implemented methods:
  - init() - Initialize filesystem
  - lookup() - Find file by name in parent directory
  - getattr() - Get file attributes (stat64)
  - opendir() - Open directory for reading
  - readdir() - List directory contents
  - releasedir() - Close directory
  - open() - Open file for reading
  - read() - Read file content (using ZeroCopyWriter)
  - release() - Close file
  - statfs() - Filesystem statistics
SQLite Backend Integration ✅
- Database: data/users/warren.sqlite (12MB, 12659 nodes)
- Tables used:
  - file_nodes - node_id, label, node_type, file_size, parent_id
  - file_locations - file_uuid, location
- UUID-to-Inode mapping: First 8 bytes of UUID → u64 inode
Compilation Success ✅
- 11 FUSE tests passed
- Zero warnings in fuse module
- Type conversions fixed (rusqlite::Error → io::Error)

Next Steps (Phase 2)

Step 1: Real Mount Implementation

Current Placeholder:

pub fn mount(&self, mount_path: &Path) -> Result<()> {
    println!("=== Mounting MarkBase FUSE ===");
    println!("User: {}", self.user_id);
    println!("Database: {}", self.db_path.display());
    println!("Backend: {}", self.backend.name());
    println!("Mount path: {}", mount_path.display());
    
    Ok(())
}

Required Implementation:

use fuse_backend_rs::transport::{FuseSession, FuseChannel};
use fuse_backend_rs::api::server::Server;

pub fn mount(&self, mount_path: &Path) -> Result<()> {
    // 1. Create FUSE session
    let session = FuseSession::new(mount_path, "MarkBase", "", false)?;
    
    // 2. Mount filesystem
    session.mount()?;
    
    // 3. Create server with filesystem
    let server = Server::new(Arc::new(self.clone()));
    
    // 4. Spawn thread to handle requests
    thread::spawn(|| {
        let channel = session.new_channel()?;
        server.svc_loop(channel)?;
        Ok(())
    });
    
    Ok(())
}

Step 2: Backend Selection

go-nfsv4 Binary Usage:

# FSKit backend (macOS 26+, fastest)
/Library/Application Support/fuse-t/bin/go-nfsv4 \
  --backend fskit \
  --volname "MarkBase_warren" \
  --rwsize 65536 \
  --attrcache \
  --noatime

# NFSv4 backend (all macOS versions)
/Library/Application Support/fuse-t/bin/go-nfsv4 \
  --backend nfs \
  --listen_addr 127.0.0.1 \
  --volname "MarkBase_warren"

Backend Auto-Selection Logic:

macOS 26+ → FSKit (direct path, ~650MB/s)
macOS <26 → NFSv4 (TCP/IP, ~550MB/s)

Step 3: Warren User Test (12659 nodes)

Test Plan:

Mount warren filesystem:

cargo run -- fuse --mount --user warren --dir /Volumes/MarkBase_warren

Verify mount:

ls -la /Volumes/MarkBase_warren/
ls -la /Volumes/MarkBase_warren/Home/

Count nodes:

find /Volumes/MarkBase_warren -type f | wc -l   # Expected: 11857 files
find /Volumes/MarkBase_warren -type d | wc -l   # Expected: 802 folders

Read test:

cat /Volumes/MarkBase_warren/Home/.../test_file.txt

Step 4: Performance Validation

AJA System Test (600MB/s target):

Download AJA System Test: https://www.aja.com/en/products/aja-system-test

Install:

hdiutil attach ~/Downloads/AJA_System_Test.dmg
cp -R /Volumes/AJA\ System\ Test/*.app /Applications/
hdiutil detach /Volumes/AJA\ System\ Test

Run 4K ProRes 4444 test:
- Target: 600MB/s sustained write
- Test file: /Volumes/MarkBase_warren/test_4k_prores.mov
- Duration: 60 seconds

Implementation Notes

Challenges:

ZeroCopyWriter Integration - read() method uses complex writer trait
- Current implementation: w.write_all(&buffer[..bytes_read])
- Optimization: Direct file copy to writer without intermediate buffer
UUID Padding - ino_to_uuid() truncates to 8 bytes
- Issue: Full UUID is 32 chars, inode only stores 8 bytes
- Solution: Use first 8 bytes for lookup, fallback to aliases_json
Thread Safety - MarkBaseFs needs Arc for concurrent access
- Current: No locking (single-threaded placeholder)
- Required: Arc<Mutex> for multi-threaded FUSE server
Backend Binary Path - go-nfsv4 is symlink
- Actual: /Library/Application Support/fuse-t/bin/go-nfsv4-1.2.6
- Symlink: /Library/Application Support/fuse-t/bin/go-nfsv4

Key Learnings:

FUSE-T uses go-nfsv4 as unified binary (NFSv4/FSKit/SMB3)
fuse-backend-rs provides FuseSession for mount management
ZeroCopyWriter is required for read/write operations
stat64 is libc::stat on macOS (not fuse_abi::stat64)

Estimated Timeline

Day 1-2: Mount implementation + backend integration
Day 3-4: Warren user test + node verification
Day 5-7: AJA System Test + performance optimization
Day 8-10: Multi-user concurrent mount (10 users)
Day 11-12: Final validation + documentation

Last Updated: 2026-05-17 11:00 Phase: 2 - Real Mount Implementation Status: FileSystem trait ✅, Mount function ⏳

5.4 KiB Raw Permalink Blame History