MarkBase架构升级:Multi-Volume Virtual Tree + Dual-View Management + Git Remote修正
核心功能: - ✅ Categories/Series双视图管理(category_view.rs + import_markdown.rs) - ✅ FUSE Multi-Volume支持(tree_type参数) - ✅ SSH/SFTP/SCP/rsync协议完整实现(4042行) - ✅ NFS/SMB Module Phase 1-3完成 - ✅ Archive Module Phase 1-4完成(2916行) - ✅ Download Center API完整实现 - ✅ S3兼容API实现(560行) Git配置修正: - ✅ 删除错误origin(gitea.momentry.ddns.net) - ✅ 删除m5max128(指向机器名) - ✅ 设置origin = m5max128gitea.momentry.ddns.net/admin/markbase - ✅ 设置m4minigitea = m4minigitea.momentry.ddns.net/warren/markbase 数据清理: - ✅ 删除38个临时SQLite(保留accusys.sqlite、demo.sqlite) - ✅ 删除.bak、test_*.bin、调试脚本等临时文件 - ✅ 删除临时目录(build/、download files/、raid_test/等) - ✅ 更新.gitignore排除临时文件 架构优化: - 52个文件修改,2434行新增,4739行删除 - Workspace成员整合(16个crate) - 数据库状态:accusys.sqlite保留(主demo测试) 远程同步: - ✅ 准备推送到m5max128gitea(远程Gitea) - ✅ 准备推送到m4minigitea(本地Gitea)
This commit is contained in:
936
docs/fskit-research/ZFS_MVP_ARCHITECTURE.md
Normal file
936
docs/fskit-research/ZFS_MVP_ARCHITECTURE.md
Normal file
@@ -0,0 +1,936 @@
|
||||
# OpenZFS架构分析 ⭐⭐⭐⭐⭐
|
||||
|
||||
## 概述
|
||||
|
||||
本文档分析OpenZFS完整架构,为MarkBase ZFS MVP实现提供技术基础。
|
||||
|
||||
**关键发现**:
|
||||
- ZPL Layer是FSKit集成点
|
||||
- libzfs提供所有MVP operations
|
||||
- macOS已有OpenZFS kernel extension
|
||||
- MVP工作量:700行(2-3 weeks)
|
||||
- **跳过kernel porting,直接使用libzfs** ⭐⭐⭐⭐⭐
|
||||
|
||||
---
|
||||
|
||||
## 1. Kernel Module Architecture
|
||||
|
||||
```
|
||||
OpenZFS Kernel Module Stack
|
||||
│
|
||||
├── spl.ko (Solaris Portability Layer)
|
||||
│ ├── Kernel abstraction layer
|
||||
│ ├── Solaris API compatibility
|
||||
│ ├── Memory management (kmem)
|
||||
│ ├── Thread management
|
||||
│ └── Mutexes/condition variables
|
||||
│
|
||||
├── zfs.ko (ZFS Core Module)
|
||||
│ ├── All ZFS functionality
|
||||
│ ├── Pool management
|
||||
│ ├── File system operations
|
||||
│ ├── Data management
|
||||
│ └── Compression/encryption
|
||||
│
|
||||
└── Supporting Modules
|
||||
├── zcommon.ko (Common utilities)
|
||||
├── icp.ko (Integrity Checking)
|
||||
├── zstd.ko (ZSTD compression)
|
||||
├── avl.ko (AVL tree)
|
||||
├── nvpair.ko (Name-value pairs)
|
||||
└── lua.ko (Lua scripting)
|
||||
```
|
||||
|
||||
**说明**:
|
||||
- **spl.ko**: Solaris Portability Layer,提供Solaris API兼容性
|
||||
- **zfs.ko**: ZFS核心模块,包含所有ZFS功能
|
||||
- **macOS现状**: org.openzfsonosx.zfs.kext已存在,可直接使用
|
||||
|
||||
---
|
||||
|
||||
## 2. ZFS Layer Architecture ⭐⭐⭐⭐⭐
|
||||
|
||||
```
|
||||
Application Layer
|
||||
↓
|
||||
VFS Layer (Virtual File System)
|
||||
↓
|
||||
ZPL - ZFS POSIX Layer ⭐⭐⭐⭐⭐ (FSKit Integration Point)
|
||||
│ ├── zfs_vfsops.c (VFS operations)
|
||||
│ │ └── Mount/unmount dataset
|
||||
│ │
|
||||
│ ├── zfs_vnops.c (Vnode operations)
|
||||
│ │ ├── File read/write
|
||||
│ │ ├── Directory operations
|
||||
│ │ ├── Lookup/create/remove
|
||||
│ │ └── Attribute operations
|
||||
│ │
|
||||
│ └── Key Functions → FSKit Operations:
|
||||
│ ├── zfs_mount() → FSVolume.activate
|
||||
│ ├── zfs_lookup() → FSVolume.lookupItem
|
||||
│ ├── zfs_readdir() → FSVolume.enumerateDirectory
|
||||
│ ├── zfs_read() → FSVolume.read
|
||||
│ ├── zfs_write() → FSVolume.write
|
||||
│ ├── zfs_create() → FSVolume.createItem
|
||||
│ └── zfs_getattr() → FSVolume.attributes
|
||||
│ ↓
|
||||
DSL - Dataset/Snapshot Layer ⭐⭐⭐⭐
|
||||
│ ├── dsl_pool.c (Pool management)
|
||||
│ ├── dsl_dataset.c (Dataset operations)
|
||||
│ ├── dsl_dir.c (Directory hierarchy)
|
||||
│ ├── dsl_destroy.c (Destroy operations)
|
||||
│ └── dsl_prop.c (Property management)
|
||||
│ ↓
|
||||
DMU - Data Management Unit ⭐⭐⭐⭐⭐
|
||||
│ ├── dmu.c (Object management)
|
||||
│ ├── dbuf.c (Buffer management)
|
||||
│ ├── dnode.c (Data nodes)
|
||||
│ ├── dmu_tx.c (Transactions)
|
||||
│ └── dmu_objset.c (Object sets)
|
||||
│ ↓
|
||||
ZIL - ZFS Intent Log ⭐⭐⭐⭐
|
||||
│ ├── zil.c (Intent log core)
|
||||
│ ├── Synchronous write logging
|
||||
│ └── Crash recovery
|
||||
│ ↓
|
||||
ARC - Adaptive Replacement Cache ⭐⭐⭐⭐⭐
|
||||
│ ├── arc.c (Memory cache)
|
||||
│ ├── LRU/LFU lists
|
||||
│ ├── Ghost lists
|
||||
│ └── L2ARC (SSD cache)
|
||||
│ ↓
|
||||
SPA - Storage Pool Allocator ⭐⭐⭐⭐⭐
|
||||
│ ├── spa.c (Pool management)
|
||||
│ ├── Pool import/export ⭐⭐⭐⭐⭐ (MVP critical)
|
||||
│ ├── Pool creation/destroy
|
||||
│ └── Pool health monitoring
|
||||
│ ↓
|
||||
VDEV - Virtual Device Layer ⭐⭐⭐⭐⭐
|
||||
│ ├── vdev.c (Device management)
|
||||
│ ├── vdev_label.c (Device labels)
|
||||
│ ├── vdev_queue.c (I/O scheduling)
|
||||
│ │
|
||||
│ ├── Leaf Vdevs:
|
||||
│ │ ├── vdev_disk.c (Physical disk)
|
||||
│ │ ├── vdev_file.c (File backend)
|
||||
│ │
|
||||
│ └── Top-Level Vdevs (Redundancy):
|
||||
│ │ ├── vdev_mirror.c (RAID 1)
|
||||
│ │ ├── vdev_raidz.c (RAIDZ1/2/3)
|
||||
│ │ └── vdev_draid.c (Distributed RAID)
|
||||
│ ↓
|
||||
Physical Storage (HDD/SSD/NVMe)
|
||||
```
|
||||
|
||||
### Layer职责说明
|
||||
|
||||
**ZPL (ZFS POSIX Layer)** ⭐⭐⭐⭐⭐:
|
||||
- FSKit集成点
|
||||
- 提供POSIX语义
|
||||
- 实现VFS接口
|
||||
- 关键文件:zfs_vfsops.c, zfs_vnops.c
|
||||
|
||||
**DSL (Dataset Layer)** ⭐⭐⭐⭐:
|
||||
- Dataset管理
|
||||
- Snapshot管理
|
||||
- 属性管理
|
||||
- 关键文件:dsl_dataset.c, dsl_pool.c
|
||||
|
||||
**DMU (Data Management Unit)** ⭐⭐⭐⭐⭐:
|
||||
- 数据对象管理
|
||||
- Buffer管理
|
||||
- 事务管理
|
||||
- 关键文件:dmu.c, dbuf.c
|
||||
|
||||
**ZIL (ZFS Intent Log)** ⭐⭐⭐⭐:
|
||||
- 意图日志
|
||||
- 同步写入保证
|
||||
- 崩溃恢复
|
||||
- 关键文件:zil.c
|
||||
|
||||
**ARC (Adaptive Replacement Cache)** ⭐⭐⭐⭐⭐:
|
||||
- 内存缓存
|
||||
- LRU/LFU算法
|
||||
- L2ARC SSD缓存
|
||||
- 关键文件:arc.c
|
||||
|
||||
**SPA (Storage Pool Allocator)** ⭐⭐⭐⭐⭐:
|
||||
- Pool管理(MVP关键)
|
||||
- Pool import/export
|
||||
- 健康监控
|
||||
- 关键文件:spa.c
|
||||
|
||||
**VDEV (Virtual Device)** ⭐⭐⭐⭐⭐:
|
||||
- 设备抽象
|
||||
- 冗余管理
|
||||
- I/O调度
|
||||
- 关键文件:vdev.c, vdev_*.c
|
||||
|
||||
---
|
||||
|
||||
## 3. MVP Architecture ⭐⭐⭐⭐⭐
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ macOS Application Layer │
|
||||
│ └── Normal file operations │
|
||||
└─────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ FSKit Layer (Swift) │
|
||||
│ ├── FSUnaryFileSystem │
|
||||
│ │ ├── probeResource() → Detect ZFS pool │
|
||||
│ │ ├── loadResource() → Import dataset │
|
||||
│ │ └── unloadResource() → Export pool │
|
||||
│ │ │
|
||||
│ ├── ZFSVolume (FSVolume.Operations) ⭐⭐⭐⭐⭐ │
|
||||
│ │ ├── activate() → Mount dataset │
|
||||
│ │ ├── lookupItem() → Find file │
|
||||
│ │ ├── enumerateDirectory() → List files │
|
||||
│ │ ├── attributes() → Get metadata │
|
||||
│ │ ├── createItem() → Create file/dir │
|
||||
│ │ ├── removeItem() → Delete file/dir │
|
||||
│ │ ├── open/close/read/write → I/O ops │
|
||||
│ │ │
|
||||
│ └── ZFSItem (FSItem) │
|
||||
│ ├── name, id, attributes, type │
|
||||
└─────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ libzfs Wrapper (C + Swift FFI) │
|
||||
│ ├── libzfs_wrapper.c (约200行) ⭐⭐⭐⭐⭐ │
|
||||
│ │ ├── zpool_import_wrapper() │
|
||||
│ │ ├── zfs_open_wrapper() │
|
||||
│ │ ├── zfs_read_wrapper() │
|
||||
│ │ ├── zfs_write_wrapper() │
|
||||
│ │ ├── zfs_list_wrapper() │
|
||||
│ │ ├── zfs_lookup_wrapper() │
|
||||
│ │ ├── zfs_create_wrapper() │
|
||||
│ │ └── zfs_getattr_wrapper() │
|
||||
│ │ │
|
||||
│ └── Swift FFI (Swift ↔ C interop) │
|
||||
└─────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ libzfs + libspl (C Libraries) │
|
||||
│ ├── libzfs.dylib │
|
||||
│ │ ├── libzfs_dataset.c │
|
||||
│ │ ├── libzfs_pool.c │
|
||||
│ │ └── libzfs_util.c │
|
||||
│ │ │
|
||||
│ └── libspl.dylib │
|
||||
│ ├── Platform abstraction │
|
||||
│ ├── Memory management │
|
||||
│ └── Threading support │
|
||||
└─────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ OpenZFS Kernel Extension │
|
||||
│ ├── org.openzfsonosx.zfs.kext ⭐⭐⭐⭐⭐ │
|
||||
│ │ ├── spl.kext (Portability) │
|
||||
│ │ ├── zfs.kext (ZFS Core) │
|
||||
│ │ ├── Already installed on macOS │
|
||||
│ │ ├── Pool import/export │
|
||||
│ │ ├── Dataset mount/unmount │
|
||||
│ │ ├── File read/write │
|
||||
│ │ ├── Directory listing │
|
||||
│ │ ├── ARC cache │
|
||||
│ │ ├── ZIL logging │
|
||||
│ │ ├── SPA pool management │
|
||||
│ │ └── VDEV device management │
|
||||
└─────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Physical Storage │
|
||||
│ ├── HDDs, SSDs, NVMe drives │
|
||||
│ └── Existing ZFS pools (User-created) │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### MVP实现策略 ⭐⭐⭐⭐⭐
|
||||
|
||||
**关键发现**:
|
||||
1. **跳过Kernel Porting** ⭐⭐⭐⭐⭐
|
||||
- macOS已有OpenZFS kext(org.openzfsonosx.zfs.kext)
|
||||
- 不需要重新实现kernel模块
|
||||
- 直接使用现有kernel extension
|
||||
|
||||
2. **FSKit Layer集成** ⭐⭐⭐⭐⭐
|
||||
- 只需实现FSVolume.Operations(13个方法)
|
||||
- Swift代码量:约290行
|
||||
- C wrapper代码量:约200行
|
||||
|
||||
3. **libzfs Wrapper** ⭐⭐⭐⭐⭐
|
||||
- 封装libzfs C API
|
||||
- 提供Swift FFI接口
|
||||
- 工作量:约200行C代码
|
||||
|
||||
**总工作量**:
|
||||
- Swift代码:约290行
|
||||
- C wrapper:约200行
|
||||
- 测试和文档:约100行
|
||||
- **总计:约700行代码**
|
||||
- **时间:2-3 weeks** ⭐⭐⭐⭐⭐
|
||||
|
||||
---
|
||||
|
||||
## 4. Data Flow - Read Path ⭐⭐⭐⭐⭐
|
||||
|
||||
```
|
||||
Application: read(fd, buffer, size)
|
||||
↓
|
||||
VFS Layer: vfs_read()
|
||||
↓
|
||||
ZPL Layer: zfs_read() → FSKit.read()
|
||||
↓
|
||||
DMU Layer: dmu_read()
|
||||
↓
|
||||
ARC Layer: arc_read() ← Cache Hit?
|
||||
↓ (Cache Miss)
|
||||
SPA Layer: spa_read()
|
||||
↓
|
||||
VDEV Layer: vdev_read()
|
||||
├── Mirror: read from one disk
|
||||
├── RAIDZ: read + parity reconstruction
|
||||
└── Stripe: direct read
|
||||
↓
|
||||
Leaf Vdev: vdev_disk_read()
|
||||
↓
|
||||
Kernel Block Layer
|
||||
↓
|
||||
Physical Disk: Read sectors
|
||||
↓ (Data flows back)
|
||||
ARC (cached) → DMU → ZPL → Application buffer
|
||||
```
|
||||
|
||||
### Read Path性能优化
|
||||
|
||||
**ARC缓存** ⭐⭐⭐⭐⭐:
|
||||
- LRU(Least Recently Used)
|
||||
- LFU(Least Frequently Used)
|
||||
- Ghost lists(预测缓存)
|
||||
- L2ARC(SSD二级缓存)
|
||||
|
||||
**MVP优化重点**:
|
||||
1. 依赖ARC自动缓存
|
||||
2. 不需要手动实现缓存
|
||||
3. libzfs自动处理缓存逻辑
|
||||
|
||||
---
|
||||
|
||||
## 5. Data Flow - Write Path ⭐⭐⭐⭐
|
||||
|
||||
```
|
||||
Application: write(fd, buffer, size)
|
||||
↓
|
||||
VFS Layer: vfs_write()
|
||||
↓
|
||||
ZPL Layer: zfs_write() → FSKit.write()
|
||||
↓
|
||||
DMU Layer: dmu_write()
|
||||
├── Create transaction
|
||||
├── Allocate blocks
|
||||
└── Fill data
|
||||
↓
|
||||
ZIL Layer: zil_commit() (if sync write)
|
||||
├── Log intent
|
||||
└── Guarantee durability
|
||||
↓
|
||||
ARC Layer: arc_write()
|
||||
├── Cache data
|
||||
└── Mark dirty
|
||||
↓
|
||||
SPA Layer: spa_write()
|
||||
├── Allocate space
|
||||
└── Schedule I/O
|
||||
↓
|
||||
VDEV Layer: vdev_write()
|
||||
├── Mirror: write to all disks
|
||||
├── RAIDZ: write + parity
|
||||
└── Stripe: direct write
|
||||
↓
|
||||
Leaf Vdev: vdev_disk_write()
|
||||
↓
|
||||
Kernel Block Layer
|
||||
↓
|
||||
Physical Disk: Write sectors
|
||||
```
|
||||
|
||||
### Write Path关键点
|
||||
|
||||
**ZIL(Intent Log)** ⭐⭐⭐⭐:
|
||||
- 同步写入保证
|
||||
- 崩溃恢复
|
||||
- 性能权衡(可配置)
|
||||
|
||||
**DMU Transaction** ⭐⭐⭐⭐⭐:
|
||||
- 原子性保证
|
||||
- Copy-on-Write
|
||||
- 数据一致性
|
||||
|
||||
**MVP简化**:
|
||||
1. 依赖ZIL自动处理
|
||||
2. 不需要手动实现transaction
|
||||
3. libzfs自动保证一致性
|
||||
|
||||
---
|
||||
|
||||
## 6. Data Flow - Snapshot Path ⭐⭐⭐
|
||||
|
||||
```
|
||||
Application: zfs snapshot pool/dataset@snap1
|
||||
↓
|
||||
ZFS CLI: zfs_snapshot()
|
||||
↓
|
||||
DSL Layer: dsl_dataset_snapshot()
|
||||
├── Create snapshot dataset
|
||||
├── Copy metadata
|
||||
└── No data copy (COW)
|
||||
↓
|
||||
DMU Layer: dmu_objset_snapshot()
|
||||
├── Snapshot objset
|
||||
└── Reference existing blocks
|
||||
↓
|
||||
SPA Layer: spa_snapshot()
|
||||
├── Update pool metadata
|
||||
└── No space allocation
|
||||
```
|
||||
|
||||
### Snapshot特性 ⭐⭐⭐⭐⭐
|
||||
|
||||
**Copy-on-Write (COW)**:
|
||||
- 快照不占用额外空间
|
||||
- 只在数据修改时分配新块
|
||||
- 快照创建瞬间完成
|
||||
|
||||
**MVP策略**:
|
||||
- **延后实现**(Level 5)
|
||||
- 需要约300行代码
|
||||
- 时间:约2-3 days
|
||||
|
||||
---
|
||||
|
||||
## 7. MVP Operations映射 ⭐⭐⭐⭐⭐
|
||||
|
||||
### FSKit → libzfs → Kernel 映射表
|
||||
|
||||
```
|
||||
FSKit Operation libzfs Function Kernel Layer
|
||||
─────────────────────────────────────────────────────────────
|
||||
FSVolume.activate() → zfs_mount() → ZPL: zfs_mount()
|
||||
FSVolume.lookupItem() → zfs_lookup() → ZPL: zfs_lookup()
|
||||
FSVolume.enumerateDir → zfs_readdir() → ZPL: zfs_readdir()
|
||||
FSVolume.read() → zfs_read() → ZPL/DMU: dmu_read()
|
||||
FSVolume.write() → zfs_write() → ZPL/DMU: dmu_write()
|
||||
FSVolume.createItem() → zfs_create() → ZPL: zfs_create()
|
||||
FSVolume.removeItem() → zfs_remove() → ZPL: zfs_remove()
|
||||
FSVolume.attributes() → zfs_getattr() → ZPL: zfs_getattr()
|
||||
FSVolume.open() → zfs_open() → Dataset handle
|
||||
FSVolume.close() → zfs_close() → Handle cleanup
|
||||
FSVolume.deactivate() → zfs_unmount() → ZPL: zfs_unmount()
|
||||
```
|
||||
|
||||
### libzfs API映射详细说明
|
||||
|
||||
**1. Pool Operations**:
|
||||
```c
|
||||
// Pool import (MVP关键)
|
||||
int zpool_import(libzfs_handle_t *, nvlist_t *, const char *, const char *);
|
||||
|
||||
// Pool export
|
||||
int zpool_export(zpool_handle_t *, boolean_t, const char *);
|
||||
|
||||
// Pool list
|
||||
zpool_list_t * zpool_list_import(libzfs_handle_t *, char **);
|
||||
```
|
||||
|
||||
**2. Dataset Operations**:
|
||||
```c
|
||||
// Dataset open
|
||||
zfs_handle_t * zfs_open(libzfs_handle_t *, const char *, int);
|
||||
|
||||
// Dataset close
|
||||
void zfs_close(zfs_handle_t *);
|
||||
|
||||
// Dataset list
|
||||
int zfs_iter_filesystems(zfs_handle_t *, zfs_iter_f, void *);
|
||||
```
|
||||
|
||||
**3. File Operations**:
|
||||
```c
|
||||
// Read file
|
||||
int zfs_read(zfs_handle_t *, void *, uint64_t, uint64_t *);
|
||||
|
||||
// Write file
|
||||
int zfs_write(zfs_handle_t *, const void *, uint64_t, uint64_t *);
|
||||
|
||||
// Create file
|
||||
int zfs_create(zfs_handle_t *, const char *, zfs_type_t, nvlist_t *);
|
||||
|
||||
// Delete file
|
||||
int zfs_destroy(zfs_handle_t *, boolean_t);
|
||||
```
|
||||
|
||||
**4. Attribute Operations**:
|
||||
```c
|
||||
// Get attributes
|
||||
int zfs_prop_get(zfs_handle_t *, zfs_prop_t, char *, size_t,
|
||||
zprop_source_t *, char *, size_t, boolean_t);
|
||||
|
||||
// Set attributes
|
||||
int zfs_prop_set(zfs_handle_t *, const char *, const char *);
|
||||
```
|
||||
|
||||
### MVP实现优先级 ⭐⭐⭐⭐⭐
|
||||
|
||||
**Level 1 - 立即实现(约150行)**:
|
||||
1. ✅ zpool_import - Pool导入
|
||||
2. ✅ zfs_mount - Dataset挂载
|
||||
3. ✅ zfs_lookup - 文件查找
|
||||
4. ✅ zfs_readdir - 目录列举
|
||||
5. ✅ zfs_read - 文件读取
|
||||
6. ✅ zfs_getattr - 属性获取
|
||||
|
||||
**Level 2 - 次要实现(约80行)**:
|
||||
7. ✅ zfs_create - 文件创建
|
||||
8. ✅ zfs_write - 文件写入
|
||||
9. ✅ zfs_remove - 文件删除
|
||||
|
||||
**Level 3 - 延后实现(约60行)**:
|
||||
10. ⏸️ zfs_setattr - 属性设置
|
||||
11. ⏸️ zfs_snapshot - 快照创建
|
||||
12. ⏸️ zfs_rollback - 快照回滚
|
||||
|
||||
**总计**:13个operations,约290行Swift + 200行C wrapper
|
||||
|
||||
---
|
||||
|
||||
## 8. MVP vs 完整ZFS对比 ⭐⭐⭐⭐⭐
|
||||
|
||||
### 功能对比表
|
||||
|
||||
| 功能模块 | MVP实现 | 完整ZFS | 工作量对比 |
|
||||
|---------|---------|---------|-----------|
|
||||
| **Pool管理** | Import only | Create/Destroy/Vdev | 90%减少 |
|
||||
| **Dataset管理** | Mount only | Create/Destroy/Snapshot | 80%减少 |
|
||||
| **File Ops** | Read/Write/Create | Full POSIX | 50%减少 |
|
||||
| **Attributes** | Basic | Full + xattr | 60%减少 |
|
||||
| **Snapshot** | ❌ None | ✅ Full | 100%减少 |
|
||||
| **Compression** | ❌ None | ✅ lz4/zstd | 100%减少 |
|
||||
| **Send/Receive** | ❌ None | ✅ Full | 100%减少 |
|
||||
| **ARC Cache** | ✅ Use kernel | ✅ Full | 0%减少(自动) |
|
||||
| **ZIL Log** | ✅ Use kernel | ✅ Full | 0%减少(自动) |
|
||||
| **VDEV RAID** | ✅ Use kernel | ✅ Full | 0%减少(自动) |
|
||||
|
||||
### 代码量对比
|
||||
|
||||
| 组件 | MVP | 完整ZFS | 减少比例 |
|
||||
|-----|-----|---------|---------|
|
||||
| **FSKit Layer** | 约290行 | - | - |
|
||||
| **libzfs Wrapper** | 约200行 | - | - |
|
||||
| **Kernel Module** | 0行(使用现有) | 约135,000行 | 100%减少 |
|
||||
| **总计** | **约700行** | **约135,000行** | **99%减少** |
|
||||
|
||||
### 时间对比
|
||||
|
||||
| 项目 | MVP | 完整ZFS | 减少比例 |
|
||||
|-----|-----|---------|---------|
|
||||
| **开发时间** | 2-3 weeks | 15-20 months | 95%减少 |
|
||||
| **测试时间** | 1 week | 6-12 months | 92%减少 |
|
||||
| **总计** | **3-4 weeks** | **21-32 months** | **94%减少** |
|
||||
|
||||
---
|
||||
|
||||
## 9. MVP实现路径 ⭐⭐⭐⭐⭐
|
||||
|
||||
### Phase 1: Pool Import & Mount(约100行)
|
||||
|
||||
**目标**:导入现有ZFS pool,挂载dataset
|
||||
|
||||
**实现**:
|
||||
```swift
|
||||
// 1. probeResource - 检测ZFS pool
|
||||
func probeResource(_ resource: FSResource,
|
||||
replyHandler: @escaping (FSResourceProbeResult) -> Void) {
|
||||
// 调用zpool_list_import
|
||||
// 检测设备是否为ZFS pool
|
||||
}
|
||||
|
||||
// 2. loadResource - 加载dataset
|
||||
func loadResource(_ resource: FSResource,
|
||||
replyHandler: @escaping (FSResource) -> Void) {
|
||||
// 调用zpool_import
|
||||
// 导入pool
|
||||
// 返回FSResource
|
||||
}
|
||||
|
||||
// 3. activate - 挂载dataset
|
||||
func activate(options: FSTaskOptions) async throws -> FSItem {
|
||||
// 调用zfs_mount
|
||||
// 返回root item
|
||||
}
|
||||
```
|
||||
|
||||
**libzfs Wrapper**(约50行C):
|
||||
```c
|
||||
// zpool_import_wrapper
|
||||
int zpool_import_wrapper(const char *pool_name) {
|
||||
// 封装zpool_import
|
||||
}
|
||||
|
||||
// zfs_mount_wrapper
|
||||
int zfs_mount_wrapper(const char *dataset_name) {
|
||||
// 封装zfs_mount
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: File Read & Directory List(约80行)
|
||||
|
||||
**目标**:读取文件,列举目录
|
||||
|
||||
**实现**:
|
||||
```swift
|
||||
// 4. lookupItem - 查找文件
|
||||
func lookupItem(named name: FSFileName,
|
||||
inDirectory directory: FSItem) async throws -> (FSItem, FSFileName) {
|
||||
// 调用zfs_lookup
|
||||
// 返回找到的item
|
||||
}
|
||||
|
||||
// 5. enumerateDirectory - 列举目录
|
||||
func enumerateDirectory(_ directory: FSItem,
|
||||
startingAt cookie: FSDirectoryCookie,
|
||||
verifier: FSDirectoryVerifier,
|
||||
attributes: FSItem.GetAttributesRequest?,
|
||||
packer: FSDirectoryEntryPacker) async throws -> FSDirectoryVerifier {
|
||||
// 调用zfs_readdir
|
||||
// 使用packer添加每个item
|
||||
}
|
||||
|
||||
// 6. read - 读取数据
|
||||
func read(_ handle: FSFileHandle,
|
||||
buffer: UnsafeMutableRawBufferPointer,
|
||||
options: FSTaskOptions) async throws -> Int {
|
||||
// 调用zfs_read
|
||||
// 返回读取的字节数
|
||||
}
|
||||
```
|
||||
|
||||
**libzfs Wrapper**(约30行C):
|
||||
```c
|
||||
// zfs_lookup_wrapper
|
||||
int zfs_lookup_wrapper(const char *path, zfs_handle_t **handle) {
|
||||
// 封装zfs_lookup
|
||||
}
|
||||
|
||||
// zfs_readdir_wrapper
|
||||
int zfs_readdir_wrapper(zfs_handle_t *handle, zfs_readdir_cb cb, void *arg) {
|
||||
// 封装zfs_readdir
|
||||
}
|
||||
|
||||
// zfs_read_wrapper
|
||||
int zfs_read_wrapper(zfs_handle_t *handle, void *buf, uint64_t len) {
|
||||
// 封装zfs_read
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: File Write & Create(约80行)
|
||||
|
||||
**目标**:写入文件,创建文件/目录
|
||||
|
||||
**实现**:
|
||||
```swift
|
||||
// 7. createItem - 创建文件
|
||||
func createItem(named name: FSFileName,
|
||||
type: FSItem.ItemType,
|
||||
inDirectory directory: FSItem,
|
||||
attributes: FSItem.SetAttributesRequest) async throws -> (FSItem, FSFileName) {
|
||||
// 调用zfs_create
|
||||
// 返回新创建的item
|
||||
}
|
||||
|
||||
// 8. write - 写入数据
|
||||
func write(_ handle: FSFileHandle,
|
||||
buffer: UnsafeRawBufferPointer,
|
||||
options: FSTaskOptions) async throws -> Int {
|
||||
// 调用zfs_write
|
||||
// 返回写入的字节数
|
||||
}
|
||||
|
||||
// 9. removeItem - 删除文件
|
||||
func removeItem(_ item: FSItem,
|
||||
named name: FSFileName,
|
||||
fromDirectory directory: FSItem) async throws {
|
||||
// 调用zfs_remove
|
||||
}
|
||||
```
|
||||
|
||||
**libzfs Wrapper**(约30行C):
|
||||
```c
|
||||
// zfs_create_wrapper
|
||||
int zfs_create_wrapper(const char *path, zfs_type_t type) {
|
||||
// 封装zfs_create
|
||||
}
|
||||
|
||||
// zfs_write_wrapper
|
||||
int zfs_write_wrapper(zfs_handle_t *handle, const void *buf, uint64_t len) {
|
||||
// 封装zfs_write
|
||||
}
|
||||
|
||||
// zfs_remove_wrapper
|
||||
int zfs_remove_wrapper(const char *path) {
|
||||
// 封装zfs_remove
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Attributes & Cleanup(约30行)
|
||||
|
||||
**目标**:属性操作,清理资源
|
||||
|
||||
**实现**:
|
||||
```swift
|
||||
// 10. attributes - 获取属性
|
||||
func attributes(_ item: FSItem) async throws -> FSItem.Attributes {
|
||||
// 调用zfs_getattr
|
||||
// 返回Attributes对象
|
||||
}
|
||||
|
||||
// 11. deactivate - 停用volume
|
||||
func deactivate(options: FSDeactivateOptions = []) async throws {
|
||||
// 调用zfs_unmount
|
||||
// 清理资源
|
||||
}
|
||||
|
||||
// 12. synchronize - 同步数据
|
||||
func synchronize(flags: FSSynchronizeFlags) async throws {
|
||||
// 调用zfs_sync
|
||||
}
|
||||
|
||||
// 13. reclaimItem - 回收资源
|
||||
func reclaimItem(_ item: FSItem) async throws {
|
||||
// 清理item资源
|
||||
}
|
||||
```
|
||||
|
||||
**libzfs Wrapper**(约20行C):
|
||||
```c
|
||||
// zfs_getattr_wrapper
|
||||
int zfs_getattr_wrapper(zfs_handle_t *handle, zfs_attr_t *attr) {
|
||||
// 封装zfs_prop_get
|
||||
}
|
||||
|
||||
// zfs_unmount_wrapper
|
||||
int zfs_unmount_wrapper(const char *dataset_name) {
|
||||
// 封装zfs_unmount
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. 依赖关系图 ⭐⭐⭐⭐⭐
|
||||
|
||||
### macOS环境依赖
|
||||
|
||||
```
|
||||
MarkBase ZFS MVP
|
||||
│
|
||||
├── FSKit.framework (macOS Sequoia 15.4+)
|
||||
│ ├── FSUnaryFileSystem
|
||||
│ ├── FSVolume.Operations
|
||||
│ ├── FSItem
|
||||
│ └── FSTaskOptions
|
||||
│
|
||||
├── libzfs.dylib (OpenZFS on macOS)
|
||||
│ ├── libzfs_dataset.c
|
||||
│ ├── libzfs_pool.c
|
||||
│ ├── libzfs_util.c
|
||||
│ └── libzfs_diff.c
|
||||
│
|
||||
├── libspl.dylib (Solaris Portability Layer)
|
||||
│ ├── libspl.c
|
||||
│ ├── libspl_mem.c
|
||||
│ └── libspl_thread.c
|
||||
│
|
||||
└── OpenZFS Kernel Extension (org.openzfsonosx.zfs.kext)
|
||||
├── spl.kext
|
||||
├── zfs.kext
|
||||
├── zcommon.kext
|
||||
├── icp.kext
|
||||
└── zstd.kext
|
||||
```
|
||||
|
||||
### 编译依赖
|
||||
|
||||
**Swift Package**:
|
||||
```swift
|
||||
// Package.swift
|
||||
dependencies: [
|
||||
.target(name: "libzfs_wrapper", dependencies: ["libzfs", "libspl"]),
|
||||
]
|
||||
```
|
||||
|
||||
**Link Libraries**:
|
||||
```swift
|
||||
// linker flags
|
||||
-linker-options: ["-lzfs", "-lspl"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. 测试策略 ⭐⭐⭐⭐⭐
|
||||
|
||||
### 单元测试
|
||||
|
||||
**测试libzfs wrapper**:
|
||||
```swift
|
||||
func testZpoolImport() {
|
||||
let pool = try zpool_import_wrapper("testpool")
|
||||
XCTAssertNotNil(pool)
|
||||
}
|
||||
|
||||
func testZfsMount() {
|
||||
let dataset = try zfs_mount_wrapper("testpool/dataset1")
|
||||
XCTAssertNotNil(dataset)
|
||||
}
|
||||
|
||||
func testZfsRead() {
|
||||
let data = try zfs_read_wrapper("/testpool/dataset1/file.txt")
|
||||
XCTAssertNotNil(data)
|
||||
}
|
||||
```
|
||||
|
||||
### 集成测试
|
||||
|
||||
**测试FSKit Volume**:
|
||||
```swift
|
||||
func testActivate() async throws {
|
||||
let volume = ZFSVolume()
|
||||
let rootItem = try await volume.activate(options: FSTaskOptions())
|
||||
XCTAssertNotNil(rootItem)
|
||||
XCTAssertEqual(rootItem.name, "dataset1")
|
||||
}
|
||||
|
||||
func testLookupItem() async throws {
|
||||
let volume = ZFSVolume()
|
||||
let item = try await volume.lookupItem(named: "file.txt", inDirectory: rootItem)
|
||||
XCTAssertNotNil(item)
|
||||
}
|
||||
```
|
||||
|
||||
### 端到端测试
|
||||
|
||||
**测试完整流程**:
|
||||
```bash
|
||||
# 1. 创建测试pool
|
||||
zpool create testpool /dev/disk0
|
||||
|
||||
# 2. 创建dataset
|
||||
zfs create testpool/dataset1
|
||||
|
||||
# 3. 创建测试文件
|
||||
echo "test content" > /testpool/dataset1/file.txt
|
||||
|
||||
# 4. 挂载FSKit volume
|
||||
# (通过Finder或命令行挂载)
|
||||
|
||||
# 5. 验证文件内容
|
||||
cat /Volumes/testpool_dataset1/file.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. 风险评估 ⭐⭐⭐⭐⭐
|
||||
|
||||
### 技术风险
|
||||
|
||||
| 风险 | 等级 | 影响 | 缓解策略 |
|
||||
|-----|------|------|---------|
|
||||
| **FSKit API变化** | 中 | 高 | 使用稳定API,参考HelloFS |
|
||||
| **libzfs版本兼容** | 中 | 中 | 链接特定版本libzfs |
|
||||
| **Kernel Extension加载失败** | 低 | 高 | 检测kext状态,提示用户 |
|
||||
| **性能问题** | 低 | 中 | 依赖ARC自动优化 |
|
||||
| **数据损坏** | 低 | 极高 | 依赖ZFS checksum,只读测试先 |
|
||||
|
||||
### 许可证风险
|
||||
|
||||
| 风险 | 等级 | 影响 | 缓解策略 |
|
||||
|-----|------|------|---------|
|
||||
| **CDDL/GPLv2兼容** | 低 | 中 | CLI Wrapper方式规避 |
|
||||
| **Apple专利风险** | 极低 | 高 | FSKit是公开API |
|
||||
|
||||
### 用户风险
|
||||
|
||||
| 风险 | 等级 | 影响 | 缓解策略 |
|
||||
|-----|------|------|---------|
|
||||
| **数据丢失** | 低 | 极高 | 只读模式测试,完整备份 |
|
||||
| **系统崩溃** | 低 | 高 | 用户空间实现,隔离kernel |
|
||||
| **性能下降** | 中 | 中 | 依赖ARC缓存 |
|
||||
|
||||
---
|
||||
|
||||
## 13. 总结 ⭐⭐⭐⭐⭐
|
||||
|
||||
### 关键发现
|
||||
|
||||
1. **跳过Kernel Porting** ⭐⭐⭐⭐⭐
|
||||
- macOS已有OpenZFS kext
|
||||
- 直接使用现有kernel extension
|
||||
- 节省99%工作量
|
||||
|
||||
2. **FSKit集成点** ⭐⭐⭐⭐⭐
|
||||
- ZPL Layer提供POSIX语义
|
||||
- FSVolume.Operations只需13个方法
|
||||
- Swift代码量:约290行
|
||||
|
||||
3. **libzfs Wrapper** ⭐⭐⭐⭐⭐
|
||||
- 封装现有C API
|
||||
- 提供Swift FFI接口
|
||||
- C代码量:约200行
|
||||
|
||||
4. **MVP工作量** ⭐⭐⭐⭐⭐
|
||||
- 总代码量:约700行
|
||||
- 开发时间:2-3 weeks
|
||||
- 测试时间:1 week
|
||||
|
||||
### MVP可行性评估
|
||||
|
||||
**技术可行性**:⭐⭐⭐⭐⭐
|
||||
- FSKit API可用(macOS 15.4+)
|
||||
- libzfs成熟稳定
|
||||
- Kernel extension已存在
|
||||
|
||||
**工作量评估**:⭐⭐⭐⭐⭐
|
||||
- 700行代码(vs 135,000行)
|
||||
- 3-4 weeks(vs 21-32 months)
|
||||
- 99%工作量减少
|
||||
|
||||
**风险等级**:⭐⭐⭐⭐(低风险)
|
||||
- 用户空间实现
|
||||
- 隔离kernel
|
||||
- 依赖成熟组件
|
||||
|
||||
### 推荐下一步
|
||||
|
||||
**立即开始** ⭐⭐⭐⭐⭐:
|
||||
1. 创建HelloFS测试项目
|
||||
2. 实现libzfs wrapper(约200行)
|
||||
3. 实现FSVolume.Operations(约290行)
|
||||
4. 测试基本功能(pool import, file read)
|
||||
|
||||
**延后实现**:
|
||||
- Snapshot操作(Level 5)
|
||||
- Compression支持(Level 6)
|
||||
- Send/Receive(Level 7)
|
||||
|
||||
---
|
||||
|
||||
**文档版本**:1.0
|
||||
**最后更新**:2026-06-11
|
||||
**作者**:MarkBase Team
|
||||
**参考**:OpenZFS源码,FSKit文档,KhaosT/FSKitSample
|
||||
Reference in New Issue
Block a user