admin/markbase
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
377 Commits 1 Branch 0 Tags
1d9e140e6cf84afe5aa7af47d6ee7cbb04dfd098
Commit Graph

377 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Warren
1d9e140e6c Fix Backup/Restore API compilation errors 
- chrono timestamp_opt API: use TimeZone trait method
- VfsError::Io/NotFound: use String literals
- SendFormat: add PartialEq derive
- VfsRaidConfig tests: add disk_paths field
- BackupStats test: use relative timestamps
- HashSet file tracking: use (String, u64) tuple
- BackupStream::receive: clone format before use
- collect_file_data: fix temporary lifetime

All tests pass: 495 markbase-core + 201 smb-server = 696 total
 2026-06-24 02:37:03 +08:00
Warren
5f12e9f5d7 Implement scrub scheduler + dedup repair: Phase 5-6 complete 
Phase 5: Background scrub scheduler (~220 lines)
- ScrubScheduler: periodic scrub at configurable interval
- ScrubSchedulerConfig: interval_secs, scrub_on_startup, repair_enabled
- start/stop/run_once methods
- ScrubStats: running, scrub_count, last/next scrub time
- 6 unit tests: default config, start/stop, stats, timestamp format

Phase 6: Dedup repair integration (~30 lines)
- DedupStore::get_block_by_checksum(): retrieve by SHA-256 hash
- DedupStore::has_block_by_checksum(): check existence
- DedupStore::repair_from_checksum(): repair corrupted block
- checksum::repair_block_from_dedup(): integration hook

Tests: 471 passed (+6 new scrub_scheduler tests)

Files:
- markbase-core/src/vfs/scrub_scheduler.rs (NEW)
- markbase-core/src/vfs/dedup.rs (MOD +30 lines)
- markbase-core/src/vfs/checksum.rs (MOD +20 lines)
- markbase-core/src/vfs/mod.rs (MOD +1 line)
 2026-06-24 01:46:08 +08:00
Warren
ffc3f03744 Implement block-level checksum: Phase 1-4 complete 
Phase 1: VfsBlockChecksum struct + JSON storage (~240 lines)
- VfsBlockChecksum: offset + SHA-256 hash
- VfsChecksumFile: block_size + algorithm + blocks + file_size
- compute_block_hash() + verify_block_hash()
- ChecksumMode: Lazy (default) + OnRead
- ScrubResult: total/verified/corrupted/repaired blocks metrics

Phase 2: ChecksumFile wrapper (~180 lines)
- VfsFile wrapper with transparent checksum
- Lazy verification (only on scrub)
- Cache of verified blocks
- Update checksum on flush()
- read_at/write_at support

Phase 3: Scrub API (~150 lines)
- scrub_file(): verify single file integrity
- scrub_all(): recursive directory scrub
- create_checksums_for_file(): generate checksums
- repair_block(): placeholder for RAID/Dedup

Phase 4: RAID repair integration (~160 lines)
- repair_block_from_parity(): reconstruct from RAID parity
- reconstruct_from_p(): XOR reconstruction for RaidZ1
- reconstruct_from_pq/pqr(): placeholder for RaidZ2/3

Tests: 15 checksum tests pass (465 total)

Files:
- markbase-core/src/vfs/checksum.rs (NEW)
- markbase-core/src/vfs/checksum_file.rs (NEW)
- markbase-core/src/vfs/raid.rs (MOD +160 lines)
- markbase-core/src/vfs/mod.rs (MOD +2 lines)
 2026-06-24 01:41:56 +08:00
Warren
7c4476e19c Implement at-rest encryption: AES-256-GCM VFS layer 
- Added encrypted_fs.rs module for transparent file encryption
- EncryptedVfs wraps any VfsBackend with AES-256-GCM encryption
- Per-file key derivation from master key + file path (SHA-256)
- File format: MBE1 magic + version + nonce + original_size + ciphertext + tag
- EncryptedFile transparently decrypts on read, encrypts on flush
- 5 unit tests: roundtrip, different keys, key derivation, header format, password config

Tests: 457 markbase-core (+5 new), 201 smb-server (658 total)
 2026-06-24 00:57:53 +08:00
Warren
57fd6a475f macOS Time Machine AFP monitoring: backup_time update on file modification 
- Added afp_monitor.rs module to track AFP_AfpInfo backup_time
- Open struct now has 'modified' flag to track file modifications
- write.rs sets modified=true on successful write
- close.rs calls AfpMonitor::update_backup_time() on modified files
- create.rs calls AfpMonitor::init_afp_info() on new file creation
- AFP_AfpInfo stored as xattr com.apple.aapl.AfpInfo
- backup_time updated to current epoch time on modification

Also includes:
- LZ4 compression using lz4_flex crate
- Case sensitivity conditional on backend capabilities
- LDAP cfg feature gate fix
- RAID rebuild reconstruction implementation
- DOS attributes xattr persistence
- Snapshot disk persistence

Tests: 201 smb-server, 452 markbase-core (653 total)
 2026-06-24 00:46:33 +08:00
Warren
5300b672cb Compound request integration tests: stitch_responses, capture_file_id, inherit_context, CREATE+CLOSE chain
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-23 10:46:30 +08:00
Warren
637227f4e4 SMB: reusable read buffer in VfsHandle (avoid per-read allocation + zero-init) 
- Add FileAndBuf struct wrapping file + reusable read_buf Vec
- read(): reuse Vec capacity across calls, use unsafe set_len to skip memset
- ~15% read throughput improvement (2.6 → 3.0 GB/s on localhost smbclient)
 2026-06-23 10:05:39 +08:00
Warren
d4f60929fa SMB performance optimization: pread/pwrite, tokio::sync::Mutex, direct response, fast-path 
- VfsFile trait: add read_at()/write_at() with seek+read default impl
- LocalFs: override with real pread/pwrite (FileExt::read_at/write_at) — 1 syscall vs 2
- smb_server_backend: use read_at/write_at + tokio::sync::Mutex (non-blocking async)
- read handler: build response directly, avoid Bytes→Vec<u8> copy + intermediate struct
- oplock break: fast-path skip when ≤1 open entry (single-user scenario)
 2026-06-23 09:58:19 +08:00
Warren
e7863a3034 Fix macOS SMB mount: AAPL caps, credit grant, file_index, QueryDirectory padding 
- AAPL: Restore UNIX_BASED+NFS_ACE server_caps, RESOLVE_ID+FULL_SYNC volume_caps (Samba baseline)
- Credit: Grant min 1 credit in dispatch response for smbclient compatibility
- file_index: Assign 1-based index per entry in list_dir (both VFS and local backends)
- smb_match(): Add wildcard pattern filter (*/?) for macOS single-entry QueryDirectory probes
- FILE_ID_BOTH_DIR_INFORMATION: Add 2-byte Reserved2 padding between ShortName and FileId
- macOS Sequoia 15.5 mount_smbfs now succeeds (tested: ls, cat, read)
 2026-06-23 09:44:01 +08:00
Warren
8ef1406ed3 SMB fixes: IPC$ ShareMode=Public, capabilities=0, FILE_ID_BOTH_DIRECTORY_INFORMATION Reserved2 removed, NextEntryOffset=0 for last entry, debug logging 2026-06-23 03:22:39 +08:00
Warren
bb796ec6b9 Fix smb-server xattr: add root_path field for absolute path storage 2026-06-22 16:25:33 +08:00
Warren
9dd2eefeea Fix smb-server xattr: dereference Arc<Dir> before as_std_path() 2026-06-22 15:41:03 +08:00
Warren
0c4459ae66 Fix smb-server xattr: use PathBuf for absolute paths 2026-06-22 15:39:37 +08:00
Warren
5b0086f6f0 Implement Time Machine xattr support (Phase 4.1 complete) 2026-06-22 15:30:44 +08:00
Warren
3029327d5e Implement SMB AFP_Resource Stream via AppleDouble files (Phase 3 complete) 2026-06-22 15:27:28 +08:00
Warren
1c8c47d5fa Implement SMB AFP_AfpInfo read/write via xattr (Phase 2.8 complete)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 15:16:59 +08:00
Warren
25991c71b2 Update Cargo.lock for new dependencies (xattr, smb-server AAPL modules)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 14:22:16 +08:00
Warren
866d0536c8 Add SMB AAPL Extensions Phase 1-6 + VFS xattr support 
Phase 1: AAPL Create Context negotiation
Phase 2: AFP_AfpInfo Stream structure (Finder info + creation time)
Phase 2.5: SMB Named Stream Backend (NamedStreamPath)
Phase 2.6: Backend Named Stream Support in handlers
Phase 2.7: VFS Extended Attributes (get/set/remove/list_xattr)
Phase 4: Time Machine share config (time_machine field)
Phase 5: Server/Volume Capabilities
Phase 6: macOS Unicode mapping (private range ↔ ASCII)

Tests: 174 smb-server tests pass, 52 VFS tests pass
 2026-06-22 14:21:53 +08:00
Warren
64709ec529 Add CTDB Phase 1-5: TDB storage + Node management + Control protocol + IP manager + Recovery 2026-06-22 14:21:39 +08:00
Warren
a8d81f2a9c Revert "Remove Download Center routes from server.rs (dead code cleanup)" 
This reverts commit 20b208bb7f.
 2026-06-22 14:12:14 +08:00
Warren
20b208bb7f Remove Download Center routes from server.rs (dead code cleanup)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
Removed routes:
- /api/v2/products/* (CRUD + file assignment)
- /api/v2/download/* (file download + stats)
- /api/v2/files/:user_id (list + info via download module)
- /upload, /files, /products (HTML pages)
Kept: /api/v2/upload-unlimited, /downloads, category/series APIs
 2026-06-22 11:00:41 +08:00
Warren
60e4329eed Add VirtualFs tag-mode WebDAV + MyFiles UI + Admin WebDAV endpoint
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- VirtualFs: SQLite-backed virtual folders (tag mode), 16 unit tests
- MyFiles module: API endpoints + Web UI for folder/tag management
- Admin WebDAV: /admin-webdav/*path with Basic Auth + URI prefix rewrite
- CLI: webdav-folder/tag/untag/list/start --virtual-mode commands
- Deployed and tested on M5Max48: PROPFIND, PUT, GET, DELETE all working
 2026-06-22 10:38:25 +08:00
Warren
37d0fe1a3c Fix duplicate derive(Clone)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:28:33 +08:00
Warren
4003864d28 Fix WebDAV: add Clone to WebdavCredentials
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:26:54 +08:00
Warren
8039f0d375 Fix WebDAV auth: use map_or for password check
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:25:53 +08:00
Warren
3d395584a8 Fix WebDAV: middleware use extensions().get() to not consume
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:23:57 +08:00
Warren
cf57d46ca5 Fix WebDAV: handle_dav extract WebdavCredentials Extension
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:22:01 +08:00
Warren
8a5a23a309 Fix WebDAV Extension layer order
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:20:34 +08:00
Warren
a7f50ff747 Update WebDAV: root path + 0.0.0.0 bind
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:17:45 +08:00
Warren
41f0217450 Update Caddyfile: studio.momentry.ddns.net/demo WebDAV config
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 06:51:51 +08:00
Warren
e7a9f886ed Fix web server bind to 0.0.0.0 for external access
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 06:20:17 +08:00
Warren
cd184daa20 Update AGENTS.md: CTDB architecture analysis summary
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 05:37:12 +08:00
Warren
060f43f0c4 Add CTDB architecture analysis document
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 05:36:23 +08:00
Warren
63b765f68e Update AGENTS.md: Phase 5 complete summary
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 05:31:26 +08:00
Warren
e9eca1b492 Add DFS Referral Support (Phase 5)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 05:30:16 +08:00
Warren
4db72fff4a Update AGENTS.md: Phase 6 complete summary
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 05:22:54 +08:00
Warren
52c38b1919 Add SMB Configuration Templates (Phase 6)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 05:22:14 +08:00
Warren
054bf55490 Update AGENTS.md: Phase 1-4 complete summary
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 05:15:41 +08:00
Warren
e267b43424 Add Compound Request tests (Phase 4)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 05:13:02 +08:00
Warren
c89f6c96ae Update AGENTS.md: Phase 1-3 complete summary
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 04:43:49 +08:00
Warren
ebe976eee4 Implement Write/Read Cache (Phase 3)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 04:42:55 +08:00
Warren
9ae0402318 Document NTLMv2+LDAP incompatibility and skip Phase 2.3
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 04:34:15 +08:00
Warren
3c5de4e6a3 Update AGENTS.md: LDAP Provider Phase 2.1-2.2 complete
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 04:13:59 +08:00
Warren
88590d3611 Add LDAP CLI parameters to SMB server (Phase 2.2)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 04:13:10 +08:00
Warren
912bc21929 Implement LDAP Provider Phase 2.1: DataProvider trait with OpenLDAP/AD support
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 03:34:17 +08:00
Warren
4ab282bbff Update AGENTS.md: SMB3 encryption Phase 1 complete (v1.51)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 03:19:23 +08:00
Warren
382ea2e28b Phase 1.3: SMB3 packet encryption handling complete
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- Add handle_encrypted_frame() to dispatch.rs
- Detect TRANSFORM_HEADER magic (0x534D4220)
- Decrypt incoming packets using session.encryption_key
- Encrypt outgoing responses
- All encryption tests pass (3 passed)

Phase 1 SMB3 encryption complete: ~380 lines total
 2026-06-22 03:18:22 +08:00
Warren
98239c09d4 Phase 1.2: SMB3 encryption negotiation + session state
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Add encryption_supported and encryption_cipher to Connection state
- Add encryption_key and encryption_enabled to Session state
- Add EncryptionCapabilities context to NegotiateResponse (SMB 3.1.1)
- Derive encryption_key from session_base_key in session_setup
- Export derive_encryption_key as public method
- Fix Session::new() signature with 8 parameters
- All encryption tests pass (3 passed)
 2026-06-22 02:56:02 +08:00
Warren
104e7f5f9c Phase 1.1: SMB3 encryption module (AES-CTR + HMAC)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- Add encryption.rs with Smb3Encryption struct
- Implement AES-128-CTR + HMAC-SHA256 (simplified approach)
- Add TransformHeader struct for SMB2 TRANSFORM_HEADER
- 3 unit tests pass (encrypt/decrypt roundtrip + signature verification)
- Total: ~180 lines of code
 2026-06-22 02:20:59 +08:00
Warren
097521b35d P2: Fix S3 multipart route - use query param for action
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Change route from /s3/multipart/:bucket/*key/init to /s3/multipart/:bucket/*key?action=init
- Add multipart_handler to unify all multipart operations
- Use Response type instead of impl IntoResponse for type compatibility
 2026-06-22 01:22:16 +08:00