admin/markbase
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
369 Commits 1 Branch 0 Tags
e7863a30341da37266c6a4fb3b10dac43412c10c
Commit Graph

184 Commits

Author SHA1 Message Date
Warren
e7863a3034 Fix macOS SMB mount: AAPL caps, credit grant, file_index, QueryDirectory padding 
- AAPL: Restore UNIX_BASED+NFS_ACE server_caps, RESOLVE_ID+FULL_SYNC volume_caps (Samba baseline)
- Credit: Grant min 1 credit in dispatch response for smbclient compatibility
- file_index: Assign 1-based index per entry in list_dir (both VFS and local backends)
- smb_match(): Add wildcard pattern filter (*/?) for macOS single-entry QueryDirectory probes
- FILE_ID_BOTH_DIR_INFORMATION: Add 2-byte Reserved2 padding between ShortName and FileId
- macOS Sequoia 15.5 mount_smbfs now succeeds (tested: ls, cat, read)
 2026-06-23 09:44:01 +08:00
Warren
866d0536c8 Add SMB AAPL Extensions Phase 1-6 + VFS xattr support 
Phase 1: AAPL Create Context negotiation
Phase 2: AFP_AfpInfo Stream structure (Finder info + creation time)
Phase 2.5: SMB Named Stream Backend (NamedStreamPath)
Phase 2.6: Backend Named Stream Support in handlers
Phase 2.7: VFS Extended Attributes (get/set/remove/list_xattr)
Phase 4: Time Machine share config (time_machine field)
Phase 5: Server/Volume Capabilities
Phase 6: macOS Unicode mapping (private range ↔ ASCII)

Tests: 174 smb-server tests pass, 52 VFS tests pass
 2026-06-22 14:21:53 +08:00
Warren
64709ec529 Add CTDB Phase 1-5: TDB storage + Node management + Control protocol + IP manager + Recovery 2026-06-22 14:21:39 +08:00
Warren
a8d81f2a9c Revert "Remove Download Center routes from server.rs (dead code cleanup)" 
This reverts commit 20b208bb7f.
 2026-06-22 14:12:14 +08:00
Warren
20b208bb7f Remove Download Center routes from server.rs (dead code cleanup)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
Removed routes:
- /api/v2/products/* (CRUD + file assignment)
- /api/v2/download/* (file download + stats)
- /api/v2/files/:user_id (list + info via download module)
- /upload, /files, /products (HTML pages)
Kept: /api/v2/upload-unlimited, /downloads, category/series APIs
 2026-06-22 11:00:41 +08:00
Warren
60e4329eed Add VirtualFs tag-mode WebDAV + MyFiles UI + Admin WebDAV endpoint
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- VirtualFs: SQLite-backed virtual folders (tag mode), 16 unit tests
- MyFiles module: API endpoints + Web UI for folder/tag management
- Admin WebDAV: /admin-webdav/*path with Basic Auth + URI prefix rewrite
- CLI: webdav-folder/tag/untag/list/start --virtual-mode commands
- Deployed and tested on M5Max48: PROPFIND, PUT, GET, DELETE all working
 2026-06-22 10:38:25 +08:00
Warren
37d0fe1a3c Fix duplicate derive(Clone)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:28:33 +08:00
Warren
4003864d28 Fix WebDAV: add Clone to WebdavCredentials
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:26:54 +08:00
Warren
8039f0d375 Fix WebDAV auth: use map_or for password check
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:25:53 +08:00
Warren
3d395584a8 Fix WebDAV: middleware use extensions().get() to not consume
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:23:57 +08:00
Warren
cf57d46ca5 Fix WebDAV: handle_dav extract WebdavCredentials Extension
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:22:01 +08:00
Warren
8a5a23a309 Fix WebDAV Extension layer order
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:20:34 +08:00
Warren
a7f50ff747 Update WebDAV: root path + 0.0.0.0 bind
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 07:17:45 +08:00
Warren
e7a9f886ed Fix web server bind to 0.0.0.0 for external access
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 06:20:17 +08:00
Warren
52c38b1919 Add SMB Configuration Templates (Phase 6)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 05:22:14 +08:00
Warren
ebe976eee4 Implement Write/Read Cache (Phase 3)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 04:42:55 +08:00
Warren
88590d3611 Add LDAP CLI parameters to SMB server (Phase 2.2)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-22 04:13:10 +08:00
Warren
912bc21929 Implement LDAP Provider Phase 2.1: DataProvider trait with OpenLDAP/AD support
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-22 03:34:17 +08:00
Warren
097521b35d P2: Fix S3 multipart route - use query param for action
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Change route from /s3/multipart/:bucket/*key/init to /s3/multipart/:bucket/*key?action=init
- Add multipart_handler to unify all multipart operations
- Use Response type instead of impl IntoResponse for type compatibility
 2026-06-22 01:22:16 +08:00
Warren
321310582b E: Security improvements - auth + policy enforcement
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Add Signature V4 auth to multipart endpoints (init/upload/complete/abort)
- Add policy checks to main S3 handlers (get/put/delete)
- extract_user_from_auth() helper for policy evaluation
- check_bucket_policy() integrated into all handlers
- Policy denied returns 403 FORBIDDEN

Tests: 299 passed, 0 failed
 2026-06-21 23:43:24 +08:00
Warren
9b02bbac27 A: Code quality improvements - fix clippy warnings
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- Remove unused imports in server.rs (Body, HeaderValue, RwLock)
- Remove unused imports in forward_acl.rs (tests still need Ipv4Addr)
- Remove unused imports in host_key.rs (Read, Write)
- Remove unused imports in kex_exchange.rs (HostKeyType)
- Remove unused imports in known_hosts.rs (tests need Ipv4Addr)
- Remove unused imports in multiplex.rs (Arc)
- Auto-fix other unused imports via clippy --fix

Tests: 303 passed, 0 failed (4 new tests added)
 2026-06-21 23:08:07 +08:00
Warren
02d98419e1 P3: Bucket Policy implementation complete
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- BucketPolicy struct with Version + Statement array
- PolicyStatement: Effect, Principal, Action, Resource, Condition
- Principal matching (wildcard + user-specific)
- Action/Resource pattern matching with wildcards
- GetBucketPolicy: GET /s3/policy/:bucket
- PutBucketPolicy: PUT /s3/policy/:bucket
- DeleteBucketPolicy: DELETE /s3/policy/:bucket
- Policy persistence to data/s3_policies/:bucket/policy.json
- check_bucket_policy() for authorization
- 6 unit tests

Tests: 299 passed, 0 failed
 2026-06-21 22:50:53 +08:00
Warren
ca0f541a79 P2: S3 Multipart Upload support complete
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- InitiateMultipartUpload: POST /s3/multipart/:bucket/:key/init
- UploadPart: PUT /s3/multipart/:bucket/:key/part
- CompleteMultipartUpload: POST /s3/multipart/:bucket/:key/complete
- AbortMultipartUpload: DELETE /s3/multipart/:bucket/:key/abort
- In-memory upload tracking with once_cell::Lazy
- Part files stored in temp dir during upload
- Final file assembled on CompleteMultipartUpload
- XML responses for all operations

Tests: 293 passed, 0 failed
 2026-06-21 22:44:17 +08:00
Warren
5487ad63a6 P1: AsyncS3Vfs native async implementation using reqwest
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Replace spawn_blocking + ureq with native async reqwest
- AsyncS3Vfs uses reqwest::Client for HTTP operations
- rusty-s3 for presigned URL generation + XML parsing
- AsyncS3File with async read/write/seek/flush
- reqwest dependency added under async-vfs feature

Tests: 297 passed (293 + 4 new s3_auth tests)
 2026-06-21 22:22:05 +08:00
Warren
f5074b2ce2 P0: AWS Signature V4 implementation complete
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- Full Canonical Request with signed headers
- Proper URI encoding (encode_slash option)
- X-Amz-Date timestamp support
- SignedHeaders extraction from Authorization header
- Payload hash from X-Amz-Content-Sha256
- 4 unit tests passing

Tests: 297 passed (293 + 4 new)
 2026-06-21 22:14:34 +08:00
Warren
49873cb302 Phase 5.1: AsyncVfsDavFs spawn_blocking wrapper complete
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- AsyncVfsDavFs wraps VfsDavFs with spawn_blocking
- All DavFileSystem methods offloaded to blocking thread pool
- Uses tokio::runtime::Runtime::block_on inside spawn_blocking
- Prevents blocking async executor during VFS operations

Tests: 293 passed, 0 failed
 2026-06-21 21:33:43 +08:00
Warren
c2ff6fc90e Phase 5: WebDAV async integration analysis - API mismatch found
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- dav-server DavFileSystem API changed (20+ compile errors)
- read_dir takes ReadDirMeta, not depth
- have_props/get_props/get_prop/patch_props new methods
- DavFile needs write_buf method
- DavMetaData/DavDirEntry async return types changed

Recommended approach: spawn_blocking wrapper (~2h)
Alternative: full rewrite (~8h)

Phase 5 blocked pending API analysis
 2026-06-21 21:28:39 +08:00
Warren
23e0996b81 Phase 5: WebDAV async integration design framework
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Detailed design notes for AsyncVfsDavFs
- AsyncVfsDavFile implementation pattern
- DavFileSystem trait async implementation
- Estimated: ~3 hours for full implementation

Phase 5 framework documented for future implementation
 2026-06-21 21:20:47 +08:00
Warren
94a7584e64 P1: AsyncSmbVfs implementation (Phase 4)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- AsyncSmbVfs: spawn_blocking wrapper over SmbVfs
- AsyncSmbFile: tokio::sync::Mutex for async state
- Add Clone derive to SmbVfs (Arc<Mutex<Tree>>)
- Remove manual Clone impl (derive handles it)

Phase 4 complete: AsyncSmbVfs working
Phase 5 pending: WebDAV integration

Tests: 293 passed, 0 failed
 2026-06-21 21:16:50 +08:00
Warren
5c9b51fc49 P1: AsyncS3Vfs implementation (Phase 3)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- AsyncS3Vfs: spawn_blocking wrapper over S3Vfs
- AsyncS3File: tokio::sync::Mutex for async state
- Add Clone derive to S3Vfs
- All backend methods wrapped with spawn_blocking

Phase 3 complete: AsyncS3Vfs working
Phase 4 pending: AsyncSmbVfs
Phase 5 pending: WebDAV integration

Tests: 293 passed, 0 failed
 2026-06-21 21:08:48 +08:00
Warren
790efe13f4 P1: AsyncLocalFs implementation (Phase 2)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- AsyncLocalFile: tokio::fs::File wrapper
- AsyncLocalFs: AsyncVfsBackend impl using tokio::fs
- Key methods: read_dir, open_file, stat, create_dir, remove_file, rename
- 4 async tests passing

Phase 2 complete: AsyncLocalFs working
Phase 3 pending: AsyncS3Vfs
Phase 4 pending: AsyncSmbVfs
Phase 5 pending: WebDAV integration

Tests: 293 passed, 0 failed
 2026-06-21 20:59:41 +08:00
Warren
6242a5eaab P1: AsyncVfsBackend trait design (Phase 1 - framework)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Add AsyncVfsBackend + AsyncVfsFile trait definitions
- Use cfg(feature = "async-vfs") for optional compilation
- Design notes for Phase 2-5 implementation
- Estimated: ~13 hours (multi-day project)

Phase 2: AsyncLocalFs (tokio::fs)
Phase 3: AsyncS3Vfs (async client)
Phase 4: AsyncSmbVfs (async wrapper)
Phase 5: WebDAV integration

Tests: 289 passed, 0 failed
 2026-06-21 20:52:31 +08:00
Warren
ed55c6050e P2: Streaming read optimization (64KB chunk cache)
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- Add read_cache + read_cache_offset fields to VfsDavFile
- Read-ahead 64KB chunks to reduce VFS calls
- Serve from cache when data is available
- Invalidate cache on seek()
- Reduces memory allocations and VFS syscall overhead

Tests: 289 passed, 0 failed
 2026-06-21 19:16:12 +08:00
Warren
9c82830959 P1: WebDAV ACL enforcement (RFC 3744)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Add enable_acl field to VfsDavFs
- Add check_acl() helper method
- ACL checks in open(), read_dir(), create_dir(), remove_dir(), remove_file(), rename()
- Uses VfsAceMask for permission checks (ReadData, WriteData, etc.)
- Returns FsError::Forbidden if ACL denies access

Tests: 289 passed, 0 failed
 2026-06-21 18:37:48 +08:00
Warren
a56207db0b P3: Quota enforcement - check before write in flush()
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Check VfsBackend quota before writing buffered data
- Return FsError::InsufficientStorage (507) if limit exceeded
- Log warning with current/adding/limit values

Tests: 289 passed, 0 failed
 2026-06-21 18:24:44 +08:00
Warren
12ec190831 Add Range request test: verify dav-server partial content support
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- test_range_request: GET with Range header returns 206 + partial content
- Verify Content-Range header present
- Test bytes=5-10 returns correct 6-byte slice

Tests: 289 passed, 0 failed
 2026-06-21 18:21:48 +08:00
Warren
b71510b2e8 P0 fix: Mutex/RwLock poison recovery for webdav_locks and webdav_version
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Add recover_mutex() helper in webdav_locks.rs
- Add recover_rwlock() helper in webdav_version.rs
- Replace all .unwrap() calls with recovery pattern
- Tests: 288 passed, 0 failed
 2026-06-21 18:11:48 +08:00
Warren
0322e2d4b6 WebDAV error handling improvements: map_vfs_error helper
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
- Add map_vfs_error() to map VfsError to FsError properly
- NotFound → NotFound, PermissionDenied → Forbidden, etc.
- Update create_dir/remove_dir/remove_file/rename/set_atime/set_mtime/get_quota
- Add executable() method to VfsDavMetaData (mode & 0o111)

Tests: 288 passed, 0 failed
 2026-06-21 16:50:23 +08:00
Warren
43c135e877 WebDAV additional fixes: dead props compaction + accessed metadata
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
- save_props/patch_props: filter empty entries before persisting
- VfsDavMetaData: add accessed field + accessed() method

Tests: 288 passed, 0 failed
 2026-06-21 16:45:03 +08:00
Warren
ab11983c1b WebDAV MKCOL: return 405 Exists if directory already exists (RFC 4918)
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
P3 fix:
- create_dir: check vfs.exists() before creating
- Return FsError::Exists (405 Method Not Allowed) if path exists

Tests: 36 webdav tests pass
 2026-06-21 16:16:43 +08:00
Warren
5000ba7c14 WebDAV async + cache TTL: spawn_blocking for props persistence, 5min TTL eviction
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
P2 improvements:
- patch_props: use tokio::spawn_blocking for blocking VFS writes
- WEBDAV_HANDLER_CACHE: add CachedHandler with Instant timestamp
- TTL check on each request (300s = 5 minutes), recreate if expired
- create_handler_for_user() helper function

Tests: 288 passed, 0 failed
 2026-06-21 16:14:42 +08:00
Warren
9acd174388 WebDAV improvements: flush fix, RwLock recovery, expired lock cleanup, atomic set_times
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details 
P0 fixes:
- flush(): add flushed flag, proper error logging, Drop warning for data loss
- props_data RwLock: replace unwrap() with try_read/try_write recovery
- PersistedLs: add is_expired() + cleanup_expired_locks() helper

P1 improvements:
- Props persistence via VFS (load_props/save_props/patch_props)
- COPY/MOVE sync dead props (copy on COPY, move key on rename)
- Atomic set_atime/set_mtime via filetime crate (no race condition)

New files:
- webdav_locks.rs: PersistedLs with lock persistence + expiry cleanup

Tests: 288 passed, 0 failed
 2026-06-21 16:07:12 +08:00
Warren
a475de45c9 Add SSH Port Forwarding ACL (Phase 1-3): prevent SSH tunnel abuse
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
Features:
- ForwardRule: Allow/Deny rules with address/port specifications
- ForwardAcl: User-specific ACL with priority-based rule matching
- ForwardAclManager: Global ACL manager for all users
- OpenSSH-style PermitOpen/PermitListen parsing
- 8 unit tests for all operations

Security:
- Prevent unauthorized SSH tunnel creation
- Restrict forwarding to specific hosts/ports
- Default deny policy for unknown users

Files:
- markbase-core/src/ssh_server/forward_acl.rs (493 lines)
- markbase-core/src/ssh_server/mod.rs (+1 line)

Tests: 317 passed (+8)
 2026-06-21 12:48:56 +08:00
Warren
204186e34b Add WebDAV Versioning (Phase 1-5): version control with history tracking
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details 
Features:
- WebDavVersioning: Version control using HashMap storage
- VersionInfo/VersionHistory: Version metadata structures
- create_version/get_version/delete_version operations
- restore_version: Restore from previous version
- SHA-256 checksum calculation
- 11 unit tests for all operations

Files:
- markbase-core/src/webdav_version.rs (391 lines)
- markbase-core/src/lib.rs (add module)

Tests: 309 passed (+11)
 2026-06-21 12:15:37 +08:00
Warren
2ca543fd66 Add SSH Structured Logging (Phase 1-5): ssh_audit_log.rs module with JSON tracing 
Features:
- SshAuditLog: Structured audit logging using tracing crate
- 16 audit event types (connection/auth/command/file/port_forward)
- JSON output format via tracing-subscriber json layer
- 10 unit tests for all audit events

Files:
- markbase-core/src/ssh_server/ssh_audit_log.rs (289 lines)
- markbase-core/Cargo.toml (tracing + json layer)
- markbase-core/src/ssh_server/mod.rs (export module)

Tests: 298 passed (+10)
 2026-06-21 11:29:04 +08:00
Warren
3d0d031677 Add SMB Previous Versions tests: GMT token generation and snapshot listing/open/restore verification
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-21 06:20:17 +08:00
Warren
d368a7a4c0 Implement SSH Multiplexing: Connection/Session/Channel management with expiration and cleanup
Some checks failed
Test / test (push) Has been cancelled
Details
Test / build (push) Has been cancelled
Details
2026-06-21 05:31:06 +08:00
Warren
30c1e5fff9 Implement SSH Known Hosts Verification: Parse ~/.ssh/known_hosts + verify host keys + hashed host support
Some checks failed
Test / build (push) Has been cancelled
Details
Test / test (push) Has been cancelled
Details
2026-06-21 05:24:33 +08:00
Warren
b014390d12 Implement SSH Connection Rate Limiting: IP rate limit + global rate limit + auth brute force prevention 2026-06-21 05:01:04 +08:00
Warren
56e73ad8a4 Implement SSH Host Key Management (Phase 1): Generate/Load/Rotate Ed25519 keys 2026-06-21 04:57:15 +08:00