Commit Graph

26 Commits

Author SHA1 Message Date
Lei Xue
76ab15b0df feat: add S3-compatible object storage backend
Add a new backend store that enables iSCSI targets backed by
S3-compatible object storage (AWS S3, MinIO, Ceph RGW, etc.).

The implementation uses a chunked storage strategy where the virtual
block device is divided into fixed-size chunks (default 4 MiB), each
stored as an independent S3 object. This enables efficient random
read/write access on top of object storage.

Key features:
- Chunked storage with configurable chunk size
- Sparse device support (unwritten chunks treated as zeros)
- Concurrent multi-chunk I/O via errgroup
- Per-chunk locking for safe read-modify-write
- AWS SDK v2 with default credential chain
- In-process gofakes3 test server (no Docker needed)
- 12 unit tests + 2 integration tests

Also updates CI workflow to run S3 backend tests and updates
README with S3 backend documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 16:22:57 +08:00
Lei Xue
36149cd4a9 support more SCSI commands: ReadDefectData, Sanitize, and expanded CI
New SCSI commands implemented:
- READ DEFECT DATA(10/12): returns empty defect list (virtual device)
- SANITIZE: supports OVERWRITE and BLOCK ERASE (zeros all blocks)
- EXTENDED COPY / RECEIVE COPY RESULTS: registered as unsupported

New unit tests for ReadDefectData10/12, Sanitize, and command registration.

New CI libiscsi test cases:
- PersistentReservation (PrinReadKeys, PrinReportCapabilities,
  ProutRegister, ProutReserve)
- ReadDefectData10/12 (Simple)
- CompareAndWrite (Simple)
- OrWrite (Simple, BeyondEol, ZeroBlocks)
- GetLBAStatus (Simple, BeyondEol)
- ReportSupportedOpcodes (OneCommand)

Partial fix for #55

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 13:06:35 +08:00
Lei Xue
00d0c3a306 fix UNMAP data corruption by implementing block zeroing
The UNMAP command was a no-op in all backing stores, causing unmapped
blocks to retain stale data instead of returning zeros per SCSI spec.

- Implement Unmap in FileBackingStore to zero out unmapped blocks
- Implement Unmap in IOUringBackingStore to zero out unmapped blocks
- Enable Unmap in RemBackingStore (was commented out)
- Change UnmapBlockDescriptor.TL from uint32 to uint64 to prevent
  integer overflow when converting block count to byte length with
  large block shifts

Fixes #119

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 12:29:12 +08:00
Lei Xue
00cfac3d24 optimize the perf and support more features 2026-03-14 11:45:35 +08:00
Lei Xue
1c9f8d3e06 fix part of golint 2022-12-11 09:59:06 +08:00
Utkarsh Mani Tripathi
dbb4ad7eb2 fix gofmt error
Signed-off-by: Utkarsh Mani Tripathi <utkarsh.tripathi@mayadata.io>
2019-11-21 19:27:01 +05:30
Utkarsh Mani Tripathi
d7caf89610 Implement InitSCSILuMap func for jiva
Signed-off-by: Utkarsh Mani Tripathi <utkarsh.tripathi@mayadata.io>
2019-11-19 10:19:58 +05:30
Utkarsh Mani Tripathi
1dbc82435f iSCSI target fixes for iSCSIResiduals tests causing libiscsi test suite to hang
Signed-off-by: Utkarsh Mani Tripathi <utkarsh.tripathi@mayadata.io>
2019-11-18 15:16:41 +05:30
Utkarsh Mani Tripathi
f3ef8c973d add initial implementation of remote backing store
This commit implement the BackingStore Interface for remote backing
store.

Signed-off-by: Utkarsh Mani Tripathi <utkarsh.tripathi@mayadata.io>
2019-11-18 14:52:03 +05:30
chessman
0b1ffd5704 provide offset and tl to DataSync 2019-06-27 19:54:16 +03:00
chessman
e2b7ea4891 rename: Thinprovisioning -> ThinProvisioning 2019-06-10 12:42:21 +03:00
chessman
12b06c21ef UNMAP: propagate request to backing store 2019-05-29 16:14:31 +03:00
chessman
22d47a9212 enable UNMAP when LUN is thin provisioned
- support Block Limits VPD page (0xB0)
 - add UNMAP to REPORT SUPPORTED OPERATION CODES
 - READ CAPACITY(16): set LBPME when Thin provisioning is enabled
 - move Thinprovisioning and BlockShift to config
 - add Unmap to BackingStore
2019-05-28 13:53:06 +03:00
Lei Xue
4ae643e9f8 perf 2017-07-31 21:47:38 +08:00
Lei Xue
8ce8ade3d4 update ModeSense6 function 2017-07-05 21:23:13 +08:00
Lei Xue
7a2b08ff80 support iscsi task function 2017-06-12 21:06:57 +08:00
Lei Xue
05b6f88de4 restruct the backend storage command submit 2016-11-28 11:11:24 +08:00
Lei Xue
bd7193880e basic change for godoc 2016-11-27 13:31:43 +08:00
Le Zhang
82e6d230a2 PR implementation 2016-11-24 23:16:53 +08:00
Le Zhang
547faf684d iSCSI/SCSI multi port/ALUA support
fix ALUA flag issue
fix NNA flag issue
fix fixed format sense data builder issue
2016-10-18 16:59:25 +08:00
Le Zhang
a51a1d8d00 inquiry page00h page80h page83h 2016-10-10 12:13:51 +08:00
Le Zhang
681f1003b9 bunch of bug fixes with dummy lun0 feature
1) add dummy lun0
2) fix report lun
3) fix inquiry command with correct version
2016-10-08 12:40:24 +08:00
Le Zhang
c9b93c7527 mapping lun and fix portal management 2016-10-04 13:39:55 +08:00
Lei Xue
3512ab66f5 init target and luns from config file 2016-10-01 20:59:30 +08:00
Lei Xue
f97ab027de add basic command framework and apiserver 2016-08-27 20:07:03 +08:00
Lei Xue
c5d68b38b2 run the basic process with iscsi driver 2016-05-02 22:11:33 +08:00