ac75faa0cc
Some checks failed
CI / build-and-test (push) Has been cancelled
- E4B-MarkBase model (42 layers, 4.4GB) loaded successfully - All Phase 1-6 tests passed (model loading, forward pass, vision/audio towers, token generation, performance) - All stress tests passed (5/5 in 127.6s) - Concurrent inference - Memory stress (67.5 tok/s, 0 NaN) - Continuous generation - Batch processing - Long-running stability - Swift Metal inference engine with multimodal support
117 lines
1.8 KiB
Markdown
117 lines
1.8 KiB
Markdown
# 部署指南
|
||
|
||
## 系統要求
|
||
|
||
- macOS 15.0+
|
||
- Apple Silicon (M1/M2/M3)
|
||
- Swift 6.0+
|
||
- 內存: 16GB+ (E4B), 32GB+ (12B)
|
||
|
||
## 模型準備
|
||
|
||
### 下載模型
|
||
|
||
```bash
|
||
# 使用 huggingface-cli
|
||
huggingface-cli download mlx-community/gemma-4-e4b-it-4bit
|
||
|
||
# 或使用 12B
|
||
huggingface-cli download mlx-community/gemma-4-12b-it-4bit
|
||
```
|
||
|
||
### 目錄結構
|
||
|
||
```
|
||
model/
|
||
├── model.safetensors # 模型權重
|
||
├── config.json # 模型配置
|
||
└── tokenizer.json # Tokenizer
|
||
```
|
||
|
||
## 部署步驟
|
||
|
||
### 1. 構建
|
||
|
||
```bash
|
||
cd MarkBase12B
|
||
swift build -c release
|
||
```
|
||
|
||
### 2. 運行
|
||
|
||
```bash
|
||
# 基本運行
|
||
swift run -c release G12BServer ./model
|
||
|
||
# 指定端口
|
||
swift run -c release G12BServer ./model 8080
|
||
|
||
# 後台運行
|
||
nohup swift run -c release G12BServer ./model 8080 > server.log 2>&1 &
|
||
```
|
||
|
||
### 3. 驗證
|
||
|
||
```bash
|
||
curl http://localhost:8080/health
|
||
```
|
||
|
||
## Docker 部署 (可選)
|
||
|
||
```dockerfile
|
||
FROM swift:5.9
|
||
|
||
WORKDIR /app
|
||
COPY . .
|
||
RUN swift build -c release
|
||
|
||
EXPOSE 8080
|
||
CMD ["swift", "run", "-c", "release", "G12BServer", "./model", "8080"]
|
||
```
|
||
|
||
## 性能調優
|
||
|
||
### Buffer Pool
|
||
|
||
緩衝區池已自動啟用，可減少內存分配開銷。
|
||
|
||
### Kernel 融合
|
||
|
||
融合 kernels 已自動加載，減少 dispatch 次數。
|
||
|
||
### 基準測試
|
||
|
||
```bash
|
||
swift run G12BServer ./model markbase --benchmark
|
||
```
|
||
|
||
## 監控
|
||
|
||
### 日誌
|
||
|
||
服務器輸出到 stdout，可重定向到文件：
|
||
|
||
```bash
|
||
swift run G12BServer ./model > server.log 2>&1
|
||
```
|
||
|
||
### 指標
|
||
|
||
未來版本將支持 Prometheus 指標。
|
||
|
||
## 故障排除
|
||
|
||
### 常見問題
|
||
|
||
1. **模型未找到**
|
||
- 檢查路徑是否正確
|
||
- 確認 safetensors 文件存在
|
||
|
||
2. **內存不足**
|
||
- 減少 max_context_length
|
||
- 使用較小模型
|
||
|
||
3. **編譯失敗**
|
||
- 確認 Swift 版本 >= 6.0
|
||
- 確認 macOS 版本 >= 15.0
|