feat: Phase 2.6 edges migration to Qdrant (TKG-only architecture)
Phase 2.6.1: co_occurrence_edges migration - build_co_occurrence_edges_from_qdrant() - Qdrant embeddings → frame grouping → YOLO objects - Result: 6679 edges (vs 6701 PostgreSQL) Phase 2.6.2: face_face_edges migration - build_face_face_edges_from_qdrant() - Qdrant embeddings → frame grouping → face pairs - mutual_gaze detection preserved - Result: 6 edges (exact match) Phase 2.6.3: speaker_face_edges migration - build_speaker_face_edges_from_qdrant() - Qdrant embeddings → trace_id frame ranges - SPEAKS_AS edge creation Architecture: - All edges use Qdrant payload (no face_detections queries) - PostgreSQL fallback for empty Qdrant - Estimated 3.6x performance improvement Testing: - Playground (3003): ✓ All Phase 2.6 logs verified - Edge counts: ✓ Close match with PostgreSQL - Fallback: ✓ Working Docs: - docs_v1.0/DESIGN/TKG_PHASE2_6_EDGES_MIGRATION.md - docs_v1.0/M4_workspace/2026-06-21_phase2_6_test.md
This commit is contained in:
Accusys
293
v1.1/checksums_v1.11.sha256
Normal file
293
v1.1/checksums_v1.11.sha256
Normal file
@@ -0,0 +1,293 @@
|
||||
2bfe6a1c1263f35916d4a28981814515fc40cb473f7bbc801f84842904c888f6 add_yolo_to_chunks.py
|
||||
f61f7126698018b346c8bafc45501708c17e3b45d9db54be5f0109afeee63176 age_benchmark.py
|
||||
8efb13239db2a25a728abbdebd92affe685b69402a277cceb0d76e62ed9451ac analyze_asr_lip.py
|
||||
432b3e3b30578e71ef973aca758bd1964102cbbb19530620df8ac02df00eefb8 analyze_video_faces.py
|
||||
732609ef1882e14dc7ed60488697f6ae7e2607ec90b240a86ea9e585f052b9be apply_asr_corrections.py
|
||||
790bd25424e93ca5a0743ea1a740a9a70f6ae6f8a9ca411012eb1e9b03907eb4 asr_benchmark_runner.py
|
||||
18744dc3bebdce0d89ea7076b5e43febd35ad3c84064bb52adde4d128d50bc9f asr_face_stats.py
|
||||
1577d055328a73561f9ccfaf0c54727532e3dddcd1bf0f33e3c38081415cced8 asr_model_benchmark.py
|
||||
fcbb81639f53e9e08bee436853c84d918c0eeac09d985b34634d5ddc00055b61 asr_processor_base.py
|
||||
25948a204e45ce844d43606b7e45c9532321d48df44887d261fc886748276b10 asr_processor_contract_v1.py
|
||||
e9209cf028a11bdc45514124826374e58458ee06b054cfedffe8013d751735ea asr_processor_contract_v2.py
|
||||
407dd0ec772027e0df27af0b66ea8130cb390595ccdeca4350e7bdc210acee6c asr_processor_debug.py
|
||||
dcee1b80071b47c974bcffe3d27ec2f2269f4b8de7e7409ceaec7e6f271d31aa asr_processor_legacy_v2.py
|
||||
10728a05a6ff2d56a70bb831abb51e05b03309e45bc5fa068c5a0702a4c73769 asr_processor_legacy.py
|
||||
9106bfe07de9cfc920f4f4d2f821dc024df612f4c2a8f5f75d35f012d26440f0 asr_processor_simplified.py
|
||||
7eabdcf7320302ee65c67e801f3ac7ca5801abc76165faa182348d30a8113e9f asr_processor_small_multilingual.py
|
||||
2714f7be88f286635ea8465daf8fa969e6b27d2b2d1f73ac5e98f5e496139cad asr_processor_small.py
|
||||
1089ff10b9b0a9f528cac79580aec25e33f8eeea485ac44b6aaf8c7c0cab5b42 asr_processor_v2.py
|
||||
b9e826f23f080ae67f5961ad750ec2a6834cd18335955c3b3175b8cd06ebd6d3 asr_processor.py
|
||||
5431b57d4369a841d51a6d6c5e1fb5e6c2932cb97cb4601f5e1b41ffe9f7ecaf asr_side_by_side_comparison.py
|
||||
6c11efc3d40e559bfbeadcbf4f51eb353b744cc4f765bd8abc472a701e3f33cb asrx_processor_contract_v1.py
|
||||
93501463af84d6541405057da3783d40492aec5e536b4210dcaffe460cdb5503 asrx_processor_custom.py
|
||||
6adfbee842d134b9d180e2d1104694ed5cdc1fa4febcd0c502801b8f87b3ce66 asrx_processor_simplified.py
|
||||
60fc3465f9c461583f8d0b888e85b3a6e04e1f252a1e1c21d036b52e1ce4b43c asrx_processor_v2_noalign.py
|
||||
82d65b71bd86874e484870c40214d3fbd9343c39d5d635896fb4d257d13a410f asrx_processor_v2_transcribe.py
|
||||
5a0c9905a2e10c847aa74f108e4054de4704bbafb2004589db15bf33833ea3c7 asrx_processor_v2.py
|
||||
b16b00cf9e5de96abc512022af9bb81196405b10988f5a39dfd3a9b6471f1155 asrx_processor.py
|
||||
f11b67ada6167540d2f95cb2af93d0e3a0de55bce659745baa37c4aa4805212e audio_taxonomy_processor_v2.py
|
||||
ded810b81cda24e31e82de14ba9846770ee2b18d84d52b9d570de5877e9e2513 audio_taxonomy_processor.py
|
||||
f7c53be5a031a8bff15c3165543586529932d81c4312521654d132b1f0ed6bc3 auto_identify_persons.py
|
||||
5497a6f1f7ae267c796a398a9f020ea485aa45f980f2eca932b904ad61ce9b40 backfill_demographics.py
|
||||
39a479ca4f8986f3255b0bcd0d9162a1f2ae339bb4dcf081f931ff9b304797a1 backfill_frame_data.py
|
||||
77a98d9b7cb97eceae4c0fcf2c353933e0fb36ee7406b57d59b1e216b1a44601 build_docs.py
|
||||
308c8e3f3d45ee273504f9f415eaf6c025f06aaf1cca33156a66431ed6e64f43 build_semantic_index_poc.py
|
||||
4eb37768edd252d94f0d751f219c317e905bc093f414b2a6350efb8294131138 build_semantic_index.py
|
||||
debbd058957d09c2397f3f4c028edaa0a658002921dcca95eae2a20070ba95fb caption_processor_contract_v1.py
|
||||
7236cdb5deaeada266cc246ee11380248bb9f2255888c25a152b2f6ab1f981cc caption_processor.py
|
||||
e73cbb688dade5c5b6fc4276f0c78b377903ff83f3830b63d8bcdacd8da8aecf check_all_stamps.py
|
||||
7ecdbd4b1f94be8ebab9935ea210a868330e7030b6e19c73229c579c1189fd5c check_architecture_all.py
|
||||
7179ed1a87241904af29542f9018398f8afd9b9dd89af7bb11909310ab7b49e0 check_architecture_docs.py
|
||||
7e6bd7d14582e494baf8b28354bbded3f79b43f0bd271ab33874da55b9086311 check_code_document_consistency.py
|
||||
5ffca7c55edafad755e84499981553fcb48ce6056ca7b04130acafb9e6a9b1c3 check_frame_112_36.py
|
||||
f49c7b0cfa53b657f69b2ad97a6e18393741cc2151b32c9d7dde2e078b75953f check_frame_91_59.py
|
||||
d2cb7475262ee711a4b06e53559f0927242be4a924a56e7fe212225f318f4193 chinese_vector_test.py
|
||||
ecde3d3df773916f62de4e34f8d8693feaedf112a3ef9955e22417c8421722bd chunk_statistics.py
|
||||
2588ecf27c13020d894e46ba70a76de89f09556b475f555dae59db36da0b90a0 clean_sentence_text.py
|
||||
98ab1129032f42fddc020f9b3492d1fc133851d1af33ddeb57e2385d88425af4 clip_logo_integration.py
|
||||
bf6f74c09b8f8c7f25c5fffb9c36f16a8afb483a7b65903cfc75e2ea641bdf49 compare_asr_content.py
|
||||
1f2caadcded724aa04a929018a35ace53dd79d172f5ee2720308fbd4581b0c6c compare_asr_models.py
|
||||
1ed8a9530f40e304b556ff76c7cac40468c86a0cd32ff2a8bc7bf2a69669121d compare_models_gun_test.py
|
||||
6bf790fe75a7a2a5220052ca14c31e90a97eabc4558cd5e9059280913862a81e compare_search.py
|
||||
875e7a598982c8ad7222a51b7b147e91cd5e1a930f41214b3942107cb932fc5c compare_segmentation.py
|
||||
e432b6f2364d5a9aaf207a1de0dca3fb14ab8d118c53ee34306abfe6fd211ba8 comprehensive_search_test.py
|
||||
43df85cf860ac28e083de35b511bb2a7b91ed48f596757f52f19487768987500 coreml_embed_server.py
|
||||
9149ccc8de5adfec69c6f3f2ec502ae7d5e7844518a228ba587af2e08cb38805 crop_opencv_stamp.py
|
||||
fc36ecbb1455d959456945266e193b601a29c4210b4938a3f0d4a9aaf44b5cee crop_real_stamps.py
|
||||
34a694624ce94d916b06a847bc4d41e7665985b85e55a626a4bc3a4370c21acf crop_stamp_112_36.py
|
||||
27099dc9c8ee52a6949ce18c505089afef1720fe70858b90d0801972c3b43fff crop_stamp_closeup.py
|
||||
01b5a3b091ebcffc0c1e2637b7af8192ba597239fa80d152738e3b8cfdf8174d crop_stamp.py
|
||||
71b2a362b5395c6e4d70e62766820db92d94eaf140d98eecb2880bcd98d55be9 crop_top_candidates.py
|
||||
60f18c5fa03ffbc80c209337cd1c8b6acd0b8471e600119340aa8cdfeef14f5b cut_benchmark_runner.py
|
||||
deba86a1645ca5b1acf413dd9edfad77b93ff213897d739a32de1ba629bfce52 cut_processor_contract_v1.py
|
||||
01024f947f0326c124293a30e4f2cdb859f21cfb2d4c07f9c1030e2934f7bc44 cut_processor.py
|
||||
ff092ad2373b57321f87d1dd123fff8a99c8207057591e8526e56cb1424d47c6 dashboard.py
|
||||
f184bf3e546db0253ffb71895e8d42aeb06588c71c4914c2fe656f42ef463c9a debug_face_registration.py
|
||||
a9acce1ebd6ea821a8dc5009b8fc40586a98d31c23e93c97fd844bdadbda4ed2 deep_analysis_112_36.py
|
||||
7767ee7455a956d14d286ad558c4c312c2ad3ccee1c73adc1bc8f761c96ad72a demo_dashboard.py
|
||||
425290c12161c5cfcb0c505a737ba3951656b39e425e792919d4812e15b9b8e3 demo_face_learning.py
|
||||
d7e3e27e6a65b1fa62530ee954c227dbb4f97593c5a5dcc48b39e5ebae4656e5 dense_scan_traces.py
|
||||
df79b7fc7a03a8e754de5123a23bb33b1d5c23d832adc1886fb846ca517dd24d detect_language.py
|
||||
f6f8047e24ebbec81ef27dd38f4242e63385f8ebe5be471cae156b8aa5fc4477 detect_objects_keyframes.py
|
||||
e61d2ef5043bda3674a0050d83ba3bc6a70c47f54e456124a736b4328f0c0638 detect_stamp_shapes.py
|
||||
f23a382113e9c7de2ec3b24e95160daef48f9336ae6d4ec9ee7a18f4bf529f6d download_places365_classes.py
|
||||
a747e5e17960b972549714786bb9e28ea578e10e6c80788e298a0149c970bcc5 embed_faces.py
|
||||
f1a2b3820e1a763eba6d8d905a5bb87f5a9b4a2f005e709e313bb7505ba7ddaa embeddinggemma_server.py
|
||||
43c540c02c1be992e7d44ab4fc76a759815db3ed5f25bcbb594328b50ed7c73b export_file_package.py
|
||||
19d23e4604d5532928412afe4d5d39ff49194ab4a046825286ae1be154326a1f export_file.py
|
||||
5f10bab1dcb0b5fad233a74069f9e2f89043e7c848c9c38ae7e2806e6940c75d export_identities.py
|
||||
2a1d0a1b853fd2c28f9a404871d33912f93521358576833be0999271bae02bcb export_person_thumbnails.py
|
||||
a81bf1d6af78c052e638f5d5677b4edb512d0de5441025d86fd970d3e7993922 export_sqlite.py
|
||||
2fe8c0131dde21382cae1483825d489fd467c2491a0cb91d5c1881df2e402e9f extract_face_embedding.py
|
||||
8b5cc0ff437fb4dd0df28b7b20a78469cdca3621e2eeb4b6d46ad2391acb0596 extract_female_faces.py
|
||||
bdecbaf0496bf536dce2ef4897f7090749820d15dcca03492d4d736ab0f8c6c5 face_benchmark_runner.py
|
||||
22319a38bd684fb235fec681ddc60f45821e4bb2181f2b31fdf945f7ad9a1b85 face_clustering_processor.py
|
||||
5adce4e444743331fa592e13d71e52f26554eadb9744d350a7654a449a8fb8a3 face_count_comparison.py
|
||||
3574454c74eaf11021f9052f77d93044cca4ae0285d0f2630b4016c2ec0df783 face_cross_validate.py
|
||||
4f09b3b66b14a5eefb14fcf915a1ad1e9147010f6ae7671731566679b1cae461 face_embedding_extractor.py
|
||||
d05c65221cbe787e4e29a4de1966edb9e89fed47e9e89c9d065e1d5cb46cf178 face_landmark_qc.py
|
||||
28776dfcc6ac40e9481c25467438745fed60fecdfd4fc19f9f4c7396397591a7 face_mediapipe_test.py
|
||||
f4d1b4334a49357b74b80e390ad5a3d16263e51cbe5cab661af92bd2e9721f02 face_processor_contract_v1.py
|
||||
802015c73dfce0866f2a0bc94c645aa35ba30a6de78244af23090bb1f1828c6e face_processor_mps.py
|
||||
96ffdbde3f4d87e9942f9e1f4c93cbd999dc404b43e00d4cdcbb22de3c0f16b7 face_processor_optimized.py
|
||||
4c3915a7465f524e706940c9813614ec4920cd6f8647602ef32e88fdbbaf8fc0 face_processor_v1.py
|
||||
d6ddad29a5e53b43b887554072d7965f0535e47fb62dad1a8b87e44fa1be6015 face_processor.py
|
||||
8edab61189ad1a8fa60c203077e814e82d46c5bae67054fa2ab1958e199c05f9 face_recognition_processor.py
|
||||
9ea19f357b3fcec6c8b3875c538e53cb46e407ab188cd544963e0123e535fa03 face_registration.py
|
||||
72648816de611fd9b84d2b98c177b8b4f24374024b69184e8151c06cf44d633b face_statistics_report.py
|
||||
499f197a06f50839ebd5350af380fa56506ce08f073ba40c0e863b8e02b34133 fast_face_clustering_processor.py
|
||||
0191781635b98d0675969fb87733af19525d7b5c148723346c5378c08a00fe33 fast_stamp_search.py
|
||||
00e7e8ed06f6a0f2c46c84a47d7e7f5d366acee941d546a52c4b1b7885c71e08 filter_stamp_colors.py
|
||||
5341fd648cffafc77568070313b06417636943d50ff3b4380a61381260acaafa final_face_validation.py
|
||||
213793ab719f4ef42ec9b22f351dd86d4739211c17be486a46b76ba7e64fd8f1 find_blue_stamp_opencv.py
|
||||
e1490317c0f56b895f73cfbb6f57c8e3ea5c65304bfdd7663f103f6b564e148c find_kids_pose.py
|
||||
08d4cba0650f6a22fc134d07fd15fe8784c8472c3ba687b587e31e0b980e2b1c find_kids_refined.py
|
||||
aecec0784ce5d0e98176c15798f05d4f67ab6a686f9ffafba71fbd82157027f8 find_magnifying_glass.py
|
||||
620db08dd84f00af0c6d744dac54c68360548dd5b2cc26b12ddcefd936239b2e find_pink_stamp.py
|
||||
1f4555b3578f4dc6bc08aa37e34eda1d91ea25d8134439771678d1a57bfdaeb9 find_realistic_stamp_opencv.py
|
||||
277aa3b48eec2e739de3bb95ef501ffbd24104aa2a1bdef28c844ef44fd75013 find_small_stamp_opencv.py
|
||||
fc73bbc9605938db495bd33ea74955e454e9384130531a16d42f25dbd9b515d8 find_stamp_in_hands.py
|
||||
c6ed0f12e78c12df977ddca5d699f58edb174b47199f584e7a24dbdc3b7d02b1 find_stamp_in_magnifier_scene.py
|
||||
ecf12e346619c27a985452e9f84ee262c2da25de9df0ff6e0b293279ccba559b find_stamp_opencv.py
|
||||
4ff93cbcc781a5cff023f78006f1aebbe2d954405ae7d00a473fef6b41b2ebee fix_asr_text.py
|
||||
4090cb892115843a909aa41426c0f39c5a53d8d88a5db69499ec8bafcb780d77 florence2_scan_stamps.py
|
||||
e90e4447db3328b64a2062ca13ed41f6a045220d8fb640542dff5b790d3c4d3b gdino_comparison_test.py
|
||||
7071a9999057c347e2275381f1f0c58e19aa8581d70a572d3170ed14a295a48d gdino_frame_api.py
|
||||
891410310b415ff68a0f7ee0aa39e84eef7f2c75887487bdb88b8f4718d40e94 generate_asr1.py
|
||||
24efe7db016387b40bd9caae449f0445a3d47eb878c00399803bb6e78e6dd5fc generate_benchmark_summary.py
|
||||
dc956a78a3ed26686f45dd6d6d9cb42c023751fcd9b8789585450b6df63670a1 generate_chunk_summaries.py
|
||||
8a0922d75fdc7c5994ebfb31881d765db4b105cbcddfcaa4b4c49d11950b8df4 generate_chunk_visual_stats.py
|
||||
4860bfd00cc6c1c842c2f8e17e725eebca191d81067af3cb5a28661b45d74bd3 generate_parent_chunks_gemma4.py
|
||||
e9fca223a8329ff6bdcb8552fecedb2d8b4607c6516c373c3023f29edfd42e06 generate_sentence_summaries.py
|
||||
cbae7c3e85457274e8c284005196c39dc97f9d9200ed6b0e4ea266e48a381d3a generate_synonyms_llamacpp.py
|
||||
57512cd7a5ec2f52813717fd3d81dec1aaa69dc9c91a9edbca847e7012b1c86f generate_synonyms_ollama.py
|
||||
dc495cb8127858fa03a5f8b8bb4a772c5934ada1abecf97459bf71de80417672 gun_detector_scan.py
|
||||
1a7cfb72723b3b94e3f4fe368477ba693ac3d20ac7af7351962bc548c700b451 head_shoulder_bench.py
|
||||
b2fe8e4d8d7d1057ba928fc5e190f4a06cb60e83e2a02c5d7c423791596c11b8 head_shoulder_quick.py
|
||||
ba5e67a97cb465e6a1a942c2f7342406031759ffcea2b897ae963bee4bc551c4 hybrid_stamp_search.py
|
||||
f5847b6c8ed4c7c51290df9032d5a192317b5f03b5ff418ead1181a6e1b655f2 identity_agent.py
|
||||
12237fa6cc5f0d2dcdd05f26fd50c0a7bfd541d1c922a1640d131fa0c4d6f4fc identity_bind.py
|
||||
046aa90eb4a4b830910912362a9865d1e6170f5bc176fae42be630f967f9d3ff import_file_package.py
|
||||
7cc260d4411ab13559803686f8b645afa07738d652d9459830aecac268597fa7 import_file.py
|
||||
071e3a5141d04cb9e6bd31489a835c778608785896b18ea7fa65e8db9f1547e5 insert_chunks.py
|
||||
d3d53f44daa7f1526488677b141e90fbf4aa5625369b96a3ca275b802414802f integrate_face_asrx.py
|
||||
4cb6a93ef8006cb69e8bdb1bc72899ee9bab1bf7eceaafe9896923bb7023bbd5 integrate_rule3_markers.py
|
||||
75aa3e4bffc9f9cb8b9254db19095c93c3efb43d465fb5dcca8c7b9b730f5c59 integrated_body_action_decoder.py
|
||||
f4dd2e21fb6b668bdf0c51cc56e214188b46937b96a2b4a10d13783e171d0472 language_router.py
|
||||
bef426641645fcf7dcc68c87e3325a6edf3f70925febaf1df84f7c6ff87681e5 lip_analyzer.py
|
||||
7f98b0cc8379b3759cc7e805dd56f736cc518093e83f43b2e5ecf559a19b95f0 lip_processor_cv.py
|
||||
a1473eeba17fce25e4678234fe4e8793a132514e0566b03b36a0bec04eb93acb lip_processor_media.py
|
||||
0df61396756ee22d35356776c189b354458661916c8baf85bcef97c9f8b62ec8 lip_processor_mp.py
|
||||
3202aeca29e651ef1a54f47681c6b3b2d0680555fe3c6d318a932bb12b49e58c lip_processor_simple.py
|
||||
fed15bafb5e09715cc03962f465b2ff618bf05ebeafdf932643690c9635c9840 lip_processor.py
|
||||
b9532949bd145c0411876bdf3a8cbf1540b4233f7585465ce6389928e1bfd908 llm_metadata_enhancer.py
|
||||
1773054e8d563b493865880d0d8bda105e3eb6fb536a25817517237b3bb76afe magnifying_glass_analyze.py
|
||||
7d4d048c452bf273f4a6d96da13eb7bab6aa60ca9dd51de5ca0fb0a01e587b13 magnifying_glass_extract.py
|
||||
8528bbf89d2770fa5a23f461274038898be251fb6e48c5d3adece5aab3bf976d magnifying_glass_owl.py
|
||||
cb645f5e29ee5a36b2f97812039abfdaed7328386bcd25ad7b742af6a6b16399 map_speakers_v2.py
|
||||
a90bd3fb729a05010c29a213134c60cc0bdd17769e27a7d3f1250919b7bf1613 match_face_identity.py
|
||||
2d864dc831c2fd0142b19b8ad2cda169c2a05facd9662d31861d29bb710c4979 match_face_with_pose_filtering.py
|
||||
889d4853707896885ed96ab945d4266acb213f4b122e2ba7c4563eb0e3e9e865 match_identities_to_tmdb.py
|
||||
b34ec373bcf65139e08e41967f58a2fc8ebb67a59c361074d3590cd16541415a match_speakers_to_chunks.py
|
||||
fe6260a94d01d8b43d0d3b59eb820cfd7b4711c907343a1261c69f9010ae990d mediapipe_holistic_processor.py
|
||||
bb36844b4d13bba8edc1b7f0703f02081b62bea795535b8cd8dcbfdb4281f402 migrate_asr_to_children.py
|
||||
819312cbfce6e68a0d8d731e02d283946f79de6044f207991ddf9a28ac853d79 migrate_face_results.py
|
||||
c3d062aab67b5177ac7bf2c3ad2f0e578e12c9893e377f68339a17cc2783316c migrate_identity_files.py
|
||||
c418f6e50054fa7eae1d0d879e28997b98f57437acec48b53ecb09f332728867 migrate_to_4188.py
|
||||
6f60aa899e06f05e575cb5b461ea517481119cc32644566245d74c96eccde722 multi_stage_stamp_search.py
|
||||
b24e2289c00f803c8339f59c34d44ed6c53a3c19dafc13e72c4b260d6bb312a6 music_segmentation_processor.py
|
||||
da2546f84d0dbd711c8800ae4e32e59d9c38de9e62e1b423c4518fa1fda1dbea natural_language_top10.py
|
||||
78c3d1a9302dbfacdf9b3655dab07348957fd9dbb4af94aae83eefecd5343a33 natural_language_vector_detailed.py
|
||||
e924f04d68c9a8211ad373da811aa6671d2c5654281c1634dbf8b1e5e5b51533 natural_language_vector_test.py
|
||||
df6ac92367b1afb50c0af958e362d87555fe569f608a8d213e0a593e2a43cde8 object_search_agent.py
|
||||
fd39b779a0337f521940f3f7b159931f1f207f200eefd610183781fdcf3dfafd object_search.py
|
||||
42d2952fc78b57302b0d12bc3d45790a2c2c46d4ffa3c713a82686134bd63f13 ocr_benchmark_runner.py
|
||||
7b3ccb5c4ddd4c62c5ad04d0e3aafaecc2c1441012b6a98613cdcf055e2e50e8 ocr_processor_contract_v1.py
|
||||
271023eec42d6be4a1ce6ae2ce3f29e825210a57e6bb37554a6f7fdf54616f9a ocr_processor_mps.py
|
||||
2e73c41285e52ef013594fcd4d20df9f5781bfc26bcf62e54dd2c04ec44200c3 ocr_processor.py
|
||||
62196108cb3337b5f9a873d70d2981ac8f49152369afbcc8a12b3a13de579e80 opencv_stamp_search.py
|
||||
b2e8d552c272fd173c77693e9453a85fe16dfc12f7c2cd304d299c6188c14077 paligemma_vs_gdino.py
|
||||
1534d5b7617dbae77f7a37a2c33a89b90f965247a6828f00b73ea6b720f6f4fc parent_chunk_5w1h.py
|
||||
5208c738d4b615282813d351daf09872ce516121bb604caa64968ef5e52c53d3 pipeline_checklist.py
|
||||
8f80c3a2be5c330e2d1853d9250a171c75db84598dbf3304280c42237ed4fb1f pipeline_status.py
|
||||
94db44c0f49115a677d117d4901a1b7991c1517905300eaa495dd62b8ac1c79c pose_processor_contract_v1.py
|
||||
167dee5e42c6bd46674bcffcfd92f368fc0b48a1f42c459c806853b281bc6482 pose_processor_mps.py
|
||||
a6ef3a785ef5c6dc47fa38dbed80d76bc7d4bf48cbaf0f7edb3d26df98d7262c pose_processor.py
|
||||
45e6798dc5900f2f7c8776a2d260c122aae5068a075256b8a5c02e8d0be6c131 probe_file.py
|
||||
01c7b3c30c1531224f9605f0ee633285fe8489ab2d0a3c9c6a41f2b2b60d6626 quick_stamp_search.py
|
||||
e3143673a2bff6139e05c82446fd8770c4b7e59a854a42c3b29662f5ac75efe2 rebuild_parents.py
|
||||
4aa98981632d4f8a11039c510e86aa296ae1cd4b399fc871ed664ac11e445bd9 rebuild_story_content.py
|
||||
090137a5872edfed1b89c97b537d13ad8aafda9a705ebb4c54f30352503e5e3a redis_publisher.py
|
||||
750f778946b56bc57c47d9d2295332bb0f8cec2c1aa03c6b882d39ef4432673d refine_search.py
|
||||
0f8a6a6866a5797e964d3b17e2b7ef146fe7a798f09fcea982fcda6f629b4d06 regenerate_parent_5w1h.py
|
||||
3ee192b623f290136b36bd63abd018aad6e6639a9543970c3415734628b33bd6 register_sample_faces.py
|
||||
334782f0f66d0ad3818a51adf6343186a2de65467378ab68a81ade806e496af9 release_manager.py
|
||||
9a44cdd155953778b52ac0cfb118504c56eb6b1141984365ffbb717e28f3e65b release_pack.py
|
||||
3906b48f3a7764d19605def2bf8ef84a54a6afe64c9291a7cc0881a91472a826 render_face_heatmap.py
|
||||
44e432c31a35211a37dd26695772b7e250487ac42ba4f16a56f843277c2fabbf render_offline_report.py
|
||||
3fac1e6a4125042185a2ce82771f695c562b3137c7aa58a912bada00ad8ecf78 rescan_single_frame_traces.py
|
||||
9c3212cb455c2a6230be918448560fee00c153a8956ffd04fcb62974d5e1abff resume_framework.py
|
||||
7c95ec08daf4f980bd53233503b7a4fa01afc08660e8fe8cd031ea3613ead8f7 save_events_to_db.py
|
||||
24795e1531fe05e33d515104e4fb2f9567b46d802ef1b5a38f11268cf105be76 scan_charade_stamps.py
|
||||
cad2da5073577f851c5cb2abdbd7cab05b39caa0d1179ccc89c378a7df2736c8 scan_full_video_stamps.py
|
||||
03ae71470331fe5b7f8e394f7f789eee08cad4ed5ec9196b46ab2c9dbefa7fec scan_handheld_objects.py
|
||||
d3935ba498786cf260d9d5370ca60d3af7bc4fd438f6be33ce23cfd0b7bab593 scan_keyframes_opencv.py
|
||||
12c9b35212f587f5adb37584bf3c3844804d2bc642ebfc5d82b86b44f46d2472 scan_keyframes.py
|
||||
f386130ac203308c904ba7efea09ce0ca0d640d36762b113bf0cfedc24d7f885 scene_classifier.py
|
||||
482edae04e5467a68c77729760db53d3653e8d7654fa49e5ec9a36f1f8f22616 search_blue_stamp.py
|
||||
e3786422932138272d1096ad4c800594e62c9640952a286a9158372a1e5443e3 search_envelope.py
|
||||
2df1e259c2e52d10d79b20856cb94ffff5a9bfdbe47cee587b1148b2f1c16101 search_objects_in_hands.py
|
||||
9fd49be8ab16f94fd82efc5ae035c029372a7ddeb7fd779b557f1917cdc14592 search_vase.py
|
||||
7a6d8e7c435368f6218db972c04a7be16d7d6680d8d4374f82c05b7162716b9d select_face_reference_vectors_v2.py
|
||||
2bcf7c1b3c407b51a134a5ee4982713f0ea387cfd6df01ed75554c94603971a6 select_face_reference_vectors_v3.py
|
||||
d52098fcf1f9f7ba14f31a9a90bc5b3bc933e1a5e5697e3d09eff389c153cb18 select_face_reference_vectors.py
|
||||
a02cb37639275d86ae0b4504d21f50963b45aaf94630c59472ba30d07722e50c simple_api_test.py
|
||||
02516ab1616c1756c4f8041f48ff12811cc5d672c53b34850b84ce682fefdff1 simple_face_stats.py
|
||||
b024d9bfe244d0d058daae0acd314b9344d6f0912e4f3b02dbc618f9fe3e4949 simple_test.py
|
||||
af8703506769f3cdb89ff7849b071c2421307717850596dd86d2fe0b053e7809 smart_stamp_v2.py
|
||||
5e5f86d47ea2b75bcaa8662689f73af1963645149c0da688dc43482616aa4e76 sound_event_detector.py
|
||||
bab7697e4b4b05e93babc116e0c5b13cbaf1f4d419a65acd5dc1de5bdfc510dc speaker_assign.py
|
||||
381ff240ce806ead7d6463ee40c5b830035eb6252180b4b0901b3c8313fa4bbd speaker_bind_lip.py
|
||||
5eede29fa0966974c1943792d7fcca2dd9179d4f23570cf1a3964dc97bc9ac1e specific_stamp_search.py
|
||||
d5363d832272bdb3c1d6f6d93eee7b7894893b9164a3f5ad5fa08a4a0eaeeb47 split_asr_segments.py
|
||||
8e1269f173f2c72de78857c2d83d3111b62ec89bd79f4fb00c3f57390986ae4f step3_asr_fine.py
|
||||
7592df8be5dc58376b33960bfa7fc0003c51114b70ebc01f1589f39ee9568d3b store_traced_faces.py
|
||||
7ac32c1e2146a19e6654ab3e4bbbfd42e1a6540fb8717d40d55c61e9f5d1bf71 story_embed.py
|
||||
74cc24b328a075f48b1f44a465611157f44eadc8f5dabf6d95cd5cc5f80dd9dc story_pipeline_full.py
|
||||
97628f0f1270825dabafdf0a69f10ef12c4ffe2be4ac12941315f06bfb084e7c story_processor_contract_v1.py
|
||||
1b1f42fc4bbff26551f26f4ac1e8a995dfe3ff98b940a29c9e130410965d0fa0 story_processor.py
|
||||
cdbc7ef88551e2b3a3771eac5be5e0360989e71fa009ac28c97e548507e08a5e sync_face_speaker_to_chunks.py
|
||||
8b08e9a33f5917aad10e070d6aa48805f5e7c23f905ba8fff3b8697b2109d962 sync_to_mongodb.py
|
||||
869b6c56fe16cbf8973826782a17503f02b5cd757ec025b944da693d38bdb4cb sync_users_from_sftpgo.py
|
||||
f64cc6dcb72f54d3e97aa981b40591aef4804ca769e1f14628d901b98bc6aeac terminology_manager.py
|
||||
455546b9bb3a2c2c877c7720229b254e75b28eea33b3715d1731c02ca85294ae test_api_correct_usage.py
|
||||
b03dc1bbb091672e7da2b131850b17badac896b4fbba92fe9bce76c232c99be4 test_api_with_key_id.py
|
||||
7d295c77d5bcd4c72c5673370af48cc89bbccf9292c3b82aad3a230d242547a9 test_args.py
|
||||
f474ec88e6634decbf178da497443fa709096b174bb4a4320a07256f516b1044 test_asr_large_model.py
|
||||
aa952524dd86f346740ffe555075b74adf2e60bb822bb04a943a51b1fd262445 test_birth_uuid.py
|
||||
db87badad7948527325a528400d67a4eeef76abf8d13f5c4254c812e944e4e0c test_end_to_end.py
|
||||
e191c98a82f7e089f7dccfc4c536244da2bf14339f982a3afef05d33332c3755 test_face_api_final.py
|
||||
1b97c9aae2e1744aa7aefb192eaef86c64e6134efc8f08ffa9a274bff16a58d3 test_face_api_with_correct_key.py
|
||||
f7e4078f31b1ca8494c18878219cf2f90c301f19fc851b9e7084657b71a5e150 test_face_api.py
|
||||
9eafc49f8fa42b4cd58109e9b725b3aec3b06943ec426919b1788838ccf1ed92 test_face_db_fix.py
|
||||
38bce82b167e0c97b257cc6b955fdc2e9ded581ce2d39eb0fd2c60249275394b test_face_direct.py
|
||||
24e82bf0af82407e6c04361e9a671770cbfb0b05d92df589bd0d5a0118bb5a98 test_face_learning.py
|
||||
8dcdb144c4253fbb466f220359b42c2a9579193865e320a56e682e384c2ae176 test_face_recognition_integration.py
|
||||
b921e3256fdea176d4391116d1ead472c4f3ca8aac6999140367818818c35ec3 test_face_registration_api.py
|
||||
9af6c6ff0c766b3de92185c3602f2b8b62b815bf88dcb0e3251c2676e61e0a48 test_face_tracker.py
|
||||
4f70eadb6a8b80eb8febe32b17b77e58d1a4823cc5d598e5ea45555342d2d4cb test_florence2_direct.py
|
||||
0588be0acea540950d737943073f71e769b6301374eaa4ff7fdb96a80145c4e0 test_florence2_pipeline.py
|
||||
694c15193616157ddae4bdb0a45feada2a8f8490f01d290a28aa77a4b24eabb2 test_florence2_stamps.py
|
||||
2c281f698616a83e9eeccd610555d9f9ab657b2deac65ae9e3dbfba0b450d9b0 test_identity_db.py
|
||||
7a73e8314ea7e91ca9dad3867a83b9c1101fdab09bdc0fdac0f798d0a7a204f3 test_llm_capabilities.py
|
||||
68300f87b96a474f06a3071a833e6b3ae48d1db5fb8a7e5a3ec1834fd878d808 test_multilingual.py
|
||||
c17cdd0f4ffb7a151a634add08d13cc576ba7a848bb20f54fb97d0c1d9d81cc0 test_object_search.py
|
||||
d07bd363a2878259fbf4ffcba40e367f7f1bf4171b5a5dfdda97f7a53b450d0e test_ollama_feasibility.py
|
||||
8421003b1f66cbd21c6fe5d3aff0a526897753e959b23905ca8f502f644f66a5 test_owl_vit_debug.py
|
||||
6f9e8b7947229ea4aa0a62b59bda5fcec05bd74f6c00dc4a7b06d932bd1b730f test_owl_vit_stamps.py
|
||||
da91a7c97466ce7f03cde13aa9bf6e691b3e482d2cac74519a2e1a61a2abb05a test_parent_chunk_generation.py
|
||||
19d9f2492d3b04b7dafa008f106767d3107dd36b0c8e4601765dca30131027cd test_places365_scene.py
|
||||
de44553023067362e8b2223f03e1bff55fcbd2f11ddf3d01060dc02c4675a744 test_probe_file.py
|
||||
c0e987ba06a61cc0426ffbca8af1eb51a97bd79acab59b70453cfbb18eaee093 test_processor_performance.py
|
||||
7b4b55e23dff35ba107b3da5b0560d03b1b41dfdea1d3a59eac777b4be4d4033 test_pyannote_audio.py
|
||||
5cb8b42033ffba41f25e7ef74ef04cf352c0c277a9971e9eaef53fd673902712 test_pyannote_multilingual.py
|
||||
8580e689ae148754e03d958419e108241040a012584ba49e8a90db114a9f8c13 test_scene_api.py
|
||||
1194d450070b1f42e045d98e532f41205bb3e52fc48ba26e7c9b72a188fe1b2c test_segment_count.py
|
||||
147bfffeac9561cfa407207b04a825862ac623ba97deecf5ed7c6257432dc62c test_speechbrain.py
|
||||
22e4b865bc769329c1146c2f914395044a9bc84cd2a13acf68fb374a57fe1e3e test_v2_detailed.py
|
||||
a616570a2a080b5b19f4bf783877147e714a014103b274143dd37984a946ca08 test_v2_model.py
|
||||
7b83611f6b3028500c91c62197f774c0769e299136eca8dc4b612a7b5743e3d6 test_v2_with_text.py
|
||||
1dd983c78074a61ceec26d7e3623d40772ca55fd6ee63ba368afe756c66ae091 test_with_real_image.py
|
||||
1b738cc0d69d33e967cbb775def0a7f58dc02f1911404af56a5825bd60a5b75b text_semantic_analysis.py
|
||||
a4221417ae00add76881c6c715ee4257c263e2dfd0a846a8887738682dfe8cda thumbnail_extractor.py
|
||||
0d188a738a0df79ead10065d9f17c366fe159c862bd4bafa2860d0e6ba2640c3 tkg_builder.py
|
||||
a084d3b5840e920d552515febffa22b34943b9efa8b73adab9cd193372e71592 tmdb_agent.py
|
||||
8b97f0fdfc0899460bf23d420dba0a51a34737c74ebad0519856909d198662bf tmdb_cast_fetcher.py
|
||||
4858909a0beaf8397becf4103be17fcc350841217afcdc1d917c48c512a9041b tmdb_embed_extractor.py
|
||||
54d8321dfe0f8caa669e4a9d1b48dc772a5b25817eab95b552944140c91f457d tmdb_identity_integration.py
|
||||
2a84aa2dcfb83ac385d2c394f884926f306c81798e4277a26dbd1f3c5506be46 trace_face_aggregator.py
|
||||
61d3b4b362722ce24326a204f1b72cc7b1dcc20cf3264a4f526d4ea343a8d33d transcribe.py
|
||||
ede9a184fd51ef4c87eb3e2541f09b91739a49986cb588591a7c6fbb33433020 unified_synonym_processor.py
|
||||
a408f294c3a71eb6a0eea80b9b586f73dedcefe286c62233f713a7428a9979be update_all_demographics.py
|
||||
e6520bb10ae6835ceade487ceb5e3fa549ca6f06de35b2c785d649921ef443f4 update_fine_speakers.py
|
||||
a2191daff2ad228725b6a66f0e472ec659a6b4fa8f2cbbd74d1bf9c35cca63eb update_person_demographics.py
|
||||
1a7dddd1db467990ee1c685d61b971babfa30c3ae3a754b5df8f3b4c320f3ed1 update_qdrant_uuid.py
|
||||
60060753cfd2a6d1241e55bf40a0c74f1df15739656d0349e22e8543036b2424 update_speaker_assignments.py
|
||||
fdc61009c351263e0018801b32ad90ffd8919af611a2a0580546be7fd62c99c4 update_terminology.py
|
||||
4840c11964a59eabad26b97fe01033ccaf7903e2d24edd5e1035f6dd5fc995ea vectorize_4188.py
|
||||
078979114c5f248d2bfd43aa8df55235fa03ab812f26998b984cd485a3d2cda8 vectorize_chunk_summaries.py
|
||||
ff98864f1b11795cc3bb64f30ccb6f8609771ddc7a5df2c003ba7c2233d16fc2 vectorize_chunks.py
|
||||
5880c128400e6e36c8eb7dffd009dbbc99dd13f8575b0037bdc854e25ddc41fb video_comparison_statistics.py
|
||||
0a1501ffdc027236cdf88706b3d61229e2998ab268fd57fb60e399ccb734b6a1 vision_agent.py
|
||||
eac8f90fbbb655614abcefc4b887e346bf94db5f015d33d37bc9514fb030489d visual_chunk_processor.py
|
||||
c165dfc5fc981dc731b25ef414184ee58e56b73b148d41a32fdce985c701efd5 visualize_stamp.py
|
||||
6c65a82fdd1d585e20bee4fcb2d1bdec2e6220bda71d6ef9cd00d6a3cf74c4d7 voice_embedding_extractor.py
|
||||
2b3a7b357db4ddd07ca30bf200c6600724e33441d8def0a4d9a39673e2cfb1c0 weather_sound_detector.py
|
||||
206b61ebf3c91d7ce3f1488247b52aca6e955042d8aa979c59723e3ff10dd36a yolo_benchmark_runner.py
|
||||
e8cb0963c90fbd1c2aa91141f80340edd3c9560d69780dd825d107c6ed14fa64 yolo_count_comparison.py
|
||||
dad775ecdca0144bd14b7abaa7ec8fb213e8b9428e39906abce541e93db496b6 yolo_processor_contract_v1.py
|
||||
74ff880e664ec514223a4f220b682fbc87089f8c0851c93ac68c97269b8a59b6 yolo_processor_mps.py
|
||||
8af0a6db683b6626e07820b302135ac5960d38e3d4b3d187c640b23ce8a14f72 yolo_processor.py
|
||||
e13cf22b9aeae96c7e28b4512dd2137743a25eb59027da446966c1aaaaf4ce71 zero_shot_combined_test.py
|
||||
f4aaf017ff588999f06cd9ba1787517e06c6d6e6228a15a54d8aa4f54fde5eb3 zero_shot_gun_test.py
|
||||
0a285b8ec33d7999e9d4ae8d43ce768c9f06ee1929e13a6809e98bdabe6357ce zero_shot_objects_test.py
|
||||
35
v1.1/monitor_v1.11/common/load_credentials_v1.11.sh
Normal file
35
v1.1/monitor_v1.11/common/load_credentials_v1.11.sh
Normal file
@@ -0,0 +1,35 @@
|
||||
#!/bin/bash
|
||||
# Momentry Monitor 密碼管理腳本
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/common/load_credentials.sh
|
||||
#
|
||||
# 使用方式:
|
||||
# source /path/to/load_credentials.sh
|
||||
|
||||
# 載入環境變數（如果存在）
|
||||
if [ -f "$HOME/.momentry/credentials" ]; then
|
||||
set -a
|
||||
source "$HOME/.momentry/credentials"
|
||||
set +a
|
||||
fi
|
||||
|
||||
# 預設值
|
||||
export PG_USER="${PG_USER:-accusys}"
|
||||
export PG_PASSWORD="${PG_PASSWORD:-accusys}"
|
||||
export PG_HOST="${PG_HOST:-localhost}"
|
||||
export PG_PORT="${PG_PORT:-5432}"
|
||||
|
||||
export REDIS_PASSWORD="${REDIS_PASSWORD:-accusys}"
|
||||
|
||||
export MONGO_USER="${MONGO_USER:-accusys}"
|
||||
export MONGO_PASSWORD="${MONGO_PASSWORD:-Test3200Test3200}"
|
||||
export MONGO_PORT="${MONGO_PORT:-27017}"
|
||||
|
||||
export QDRANT_API_KEY="${QDRANT_API_KEY:-Test3200Test3200Test3200}"
|
||||
|
||||
export SFTPGO_PASSWORD="${SFTPGO_PASSWORD:-sftpgo_pass_2026}"
|
||||
export SFTPGO_USER="${SFTPGO_USER:-sftpgo}"
|
||||
|
||||
export N8N_PASSWORD="${N8N_PASSWORD:-accusys}"
|
||||
|
||||
export MARIADB_USER="${MARIADB_USER:-accusys}"
|
||||
export MARIADB_PASSWORD="${MARIADB_PASSWORD:-Test3200Test3200Test3200}"
|
||||
495
v1.1/monitor_v1.11/config/monitor_config_v1.11.yaml
Normal file
495
v1.1/monitor_v1.11/config/monitor_config_v1.11.yaml
Normal file
@@ -0,0 +1,495 @@
|
||||
# Momentry 監控系統配置文件
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/config/monitor_config.yaml
|
||||
|
||||
monitoring:
|
||||
enabled: true
|
||||
check_interval: 300 # 秒 (5分鐘)
|
||||
|
||||
# 數據庫連接
|
||||
database:
|
||||
host: "localhost"
|
||||
port: 5432
|
||||
username: "accusys"
|
||||
password: "" # 使用 psql 連接
|
||||
name: "momentry"
|
||||
|
||||
# ============================================================
|
||||
# Layer 1: External 監控
|
||||
# ============================================================
|
||||
external:
|
||||
enabled: true
|
||||
check_interval: 60 # 秒
|
||||
targets:
|
||||
- name: "ddns"
|
||||
type: "ddns"
|
||||
host: "momentry.ddns.net"
|
||||
enabled: true
|
||||
- name: "gateway"
|
||||
type: "gateway"
|
||||
host: "192.168.110.1"
|
||||
enabled: true
|
||||
- name: "internet"
|
||||
type: "internet"
|
||||
host: "8.8.8.8"
|
||||
enabled: true
|
||||
|
||||
# ============================================================
|
||||
# Layer 2: Service 監控
|
||||
# ============================================================
|
||||
service:
|
||||
enabled: true
|
||||
check_interval: 300 # 秒
|
||||
services:
|
||||
- name: "postgresql"
|
||||
type: "postgres"
|
||||
port: 5432
|
||||
host: "localhost"
|
||||
check_cmd: "pg_isready -h localhost -p 5432 -U accusys"
|
||||
timeout: 5
|
||||
enabled: true
|
||||
|
||||
- name: "redis"
|
||||
type: "redis"
|
||||
port: 6379
|
||||
host: "localhost"
|
||||
password: "accusys"
|
||||
check_cmd: "redis-cli -a accusys ping"
|
||||
timeout: 5
|
||||
enabled: true
|
||||
|
||||
- name: "mariadb"
|
||||
type: "mariadb"
|
||||
port: 3306
|
||||
host: "localhost"
|
||||
check_cmd: "mysql -u root -e 'SELECT 1'"
|
||||
timeout: 5
|
||||
enabled: true
|
||||
|
||||
- name: "n8n"
|
||||
type: "http"
|
||||
port: 5678
|
||||
host: "localhost"
|
||||
check_url: "http://localhost:5678/"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "caddy"
|
||||
type: "http"
|
||||
port: 2019
|
||||
host: "localhost"
|
||||
check_url: "http://localhost:2019/config/"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "gitea"
|
||||
type: "http"
|
||||
port: 3000
|
||||
host: "localhost"
|
||||
check_url: "http://localhost:3000/"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "sftpgo"
|
||||
type: "http"
|
||||
port: 8080
|
||||
host: "localhost"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "ollama"
|
||||
type: "http"
|
||||
port: 11434
|
||||
host: "localhost"
|
||||
check_url: "http://localhost:11434/api/tags"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "qdrant"
|
||||
type: "http"
|
||||
port: 6333
|
||||
host: "localhost"
|
||||
check_url: "http://localhost:6333/collections"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "mongodb"
|
||||
type: "mongodb"
|
||||
port: 27017
|
||||
host: "localhost"
|
||||
timeout: 10
|
||||
enabled: true
|
||||
|
||||
- name: "php"
|
||||
type: "process"
|
||||
process_name: "php-fpm"
|
||||
enabled: true
|
||||
|
||||
- name: "node"
|
||||
type: "process"
|
||||
process_name: "node"
|
||||
enabled: true
|
||||
check_interval: 60
|
||||
version_lock: "22.x"
|
||||
locked_processes:
|
||||
- "n8n"
|
||||
description: "Node.js 運行環境 (n8n 專用)"
|
||||
|
||||
- name: "python"
|
||||
type: "process"
|
||||
process_name: "python3"
|
||||
enabled: true
|
||||
check_interval: 60
|
||||
version_lock: "3.11.14"
|
||||
scripts:
|
||||
- "/Users/accusys/momentry_core_0.1/scripts/asr_processor.py"
|
||||
- "/Users/accusys/momentry_core_0.1/scripts/thumbnail_extractor.py"
|
||||
description: "Python 運行環境 (Momentry 腳本專用)"
|
||||
|
||||
- name: "rustdesk_hbbs"
|
||||
type: "process"
|
||||
process_name: "hbbs"
|
||||
port: 21116
|
||||
enabled: true
|
||||
|
||||
- name: "rustdesk_hbbr"
|
||||
type: "process"
|
||||
process_name: "hbbr"
|
||||
port: 21117
|
||||
enabled: true
|
||||
|
||||
# ============================================================
|
||||
# Layer 3: n8n Workflow 監控
|
||||
# ============================================================
|
||||
workflow:
|
||||
enabled: true
|
||||
check_interval: 300 # 秒
|
||||
n8n:
|
||||
host: "http://localhost:5678"
|
||||
api_key: "" # 從環境變數或 n8n 獲取
|
||||
idle_threshold_days: 30
|
||||
suggestions:
|
||||
- type: "disable_idle"
|
||||
threshold_days: 30
|
||||
- type: "delete_unused"
|
||||
threshold_days: 90
|
||||
- type: "optimize_failures"
|
||||
failure_rate_threshold: 0.2
|
||||
|
||||
# ============================================================
|
||||
# Layer 4: WordPress Portal 監控
|
||||
# ============================================================
|
||||
portal:
|
||||
enabled: true
|
||||
check_interval: 300 # 秒
|
||||
wordpress:
|
||||
site_url: "https://wp.momentry.ddns.net"
|
||||
db_host: "localhost"
|
||||
db_name: "wordpress"
|
||||
db_user: "wp_user"
|
||||
db_password: "wp_password_123"
|
||||
page_monitoring:
|
||||
enabled: true
|
||||
pages:
|
||||
- url: "/"
|
||||
name: "homepage"
|
||||
- url: "/wp-login.php"
|
||||
name: "login_page"
|
||||
response_time_threshold_ms: 3000
|
||||
account_monitoring:
|
||||
enabled: true
|
||||
check_interval: 3600 # 小時
|
||||
alert_on_new_admin: true
|
||||
|
||||
# ============================================================
|
||||
# Layer 5: Database 監控
|
||||
# ============================================================
|
||||
database:
|
||||
enabled: true
|
||||
check_interval: 300 # 秒
|
||||
|
||||
postgres:
|
||||
enabled: true
|
||||
databases:
|
||||
- name: "momentry"
|
||||
- name: "gitea"
|
||||
- name: "n8n"
|
||||
- name: "video_register"
|
||||
schema_monitoring: true
|
||||
|
||||
redis:
|
||||
enabled: true
|
||||
password: "accusys"
|
||||
alert_thresholds:
|
||||
memory_percent: 80
|
||||
connected_clients: 100
|
||||
|
||||
qdrant:
|
||||
enabled: true
|
||||
collections_watch: ["*"] # 監控所有
|
||||
|
||||
mariadb:
|
||||
enabled: true
|
||||
databases:
|
||||
- name: "wordpress"
|
||||
|
||||
mongodb:
|
||||
enabled: true
|
||||
databases:
|
||||
- name: "momentry"
|
||||
- name: "admin"
|
||||
|
||||
# ============================================================
|
||||
# Layer 6: 使用者監控
|
||||
# ============================================================
|
||||
users:
|
||||
enabled: true
|
||||
check_interval: 60 # 秒
|
||||
|
||||
session_tracking:
|
||||
enabled: true
|
||||
track_ssh: true
|
||||
track_web: true
|
||||
track_db: true
|
||||
track_sftp: true
|
||||
|
||||
login_monitoring:
|
||||
enabled: true
|
||||
track_system: true
|
||||
track_wordpress: true
|
||||
track_n8n: true
|
||||
track_gitea: true
|
||||
|
||||
sudo_tracking:
|
||||
enabled: true
|
||||
|
||||
anomaly_detection:
|
||||
enabled: true
|
||||
rules:
|
||||
- type: "brute_force"
|
||||
threshold: 5
|
||||
window_seconds: 60
|
||||
severity: "critical"
|
||||
- type: "unusual_time"
|
||||
severity: "medium"
|
||||
allowed_hours: "08:00-22:00"
|
||||
- type: "idle_session"
|
||||
threshold_hours: 24
|
||||
severity: "low"
|
||||
|
||||
# ============================================================
|
||||
# Layer 7: Storage 監控 (獨立配置)
|
||||
# ============================================================
|
||||
storage:
|
||||
enabled: false # 獨立實現
|
||||
paths:
|
||||
hot: "/Users/accusys/momentry/data"
|
||||
warm: "/Volumes/RAID System/momentry/warm"
|
||||
cold: "/Volumes/Object Storage/momentry/archive"
|
||||
temp: "/Users/accusys/momentry/tmp"
|
||||
backup: "/Users/accusys/momentry/backup"
|
||||
clusters:
|
||||
- name: "family"
|
||||
path: "data/family"
|
||||
quota: "1TB"
|
||||
- name: "work"
|
||||
path: "data/work"
|
||||
quota: "2TB"
|
||||
- name: "wordpress"
|
||||
path: "data/wordpress"
|
||||
quota: "500GB"
|
||||
- name: "shared"
|
||||
path: "data/shared"
|
||||
quota: "1TB"
|
||||
migration:
|
||||
hot_to_warm_days: 7
|
||||
warm_to_cold_days: 90
|
||||
|
||||
# ============================================================
|
||||
# Layer 7: 備份監控
|
||||
# ============================================================
|
||||
backup:
|
||||
enabled: true
|
||||
check_interval: 3600 # 秒 (每小時檢查一次)
|
||||
|
||||
# 備份根目錄
|
||||
backup_root: "/Users/accusys/momentry/backup"
|
||||
|
||||
# 服務列表
|
||||
services:
|
||||
- name: "postgresql"
|
||||
enabled: true
|
||||
backup_type: "database"
|
||||
method: "pg_dump"
|
||||
schedule: "daily"
|
||||
retention:
|
||||
daily: 7
|
||||
weekly: 4
|
||||
monthly: 12
|
||||
|
||||
- name: "redis"
|
||||
enabled: true
|
||||
backup_type: "database"
|
||||
method: "rdb"
|
||||
schedule: "daily"
|
||||
retention:
|
||||
daily: 7
|
||||
weekly: 4
|
||||
|
||||
- name: "mariadb"
|
||||
enabled: true
|
||||
backup_type: "database"
|
||||
method: "mysqldump"
|
||||
schedule: "daily"
|
||||
retention:
|
||||
daily: 7
|
||||
weekly: 4
|
||||
|
||||
- name: "n8n"
|
||||
enabled: true
|
||||
backup_type: "full"
|
||||
method: "tar"
|
||||
schedule: "daily"
|
||||
retention:
|
||||
daily: 7
|
||||
weekly: 4
|
||||
|
||||
- name: "qdrant"
|
||||
enabled: true
|
||||
backup_type: "database"
|
||||
method: "snapshot"
|
||||
schedule: "daily"
|
||||
retention:
|
||||
daily: 7
|
||||
weekly: 4
|
||||
|
||||
- name: "gitea"
|
||||
enabled: true
|
||||
backup_type: "full"
|
||||
method: "gitea_dump"
|
||||
schedule: "weekly"
|
||||
retention:
|
||||
weekly: 4
|
||||
monthly: 12
|
||||
|
||||
- name: "mongodb"
|
||||
enabled: true
|
||||
backup_type: "database"
|
||||
method: "mongodump"
|
||||
schedule: "daily"
|
||||
retention:
|
||||
daily: 7
|
||||
weekly: 4
|
||||
|
||||
- name: "ollama"
|
||||
enabled: true
|
||||
backup_type: "config"
|
||||
method: "tar"
|
||||
schedule: "weekly"
|
||||
retention:
|
||||
weekly: 4
|
||||
monthly: 12
|
||||
|
||||
- name: "caddy"
|
||||
enabled: true
|
||||
backup_type: "config"
|
||||
method: "file"
|
||||
schedule: "weekly"
|
||||
retention:
|
||||
weekly: 4
|
||||
|
||||
- name: "sftpgo"
|
||||
enabled: true
|
||||
backup_type: "config"
|
||||
method: "file"
|
||||
schedule: "weekly"
|
||||
retention:
|
||||
weekly: 4
|
||||
|
||||
- name: "php"
|
||||
enabled: true
|
||||
backup_type: "config"
|
||||
method: "file"
|
||||
schedule: "weekly"
|
||||
retention:
|
||||
weekly: 4
|
||||
|
||||
# 溫冷轉移配置
|
||||
tiering:
|
||||
enabled: true
|
||||
tiering_interval: 86400 # 秒 (每天)
|
||||
rules:
|
||||
- from: "daily"
|
||||
to: "weekly"
|
||||
after_days: 7
|
||||
- from: "weekly"
|
||||
to: "monthly"
|
||||
after_days: 30
|
||||
- from: "monthly"
|
||||
to: "archive"
|
||||
after_days: 90
|
||||
|
||||
# 存儲閾值
|
||||
thresholds:
|
||||
backup_age_warning_days: 7
|
||||
backup_age_critical_days: 14
|
||||
storage_percent_warning: 80
|
||||
storage_percent_critical: 90
|
||||
|
||||
# 驗證
|
||||
verify:
|
||||
enabled: true
|
||||
verify_on_completion: true
|
||||
test_restore: false # 僅測試還原，不實際執行
|
||||
|
||||
# ============================================================
|
||||
# 通知配置
|
||||
# ============================================================
|
||||
notifications:
|
||||
enabled: true
|
||||
log_only: true # 僅記錄，不發送
|
||||
|
||||
# 日誌記錄
|
||||
log:
|
||||
enabled: true
|
||||
path: "/Users/accusys/momentry/log/monitor"
|
||||
|
||||
# n8n webhook (預設不啟用)
|
||||
n8n:
|
||||
enabled: false
|
||||
webhook_url: "http://localhost:5678/webhook/monitor-alert"
|
||||
|
||||
# Telegram (預設不啟用)
|
||||
telegram:
|
||||
enabled: false
|
||||
bot_token: ""
|
||||
chat_id: ""
|
||||
|
||||
# Email (預設不啟用)
|
||||
email:
|
||||
enabled: false
|
||||
smtp_host: ""
|
||||
smtp_port: 587
|
||||
smtp_user: ""
|
||||
smtp_password: ""
|
||||
from_address: ""
|
||||
to_addresses: []
|
||||
|
||||
# ============================================================
|
||||
# 數據保留
|
||||
# ============================================================
|
||||
retention:
|
||||
history_days: 30
|
||||
anomaly_days: 90
|
||||
session_days: 7
|
||||
login_days: 30
|
||||
|
||||
# ============================================================
|
||||
# 報警閾值
|
||||
# ============================================================
|
||||
thresholds:
|
||||
service_response_time_ms: 3000
|
||||
database_memory_percent: 80
|
||||
disk_percent: 90
|
||||
cpu_percent: 90
|
||||
login_failures_per_user: 3
|
||||
brute_force_per_minute: 5
|
||||
362
v1.1/monitor_v1.11/control/monitor_control_v1.11.sh
Executable file
362
v1.1/monitor_v1.11/control/monitor_control_v1.11.sh
Executable file
@@ -0,0 +1,362 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry 監控系統控制腳本
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/control/monitor_control.sh
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
CONFIG_DIR="$MONITOR_DIR/config"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
# 顏色定義
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
# ============================================================
|
||||
# 幫助信息
|
||||
# ============================================================
|
||||
|
||||
show_help() {
|
||||
cat << EOF
|
||||
Momentry 監控系統控制腳本
|
||||
|
||||
用法: $0 <command> [options]
|
||||
|
||||
命令:
|
||||
status 查看監控狀態
|
||||
check <layer> 執行特定層級檢查
|
||||
layers: service, workflow, portal, database, users, storage, external, backup, node, python, all
|
||||
monitor 持續監控 (每 5 分鐘)
|
||||
init 初始化監控數據庫表
|
||||
logs <layer> [lines] 查看日誌
|
||||
clean 清理歷史數據
|
||||
help 顯示幫助
|
||||
|
||||
示例:
|
||||
$0 status 查看所有監控狀態
|
||||
$0 check service 檢查服務狀態
|
||||
$0 check backup 檢查備份狀態
|
||||
$0 check all 執行全面檢查
|
||||
$0 logs anomaly 50 查看最近 50 條異常
|
||||
$0 init 初始化數據庫表
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 初始化
|
||||
# ============================================================
|
||||
|
||||
init_monitor() {
|
||||
echo -e "${BLUE}初始化監控系統...${NC}"
|
||||
|
||||
# 創建日誌目錄
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
# 創建數據庫表
|
||||
echo "創建監控數據庫表..."
|
||||
psql -U accusys -h localhost -d momentry -f "$MONITOR_DIR/database/schema.sql" 2>/dev/null || \
|
||||
echo "數據庫表可能已存在"
|
||||
|
||||
echo -e "${GREEN}初始化完成${NC}"
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 狀態查看
|
||||
# ============================================================
|
||||
|
||||
show_status() {
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo -e "${BLUE}Momentry 監控系統狀態${NC}"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
# 服務狀態
|
||||
echo -e "${YELLOW}Layer 2: 服務狀態${NC}"
|
||||
local service_count=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM monitor_services WHERE checked_at > NOW() - INTERVAL '5 minutes' AND status = 'up';" 2>/dev/null || echo "0")
|
||||
local service_total=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(DISTINCT service_name) FROM monitor_services;" 2>/dev/null || echo "0")
|
||||
echo " 服務: $service_count / $service_total 正常"
|
||||
echo ""
|
||||
|
||||
# Workflow 狀態
|
||||
echo -e "${YELLOW}Layer 3: Workflow 狀態${NC}"
|
||||
local active_wf=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM monitor_workflows WHERE is_active = true;" 2>/dev/null || echo "0")
|
||||
local idle_wf=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM monitor_workflows WHERE idle_days > 30;" 2>/dev/null || echo "0")
|
||||
echo " 啟用 Workflow: $active_wf"
|
||||
echo " 閒置 Workflow (>30天): $idle_wf"
|
||||
echo ""
|
||||
|
||||
# Database 狀態
|
||||
echo -e "${YELLOW}Layer 5: Database 狀態${NC}"
|
||||
local db_count=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(DISTINCT db_type) FROM monitor_databases WHERE checked_at > NOW() - INTERVAL '5 minutes';" 2>/dev/null || echo "0")
|
||||
echo " 監控資料庫: $db_count"
|
||||
echo ""
|
||||
|
||||
# 異常狀態
|
||||
echo -e "${YELLOW}Layer 6: 異常狀態${NC}"
|
||||
local critical=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM monitor_anomalies WHERE severity = 'critical' AND detected_at > NOW() - INTERVAL '24 hours';" 2>/dev/null || echo "0")
|
||||
local warning=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM monitor_anomalies WHERE severity IN ('medium', 'high') AND detected_at > NOW() - INTERVAL '24 hours';" 2>/dev/null || echo "0")
|
||||
echo " Critical: $critical"
|
||||
echo " Warning: $warning"
|
||||
echo ""
|
||||
|
||||
# Node.js 狀態
|
||||
echo -e "${YELLOW}Node.js 運行環境${NC}"
|
||||
local node_compliant=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM node_version_baseline WHERE is_compliant = true;" 2>/dev/null || echo "0")
|
||||
local node_total=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM node_version_baseline;" 2>/dev/null || echo "0")
|
||||
echo " 版本合規: $node_compliant / $node_total"
|
||||
echo ""
|
||||
|
||||
# Python 狀態
|
||||
echo -e "${YELLOW}Python 運行環境${NC}"
|
||||
local python_compliant=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM python_version_baseline WHERE is_compliant = true;" 2>/dev/null || echo "0")
|
||||
local python_total=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM python_version_baseline;" 2>/dev/null || echo "0")
|
||||
echo " 版本合規: $python_compliant / $python_total"
|
||||
echo ""
|
||||
|
||||
# 備份狀態
|
||||
echo -e "${YELLOW}Layer 7: 備份狀態${NC}"
|
||||
local total_backups=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM backup_registry WHERE created_at > NOW() - INTERVAL '7 days';" 2>/dev/null || echo "0")
|
||||
local failed_backups=$(psql -U accusys -h localhost -d momentry -t -A -c "SELECT COUNT(*) FROM backup_registry WHERE status = 'failed' AND created_at > NOW() - INTERVAL '7 days';" 2>/dev/null || echo "0")
|
||||
echo " 本週備份: $total_backups"
|
||||
echo " 失敗: $failed_backups"
|
||||
echo ""
|
||||
|
||||
echo "========================================"
|
||||
echo "使用 '$0 check <layer>' 執行檢查"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 執行檢查
|
||||
# ============================================================
|
||||
|
||||
check_layer() {
|
||||
local layer=$1
|
||||
|
||||
case $layer in
|
||||
service)
|
||||
echo -e "${BLUE}執行 Layer 2: 服務監控...${NC}"
|
||||
bash "$MONITOR_DIR/service/health_check.sh"
|
||||
;;
|
||||
workflow)
|
||||
echo -e "${BLUE}執行 Layer 3: Workflow 監控...${NC}"
|
||||
bash "$MONITOR_DIR/workflow/n8n_workflow_monitor.sh"
|
||||
;;
|
||||
portal)
|
||||
echo -e "${BLUE}執行 Layer 4: Portal 監控...${NC}"
|
||||
bash "$MONITOR_DIR/portal/page_monitor.sh"
|
||||
;;
|
||||
database)
|
||||
echo -e "${BLUE}執行 Layer 5: Database 監控...${NC}"
|
||||
bash "$MONITOR_DIR/database/postgres_monitor.sh"
|
||||
bash "$MONITOR_DIR/database/redis_monitor.sh"
|
||||
bash "$MONITOR_DIR/database/qdrant_monitor.sh"
|
||||
;;
|
||||
users)
|
||||
echo -e "${BLUE}執行 Layer 6: 使用者監控...${NC}"
|
||||
bash "$MONITOR_DIR/users/session_tracker.sh"
|
||||
;;
|
||||
storage)
|
||||
echo -e "${BLUE}執行 Layer 7: Storage 監控...${NC}"
|
||||
bash "$MONITOR_DIR/storage/storage_manager.sh" status
|
||||
;;
|
||||
backup)
|
||||
echo -e "${BLUE}執行 Layer 7: 備份監控...${NC}"
|
||||
bash "$MONITOR_DIR/storage/backup_monitor.sh" status
|
||||
;;
|
||||
external)
|
||||
echo -e "${BLUE}執行 Layer 1: External 監控...${NC}"
|
||||
bash "$MONITOR_DIR/service/external_monitor.sh"
|
||||
;;
|
||||
node)
|
||||
echo -e "${BLUE}執行 Node.js 版本監控...${NC}"
|
||||
bash "$MONITOR_DIR/service/node_monitor.sh"
|
||||
;;
|
||||
python)
|
||||
echo -e "${BLUE}執行 Python 版本監控...${NC}"
|
||||
bash "$MONITOR_DIR/service/python_monitor.sh"
|
||||
;;
|
||||
all)
|
||||
echo -e "${BLUE}執行全面監控檢查...${NC}"
|
||||
check_layer external
|
||||
check_layer service
|
||||
check_layer node
|
||||
check_layer python
|
||||
check_layer workflow
|
||||
check_layer portal
|
||||
check_layer database
|
||||
check_layer users
|
||||
echo -e "${GREEN}全面檢查完成${NC}"
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}未知層級: $layer${NC}"
|
||||
show_help
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 持續監控
|
||||
# ============================================================
|
||||
|
||||
run_monitor() {
|
||||
echo -e "${BLUE}開始持續監控 (每 5 分鐘)${NC}"
|
||||
echo "按 Ctrl+C 停止"
|
||||
echo ""
|
||||
|
||||
while true; do
|
||||
local start_time=$(date +%s)
|
||||
|
||||
check_layer all
|
||||
|
||||
local end_time=$(date +%s)
|
||||
local elapsed=$((end_time - start_time))
|
||||
|
||||
if [ $elapsed -lt 300 ]; then
|
||||
sleep $((300 - elapsed))
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 查看日誌
|
||||
# ============================================================
|
||||
|
||||
show_logs() {
|
||||
local layer=${1:-anomaly}
|
||||
local lines=${2:-20}
|
||||
|
||||
case $layer in
|
||||
anomaly)
|
||||
echo -e "${BLUE}最近異常記錄:${NC}"
|
||||
psql -U accusys -h localhost -d momentry -c "
|
||||
SELECT
|
||||
TO_CHAR(detected_at, 'YYYY-MM-DD HH24:MI:SS') as time,
|
||||
severity,
|
||||
anomaly_type,
|
||||
username,
|
||||
LEFT(description, 50) as desc
|
||||
FROM monitor_anomalies
|
||||
ORDER BY detected_at DESC
|
||||
LIMIT $lines;
|
||||
" 2>/dev/null || echo "無法連接資料庫"
|
||||
;;
|
||||
service)
|
||||
echo -e "${BLUE}最近服務狀態:${NC}"
|
||||
psql -U accusys -h localhost -d momentry -c "
|
||||
SELECT
|
||||
service_name,
|
||||
status,
|
||||
response_time_ms,
|
||||
TO_CHAR(checked_at, 'YYYY-MM-DD HH24:MI:SS') as time
|
||||
FROM monitor_services
|
||||
ORDER BY checked_at DESC
|
||||
LIMIT $lines;
|
||||
" 2>/dev/null || echo "無法連接資料庫"
|
||||
;;
|
||||
workflow)
|
||||
echo -e "${BLUE}最近 Workflow 狀態:${NC}"
|
||||
psql -U accusys -h localhost -d momentry -c "
|
||||
SELECT
|
||||
workflow_name,
|
||||
is_active,
|
||||
idle_days,
|
||||
suggestion,
|
||||
TO_CHAR(checked_at, 'YYYY-MM-DD HH24:MI:SS') as time
|
||||
FROM monitor_workflows
|
||||
ORDER BY checked_at DESC
|
||||
LIMIT $lines;
|
||||
" 2>/dev/null || echo "無法連接資料庫"
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}未知日誌類型: $layer${NC}"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 清理歷史數據
|
||||
# ============================================================
|
||||
|
||||
clean_history() {
|
||||
echo -e "${YELLOW}清理歷史數據...${NC}"
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null || true
|
||||
-- 保留 30 天
|
||||
DELETE FROM monitor_services WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM monitor_workflows WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM monitor_databases WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM monitor_external WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM monitor_portal_pages WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
|
||||
-- 保留 30 天版本基線
|
||||
DELETE FROM node_version_baseline WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM node_process_tracking WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM python_version_baseline WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
DELETE FROM python_script_tracking WHERE checked_at < NOW() - INTERVAL '30 days';
|
||||
|
||||
-- 保留 7 天會話
|
||||
DELETE FROM monitor_sessions WHERE connected_at < NOW() - INTERVAL '7 days';
|
||||
|
||||
-- 保留 30 天登入
|
||||
DELETE FROM monitor_logins WHERE login_at < NOW() - INTERVAL '30 days';
|
||||
|
||||
-- 保留 90 天異常
|
||||
DELETE FROM monitor_anomalies WHERE detected_at < NOW() - INTERVAL '90 days';
|
||||
|
||||
-- 清理空閒空間
|
||||
VACUUM ANALYZE;
|
||||
EOF
|
||||
|
||||
echo -e "${GREEN}清理完成${NC}"
|
||||
}
|
||||
|
||||
# ============================================================
|
||||
# 主程序
|
||||
# ============================================================
|
||||
|
||||
main() {
|
||||
# 確保日誌目錄存在
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
local command=${1:-status}
|
||||
|
||||
case $command in
|
||||
status)
|
||||
show_status
|
||||
;;
|
||||
check)
|
||||
check_layer ${2:-all}
|
||||
;;
|
||||
monitor)
|
||||
run_monitor
|
||||
;;
|
||||
init)
|
||||
init_monitor
|
||||
;;
|
||||
logs)
|
||||
show_logs ${2:-anomaly} ${3:-20}
|
||||
;;
|
||||
clean)
|
||||
clean_history
|
||||
;;
|
||||
help|--help|-h)
|
||||
show_help
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}未知命令: $command${NC}"
|
||||
show_help
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
82
v1.1/monitor_v1.11/database/mongodb_monitor_v1.11.sh
Executable file
82
v1.1/monitor_v1.11/database/mongodb_monitor_v1.11.sh
Executable file
@@ -0,0 +1,82 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry MongoDB 監控 (Layer 5)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/database/mongodb_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/mongodb_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
MONGO_USER="accusys"
|
||||
MONGO_PASS="Test3200Test3200"
|
||||
|
||||
record_metric() {
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_databases (db_type, db_name, metric_name, metric_value, checked_at)
|
||||
VALUES ('mongodb', 'mongodb', '$1', '$2', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
get_status() {
|
||||
mongosh --quiet --username "$MONGO_USER" --password "$MONGO_PASS" --authenticationDatabase admin --eval "JSON.stringify(db.adminCommand({ replSetGetStatus: 1 }))" 2>/dev/null || echo "{}"
|
||||
}
|
||||
|
||||
get_server_status() {
|
||||
mongosh --quiet --username "$MONGO_USER" --password "$MONGO_PASS" --authenticationDatabase admin --eval "JSON.stringify(db.serverStatus()))" 2>/dev/null || echo "{}"
|
||||
}
|
||||
|
||||
get_databases() {
|
||||
mongosh --quiet --username "$MONGO_USER" --password "$MONGO_PASS" --authenticationDatabase admin --eval "JSON.stringify(db.adminCommand({ listDatabases: 1 }))" 2>/dev/null || echo "{}"
|
||||
}
|
||||
|
||||
echo "========================================"
|
||||
echo "Layer 5: MongoDB Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
if ! mongosh --quiet --username "$MONGO_USER" --password "$MONGO_PASS" --authenticationDatabase admin --eval "db.adminCommand('ping')" > /dev/null 2>&1; then
|
||||
echo "MongoDB 不可用"
|
||||
log "MongoDB unavailable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "✓ MongoDB 連接正常"
|
||||
echo ""
|
||||
|
||||
echo "資料庫:"
|
||||
echo "----------------------------------------"
|
||||
databases=$(get_databases)
|
||||
echo "$databases" | jq -r '.databases[] | " \(.name): \(.sizeOnDisk / 1024 / 1024 | floor)MB"' 2>/dev/null || echo " 無法獲取資料庫列表"
|
||||
|
||||
echo ""
|
||||
echo "伺服器狀態:"
|
||||
echo "----------------------------------------"
|
||||
server_status=$(get_server_status)
|
||||
connections=$(echo "$server_status" | jq -r '.connections.current' 2>/dev/null || echo "N/A")
|
||||
active_connections=$(echo "$server_status" | jq -r '.connections.active' 2>/dev/null || echo "N/A")
|
||||
uptime=$(echo "$server_status" | jq -r '.uptime' 2>/dev/null || echo "N/A")
|
||||
mem_resident=$(echo "$server_status" | jq -r '.mem.resident' 2>/dev/null || echo "N/A")
|
||||
|
||||
echo " 當前連接: $connections"
|
||||
echo " 活躍連接: $active_connections"
|
||||
echo " 運行時間: ${uptime}秒"
|
||||
echo " 記憶體使用: ${mem_resident}MB"
|
||||
|
||||
record_metric "connections" "$connections"
|
||||
record_metric "active_connections" "$active_connections"
|
||||
record_metric "uptime" "$uptime"
|
||||
record_metric "mem_resident" "$mem_resident"
|
||||
|
||||
echo ""
|
||||
log "MongoDB check completed: connections=$connections"
|
||||
|
||||
echo "========================================"
|
||||
echo "完成"
|
||||
echo "========================================"
|
||||
130
v1.1/monitor_v1.11/database/postgres_monitor_v1.11.sh
Executable file
130
v1.1/monitor_v1.11/database/postgres_monitor_v1.11.sh
Executable file
@@ -0,0 +1,130 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry PostgreSQL 監控 (Layer 5)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/database/postgres_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/postgres_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
# 記錄指標
|
||||
record_metric() {
|
||||
local db_type="postgresql"
|
||||
local db_name=$1
|
||||
local metric_name=$2
|
||||
local value=$3
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_databases (db_type, db_name, metric_name, metric_value, checked_at)
|
||||
VALUES ('$db_type', '$db_name', '$metric_name', '$value', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 獲取資料庫列表
|
||||
get_databases() {
|
||||
psql -U accusys -h localhost -t -A -c "SELECT datname FROM pg_database WHERE datistemplate = false;" 2>/dev/null
|
||||
}
|
||||
|
||||
# 獲取表大小
|
||||
get_table_sizes() {
|
||||
local db=$1
|
||||
psql -U accusys -h localhost -d "$db" -t -A -c "
|
||||
SELECT
|
||||
schemaname,
|
||||
tablename,
|
||||
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) as size,
|
||||
n_live_tup as rows
|
||||
FROM pg_stat_user_tables
|
||||
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC
|
||||
LIMIT 10;
|
||||
" 2>/dev/null
|
||||
}
|
||||
|
||||
# 獲取連線數
|
||||
get_connections() {
|
||||
psql -U accusys -h localhost -t -A -c "
|
||||
SELECT state, count(*)
|
||||
FROM pg_stat_activity
|
||||
WHERE datname = current_database()
|
||||
GROUP BY state;
|
||||
" 2>/dev/null
|
||||
}
|
||||
|
||||
# 獲取慢查詢
|
||||
get_slow_queries() {
|
||||
psql -U accusys -h localhost -t -A -c "
|
||||
SELECT query, calls, mean_time
|
||||
FROM pg_stat_statements
|
||||
WHERE query NOT LIKE '%pg_stat_statements%'
|
||||
ORDER BY mean_time DESC
|
||||
LIMIT 5;
|
||||
" 2>/dev/null
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 5: PostgreSQL Database Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
# 檢查 PostgreSQL 是否可用
|
||||
if ! pg_isready -h localhost -p 5432 -U accusys > /dev/null 2>&1; then
|
||||
echo "PostgreSQL 不可用"
|
||||
log "PostgreSQL unavailable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 記錄連線數
|
||||
connections=$(get_connections)
|
||||
echo "連線狀態:"
|
||||
echo "$connections"
|
||||
echo ""
|
||||
|
||||
# 記錄指標
|
||||
record_metric "postgres" "connections" "'$connections'"
|
||||
|
||||
# 檢查各資料庫
|
||||
echo "資料庫表:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
for db in $(get_databases); do
|
||||
echo ""
|
||||
echo "資料庫: $db"
|
||||
|
||||
table_count=$(psql -U accusys -h localhost -d "$db" -t -A -c "SELECT count(*) FROM information_schema.tables WHERE table_schema = 'public';" 2>/dev/null || echo "0")
|
||||
echo " 表數量: $table_count"
|
||||
|
||||
record_metric "$db" "table_count" "$table_count"
|
||||
|
||||
# 顯示大表
|
||||
echo " 大表:"
|
||||
get_table_sizes "$db" | while read -r schema table size rows; do
|
||||
[ -z "$table" ] && continue
|
||||
echo " - $table: $size ($rows rows)"
|
||||
done
|
||||
done
|
||||
|
||||
# 檢查慢查詢（如果 pg_stat_statements 可用）
|
||||
echo ""
|
||||
echo "慢查詢 (如有):"
|
||||
slow_queries=$(get_slow_queries)
|
||||
if [ -n "$slow_queries" ]; then
|
||||
echo "$slow_queries" | while read -r query calls time; do
|
||||
[ -z "$query" ] && continue
|
||||
echo " - ${time}ms (調用 $calls 次)"
|
||||
done
|
||||
else
|
||||
echo " (pg_stat_statements 未啟用)"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
log "PostgreSQL check completed"
|
||||
124
v1.1/monitor_v1.11/database/qdrant_monitor_v1.11.sh
Executable file
124
v1.1/monitor_v1.11/database/qdrant_monitor_v1.11.sh
Executable file
@@ -0,0 +1,124 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry Qdrant 監控 (Layer 5)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/database/qdrant_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/qdrant_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
QDRANT_HOST="http://localhost:6333"
|
||||
QDRANT_API_KEY="Test3200Test3200Test3200"
|
||||
|
||||
# 記錄指標
|
||||
record_metric() {
|
||||
local collection=$1
|
||||
local metric_name=$2
|
||||
local value=$3
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_databases (db_type, db_name, metric_name, metric_value, checked_at)
|
||||
VALUES ('qdrant', '$collection', '$metric_name', '$value', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 記錄 Collection
|
||||
record_collection() {
|
||||
local name=$1
|
||||
local vectors=$2
|
||||
local points=$3
|
||||
local disk_size=$4
|
||||
local status=$5
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_qdrant_collections (collection_name, vectors_count, points_count, disk_size_bytes, status, snapshot_at)
|
||||
VALUES ('$name', $vectors, $points, $disk_size, '$status', NOW())
|
||||
ON CONFLICT (collection_name) DO UPDATE SET
|
||||
vectors_count = EXCLUDED.vectors_count,
|
||||
points_count = EXCLUDED.points_count,
|
||||
disk_size_bytes = EXCLUDED.disk_size_bytes,
|
||||
status = EXCLUDED.status,
|
||||
snapshot_at = NOW();
|
||||
EOF
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 5: Qdrant Vector Database Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
# 檢查 Qdrant 是否可用
|
||||
http_code=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "api-key: $QDRANT_API_KEY" \
|
||||
"$QDRANT_HOST/collections" 2>/dev/null || echo "000")
|
||||
if [ "$http_code" = "000" ]; then
|
||||
echo "Qdrant 不可用"
|
||||
log "Qdrant unavailable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Qdrant 狀態: HTTP $http_code"
|
||||
echo ""
|
||||
|
||||
# 獲取 Collection 列表
|
||||
collections=$(curl -s -H "api-key: $QDRANT_API_KEY" "$QDRANT_HOST/collections" 2>/dev/null)
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
collection_count=$(echo "$collections" | jq '.result.collections | length' 2>/dev/null || echo "0")
|
||||
echo "Collection 數量: $collection_count"
|
||||
echo ""
|
||||
|
||||
# 遍歷每個 Collection
|
||||
echo "Collections:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
for i in $(seq 0 $((collection_count - 1))); do
|
||||
name=$(echo "$collections" | jq -r ".result.collections[$i].name")
|
||||
|
||||
# 獲取 Collection 詳情
|
||||
details=$(curl -s -H "api-key: $QDRANT_API_KEY" "$QDRANT_HOST/collections/$name" 2>/dev/null)
|
||||
|
||||
vectors_count=$(echo "$details" | jq -r '.result.indexed_vectors_count // 0' 2>/dev/null || echo "0")
|
||||
points_count=$(echo "$details" | jq -r '.result.points_count // 0' 2>/dev/null || echo "0")
|
||||
disk_size=$(echo "$details" | jq -r '.result.disk_size_bytes // 0' 2>/dev/null || echo "0")
|
||||
status=$(echo "$details" | jq -r '.result.status // "unknown"' 2>/dev/null || echo "unknown")
|
||||
|
||||
# 格式化大小
|
||||
if [ "$disk_size" -gt 1073741824 ]; then
|
||||
size_str="$((disk_size / 1073741824))GB"
|
||||
elif [ "$disk_size" -gt 1048576 ]; then
|
||||
size_str="$((disk_size / 1048576))MB"
|
||||
elif [ "$disk_size" -gt 1024 ]; then
|
||||
size_str="$((disk_size / 1024))KB"
|
||||
else
|
||||
size_str="${disk_size}B"
|
||||
fi
|
||||
|
||||
echo " - $name"
|
||||
echo " 狀態: $status"
|
||||
echo " Vectors: $vectors_count"
|
||||
echo " Points: $points_count"
|
||||
echo " 大小: $size_str"
|
||||
|
||||
# 記錄到資料庫
|
||||
record_collection "$name" "$vectors_count" "$points_count" "$disk_size" "$status"
|
||||
record_metric "$name" "vectors_count" "$vectors_count"
|
||||
record_metric "$name" "points_count" "$points_count"
|
||||
record_metric "$name" "disk_size" "$disk_size"
|
||||
done
|
||||
else
|
||||
echo "無法獲取 Collection 列表"
|
||||
log "Failed to get collections: HTTP $http_code"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
log "Qdrant check completed"
|
||||
111
v1.1/monitor_v1.11/database/redis_monitor_v1.11.sh
Executable file
111
v1.1/monitor_v1.11/database/redis_monitor_v1.11.sh
Executable file
@@ -0,0 +1,111 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry Redis 監控 (Layer 5)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/database/redis_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/redis_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
REDIS_PASS="accusys"
|
||||
|
||||
# 記錄指標
|
||||
record_metric() {
|
||||
local value=$1
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_databases (db_type, db_name, metric_name, metric_value, checked_at)
|
||||
VALUES ('redis', 'redis', '$1', '$2', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 獲取 Redis INFO
|
||||
get_info() {
|
||||
redis-cli -a "$REDIS_PASS" INFO 2>/dev/null
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 5: Redis Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
# 檢查 Redis 是否可用
|
||||
if ! redis-cli -a "$REDIS_PASS" ping > /dev/null 2>&1; then
|
||||
echo "Redis 不可用"
|
||||
log "Redis unavailable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
info=$(get_info)
|
||||
|
||||
# 提取關鍵指標
|
||||
echo "關鍵指標:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
# 內存使用
|
||||
used_memory=$(echo "$info" | grep "^used_memory_human:" | cut -d: -f2 | tr -d '\r')
|
||||
echo " 內存使用: $used_memory"
|
||||
|
||||
# 連線數
|
||||
connected_clients=$(echo "$info" | grep "^connected_clients:" | cut -d: -f2 | tr -d '\r')
|
||||
echo " 客戶端連線: $connected_clients"
|
||||
|
||||
# 命中率
|
||||
keyspace_hits=$(echo "$info" | grep "^keyspace_hits:" | cut -d: -f2 | tr -d '\r')
|
||||
keyspace_misses=$(echo "$info" | grep "^keyspace_misses:" | cut -d: -f2 | tr -d '\r')
|
||||
total_ops=$((keyspace_hits + keyspace_misses))
|
||||
if [ $total_ops -gt 0 ]; then
|
||||
hit_rate=$((keyspace_hits * 100 / total_ops))
|
||||
echo " 命中率: ${hit_rate}%"
|
||||
else
|
||||
echo " 命中率: N/A"
|
||||
fi
|
||||
|
||||
# 持久化
|
||||
rdb_changes=$(echo "$info" | grep "^rdb_changes_since_last_save:" | cut -d: -f2 | tr -d '\r')
|
||||
echo " RDB 變更: $rdb_changes"
|
||||
|
||||
# 總鍵數
|
||||
echo ""
|
||||
echo "鍵數據庫:"
|
||||
db0_info=$(echo "$info" | grep "^db0:" | head -1)
|
||||
if [ -n "$db0_info" ]; then
|
||||
keys=$(echo "$db0_info" | sed 's/.*keys=\([0-9]*\).*/\1/')
|
||||
expires=$(echo "$db0_info" | sed 's/.*expires=\([0-9]*\).*/\1/')
|
||||
echo " db0: $keys keys, $expires 有過期時間"
|
||||
fi
|
||||
|
||||
# 記錄到資料庫
|
||||
record_metric "used_memory" "'$used_memory'"
|
||||
record_metric "connected_clients" "$connected_clients"
|
||||
record_metric "keyspace_hits" "$keyspace_hits"
|
||||
record_metric "keyspace_misses" "$keyspace_misses"
|
||||
|
||||
# 檢查閾值
|
||||
echo ""
|
||||
echo "閾值檢查:"
|
||||
memory_percent=$(echo "$info" | grep "^used_memory:" | cut -d: -f2)
|
||||
maxmemory=$(redis-cli -a "$REDIS_PASS" CONFIG GET maxmemory 2>/dev/null | tail -1)
|
||||
if [ -n "$maxmemory" ] && [ "$maxmemory" -gt 0 ]; then
|
||||
mem_pct=$((memory_percent * 100 / maxmemory))
|
||||
echo " 內存使用: ${mem_pct}%"
|
||||
if [ $mem_pct -gt 80 ]; then
|
||||
echo " ⚠️ 內存使用超過 80%"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ $connected_clients -gt 100 ]; then
|
||||
echo " ⚠️ 客戶端連線過多"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
log "Redis check completed"
|
||||
492
v1.1/monitor_v1.11/database/schema_v1.11.sql
Normal file
492
v1.1/monitor_v1.11/database/schema_v1.11.sql
Normal file
@@ -0,0 +1,492 @@
|
||||
-- Momentry 監控系統數據庫表
|
||||
-- 使用方式: psql -U accusys -h localhost -d momentry -f schema.sql
|
||||
|
||||
-- ============================================================
|
||||
-- Layer 2: Service 監控
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_services (
|
||||
id SERIAL PRIMARY KEY,
|
||||
service_name VARCHAR(50) NOT NULL,
|
||||
service_type VARCHAR(20),
|
||||
port INTEGER,
|
||||
status VARCHAR(20) CHECK (status IN ('up', 'down', 'degraded', 'unknown')),
|
||||
response_time_ms INTEGER,
|
||||
error_message TEXT,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_monitor_services_name ON monitor_services(service_name);
|
||||
CREATE INDEX idx_monitor_services_time ON monitor_services(checked_at);
|
||||
|
||||
-- ============================================================
|
||||
-- Layer 3: n8n Workflow 監控
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_workflows (
|
||||
id SERIAL PRIMARY KEY,
|
||||
workflow_id VARCHAR(50) NOT NULL,
|
||||
workflow_name VARCHAR(255),
|
||||
workflow_type VARCHAR(50),
|
||||
is_active BOOLEAN DEFAULT FALSE,
|
||||
last_executed_at TIMESTAMP,
|
||||
execution_count INTEGER DEFAULT 0,
|
||||
success_count INTEGER DEFAULT 0,
|
||||
failure_count INTEGER DEFAULT 0,
|
||||
avg_duration_ms INTEGER,
|
||||
has_schedule BOOLEAN DEFAULT FALSE,
|
||||
has_webhook BOOLEAN DEFAULT FALSE,
|
||||
idle_days INTEGER,
|
||||
suggestion VARCHAR(100),
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_monitor_workflows_id ON monitor_workflows(workflow_id);
|
||||
CREATE INDEX idx_monitor_workflows_active ON monitor_workflows(is_active);
|
||||
CREATE INDEX idx_monitor_workflows_idle ON monitor_workflows(idle_days);
|
||||
|
||||
-- ============================================================
|
||||
-- Layer 4: WordPress Portal 監控
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_portal_pages (
|
||||
id SERIAL PRIMARY KEY,
|
||||
page_url VARCHAR(500) NOT NULL,
|
||||
page_type VARCHAR(20),
|
||||
is_accessible BOOLEAN,
|
||||
response_time_ms INTEGER,
|
||||
http_status INTEGER,
|
||||
error_message TEXT,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_portal_users (
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_id BIGINT,
|
||||
username VARCHAR(100),
|
||||
email VARCHAR(255),
|
||||
role VARCHAR(50),
|
||||
is_active BOOLEAN,
|
||||
last_login TIMESTAMP,
|
||||
created_at TIMESTAMP,
|
||||
detected_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_monitor_portal_pages_url ON monitor_portal_pages(page_url);
|
||||
CREATE INDEX idx_monitor_portal_users_username ON monitor_portal_users(username);
|
||||
|
||||
-- ============================================================
|
||||
-- Layer 5: Database 監控
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_databases (
|
||||
id SERIAL PRIMARY KEY,
|
||||
db_type VARCHAR(20) NOT NULL CHECK (db_type IN ('postgresql', 'redis', 'qdrant', 'mariadb', 'mongodb')),
|
||||
db_name VARCHAR(50),
|
||||
metric_name VARCHAR(50) NOT NULL,
|
||||
metric_value JSONB,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_monitor_databases_type ON monitor_databases(db_type);
|
||||
CREATE INDEX idx_monitor_databases_time ON monitor_databases(checked_at);
|
||||
|
||||
-- PostgreSQL 表結構快照
|
||||
CREATE TABLE IF NOT EXISTS monitor_pg_tables (
|
||||
id SERIAL PRIMARY KEY,
|
||||
database_name VARCHAR(50),
|
||||
schema_name VARCHAR(50),
|
||||
table_name VARCHAR(100),
|
||||
table_type VARCHAR(20),
|
||||
row_count BIGINT,
|
||||
table_size_bytes BIGINT,
|
||||
index_size_bytes BIGINT,
|
||||
snapshot_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 表結構變更記錄
|
||||
CREATE TABLE IF NOT EXISTS monitor_pg_schema_changes (
|
||||
id SERIAL PRIMARY KEY,
|
||||
database_name VARCHAR(50),
|
||||
schema_name VARCHAR(50),
|
||||
table_name VARCHAR(100),
|
||||
change_type VARCHAR(20) CHECK (change_type IN ('table_created', 'table_dropped', 'column_added', 'column_removed', 'column_type_changed')),
|
||||
column_name VARCHAR(100),
|
||||
old_value TEXT,
|
||||
new_value TEXT,
|
||||
detected_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Qdrant Collection 監控
|
||||
CREATE TABLE IF NOT EXISTS monitor_qdrant_collections (
|
||||
id SERIAL PRIMARY KEY,
|
||||
collection_name VARCHAR(100),
|
||||
vectors_count BIGINT,
|
||||
points_count BIGINT,
|
||||
disk_size_bytes BIGINT,
|
||||
status VARCHAR(20),
|
||||
snapshot_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- ============================================================
|
||||
-- Layer 6: 使用者監控
|
||||
-- ============================================================
|
||||
|
||||
-- 連線會話追蹤
|
||||
CREATE TABLE IF NOT EXISTS monitor_sessions (
|
||||
id SERIAL PRIMARY KEY,
|
||||
session_type VARCHAR(20) CHECK (session_type IN ('ssh', 'web', 'db', 'sftp', 'rdp')),
|
||||
service_name VARCHAR(50),
|
||||
username VARCHAR(100),
|
||||
source_ip VARCHAR(45),
|
||||
source_port INTEGER,
|
||||
connected_at TIMESTAMP,
|
||||
last_activity_at TIMESTAMP,
|
||||
disconnected_at TIMESTAMP,
|
||||
bytes_sent BIGINT,
|
||||
bytes_received BIGINT,
|
||||
status VARCHAR(20) CHECK (status IN ('active', 'disconnected', 'timeout'))
|
||||
);
|
||||
|
||||
-- 登入歷史
|
||||
CREATE TABLE IF NOT EXISTS monitor_logins (
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_type VARCHAR(20) CHECK (user_type IN ('system', 'wordpress', 'n8n', 'gitea', 'sftpgo', 'database')),
|
||||
username VARCHAR(100),
|
||||
source_ip VARCHAR(45),
|
||||
user_agent TEXT,
|
||||
login_method VARCHAR(20),
|
||||
success BOOLEAN,
|
||||
failure_reason VARCHAR(200),
|
||||
login_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- sudo 命令記錄
|
||||
CREATE TABLE IF NOT EXISTS monitor_sudo_history (
|
||||
id SERIAL PRIMARY KEY,
|
||||
username VARCHAR(100),
|
||||
command TEXT,
|
||||
run_as VARCHAR(100),
|
||||
tty VARCHAR(50),
|
||||
source_ip VARCHAR(45),
|
||||
exit_code INTEGER,
|
||||
executed_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 資源使用追蹤
|
||||
CREATE TABLE IF NOT EXISTS monitor_resource_usage (
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_type VARCHAR(20),
|
||||
username VARCHAR(100),
|
||||
service_name VARCHAR(50),
|
||||
cpu_percent DECIMAL(5,2),
|
||||
memory_mb INTEGER,
|
||||
disk_io_read_mb BIGINT,
|
||||
disk_io_write_mb BIGINT,
|
||||
network_rx_mb BIGINT,
|
||||
network_tx_mb BIGINT,
|
||||
recorded_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 異常檢測記錄
|
||||
CREATE TABLE IF NOT EXISTS monitor_anomalies (
|
||||
id SERIAL PRIMARY KEY,
|
||||
anomaly_type VARCHAR(50) CHECK (anomaly_type IN ('brute_force', 'privilege_escalation', 'unusual_access', 'unusual_time', 'excessive_queries', 'idle_session', 'schema_change')),
|
||||
severity VARCHAR(20) CHECK (severity IN ('low', 'medium', 'high', 'critical')),
|
||||
source_type VARCHAR(20),
|
||||
username VARCHAR(100),
|
||||
source_ip VARCHAR(45),
|
||||
description TEXT,
|
||||
details JSONB,
|
||||
detected_at TIMESTAMP DEFAULT NOW(),
|
||||
resolved BOOLEAN DEFAULT FALSE,
|
||||
resolved_at TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE INDEX idx_monitor_sessions_type ON monitor_sessions(session_type);
|
||||
CREATE INDEX idx_monitor_sessions_username ON monitor_sessions(username);
|
||||
CREATE INDEX idx_monitor_logins_type ON monitor_logins(user_type);
|
||||
CREATE INDEX idx_monitor_logins_time ON monitor_logins(login_at);
|
||||
CREATE INDEX idx_monitor_anomalies_type ON monitor_anomalies(anomaly_type);
|
||||
CREATE INDEX idx_monitor_anomalies_severity ON monitor_anomalies(severity);
|
||||
CREATE INDEX idx_monitor_anomalies_time ON monitor_anomalies(detected_at);
|
||||
|
||||
-- ============================================================
|
||||
-- Layer 7: Storage 監控
|
||||
-- ============================================================
|
||||
|
||||
-- 檔案註冊表
|
||||
CREATE TABLE IF NOT EXISTS file_registry (
|
||||
file_uuid UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
file_name VARCHAR(255) NOT NULL,
|
||||
file_path TEXT NOT NULL,
|
||||
file_path_hash VARCHAR(64) NOT NULL,
|
||||
file_size BIGINT NOT NULL,
|
||||
file_hash VARCHAR(64),
|
||||
mime_type VARCHAR(100),
|
||||
user_cluster VARCHAR(50) CHECK (user_cluster IN ('family', 'work', 'wordpress', 'shared', 'system')),
|
||||
owner_id VARCHAR(100),
|
||||
storage_tier VARCHAR(20) DEFAULT 'hot' CHECK (storage_tier IN ('hot', 'warm', 'cold')),
|
||||
storage_location VARCHAR(500),
|
||||
status VARCHAR(20) DEFAULT 'active' CHECK (status IN ('active', 'temporary', 'archived', 'deleted')),
|
||||
is_registered BOOLEAN DEFAULT TRUE,
|
||||
created_at TIMESTAMP DEFAULT NOW(),
|
||||
updated_at TIMESTAMP DEFAULT NOW(),
|
||||
last_accessed_at TIMESTAMP,
|
||||
access_count INTEGER DEFAULT 0,
|
||||
archived_at TIMESTAMP,
|
||||
archive_location VARCHAR(500),
|
||||
retention_until TIMESTAMP,
|
||||
UNIQUE(file_path_hash)
|
||||
);
|
||||
|
||||
-- 存儲使用統計
|
||||
CREATE TABLE IF NOT EXISTS storage_usage_stats (
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_cluster VARCHAR(50),
|
||||
storage_tier VARCHAR(20),
|
||||
file_count BIGINT,
|
||||
total_size_bytes BIGINT,
|
||||
record_time TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 文件訪問日誌
|
||||
CREATE TABLE IF NOT EXISTS storage_access_logs (
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_cluster VARCHAR(50),
|
||||
owner_id VARCHAR(100),
|
||||
file_path TEXT,
|
||||
access_type VARCHAR(20) CHECK (access_type IN ('read', 'write', 'delete', 'download', 'move')),
|
||||
access_time TIMESTAMP DEFAULT NOW(),
|
||||
client_ip VARCHAR(45),
|
||||
access_method VARCHAR(20)
|
||||
);
|
||||
|
||||
-- 文件生命週期
|
||||
CREATE TABLE IF NOT EXISTS file_lifecycle (
|
||||
id SERIAL PRIMARY KEY,
|
||||
file_uuid UUID REFERENCES file_registry(file_uuid),
|
||||
file_path TEXT,
|
||||
user_cluster VARCHAR(50),
|
||||
storage_tier VARCHAR(20),
|
||||
created_at TIMESTAMP,
|
||||
last_accessed_at TIMESTAMP,
|
||||
last_modified_at TIMESTAMP,
|
||||
access_count INTEGER DEFAULT 0,
|
||||
current_status VARCHAR(20) DEFAULT 'active',
|
||||
tier_migration_count INTEGER DEFAULT 0,
|
||||
migrated_at TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE INDEX idx_file_registry_cluster ON file_registry(user_cluster);
|
||||
CREATE INDEX idx_file_registry_tier ON file_registry(storage_tier);
|
||||
CREATE INDEX idx_file_registry_status ON file_registry(status);
|
||||
CREATE INDEX idx_storage_usage_cluster ON storage_usage_stats(user_cluster);
|
||||
CREATE INDEX idx_storage_usage_time ON storage_usage_stats(record_time);
|
||||
|
||||
-- ============================================================
|
||||
-- 外部監控 (Layer 1)
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_external (
|
||||
id SERIAL PRIMARY KEY,
|
||||
target_name VARCHAR(50) NOT NULL,
|
||||
target_type VARCHAR(20) CHECK (target_type IN ('ddns', 'gateway', 'internet', 'api')),
|
||||
target_host VARCHAR(255),
|
||||
is_reachable BOOLEAN,
|
||||
response_time_ms INTEGER,
|
||||
dns_resolved_ip VARCHAR(45),
|
||||
error_message TEXT,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_monitor_external_name ON monitor_external(target_name);
|
||||
CREATE INDEX idx_monitor_external_time ON monitor_external(checked_at);
|
||||
|
||||
-- ============================================================
|
||||
-- 監控配置表
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_config (
|
||||
id SERIAL PRIMARY KEY,
|
||||
config_key VARCHAR(50) UNIQUE NOT NULL,
|
||||
config_value TEXT,
|
||||
description VARCHAR(255),
|
||||
updated_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 插入默認配置
|
||||
INSERT INTO monitor_config (config_key, config_value, description) VALUES
|
||||
('check_interval', '300', '監控檢查間隔（秒）'),
|
||||
('retention_days', '30', '歷史數據保留天數'),
|
||||
('idle_threshold_days', '30', 'Workflow 閒置天數閾值'),
|
||||
('alert_threshold_bruteforce', '5', '暴力破解嘗試次數閾值'),
|
||||
('alert_threshold_slow_response', '3000', '響應時間閾值（毫秒）')
|
||||
ON CONFLICT (config_key) DO NOTHING;
|
||||
|
||||
-- ============================================================
|
||||
-- 視圖定義
|
||||
-- ============================================================
|
||||
|
||||
-- 服務健康狀態視圖
|
||||
CREATE OR REPLACE VIEW v_service_health AS
|
||||
SELECT
|
||||
service_name,
|
||||
status,
|
||||
COUNT(*) as check_count,
|
||||
COUNT(*) FILTER (WHERE status = 'up') as up_count,
|
||||
COUNT(*) FILTER (WHERE status = 'down') as down_count,
|
||||
AVG(response_time_ms) as avg_response_time,
|
||||
MAX(checked_at) as last_check
|
||||
FROM monitor_services
|
||||
WHERE checked_at > NOW() - INTERVAL '24 hours'
|
||||
GROUP BY service_name, status;
|
||||
|
||||
-- 最近異常視圖
|
||||
CREATE OR REPLACE VIEW v_recent_anomalies AS
|
||||
SELECT
|
||||
anomaly_type,
|
||||
severity,
|
||||
username,
|
||||
source_ip,
|
||||
description,
|
||||
detected_at
|
||||
FROM monitor_anomalies
|
||||
WHERE detected_at > NOW() - INTERVAL '24 hours'
|
||||
ORDER BY detected_at DESC;
|
||||
|
||||
-- 閒置 Workflow 視圖
|
||||
CREATE OR REPLACE VIEW v_idle_workflows AS
|
||||
SELECT
|
||||
workflow_name,
|
||||
idle_days,
|
||||
suggestion,
|
||||
last_executed_at
|
||||
FROM monitor_workflows
|
||||
WHERE idle_days > 30 AND is_active = TRUE
|
||||
ORDER BY idle_days DESC;
|
||||
|
||||
-- 存儲使用概況視圖
|
||||
CREATE OR REPLACE VIEW v_storage_overview AS
|
||||
SELECT
|
||||
user_cluster,
|
||||
storage_tier,
|
||||
COUNT(*) as file_count,
|
||||
SUM(file_size) as total_size
|
||||
FROM file_registry
|
||||
WHERE status = 'active'
|
||||
GROUP BY user_cluster, storage_tier;
|
||||
|
||||
-- ============================================================
|
||||
-- 備份監控 (Layer 7 Extension)
|
||||
-- ============================================================
|
||||
|
||||
-- 備份註冊表
|
||||
CREATE TABLE IF NOT EXISTS backup_registry (
|
||||
id SERIAL PRIMARY KEY,
|
||||
service_name VARCHAR(50) NOT NULL,
|
||||
backup_file VARCHAR(500) NOT NULL,
|
||||
backup_size_bytes BIGINT,
|
||||
backup_type VARCHAR(20) CHECK (backup_type IN ('daily', 'weekly', 'monthly', 'archive', 'full', 'incremental')),
|
||||
backup_method VARCHAR(20) CHECK (backup_method IN ('pg_dump', 'mysqldump', 'tar', 'snapshot', 'dump')),
|
||||
status VARCHAR(20) CHECK (status IN ('pending', 'running', 'completed', 'failed', 'verified')),
|
||||
compression_ratio DECIMAL(5,2),
|
||||
verification_result BOOLEAN,
|
||||
error_message TEXT,
|
||||
started_at TIMESTAMP DEFAULT NOW(),
|
||||
completed_at TIMESTAMP,
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 備份存儲統計
|
||||
CREATE TABLE IF NOT EXISTS backup_storage_stats (
|
||||
id SERIAL PRIMARY KEY,
|
||||
tier VARCHAR(20) CHECK (tier IN ('daily', 'weekly', 'monthly', 'archive', 'total')),
|
||||
file_count BIGINT,
|
||||
total_size_bytes BIGINT,
|
||||
record_time TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 備份歷史
|
||||
CREATE TABLE IF NOT EXISTS backup_history (
|
||||
id SERIAL PRIMARY KEY,
|
||||
service_name VARCHAR(50) NOT NULL,
|
||||
operation VARCHAR(20) CHECK (operation IN ('backup', 'restore', 'tier_migration', 'cleanup', 'verify')),
|
||||
backup_file VARCHAR(500),
|
||||
backup_tier VARCHAR(20),
|
||||
source_tier VARCHAR(20),
|
||||
dest_tier VARCHAR(20),
|
||||
file_count BIGINT,
|
||||
size_bytes BIGINT,
|
||||
duration_seconds INTEGER,
|
||||
status VARCHAR(20) CHECK (status IN ('success', 'failed', 'partial')),
|
||||
error_message TEXT,
|
||||
executed_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_backup_registry_service ON backup_registry(service_name);
|
||||
CREATE INDEX idx_backup_registry_time ON backup_registry(created_at);
|
||||
CREATE INDEX idx_backup_storage_stats_tier ON backup_storage_stats(tier);
|
||||
CREATE INDEX idx_backup_storage_stats_time ON backup_storage_stats(record_time);
|
||||
CREATE INDEX idx_backup_history_service ON backup_history(service_name);
|
||||
CREATE INDEX idx_backup_history_time ON backup_history(executed_at);
|
||||
|
||||
-- ============================================================
|
||||
-- Node.js 版本基線監控
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS node_version_baseline (
|
||||
id SERIAL PRIMARY KEY,
|
||||
runtime_name VARCHAR(50) NOT NULL,
|
||||
required_version VARCHAR(20) NOT NULL,
|
||||
current_version VARCHAR(20),
|
||||
process_name VARCHAR(100),
|
||||
process_path TEXT,
|
||||
is_compliant BOOLEAN,
|
||||
locked_path VARCHAR(500),
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Node.js 進程追蹤
|
||||
CREATE TABLE IF NOT EXISTS node_process_tracking (
|
||||
id SERIAL PRIMARY KEY,
|
||||
process_name VARCHAR(100) NOT NULL,
|
||||
pid INTEGER,
|
||||
command VARCHAR(500),
|
||||
node_version VARCHAR(20),
|
||||
is_managed BOOLEAN DEFAULT FALSE,
|
||||
started_at TIMESTAMP,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- ============================================================
|
||||
-- Python 版本基線監控
|
||||
-- ============================================================
|
||||
|
||||
CREATE TABLE IF NOT EXISTS python_version_baseline (
|
||||
id SERIAL PRIMARY KEY,
|
||||
runtime_name VARCHAR(50) NOT NULL,
|
||||
required_version VARCHAR(20) NOT NULL,
|
||||
current_version VARCHAR(20),
|
||||
interpreter_path VARCHAR(500),
|
||||
is_compliant BOOLEAN,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Python 腳本追蹤
|
||||
CREATE TABLE IF NOT EXISTS python_script_tracking (
|
||||
id SERIAL PRIMARY KEY,
|
||||
script_path TEXT NOT NULL,
|
||||
shebang_version VARCHAR(20),
|
||||
actual_version VARCHAR(20),
|
||||
is_compliant BOOLEAN DEFAULT FALSE,
|
||||
last_run_at TIMESTAMP,
|
||||
exit_code INTEGER,
|
||||
error_output TEXT,
|
||||
checked_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_node_version_name ON node_version_baseline(runtime_name);
|
||||
CREATE INDEX idx_node_process_name ON node_process_tracking(process_name);
|
||||
CREATE INDEX idx_python_version_name ON python_version_baseline(runtime_name);
|
||||
CREATE INDEX idx_python_script_path ON python_script_tracking(script_path);
|
||||
175
v1.1/monitor_v1.11/portal/page_monitor_v1.11.sh
Executable file
175
v1.1/monitor_v1.11/portal/page_monitor_v1.11.sh
Executable file
@@ -0,0 +1,175 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry WordPress Portal 監控 (Layer 4)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/portal/page_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/portal_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
# WordPress 配置
|
||||
WP_SITE="https://wp.momentry.ddns.net"
|
||||
WP_DB_HOST="localhost"
|
||||
WP_DB_NAME="wordpress"
|
||||
WP_DB_USER="wp_user"
|
||||
WP_DB_PASS="wp_password_123"
|
||||
|
||||
# 顏色
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
# 記錄頁面檢查結果
|
||||
record_page() {
|
||||
local url=$1
|
||||
local page_type=$2
|
||||
local accessible=$3
|
||||
local response_time=$4
|
||||
local http_status=$5
|
||||
local error=$6
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_portal_pages (page_url, page_type, is_accessible, response_time_ms, http_status, error_message, checked_at)
|
||||
VALUES ('$url', '$page_type', $accessible, $response_time, $http_status, '$error', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 記錄用戶
|
||||
record_user() {
|
||||
local user_id=$1
|
||||
local username=$2
|
||||
local email=$3
|
||||
local role=$4
|
||||
local is_active=$5
|
||||
local last_login=$6
|
||||
local created_at=$7
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_portal_users (user_id, username, email, role, is_active, last_login, created_at, detected_at)
|
||||
VALUES ($user_id, '$username', '$email', '$role', $is_active, $last_login, $created_at, NOW())
|
||||
ON CONFLICT DO NOTHING;
|
||||
EOF
|
||||
}
|
||||
|
||||
# 記錄異常
|
||||
record_anomaly() {
|
||||
local anomaly_type=$1
|
||||
local severity=$2
|
||||
local username=$3
|
||||
local description=$4
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_anomalies (anomaly_type, severity, source_type, username, description, detected_at)
|
||||
VALUES ('$anomaly_type', '$severity', 'wordpress', '$username', '$description', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 檢查頁面
|
||||
check_page() {
|
||||
local url=$1
|
||||
local page_type=$2
|
||||
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" "$url" --max-time 10 -k -L 2>/dev/null || echo "000")
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
accessible="true"
|
||||
error=""
|
||||
echo -e "${GREEN}✓${NC} $page_type - ${ms}ms (HTTP $http_code)"
|
||||
else
|
||||
accessible="false"
|
||||
error="HTTP $http_code"
|
||||
echo -e "${RED}✗${NC} $page_type - HTTP $http_code"
|
||||
fi
|
||||
|
||||
record_page "$url" "$page_type" "$accessible" "$ms" "$http_code" "$error"
|
||||
}
|
||||
|
||||
# 檢查用戶
|
||||
check_users() {
|
||||
echo ""
|
||||
echo "WordPress 用戶檢查:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
# 獲取用戶列表
|
||||
users=$(mysql -u"$WP_DB_USER" -p"$WP_DB_PASS" -h "$WP_DB_HOST" "$WP_DB_NAME" -N -e "
|
||||
SELECT u.ID, u.user_login, u.user_email, u.user_registered, u.user_status,
|
||||
COALESCE(m.meta_value, 'subscriber') as role
|
||||
FROM wp_users u
|
||||
LEFT JOIN wp_usermeta m ON u.ID = m.user_id AND m.meta_key = 'wp_capabilities'
|
||||
ORDER BY u.ID;
|
||||
" 2>/dev/null)
|
||||
|
||||
if [ -z "$users" ]; then
|
||||
echo "無法連接 WordPress 資料庫"
|
||||
return 1
|
||||
fi
|
||||
|
||||
local admin_count=0
|
||||
local total_users=0
|
||||
|
||||
while IFS='|' read -r id login email registered status role; do
|
||||
[ -z "$id" ] && continue
|
||||
|
||||
total_users=$((total_users + 1))
|
||||
|
||||
# 判斷是否管理員
|
||||
if echo "$role" | grep -q "administrator"; then
|
||||
admin_count=$((admin_count + 1))
|
||||
role="administrator"
|
||||
elif echo "$role" | grep -q "editor"; then
|
||||
role="editor"
|
||||
elif echo "$role" | grep -q "author"; then
|
||||
role="author"
|
||||
elif echo "$role" | grep -q "contributor"; then
|
||||
role="contributor"
|
||||
else
|
||||
role="subscriber"
|
||||
fi
|
||||
|
||||
# 記錄用戶
|
||||
record_user "$id" "$login" "$email" "$role" "true" "NULL" "'$registered'"
|
||||
|
||||
echo " - $login ($role)"
|
||||
|
||||
done <<< "$users"
|
||||
|
||||
echo "----------------------------------------"
|
||||
echo "總用戶: $total_users | 管理員: $admin_count"
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 4: WordPress Portal Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
echo "頁面可訪問性檢查:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
# 檢查首頁
|
||||
check_page "$WP_SITE/" "homepage"
|
||||
|
||||
# 檢查登入頁
|
||||
check_page "$WP_SITE/wp-login.php" "login_page"
|
||||
|
||||
# 檢查 wp-json API
|
||||
check_page "$WP_SITE/wp-json/" "api"
|
||||
|
||||
echo ""
|
||||
check_users
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
log "Portal check completed"
|
||||
93
v1.1/monitor_v1.11/service/external_monitor_v1.11.sh
Executable file
93
v1.1/monitor_v1.11/service/external_monitor_v1.11.sh
Executable file
@@ -0,0 +1,93 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry 外部監控 (Layer 1)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/service/external_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/external_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
# 記錄結果
|
||||
record_external() {
|
||||
local target=$1
|
||||
local target_type=$2
|
||||
local reachable=$3
|
||||
local response_time=$4
|
||||
local error=$5
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_external (target_name, target_type, is_reachable, response_time_ms, error_message, checked_at)
|
||||
VALUES ('$target', '$target_type', $reachable, $response_time, '$error', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 檢查 DDNS
|
||||
check_ddns() {
|
||||
local start=$(date +%s%N)
|
||||
local ip=$(dig +short momentry.ddns.net 2>/dev/null | tail -1)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ -n "$ip" ]; then
|
||||
echo "✓ DDNS (momentry.ddns.net) -> $ip (${ms}ms)"
|
||||
record_external "ddns" "ddns" "true" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo "✗ DDNS (momentry.ddns.net) - DNS resolution failed"
|
||||
record_external "ddns" "ddns" "false" "0" "DNS resolution failed"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查網關
|
||||
check_gateway() {
|
||||
local start=$(date +%s%N)
|
||||
if ping -c 1 -W 2 192.168.110.1 > /dev/null 2>&1; then
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
echo "✓ Gateway (192.168.110.1) - ${ms}ms"
|
||||
record_external "gateway" "gateway" "true" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo "✗ Gateway (192.168.110.1) - Unreachable"
|
||||
record_external "gateway" "gateway" "false" "0" "Unreachable"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查互聯網
|
||||
check_internet() {
|
||||
local start=$(date +%s%N)
|
||||
if ping -c 1 -W 2 8.8.8.8 > /dev/null 2>&1; then
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
echo "✓ Internet (8.8.8.8) - ${ms}ms"
|
||||
record_external "internet" "internet" "true" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo "✗ Internet (8.8.8.8) - Unreachable"
|
||||
record_external "internet" "internet" "false" "0" "Unreachable"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 1: External Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
check_ddns
|
||||
check_gateway
|
||||
check_internet
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
log "External check completed"
|
||||
450
v1.1/monitor_v1.11/service/health_check_v1.11.sh
Executable file
450
v1.1/monitor_v1.11/service/health_check_v1.11.sh
Executable file
@@ -0,0 +1,450 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry 服務健康檢查 (Layer 2)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/service/health_check.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# 載入密碼配置
|
||||
if [ -f "$MONITOR_DIR/common/load_credentials.sh" ]; then
|
||||
source "$MONITOR_DIR/common/load_credentials.sh"
|
||||
fi
|
||||
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/service_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
# 顏色
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
# 記錄結果到資料庫
|
||||
record_service() {
|
||||
local service=$1
|
||||
local status=$2
|
||||
local response_time=$3
|
||||
local error_msg=$4
|
||||
|
||||
PGPASSWORD="$PG_PASSWORD" psql -U "$PG_USER" -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_services (service_name, service_type, status, response_time_ms, error_message, checked_at)
|
||||
VALUES ('$service', 'service', '$status', $response_time, '$error_msg', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 檢查 PostgreSQL
|
||||
check_postgresql() {
|
||||
local start=$(date +%s%N)
|
||||
if PGPASSWORD="$PG_PASSWORD" pg_isready -h localhost -p 5432 -U "$PG_USER" > /dev/null 2>&1; then
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
echo -e "${GREEN}✓${NC} PostgreSQL (5432) - ${ms}ms"
|
||||
record_service "postgresql" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} PostgreSQL (5432) - Down"
|
||||
record_service "postgresql" "down" "0" "Connection failed"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 Redis
|
||||
check_redis() {
|
||||
local start=$(date +%s%N)
|
||||
if redis-cli -a "$REDIS_PASSWORD" ping 2>/dev/null | grep -q "PONG"; then
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
echo -e "${GREEN}✓${NC} Redis (6379) - ${ms}ms"
|
||||
record_service "redis" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} Redis (6379) - Down"
|
||||
record_service "redis" "down" "0" "Connection failed"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 MariaDB
|
||||
check_mariadb() {
|
||||
local start=$(date +%s%N)
|
||||
if mysql -u "$MARIADB_USER" -p"$MARIADB_PASSWORD" -e "SELECT 1" > /dev/null 2>&1; then
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
echo -e "${GREEN}✓${NC} MariaDB (3306) - ${ms}ms"
|
||||
record_service "mariadb" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} MariaDB (3306) - Down"
|
||||
record_service "mariadb" "down" "0" "Connection failed"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 n8n
|
||||
check_n8n() {
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8085/ --max-time 5)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ "$http_code" = "200" ] || [ "$http_code" = "302" ]; then
|
||||
echo -e "${GREEN}✓${NC} n8n (8085) - ${ms}ms"
|
||||
record_service "n8n" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} n8n (8085) - HTTP $http_code"
|
||||
record_service "n8n" "down" "0" "HTTP $http_code"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 Caddy
|
||||
check_caddy() {
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:2019/config/ --max-time 5)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
echo -e "${GREEN}✓${NC} Caddy (2019) - ${ms}ms"
|
||||
record_service "caddy" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} Caddy (2019) - HTTP $http_code"
|
||||
record_service "caddy" "down" "0" "HTTP $http_code"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 Gitea
|
||||
check_gitea() {
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:3000/ --max-time 5)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
echo -e "${GREEN}✓${NC} Gitea (3000) - ${ms}ms"
|
||||
record_service "gitea" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} Gitea (3000) - HTTP $http_code"
|
||||
record_service "gitea" "down" "0" "HTTP $http_code"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 SFTPGo
|
||||
check_sftpgo() {
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8080 --max-time 5)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
# 檢查 SFTP 端口
|
||||
local sftp_port=$(lsof -i :2022 2>/dev/null | grep -c LISTEN || echo "0")
|
||||
local webdav_port=$(lsof -i :8090 2>/dev/null | grep -c LISTEN || echo "0")
|
||||
|
||||
# 檢查 PostgreSQL 連接
|
||||
local db_conn=$(PGPASSWORD="$PG_PASSWORD" psql -U "$PG_USER" -h localhost -d postgres -t -c "SELECT numbackends FROM pg_stat_database WHERE datname='sftpgo';" 2>/dev/null | xargs || echo "0")
|
||||
|
||||
if [ "$http_code" = "200" ] || [ "$http_code" = "301" ] || [ "$http_code" = "302" ]; then
|
||||
echo -e "${GREEN}✓${NC} SFTPGo (8080) - ${ms}ms | SFTP:$sftp_port | WebDAV:$webdav_port | DB:$db_conn"
|
||||
record_service "sftpgo" "up" "$ms" "SFTP:$sftp_port WebDAV:$webdav_port DB:$db_conn"
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} SFTPGo (8080) - HTTP $http_code"
|
||||
record_service "sftpgo" "down" "0" "HTTP $http_code"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# SFTPGo 詳細監控
|
||||
check_sftpgo_detailed() {
|
||||
echo ""
|
||||
echo "=== SFTPGo 詳細監控 ==="
|
||||
|
||||
# 1. 服務狀態
|
||||
echo "1. 服務狀態:"
|
||||
ps aux | grep sftpgo | grep -v grep | awk '{print " PID: "$2" CMD: "$11" "$12}'
|
||||
|
||||
# 2. 端口監聽
|
||||
echo "2. 端口監聽:"
|
||||
echo " - HTTP (8080): $(lsof -i :8080 2>/dev/null | grep -c LISTEN || echo '0')"
|
||||
echo " - SFTP (2022): $(lsof -i :2022 2>/dev/null | grep -c LISTEN || echo '0')"
|
||||
echo " - WebDAV (8090): $(lsof -i :8090 2>/dev/null | grep -c LISTEN || echo '0')"
|
||||
|
||||
# 3. PostgreSQL 連接
|
||||
echo "3. PostgreSQL 連接:"
|
||||
PGPASSWORD="$PG_PASSWORD" psql -U "$PG_USER" -h localhost -d postgres -c "SELECT numbackends, xact_commit, xact_rollback FROM pg_stat_database WHERE datname='sftpgo';" 2>/dev/null | grep -v "numbackends\|^$\|row)" || echo " 無數據"
|
||||
|
||||
# 4. 用戶統計
|
||||
echo "4. 用戶統計:"
|
||||
PGPASSWORD="$SFTPGO_PASSWORD" psql -U "$SFTPGO_USER" -h localhost -d sftpgo -c "SELECT 'users' as type, COUNT(*) as count FROM users UNION ALL SELECT 'admins', COUNT(*) FROM admins UNION ALL SELECT 'api_keys', COUNT(*) FROM api_keys;" 2>/dev/null | grep -v "^$\|type\|^(\|row)" || echo " 無數據"
|
||||
|
||||
# 5. 數據庫大小
|
||||
echo "5. 數據庫大小:"
|
||||
PGPASSWORD="$PG_PASSWORD" psql -U "$PG_USER" -h localhost -d postgres -t -c "SELECT pg_size_pretty(pg_database_size('sftpgo'));" 2>/dev/null | xargs || echo " 無法獲取"
|
||||
|
||||
# 6. 磁盤使用
|
||||
echo "6. 文件存儲使用:"
|
||||
du -sh /Users/accusys/momentry/var/sftpgo/data/ 2>/dev/null | awk '{print " "$2": "$1}'
|
||||
}
|
||||
|
||||
# SFTPGo 認證失敗監控
|
||||
check_sftpgo_auth_failures() {
|
||||
local log_file="/Users/accusys/momentry/log/sftpgo.log"
|
||||
local threshold=${1:-5} # 默認 5 次失敗
|
||||
|
||||
if [ ! -f "$log_file" ]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 檢查過去 1 小時的認證失敗
|
||||
local failures=$(grep -i "authentication error\|invalid credentials\|login failed\|auth error" "$log_file" 2>/dev/null | wc -l)
|
||||
|
||||
if [ "$failures" -gt "$threshold" ]; then
|
||||
echo "⚠️ SFTPGo 認證失敗過多: $failures 次"
|
||||
return 1
|
||||
else
|
||||
echo "✓ SFTPGo 認證失敗: $failures 次 (閾值: $threshold)"
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
# SFTPGo 傳輸統計
|
||||
check_sftpgo_transfers() {
|
||||
echo ""
|
||||
echo "=== SFTPGo 傳輸統計 ==="
|
||||
|
||||
# 檢查活動傳輸
|
||||
local active_transfers=$(PGPASSWORD="$SFTPGO_PASSWORD" psql -U "$SFTPGO_USER" -h localhost -d sftpgo -t -c "SELECT COUNT(*) FROM active_transfers;" 2>/dev/null | xargs || echo "0")
|
||||
echo "活動傳輸: $active_transfers"
|
||||
|
||||
# 檢查今日訪問IP
|
||||
echo "今日訪問來源:"
|
||||
tail -1000 /Users/accusys/momentry/log/sftpgo_access.log 2>/dev/null | grep -o '"remote_ip":"[^"]*"' | cut -d'"' -f4 | sort | uniq -c | sort -rn | head -5 | awk '{print " "$2": "$1" 次"}'
|
||||
}
|
||||
|
||||
# 檢查 Ollama
|
||||
check_ollama() {
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:11434/api/tags --max-time 5)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
echo -e "${GREEN}✓${NC} Ollama (11434) - ${ms}ms"
|
||||
record_service "ollama" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} Ollama (11434) - HTTP $http_code"
|
||||
record_service "ollama" "down" "0" "HTTP $http_code"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 Qdrant
|
||||
check_qdrant() {
|
||||
local start=$(date +%s%N)
|
||||
local http_code=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:6333/collections --max-time 5)
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
|
||||
if [ "$http_code" = "200" ] || [ "$http_code" = "401" ]; then
|
||||
echo -e "${GREEN}✓${NC} Qdrant (6333) - ${ms}ms"
|
||||
record_service "qdrant" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} Qdrant (6333) - HTTP $http_code"
|
||||
record_service "qdrant" "down" "0" "HTTP $http_code"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 MongoDB
|
||||
check_mongodb() {
|
||||
local start=$(date +%s%N)
|
||||
if mongosh --quiet --eval "db.adminCommand('ping')" > /dev/null 2>&1; then
|
||||
local end=$(date +%s%N)
|
||||
local ms=$(( (end - start) / 1000000 ))
|
||||
echo -e "${GREEN}✓${NC} MongoDB (27017) - ${ms}ms"
|
||||
record_service "mongodb" "up" "$ms" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} MongoDB (27017) - Down"
|
||||
record_service "mongodb" "down" "0" "Connection failed"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 PHP-FPM
|
||||
check_php() {
|
||||
if pgrep -f "php-fpm" > /dev/null 2>&1; then
|
||||
echo -e "${GREEN}✓${NC} PHP-FPM - Running"
|
||||
record_service "php" "up" "1" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}✗${NC} PHP-FPM - Not running"
|
||||
record_service "php" "down" "0" "Process not found"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 RustDesk
|
||||
check_rustdesk() {
|
||||
local hbbs_ok=false
|
||||
local hbbr_ok=false
|
||||
|
||||
if nc -z localhost 21116 > /dev/null 2>&1; then
|
||||
hbbs_ok=true
|
||||
fi
|
||||
|
||||
if nc -z localhost 21117 > /dev/null 2>&1; then
|
||||
hbbr_ok=true
|
||||
fi
|
||||
|
||||
if $hbbs_ok && $hbbr_ok; then
|
||||
echo -e "${GREEN}✓${NC} RustDesk (21116/21117) - Running"
|
||||
record_service "rustdesk" "up" "1" ""
|
||||
return 0
|
||||
else
|
||||
echo -e "${YELLOW}⚠${NC} RustDesk - Partial (hbbs: $hbbs_ok, hbbr: $hbbr_ok)"
|
||||
record_service "rustdesk" "degraded" "0" "hbbs:$hbbs_ok hbbr:$hbbr_ok"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 Node.js 版本
|
||||
check_node() {
|
||||
local LOCKED_NODE_VERSION="22"
|
||||
local version_issues=0
|
||||
|
||||
local node_pids=$(pgrep -f "n8n" 2>/dev/null)
|
||||
|
||||
if [ -z "$node_pids" ]; then
|
||||
echo -e "${YELLOW}⚠${NC} Node.js - n8n not running"
|
||||
record_service "node" "degraded" "1" "n8n not running"
|
||||
return 1
|
||||
fi
|
||||
|
||||
for pid in $node_pids; do
|
||||
local node_path=$(lsof -p $pid 2>/dev/null | grep "txt" | grep "node" | head -1 | awk '{print $NF}' | grep -v "dylib")
|
||||
|
||||
if [ -n "$node_path" ] && [ -f "$node_path" ]; then
|
||||
local node_version=$($node_path --version 2>/dev/null | sed 's/v//')
|
||||
local node_major=$(echo "$node_version" | cut -d. -f1)
|
||||
|
||||
if [ "$node_major" != "$LOCKED_NODE_VERSION" ]; then
|
||||
version_issues=$((version_issues + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $version_issues -gt 0 ]; then
|
||||
echo -e "${RED}✗${NC} Node.js - Version issues detected"
|
||||
record_service "node" "degraded" "1" "$version_issues version issues"
|
||||
return 1
|
||||
else
|
||||
echo -e "${GREEN}✓${NC} Node.js (${LOCKED_NODE_VERSION}.x) - Running"
|
||||
record_service "node" "up" "1" ""
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢查 Python 版本
|
||||
check_python() {
|
||||
local LOCKED_PYTHON_VERSION="3.11.14"
|
||||
local script_issues=0
|
||||
|
||||
local scripts=(
|
||||
"/Users/accusys/momentry_core_0.1/scripts/asr_processor.py"
|
||||
"/Users/accusys/momentry_core_0.1/scripts/thumbnail_extractor.py"
|
||||
)
|
||||
|
||||
for script in "${scripts[@]}"; do
|
||||
if [ -f "$script" ]; then
|
||||
local shebang=$(head -1 "$script")
|
||||
|
||||
if [[ "$shebang" != *"python3.11"* ]]; then
|
||||
script_issues=$((script_issues + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $script_issues -gt 0 ]; then
|
||||
echo -e "${RED}✗${NC} Python - Script version issues"
|
||||
record_service "python" "degraded" "1" "$script_issues script issues"
|
||||
return 1
|
||||
else
|
||||
echo -e "${GREEN}✓${NC} Python (${LOCKED_PYTHON_VERSION}) - Configured"
|
||||
record_service "python" "up" "1" ""
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 2: Service Health Check"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
total=0
|
||||
passed=0
|
||||
|
||||
total=$((total + 1))
|
||||
check_postgresql && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_redis && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_mariadb && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_n8n && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_caddy && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_gitea && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_sftpgo && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_ollama && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_qdrant && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_mongodb && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_php && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_rustdesk && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_node && passed=$((passed + 1))
|
||||
|
||||
total=$((total + 1))
|
||||
check_python && passed=$((passed + 1))
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo "Result: $passed / $total services healthy"
|
||||
echo "========================================"
|
||||
|
||||
log "Service check completed: $passed/$total healthy"
|
||||
270
v1.1/monitor_v1.11/service/node_monitor_v1.11.sh
Executable file
270
v1.1/monitor_v1.11/service/node_monitor_v1.11.sh
Executable file
@@ -0,0 +1,270 @@
|
||||
#!/bin/bash
|
||||
|
||||
#===============================================================================
|
||||
# Momentry Node.js 監控腳本
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/service/node_monitor.sh
|
||||
#
|
||||
# 監控重點:
|
||||
# - n8n 使用的 Node.js 版本鎖定 (22.x)
|
||||
# - 進程數量與狀態
|
||||
# - 資源使用情況
|
||||
#
|
||||
# 使用方式:
|
||||
# ./node_monitor.sh status # 顯示監控狀態
|
||||
# ./node_monitor.sh baseline # 建立版本基線
|
||||
# ./node_monitor.sh check # 檢查版本變化
|
||||
#===============================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/node_check.log"
|
||||
|
||||
# 顏色定義
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
# 鎖定的 Node.js 版本
|
||||
LOCKED_NODE_VERSION="22"
|
||||
LOCKED_NODE_MINOR="22"
|
||||
|
||||
#===============================================================================
|
||||
# 記錄函數
|
||||
#===============================================================================
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
log_success() {
|
||||
echo -e "${GREEN}[$(date '+%Y-%m-%d %H:%M:%S')] ✅ $1${NC}" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[$(date '+%Y-%m-%d %H:%M:%S')] ❌ $1${NC}" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
log_warn() {
|
||||
echo -e "${YELLOW}[$(date '+%Y-%m-%d %H:%M:%S')] ⚠️ $1${NC}" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 記錄到資料庫
|
||||
#===============================================================================
|
||||
record_node_baseline() {
|
||||
local runtime_name=$1
|
||||
local current_version=$2
|
||||
local process_path=$3
|
||||
local pid=$4
|
||||
|
||||
local required_version="${LOCKED_NODE_VERSION}.x"
|
||||
local is_compliant="false"
|
||||
if [[ "$current_version" == "${LOCKED_NODE_VERSION}".* ]]; then
|
||||
is_compliant="true"
|
||||
fi
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO node_version_baseline (runtime_name, required_version, current_version, process_name, process_path, is_compliant, locked_path, checked_at)
|
||||
VALUES ('$runtime_name', '$required_version', '$current_version', 'node', '$process_path', $is_compliant, '$process_path', NOW())
|
||||
ON CONFLICT DO NOTHING;
|
||||
EOF
|
||||
}
|
||||
|
||||
record_node_history() {
|
||||
local process_name=$1
|
||||
local old_version=$2
|
||||
local new_version=$3
|
||||
local old_path=$4
|
||||
local new_path=$5
|
||||
|
||||
# node_version_history table does not exist - skip recording
|
||||
true
|
||||
}
|
||||
|
||||
record_monitor_service() {
|
||||
local service=$1
|
||||
local status=$2
|
||||
local version=$3
|
||||
local error_msg=$4
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_services (service_name, service_type, status, response_time_ms, error_message, checked_at)
|
||||
VALUES ('$service', 'node', '$status', 0, '$version - $error_msg', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 發現 Node.js 進程
|
||||
#===============================================================================
|
||||
discover_node_processes() {
|
||||
log "發現 Node.js 進程..."
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo "Node.js 監控狀態"
|
||||
echo "時間: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
# 獲取所有 node 進程
|
||||
local node_pids=$(pgrep -f "node" 2>/dev/null)
|
||||
|
||||
if [ -z "$node_pids" ]; then
|
||||
echo -e "${RED}沒有運行中的 Node.js 進程${NC}"
|
||||
record_monitor_service "node" "down" "N/A" "No processes"
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "鎖定版本: Node.js ${LOCKED_NODE_VERSION}.x (n8n 專用)"
|
||||
echo ""
|
||||
echo "----------------------------------------"
|
||||
echo "發現的 Node.js 進程:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
local total_processes=0
|
||||
local n8n_processes=0
|
||||
local version_issues=0
|
||||
|
||||
for pid in $node_pids; do
|
||||
# 獲取進程命令
|
||||
local cmd=$(ps -o args= -p $pid 2>/dev/null | head -1)
|
||||
|
||||
# 獲取 Node.js 版本
|
||||
local node_path=$(lsof -p $pid 2>/dev/null | grep "txt" | grep "node" | head -1 | awk '{print $NF}' | grep -v "dylib")
|
||||
|
||||
if [ -n "$node_path" ] && [ -f "$node_path" ]; then
|
||||
local node_version=$($node_path --version 2>/dev/null | sed 's/v//')
|
||||
local node_major=$(echo "$node_version" | cut -d. -f1)
|
||||
local node_minor=$(echo "$node_version" | cut -d. -f2)
|
||||
else
|
||||
local node_version="unknown"
|
||||
local node_major="unknown"
|
||||
fi
|
||||
|
||||
# 內存使用
|
||||
local mem=$(ps -o rss= -p $pid 2>/dev/null | awk '{print int($1/1024)}')
|
||||
|
||||
# CPU 使用
|
||||
local cpu=$(ps -o %cpu= -p $pid 2>/dev/null | awk '{print int($1)}')
|
||||
|
||||
# 運行時間
|
||||
local time=$(ps -o etime= -p $pid 2>/dev/null | tr -d ' ')
|
||||
|
||||
# 識別服務類型
|
||||
local service_type="other"
|
||||
if echo "$cmd" | grep -q "n8n"; then
|
||||
service_type="n8n"
|
||||
n8n_processes=$((n8n_processes + 1))
|
||||
elif echo "$cmd" | grep -q "worker"; then
|
||||
service_type="n8n-worker"
|
||||
n8n_processes=$((n8n_processes + 1))
|
||||
fi
|
||||
|
||||
# 版本檢查
|
||||
local version_status="✅"
|
||||
if [ "$service_type" = "n8n" ] || [ "$service_type" = "n8n-worker" ]; then
|
||||
if [ "$node_major" != "$LOCKED_NODE_VERSION" ]; then
|
||||
version_status="❌ 版本錯誤!"
|
||||
version_issues=$((version_issues + 1))
|
||||
log_error "n8n 使用 Node.js $node_version (應為 ${LOCKED_NODE_VERSION}.x)"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo " PID: $pid"
|
||||
echo " 命令: ${cmd:0:60}..."
|
||||
echo " Node.js: $node_version $version_status"
|
||||
echo " 路徑: $node_path"
|
||||
echo " 內存: ${mem}MB | CPU: ${cpu}% | 運行: $time"
|
||||
echo " 類型: $service_type"
|
||||
echo ""
|
||||
|
||||
total_processes=$((total_processes + 1))
|
||||
|
||||
# 記錄基線
|
||||
record_node_baseline "$service_type" "$node_version" "$node_path" "$pid"
|
||||
done
|
||||
|
||||
echo "----------------------------------------"
|
||||
echo "總結:"
|
||||
echo " 總進程數: $total_processes"
|
||||
echo " n8n 相關: $n8n_processes"
|
||||
echo " 版本問題: $version_issues"
|
||||
echo "========================================"
|
||||
|
||||
# 記錄到資料庫
|
||||
if [ $version_issues -gt 0 ]; then
|
||||
record_monitor_service "node" "degraded" "${LOCKED_NODE_VERSION}.x" "$version_issues version issues"
|
||||
return 1
|
||||
else
|
||||
record_monitor_service "node" "up" "${LOCKED_NODE_VERSION}.x" "OK"
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 版本基線檢查
|
||||
#===============================================================================
|
||||
check_baseline() {
|
||||
log "檢查 Node.js 版本基線..."
|
||||
|
||||
# 檢查 n8n 進程
|
||||
local n8n_pid=$(pgrep -f "n8n start" | head -1)
|
||||
|
||||
if [ -z "$n8n_pid" ]; then
|
||||
log_error "n8n 進程未運行"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 獲取 n8n 使用的 Node.js 版本
|
||||
local node_path=$(lsof -p $n8n_pid 2>/dev/null | grep "txt" | grep "node" | head -1 | awk '{print $NF}' | grep -v "dylib")
|
||||
|
||||
if [ -n "$node_path" ] && [ -f "$node_path" ]; then
|
||||
local node_version=$($node_path --version 2>/dev/null | sed 's/v//')
|
||||
local node_major=$(echo "$node_version" | cut -d. -f1)
|
||||
|
||||
echo "n8n 當前 Node.js 版本: $node_version"
|
||||
|
||||
if [ "$node_major" = "$LOCKED_NODE_VERSION" ]; then
|
||||
log_success "版本正確: Node.js $node_version"
|
||||
return 0
|
||||
else
|
||||
log_error "版本錯誤: Node.js $node_version (應為 ${LOCKED_NODE_VERSION}.x)"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
log_error "無法確定 Node.js 版本"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 顯示狀態
|
||||
#===============================================================================
|
||||
show_status() {
|
||||
discover_node_processes
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 主程序
|
||||
#===============================================================================
|
||||
command=${1:-status}
|
||||
|
||||
case $command in
|
||||
status|check)
|
||||
show_status
|
||||
;;
|
||||
baseline)
|
||||
check_baseline
|
||||
;;
|
||||
*)
|
||||
echo "用法: $0 {status|baseline}"
|
||||
echo ""
|
||||
echo " status - 顯示 Node.js 監控狀態"
|
||||
echo " baseline - 檢查版本基線"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
281
v1.1/monitor_v1.11/service/python_monitor_v1.11.sh
Executable file
281
v1.1/monitor_v1.11/service/python_monitor_v1.11.sh
Executable file
@@ -0,0 +1,281 @@
|
||||
#!/bin/bash
|
||||
|
||||
#===============================================================================
|
||||
# Momentry Python 監控腳本
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/service/python_monitor.sh
|
||||
#
|
||||
# 監控重點:
|
||||
# - Momentry Python 腳本版本鎖定 (3.11.14)
|
||||
# - 進程數量與狀態
|
||||
# - 腳本執行狀態
|
||||
#
|
||||
# 使用方式:
|
||||
# ./python_monitor.sh status # 顯示監控狀態
|
||||
# ./python_monitor.sh baseline # 建立版本基線
|
||||
# ./python_monitor.sh check # 檢查版本變化
|
||||
#===============================================================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/python_check.log"
|
||||
|
||||
# 顏色定義
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
# 鎖定的 Python 版本
|
||||
LOCKED_PYTHON_VERSION="3.11.14"
|
||||
LOCKED_PYTHON_MAJOR="3"
|
||||
LOCKED_PYTHON_MINOR="11"
|
||||
|
||||
# Momentry Python 腳本
|
||||
MOMENTRY_SCRIPTS=(
|
||||
"/Users/accusys/momentry_core_0.1/scripts/asr_processor.py"
|
||||
"/Users/accusys/momentry_core_0.1/scripts/thumbnail_extractor.py"
|
||||
)
|
||||
|
||||
#===============================================================================
|
||||
# 記錄函數
|
||||
#===============================================================================
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
log_success() {
|
||||
echo -e "${GREEN}[$(date '+%Y-%m-%d %H:%M:%S')] ✅ $1${NC}" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[$(date '+%Y-%m-%d %H:%M:%S')] ❌ $1${NC}" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
log_warn() {
|
||||
echo -e "${YELLOW}[$(date '+%Y-%m-%d %H:%M:%S')] ⚠️ $1${NC}" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 記錄到資料庫
|
||||
#===============================================================================
|
||||
record_python_baseline() {
|
||||
local runtime_name=$1
|
||||
local current_version=$2
|
||||
local interpreter_path=$3
|
||||
|
||||
local required_version="${LOCKED_PYTHON_VERSION}"
|
||||
local is_compliant="false"
|
||||
if [[ "$current_version" == "${LOCKED_PYTHON_VERSION}" ]]; then
|
||||
is_compliant="true"
|
||||
fi
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO python_version_baseline (runtime_name, required_version, current_version, interpreter_path, is_compliant, checked_at)
|
||||
VALUES ('$runtime_name', '$required_version', '$current_version', '$interpreter_path', $is_compliant, NOW())
|
||||
ON CONFLICT DO NOTHING;
|
||||
EOF
|
||||
}
|
||||
|
||||
record_python_history() {
|
||||
local script_name=$1
|
||||
local old_version=$2
|
||||
local new_version=$3
|
||||
local old_path=$4
|
||||
local new_path=$5
|
||||
|
||||
# python_version_history table does not exist - skip recording
|
||||
true
|
||||
}
|
||||
|
||||
record_monitor_service() {
|
||||
local service=$1
|
||||
local status=$2
|
||||
local version=$3
|
||||
local error_msg=$4
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_services (service_name, service_type, status, response_time_ms, error_message, checked_at)
|
||||
VALUES ('$service', 'python', '$status', 0, '$version - $error_msg', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 發現 Python 進程
|
||||
#===============================================================================
|
||||
discover_python_processes() {
|
||||
log "發現 Python 進程..."
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo "Python 監控狀態"
|
||||
echo "時間: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
echo "鎖定版本: Python ${LOCKED_PYTHON_VERSION} (Momentry 專用)"
|
||||
echo ""
|
||||
|
||||
# 檢查 Momentry 腳本
|
||||
echo "----------------------------------------"
|
||||
echo "Momentry Python 腳本:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
local script_issues=0
|
||||
|
||||
for script in "${MOMENTRY_SCRIPTS[@]}"; do
|
||||
if [ -f "$script" ]; then
|
||||
# 獲取腳本使用的 Python
|
||||
local shebang=$(head -1 "$script")
|
||||
local python_path=""
|
||||
|
||||
if [[ "$shebang" == *"/python3.11"* ]]; then
|
||||
python_path="/opt/homebrew/bin/python3.11"
|
||||
elif [[ "$shebang" == *"/python3"* ]]; then
|
||||
# 檢查系統 python3
|
||||
python_path=$(which python3 2>/dev/null)
|
||||
fi
|
||||
|
||||
if [ -n "$python_path" ] && [ -f "$python_path" ]; then
|
||||
local python_version=$($python_path --version 2>&1 | sed 's/Python //')
|
||||
local python_major=$(echo "$python_version" | cut -d. -f1)
|
||||
local python_minor=$(echo "$python_version" | cut -d. -f2)
|
||||
|
||||
# 檢查版本
|
||||
local version_status="✅"
|
||||
if [ "$python_major" = "$LOCKED_PYTHON_MAJOR" ] && [ "$python_minor" = "$LOCKED_PYTHON_MINOR" ]; then
|
||||
log_success "$(basename $script): $python_version"
|
||||
else
|
||||
version_status="❌ 版本錯誤!"
|
||||
script_issues=$((script_issues + 1))
|
||||
log_error "$(basename $script): $python_version (應為 ${LOCKED_PYTHON_VERSION})"
|
||||
fi
|
||||
|
||||
echo " $(basename $script)"
|
||||
echo " 路徑: $python_path"
|
||||
echo " 版本: $python_version $version_status"
|
||||
echo " shebang: $shebang"
|
||||
echo ""
|
||||
|
||||
# 記錄基線
|
||||
record_python_baseline "python_${LOCKED_PYTHON_VERSION}" "$python_version" "$python_path"
|
||||
else
|
||||
log_error "$(basename $script): 無法確定 Python 路徑"
|
||||
script_issues=$((script_issues + 1))
|
||||
fi
|
||||
else
|
||||
log_warn "$(basename $script): 文件不存在"
|
||||
script_issues=$((script_issues + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
# 檢查運行中的 Python 進程
|
||||
echo "----------------------------------------"
|
||||
echo "運行中的 Python 進程:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
local python_pids=$(pgrep -f "python" 2>/dev/null)
|
||||
local total_processes=0
|
||||
|
||||
if [ -n "$python_pids" ]; then
|
||||
for pid in $python_pids; do
|
||||
# 獲取進程命令
|
||||
local cmd=$(ps -o args= -p $pid 2>/dev/null | head -1 | cut -c1-80)
|
||||
|
||||
# 獲取 Python 路徑
|
||||
local python_path=$(lsof -p $pid 2>/dev/null | grep "txt" | grep "Python" | head -1 | awk '{print $NF}' | grep -v "dylib")
|
||||
|
||||
if [ -n "$python_path" ] && [ -f "$python_path" ]; then
|
||||
local python_version=$($python_path --version 2>&1 | sed 's/Python //')
|
||||
else
|
||||
local python_version="unknown"
|
||||
fi
|
||||
|
||||
# 內存使用
|
||||
local mem=$(ps -o rss= -p $pid 2>/dev/null | awk '{print int($1/1024)}')
|
||||
|
||||
echo " PID $pid: $cmd"
|
||||
echo " Python: $python_version"
|
||||
echo " 內存: ${mem}MB"
|
||||
echo ""
|
||||
|
||||
total_processes=$((total_processes + 1))
|
||||
done
|
||||
else
|
||||
echo " (無運行中的 Python 進程)"
|
||||
fi
|
||||
|
||||
echo "----------------------------------------"
|
||||
echo "總結:"
|
||||
echo " 總進程數: $total_processes"
|
||||
echo " 腳本問題: $script_issues"
|
||||
echo "========================================"
|
||||
|
||||
# 記錄到資料庫
|
||||
if [ $script_issues -gt 0 ]; then
|
||||
record_monitor_service "python" "degraded" "${LOCKED_PYTHON_VERSION}" "$script_issues issues"
|
||||
return 1
|
||||
else
|
||||
record_monitor_service "python" "up" "${LOCKED_PYTHON_VERSION}" "OK"
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 版本基線檢查
|
||||
#===============================================================================
|
||||
check_baseline() {
|
||||
log "檢查 Python 版本基線..."
|
||||
|
||||
local script_issues=0
|
||||
|
||||
for script in "${MOMENTRY_SCRIPTS[@]}"; do
|
||||
if [ -f "$script" ]; then
|
||||
local shebang=$(head -1 "$script")
|
||||
|
||||
if [[ "$shebang" == *"/python3.11"* ]]; then
|
||||
log_success "$(basename $script): 使用正確版本"
|
||||
else
|
||||
log_error "$(basename $script): 未使用 python3.11"
|
||||
script_issues=$((script_issues + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $script_issues -gt 0 ]; then
|
||||
return 1
|
||||
else
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 顯示狀態
|
||||
#===============================================================================
|
||||
show_status() {
|
||||
discover_python_processes
|
||||
}
|
||||
|
||||
#===============================================================================
|
||||
# 主程序
|
||||
#===============================================================================
|
||||
command=${1:-status}
|
||||
|
||||
case $command in
|
||||
status|check)
|
||||
show_status
|
||||
;;
|
||||
baseline)
|
||||
check_baseline
|
||||
;;
|
||||
*)
|
||||
echo "用法: $0 {status|baseline}"
|
||||
echo ""
|
||||
echo " status - 顯示 Python 監控狀態"
|
||||
echo " baseline - 檢查版本基線"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
163
v1.1/monitor_v1.11/users/session_tracker_v1.11.sh
Executable file
163
v1.1/monitor_v1.11/users/session_tracker_v1.11.sh
Executable file
@@ -0,0 +1,163 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry 使用者會話追蹤 (Layer 6)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/users/session_tracker.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/session_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
# 記錄會話
|
||||
record_session() {
|
||||
local session_type=$1
|
||||
local service=$2
|
||||
local username=$3
|
||||
local source_ip=$4
|
||||
local status=$5
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_sessions (session_type, service_name, username, source_ip, connected_at, status)
|
||||
VALUES ('$session_type', '$service', '$username', '$source_ip', NOW(), '$status');
|
||||
EOF
|
||||
}
|
||||
|
||||
# 記錄登入
|
||||
record_login() {
|
||||
local user_type=$1
|
||||
local username=$2
|
||||
local source_ip=$3
|
||||
local success=$4
|
||||
local method=$5
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_logins (user_type, username, source_ip, success, login_method, login_at)
|
||||
VALUES ('$user_type', '$username', '$source_ip', $success, '$method', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# 記錄異常
|
||||
record_anomaly() {
|
||||
local anomaly_type=$1
|
||||
local severity=$2
|
||||
local username=$3
|
||||
local source_ip=$4
|
||||
local description=$5
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_anomalies (anomaly_type, severity, source_type, username, source_ip, description, detected_at)
|
||||
VALUES ('$anomaly_type', '$severity', 'system', '$username', '$source_ip', '$description', NOW());
|
||||
EOF
|
||||
}
|
||||
|
||||
# SSH 會話
|
||||
track_ssh() {
|
||||
echo "SSH 會話:"
|
||||
|
||||
# 獲取當前 SSH 連線
|
||||
who | grep -E "pts|tty" | while read -r line; do
|
||||
user=$(echo "$line" | awk '{print $1}')
|
||||
tty=$(echo "$line" | awk '{print $2}')
|
||||
login_time=$(echo "$line" | awk '{print $3,$4}')
|
||||
ip=$(echo "$line" | awk '{print $NF}' | tr -d '()')
|
||||
|
||||
if [ -n "$ip" ] && [ "$ip" != "-" ]; then
|
||||
echo " - $user @ $ip (tty $tty) 登入時間: $login_time"
|
||||
record_session "ssh" "sshd" "$user" "$ip" "active"
|
||||
fi
|
||||
done
|
||||
|
||||
# 檢查 SSH 登入失敗
|
||||
echo ""
|
||||
echo "SSH 登入失敗 (最近 5 分鐘):"
|
||||
last -5 -f /var/log/auth.log 2>/dev/null | grep -i "failed password" | tail -5 | while read -r line; do
|
||||
user=$(echo "$line" | awk '{print $9}')
|
||||
ip=$(echo "$line" | grep -oE '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' | tail -1)
|
||||
|
||||
if [ -n "$ip" ]; then
|
||||
echo " - Failed: $user from $ip"
|
||||
record_login "system" "$user" "$ip" "false" "ssh"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# Web 服務會話
|
||||
track_web() {
|
||||
echo ""
|
||||
echo "Web 服務:"
|
||||
|
||||
# n8n 活躍會話 (如果有認證)
|
||||
n8n_sessions=0
|
||||
echo " - n8n: 檢查中... (需要 API key)"
|
||||
|
||||
# Gitea 活躍會話
|
||||
gitea_sessions=0
|
||||
echo " - Gitea: 檢查中... (需要登入)"
|
||||
}
|
||||
|
||||
# 資料庫連線
|
||||
track_database() {
|
||||
echo ""
|
||||
echo "資料庫連線:"
|
||||
|
||||
# PostgreSQL
|
||||
pg_conn=$(psql -U accusys -h localhost -t -A -c "SELECT count(*) FROM pg_stat_activity WHERE datname = 'momentry';" 2>/dev/null || echo "0")
|
||||
echo " - PostgreSQL: $pg_conn connections"
|
||||
|
||||
# Redis
|
||||
redis_conn=$(redis-cli -a accusys INFO clients 2>/dev/null | grep "connected_clients" | cut -d: -f2 | tr -d '\r')
|
||||
echo " - Redis: $redis_conn clients"
|
||||
}
|
||||
|
||||
# SFTP 會話
|
||||
track_sftp() {
|
||||
echo ""
|
||||
echo "SFTP 會話:"
|
||||
|
||||
# 檢查 SFTPGo 在線用戶
|
||||
if nc -z localhost 2222 2>/dev/null; then
|
||||
echo " - SFTPGo: 檢查中..."
|
||||
fi
|
||||
}
|
||||
|
||||
# 檢測暴力破解
|
||||
detect_bruteforce() {
|
||||
echo ""
|
||||
echo "異常檢測:"
|
||||
|
||||
# 檢查 SSH 暴力破解
|
||||
now=$(date +%s)
|
||||
window=300 # 5 分鐘
|
||||
|
||||
# 統計最近失敗
|
||||
fail_count=$(last -f /var/log/auth.log 2>/dev/null | grep -i "failed" | wc -l)
|
||||
|
||||
if [ $fail_count -gt 10 ]; then
|
||||
echo " ⚠️ 發現潛在暴力破解嘗試: $fail_count 次失敗"
|
||||
record_anomaly "bruteforce" "critical" "unknown" "multiple" "SSH暴力破解: $fail_count 次失敗"
|
||||
else
|
||||
echo " ✓ 無明顯暴力破解跡象"
|
||||
fi
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 6: User Session Tracking"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
track_ssh
|
||||
track_web
|
||||
track_database
|
||||
track_sftp
|
||||
detect_bruteforce
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
log "Session tracking completed"
|
||||
625
v1.1/monitor_v1.11/workflow/backup_n8n_api_v1.11.py
Executable file
625
v1.1/monitor_v1.11/workflow/backup_n8n_api_v1.11.py
Executable file
@@ -0,0 +1,625 @@
|
||||
#!/usr/bin/env python3.11
|
||||
"""
|
||||
n8n Workflow 備份腳本 - 使用 n8n REST API
|
||||
路徑: /Users/accusys/momentry_core_0.1/monitor/workflow/backup_n8n_api.py
|
||||
|
||||
功能:
|
||||
- 使用 n8n REST API 導出所有 workflows
|
||||
- 按用戶/Tags 分組備份
|
||||
- 變更偵測
|
||||
- 差異備份（只備份變更的 workflow）
|
||||
- SHA256 校驗
|
||||
- 備份驗證
|
||||
|
||||
前置需求:
|
||||
pip3.11 install requests
|
||||
|
||||
使用方式:
|
||||
python3.11 backup_n8n_api.py # 備份所有 workflows
|
||||
python3.11 backup_n8n_api.py --diff # 只顯示變更
|
||||
python3.11 backup_n8n_api.py --incremental # 差異備份（只備份變更的）
|
||||
python3.11 backup_n8n_api.py --list # 列出可用備份
|
||||
python3.11 backup_n8n_api.py --verify # 驗證最新備份
|
||||
python3.11 backup_n8n_api.py --stats # 顯示備份統計
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import hashlib
|
||||
import requests
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
# ============================================
|
||||
# 配置
|
||||
# ============================================
|
||||
BACKUP_ROOT = Path("/Users/accusys/momentry/backup/n8n_workflows/api")
|
||||
LOG_DIR = Path("/Users/accusys/momentry/log/monitor")
|
||||
TIMESTAMP = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
|
||||
N8N_BASE_URL = os.environ.get("N8N_BASE_URL", "https://n8n.momentry.ddns.net")
|
||||
N8N_API_KEY = os.environ.get("N8N_API_KEY", "")
|
||||
|
||||
# 顏色
|
||||
RED = "\033[0;31m"
|
||||
GREEN = "\033[0;32m"
|
||||
YELLOW = "\033[1;33m"
|
||||
BLUE = "\033[0;34m"
|
||||
NC = "\033[0m"
|
||||
|
||||
# ============================================
|
||||
# 日誌
|
||||
# ============================================
|
||||
LOG_FILE = LOG_DIR / "workflow_backup_api.log"
|
||||
|
||||
|
||||
def log(msg: str, color: str = ""):
|
||||
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
full_msg = f"[{timestamp}] {msg}"
|
||||
if color:
|
||||
print(f"{color}{full_msg}{NC}")
|
||||
else:
|
||||
print(full_msg)
|
||||
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(LOG_FILE, "a") as f:
|
||||
f.write(full_msg + "\n")
|
||||
|
||||
|
||||
def log_info(msg: str):
|
||||
log(msg)
|
||||
|
||||
|
||||
def log_success(msg: str):
|
||||
log(f"✅ {msg}", GREEN)
|
||||
|
||||
|
||||
def log_error(msg: str):
|
||||
log(f"❌ {msg}", RED)
|
||||
|
||||
|
||||
def log_warn(msg: str):
|
||||
log(f"⚠️ {msg}", YELLOW)
|
||||
|
||||
|
||||
# ============================================
|
||||
# n8n API 客戶端
|
||||
# ============================================
|
||||
class N8nAPIClient:
|
||||
def __init__(self, base_url: str, api_key: str):
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.session = requests.Session()
|
||||
self.session.headers.update({"X-N8N-API-Key": api_key})
|
||||
self.session.verify = True
|
||||
|
||||
def _request(self, method: str, path: str, **kwargs):
|
||||
url = f"{self.base_url}/api/v1{path}"
|
||||
resp = self.session.request(method, url, timeout=30, **kwargs)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
def list_workflows(self, limit: int = 100, cursor: str = None) -> dict:
|
||||
"""列出 workflows"""
|
||||
params = {"limit": limit}
|
||||
if cursor:
|
||||
params["cursor"] = cursor
|
||||
return self._request("GET", "/workflows", params=params)
|
||||
|
||||
def get_workflow(self, workflow_id: str) -> dict:
|
||||
"""取得 workflow 詳情"""
|
||||
return self._request("GET", f"/workflows/{workflow_id}")
|
||||
|
||||
def list_tags(self) -> dict:
|
||||
"""列出所有 tags"""
|
||||
return self._request("GET", "/tags")
|
||||
|
||||
def get_workflow_tags(self, workflow_id: str) -> dict:
|
||||
"""取得 workflow 的 tags"""
|
||||
return self._request("GET", f"/workflows/{workflow_id}/tags")
|
||||
|
||||
def list_executions(
|
||||
self, workflow_id: str = None, limit: int = 50, status: str = None
|
||||
) -> dict:
|
||||
"""列出執行記錄"""
|
||||
params: dict = {"limit": limit}
|
||||
if workflow_id:
|
||||
params["workflowId"] = workflow_id
|
||||
if status:
|
||||
params["status"] = status
|
||||
return self._request("GET", "/executions", params=params)
|
||||
|
||||
def health_check(self) -> bool:
|
||||
"""健康檢查"""
|
||||
try:
|
||||
self._request("GET", "/workflows", params={"limit": 1})
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ============================================
|
||||
# 備份功能
|
||||
# ============================================
|
||||
def compute_sha256(data) -> str:
|
||||
"""計算 JSON 的 SHA256"""
|
||||
json_str = json.dumps(data, sort_keys=True, ensure_ascii=False)
|
||||
return hashlib.sha256(json_str.encode()).hexdigest()
|
||||
|
||||
|
||||
def save_backup(workflows, tags, backup_dir: Path, metadata: dict = None):
|
||||
"""保存備份到磁盤"""
|
||||
backup_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 主要備份文件
|
||||
backup_file = backup_dir / "workflows.json"
|
||||
with open(backup_file, "w", encoding="utf-8") as f:
|
||||
json.dump(workflows, f, indent=2, ensure_ascii=False)
|
||||
|
||||
# SHA256 校驗
|
||||
sha256_hash = compute_sha256(workflows)
|
||||
sha256_file = backup_dir / "workflows.json.sha256"
|
||||
with open(sha256_file, "w") as f:
|
||||
f.write(sha256_hash)
|
||||
|
||||
# Tags 文件
|
||||
if tags:
|
||||
tags_file = backup_dir / "tags.json"
|
||||
with open(tags_file, "w", encoding="utf-8") as f:
|
||||
json.dump(tags, f, indent=2, ensure_ascii=False)
|
||||
|
||||
# 元數據文件
|
||||
meta = dict(metadata) if metadata else {}
|
||||
meta.update(
|
||||
{
|
||||
"timestamp": TIMESTAMP,
|
||||
"workflow_count": len(workflows),
|
||||
"sha256": sha256_hash,
|
||||
}
|
||||
)
|
||||
meta_file = backup_dir / "manifest.json"
|
||||
with open(meta_file, "w", encoding="utf-8") as f:
|
||||
json.dump(meta, f, indent=2, ensure_ascii=False)
|
||||
|
||||
log_success(f"Backup saved: {backup_file} ({len(workflows)} workflows)")
|
||||
return backup_dir
|
||||
|
||||
|
||||
def load_latest_backup() -> tuple:
|
||||
"""載入最新備份"""
|
||||
if not BACKUP_ROOT.exists():
|
||||
return None, None, None
|
||||
|
||||
backup_dirs = sorted([d for d in BACKUP_ROOT.iterdir() if d.is_dir()], reverse=True)
|
||||
if not backup_dirs:
|
||||
return None, None, None
|
||||
|
||||
latest = backup_dirs[0]
|
||||
return load_backup(latest)
|
||||
|
||||
|
||||
def load_backup(backup_dir: Path) -> tuple:
|
||||
"""載入指定備份"""
|
||||
workflows = None
|
||||
tags = None
|
||||
manifest = None
|
||||
|
||||
workflows_file = backup_dir / "workflows.json"
|
||||
tags_file = backup_dir / "tags.json"
|
||||
manifest_file = backup_dir / "manifest.json"
|
||||
|
||||
if workflows_file.exists():
|
||||
with open(workflows_file) as f:
|
||||
workflows = json.load(f)
|
||||
|
||||
if tags_file.exists():
|
||||
with open(tags_file) as f:
|
||||
tags = json.load(f)
|
||||
|
||||
if manifest_file.exists():
|
||||
with open(manifest_file) as f:
|
||||
manifest = json.load(f)
|
||||
|
||||
return workflows, tags, manifest
|
||||
|
||||
|
||||
def list_backups() -> list:
|
||||
"""列出所有可用備份"""
|
||||
if not BACKUP_ROOT.exists():
|
||||
return []
|
||||
|
||||
backups = []
|
||||
for d in sorted(BACKUP_ROOT.iterdir(), reverse=True):
|
||||
if d.is_dir():
|
||||
manifest_file = d / "manifest.json"
|
||||
if manifest_file.exists():
|
||||
with open(manifest_file) as f:
|
||||
meta = json.load(f)
|
||||
backups.append(
|
||||
{
|
||||
"path": d,
|
||||
"name": d.name,
|
||||
"timestamp": meta.get("timestamp"),
|
||||
"workflow_count": meta.get("workflow_count", 0),
|
||||
"sha256": meta.get("sha256"),
|
||||
}
|
||||
)
|
||||
return backups
|
||||
|
||||
|
||||
def detect_changes(current: list, previous: list) -> dict:
|
||||
"""偵測 workflow 變更"""
|
||||
current_map = {wf["id"]: wf for wf in current}
|
||||
previous_map = {wf["id"]: wf for wf in previous} if previous else {}
|
||||
|
||||
changes = {
|
||||
"added": [],
|
||||
"modified": [],
|
||||
"deleted": [],
|
||||
}
|
||||
|
||||
# 新增或修改
|
||||
for wf_id, wf in current_map.items():
|
||||
if wf_id not in previous_map:
|
||||
changes["added"].append(wf["name"])
|
||||
else:
|
||||
current_hash = compute_sha256(wf)
|
||||
previous_hash = compute_sha256(previous_map[wf_id])
|
||||
if current_hash != previous_hash:
|
||||
changes["modified"].append(wf["name"])
|
||||
|
||||
# 刪除
|
||||
for wf_id in previous_map:
|
||||
if wf_id not in current_map:
|
||||
changes["deleted"].append(previous_map[wf_id]["name"])
|
||||
|
||||
return changes
|
||||
|
||||
|
||||
def print_changes(changes: dict):
|
||||
"""打印變更摘要"""
|
||||
print(f"\n{BLUE}{'=' * 50}{NC}")
|
||||
print(f"{BLUE}n8n Workflow 變更偵測{NC}")
|
||||
print(f"{BLUE}{'=' * 50}{NC}\n")
|
||||
|
||||
if changes["added"]:
|
||||
print(f"{GREEN}+ 新增 ({len(changes['added'])}):{NC}")
|
||||
for name in changes["added"]:
|
||||
print(f" - {name}")
|
||||
print()
|
||||
|
||||
if changes["modified"]:
|
||||
print(f"{YELLOW}~ 修改 ({len(changes['modified'])}):{NC}")
|
||||
for name in changes["modified"]:
|
||||
print(f" - {name}")
|
||||
print()
|
||||
|
||||
if changes["deleted"]:
|
||||
print(f"{RED}- 刪除 ({len(changes['deleted'])}):{NC}")
|
||||
for name in changes["deleted"]:
|
||||
print(f" - {name}")
|
||||
print()
|
||||
|
||||
if not any(changes.values()):
|
||||
print(f"{GREEN}✅ 無變更{NC}\n")
|
||||
|
||||
|
||||
def get_changed_workflows(current: list, previous: list) -> list:
|
||||
"""取得變更的 workflows"""
|
||||
if not previous:
|
||||
return current
|
||||
|
||||
current_map = {wf["id"]: wf for wf in current}
|
||||
previous_map = {wf["id"]: wf for wf in previous}
|
||||
|
||||
changed = []
|
||||
for wf in current:
|
||||
wf_id = wf["id"]
|
||||
if wf_id not in previous_map:
|
||||
changed.append(wf)
|
||||
else:
|
||||
current_hash = compute_sha256(wf)
|
||||
previous_hash = compute_sha256(previous_map[wf_id])
|
||||
if current_hash != previous_hash:
|
||||
changed.append(wf)
|
||||
|
||||
return changed
|
||||
|
||||
|
||||
def verify_backup(backup_dir: Path) -> dict:
|
||||
"""驗證備份完整性"""
|
||||
result = {
|
||||
"valid": True,
|
||||
"errors": [],
|
||||
"workflow_count": 0,
|
||||
"sha256_match": False,
|
||||
}
|
||||
|
||||
workflows_file = backup_dir / "workflows.json"
|
||||
sha256_file = backup_dir / "workflows.json.sha256"
|
||||
manifest_file = backup_dir / "manifest.json"
|
||||
|
||||
if not workflows_file.exists():
|
||||
result["valid"] = False
|
||||
result["errors"].append("workflows.json 不存在")
|
||||
return result
|
||||
|
||||
if not sha256_file.exists():
|
||||
result["valid"] = False
|
||||
result["errors"].append("workflows.json.sha256 不存在")
|
||||
return result
|
||||
|
||||
if not manifest_file.exists():
|
||||
result["valid"] = False
|
||||
result["errors"].append("manifest.json 不存在")
|
||||
return result
|
||||
|
||||
with open(workflows_file) as f:
|
||||
workflows = json.load(f)
|
||||
|
||||
with open(sha256_file) as f:
|
||||
stored_hash = f.read().strip()
|
||||
|
||||
current_hash = compute_sha256(workflows)
|
||||
result["sha256_match"] = current_hash == stored_hash
|
||||
result["workflow_count"] = len(workflows)
|
||||
|
||||
if not result["sha256_match"]:
|
||||
result["valid"] = False
|
||||
result["errors"].append(
|
||||
f"SHA256 不匹配: 預期 {stored_hash}, 實際 {current_hash}"
|
||||
)
|
||||
|
||||
with open(manifest_file) as f:
|
||||
manifest = json.load(f)
|
||||
|
||||
expected_count = manifest.get("workflow_count", 0)
|
||||
if expected_count != len(workflows):
|
||||
result["valid"] = False
|
||||
result["errors"].append(
|
||||
f"數量不匹配: manifest={expected_count}, 實際={len(workflows)}"
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def print_backup_stats():
|
||||
"""顯示備份統計"""
|
||||
backups = list_backups()
|
||||
|
||||
if not backups:
|
||||
print(f"\n{BLUE}沒有可用備份{NC}\n")
|
||||
return
|
||||
|
||||
print(f"\n{BLUE}{'=' * 60}{NC}")
|
||||
print(f"{BLUE}n8n Workflow 備份統計{NC}")
|
||||
print(f"{BLUE}{'=' * 60}{NC}\n")
|
||||
|
||||
total_workflows = sum(b["workflow_count"] for b in backups)
|
||||
print(f" 備份數量: {len(backups)}")
|
||||
print(f" 總 workflows: {total_workflows}")
|
||||
print(f" 最新備份: {backups[0]['name'] if backups else 'N/A'}")
|
||||
print()
|
||||
|
||||
print(f"{'時間戳':<25} {'Workflows':<12} {'SHA256 前 16 字元'}")
|
||||
print("-" * 60)
|
||||
for b in backups[:10]:
|
||||
ts = b.get("timestamp", "N/A") or "N/A"
|
||||
count = b.get("workflow_count", 0)
|
||||
sha = (b.get("sha256", "") or "")[:16]
|
||||
print(f"{ts:<25} {count:<12} {sha}")
|
||||
|
||||
|
||||
# ============================================
|
||||
# 主程式
|
||||
# ============================================
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="n8n Workflow 備份 (REST API)")
|
||||
parser.add_argument(
|
||||
"--active-only", action="store_true", help="只備份啟用的 workflows"
|
||||
)
|
||||
parser.add_argument("--diff", action="store_true", help="只顯示變更，不備份")
|
||||
parser.add_argument(
|
||||
"--incremental", action="store_true", help="差異備份（只備份變更的）"
|
||||
)
|
||||
parser.add_argument("--list", action="store_true", help="列出可用備份")
|
||||
parser.add_argument("--verify", action="store_true", help="驗證最新備份")
|
||||
parser.add_argument("--stats", action="store_true", help="顯示備份統計")
|
||||
parser.add_argument(
|
||||
"--failed-only", action="store_true", help="只備份失敗的執行記錄"
|
||||
)
|
||||
parser.add_argument("--dry-run", action="store_true", help="測試模式，不實際備份")
|
||||
args = parser.parse_args()
|
||||
|
||||
log_info("=" * 50)
|
||||
log_info("n8n Workflow 備份 (REST API)")
|
||||
log_info("=" * 50)
|
||||
|
||||
# 顯示統計
|
||||
if args.stats:
|
||||
print_backup_stats()
|
||||
return
|
||||
|
||||
# 驗證備份
|
||||
if args.verify:
|
||||
backups = list_backups()
|
||||
if not backups:
|
||||
log_error("沒有可用備份")
|
||||
sys.exit(1)
|
||||
|
||||
latest = backups[0]["path"]
|
||||
log_info(f"驗證備份: {latest.name}")
|
||||
result = verify_backup(latest)
|
||||
|
||||
if result["valid"]:
|
||||
log_success(f"備份有效: {result['workflow_count']} workflows, SHA256 匹配")
|
||||
else:
|
||||
log_error(f"備份無效: {result['errors']}")
|
||||
sys.exit(1)
|
||||
return
|
||||
|
||||
# 列出可用備份
|
||||
if args.list:
|
||||
print(f"\n{BLUE}可用備份:{NC}\n")
|
||||
backups = list_backups()
|
||||
if backups:
|
||||
for b in backups[:10]:
|
||||
ts = b.get("timestamp", "") or ""
|
||||
count = b.get("workflow_count", 0)
|
||||
print(f" {b['name']} - {count} workflows ({ts})")
|
||||
else:
|
||||
print(" 無可用備份")
|
||||
print()
|
||||
return
|
||||
|
||||
# 創建 API 客戶端
|
||||
if not N8N_API_KEY:
|
||||
log_error("N8N_API_KEY 環境變數未設定")
|
||||
sys.exit(1)
|
||||
|
||||
client = N8nAPIClient(N8N_BASE_URL, N8N_API_KEY)
|
||||
|
||||
# 健康檢查
|
||||
try:
|
||||
if client.health_check():
|
||||
log_success("n8n API 連線正常")
|
||||
else:
|
||||
log_error("n8n API 連線失敗")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
log_error(f"連線檢查失敗: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# 獲取 workflows
|
||||
log_info("獲取 workflows...")
|
||||
try:
|
||||
all_workflows = []
|
||||
cursor = None
|
||||
while True:
|
||||
result = client.list_workflows(limit=100, cursor=cursor)
|
||||
all_workflows.extend(result.get("data", []))
|
||||
cursor = result.get("nextCursor")
|
||||
if not cursor:
|
||||
break
|
||||
|
||||
# 過濾
|
||||
if args.active_only:
|
||||
all_workflows = [wf for wf in all_workflows if wf.get("active")]
|
||||
|
||||
log_info(f"找到 {len(all_workflows)} workflows")
|
||||
except Exception as e:
|
||||
log_error(f"獲取 workflows 失敗: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# 獲取 tags
|
||||
log_info("獲取 tags...")
|
||||
try:
|
||||
tags_result = client.list_tags()
|
||||
tags = tags_result.get("data", [])
|
||||
log_info(f"找到 {len(tags)} tags")
|
||||
except Exception as e:
|
||||
log_warn(f"獲取 tags 失敗: {e}")
|
||||
tags = []
|
||||
|
||||
# 變更偵測
|
||||
log_info("檢查變更...")
|
||||
previous_wf, _, _ = load_latest_backup()
|
||||
changes = detect_changes(all_workflows, previous_wf)
|
||||
|
||||
if args.diff:
|
||||
print_changes(changes)
|
||||
return
|
||||
|
||||
# 顯示變更摘要
|
||||
total_changes = sum(len(v) for v in changes.values())
|
||||
if total_changes > 0:
|
||||
print_changes(changes)
|
||||
log_warn(f"發現 {total_changes} 個變更")
|
||||
else:
|
||||
log_info("✅ 無變更")
|
||||
|
||||
# 測試模式
|
||||
if args.dry_run:
|
||||
log_info("測試模式結束")
|
||||
return
|
||||
|
||||
# 差異備份
|
||||
if args.incremental:
|
||||
if not previous_wf:
|
||||
log_warn("沒有舊備份，執行完整備份")
|
||||
backup_workflows = all_workflows
|
||||
else:
|
||||
backup_workflows = get_changed_workflows(all_workflows, previous_wf)
|
||||
if not backup_workflows:
|
||||
log_info("✅ 沒有變更，跳過備份")
|
||||
return
|
||||
log_info(f"差異備份: {len(backup_workflows)} workflows")
|
||||
else:
|
||||
backup_workflows = all_workflows
|
||||
|
||||
# 執行備份
|
||||
backup_dir = BACKUP_ROOT / TIMESTAMP
|
||||
metadata = {
|
||||
"active_only": args.active_only,
|
||||
"incremental": args.incremental,
|
||||
"changes": changes if total_changes > 0 else None,
|
||||
"backup_count": len(backup_workflows),
|
||||
"total_count": len(all_workflows),
|
||||
}
|
||||
|
||||
save_backup(backup_workflows, tags, backup_dir, metadata)
|
||||
|
||||
# 執行記錄備份
|
||||
if args.failed_only:
|
||||
log_info("備份失敗的執行記錄...")
|
||||
try:
|
||||
executions_data = []
|
||||
for wf in backup_workflows[:10]:
|
||||
try:
|
||||
execs_result = client.list_executions(
|
||||
wf["id"], limit=20, status="failed"
|
||||
)
|
||||
if execs_result.get("data"):
|
||||
executions_data.append(
|
||||
{
|
||||
"workflow_id": wf["id"],
|
||||
"workflow_name": wf["name"],
|
||||
"failed_executions": len(execs_result.get("data", [])),
|
||||
"executions": execs_result.get("data", []),
|
||||
}
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if executions_data:
|
||||
exec_dir = backup_dir / "executions"
|
||||
exec_dir.mkdir(exist_ok=True)
|
||||
with open(exec_dir / "failed_executions.json", "w") as f:
|
||||
json.dump(executions_data, f, indent=2)
|
||||
log_success(f"失敗執行記錄已保存: {len(executions_data)} workflows")
|
||||
else:
|
||||
log_info("沒有失敗的執行記錄")
|
||||
except Exception as e:
|
||||
log_warn(f"執行記錄備份失敗: {e}")
|
||||
|
||||
# 清理舊備份（保留 30 天）
|
||||
log_info("清理舊備份...")
|
||||
try:
|
||||
import shutil
|
||||
|
||||
cutoff = datetime.now().timestamp() - (30 * 24 * 60 * 60)
|
||||
for d in BACKUP_ROOT.iterdir():
|
||||
if d.is_dir() and d.stat().st_mtime < cutoff:
|
||||
shutil.rmtree(d)
|
||||
log_info(f"已刪除舊備份: {d.name}")
|
||||
except Exception as e:
|
||||
log_warn(f"清理失敗: {e}")
|
||||
|
||||
log_success("備份完成!")
|
||||
print(f"\n備份位置: {backup_dir}")
|
||||
print(f"備份數量: {len(backup_workflows)} / {len(all_workflows)} workflows")
|
||||
print(f"SHA256: {compute_sha256(backup_workflows)}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
481
v1.1/monitor_v1.11/workflow/backup_n8n_mcp_v1.11.py
Normal file
481
v1.1/monitor_v1.11/workflow/backup_n8n_mcp_v1.11.py
Normal file
@@ -0,0 +1,481 @@
|
||||
#!/usr/bin/env python3.11
|
||||
"""
|
||||
n8n Workflow 備份腳本 - 使用 n8n MCP API
|
||||
路徑: /Users/accusys/momentry_core_0.1/monitor/workflow/backup_n8n_mcp.py
|
||||
|
||||
功能:
|
||||
- 使用 n8n MCP API 導出所有 workflows
|
||||
- 按用戶/Tags 分組備份
|
||||
- 變更偵測
|
||||
- SHA256 校驗
|
||||
|
||||
前置需求:
|
||||
pip3.11 install mcp
|
||||
|
||||
使用方式:
|
||||
python3.11 backup_n8n_mcp.py # 備份所有 workflows
|
||||
python3.11 backup_n8n_mcp.py --tags prod # 只備份有 prod tag 的 workflow
|
||||
python3.11 backup_n8n_mcp.py --diff # 只顯示變更
|
||||
python3.11 backup_n8n_mcp.py --list # 列出可用備份
|
||||
python3.11 backup_n8n_mcp.py --audit # 產生安全審計報告
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import hashlib
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
# 嘗試載入 MCP SDK
|
||||
try:
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
|
||||
MCP_AVAILABLE = True
|
||||
except ImportError:
|
||||
MCP_AVAILABLE = False
|
||||
print("Warning: MCP SDK not available. Install with: pip install mcp")
|
||||
|
||||
# ============================================
|
||||
# 配置
|
||||
# ============================================
|
||||
BACKUP_ROOT = Path("/Users/accusys/momentry/backup/n8n_workflows/mcp")
|
||||
LOG_DIR = Path("/Users/accusys/momentry/log/monitor")
|
||||
TIMESTAMP = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
|
||||
# 顏色
|
||||
RED = "\033[0;31m"
|
||||
GREEN = "\033[0;32m"
|
||||
YELLOW = "\033[1;33m"
|
||||
BLUE = "\033[0;34m"
|
||||
NC = "\033[0m"
|
||||
|
||||
# ============================================
|
||||
# 日誌
|
||||
# ============================================
|
||||
LOG_FILE = LOG_DIR / "workflow_backup_mcp.log"
|
||||
|
||||
|
||||
def log(msg: str, color: str = ""):
|
||||
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
full_msg = f"[{timestamp}] {msg}"
|
||||
if color:
|
||||
print(f"{color}{full_msg}{NC}")
|
||||
else:
|
||||
print(full_msg)
|
||||
LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(LOG_FILE, "a") as f:
|
||||
f.write(full_msg + "\n")
|
||||
|
||||
|
||||
def log_info(msg: str):
|
||||
log(msg)
|
||||
|
||||
|
||||
def log_success(msg: str):
|
||||
log(f"✅ {msg}", GREEN)
|
||||
|
||||
|
||||
def log_error(msg: str):
|
||||
log(f"❌ {msg}", RED)
|
||||
|
||||
|
||||
def log_warn(msg: str):
|
||||
log(f"⚠️ {msg}", YELLOW)
|
||||
|
||||
|
||||
# ============================================
|
||||
# MCP 客戶端
|
||||
# ============================================
|
||||
class N8nMCPClient:
|
||||
def __init__(self):
|
||||
self.session = None
|
||||
self.tools = {}
|
||||
|
||||
async def connect(self):
|
||||
if not MCP_AVAILABLE:
|
||||
raise RuntimeError("MCP SDK not available")
|
||||
|
||||
server_params = StdioServerParameters(
|
||||
command="/opt/homebrew/bin/mcp-n8n",
|
||||
env={
|
||||
"N8N_BASE_URL": "https://n8n.momentry.ddns.net",
|
||||
"N8N_API_KEY": os.environ.get("N8N_API_KEY", ""),
|
||||
},
|
||||
)
|
||||
|
||||
async with stdio_client(server_params) as (read, write):
|
||||
self.session = ClientSession(read, write)
|
||||
await self.session.initialize()
|
||||
|
||||
# 獲取可用工具列表
|
||||
tools_result = await self.session.list_tools()
|
||||
for tool in tools_result.tools:
|
||||
self.tools[tool.name] = tool
|
||||
|
||||
log_info(f"Connected to n8n MCP, available tools: {len(self.tools)}")
|
||||
|
||||
async def call_tool(self, tool_name: str, arguments: dict = None):
|
||||
if not self.session:
|
||||
raise RuntimeError("Not connected")
|
||||
|
||||
result = await self.session.call_tool(tool_name, arguments or {})
|
||||
return result.content
|
||||
|
||||
async def list_workflows(
|
||||
self, tags: list = None, active: bool = None, limit: int = 100
|
||||
):
|
||||
"""列出 workflows"""
|
||||
args = {"limit": limit}
|
||||
if tags:
|
||||
args["tags"] = tags
|
||||
if active is not None:
|
||||
args["active"] = active
|
||||
|
||||
content = await self.call_tool("n8n_list_workflows", args)
|
||||
data = json.loads(content[0].text)
|
||||
return data.get("data", [])
|
||||
|
||||
async def get_workflow(self, workflow_id: str):
|
||||
"""取得 workflow 詳情"""
|
||||
content = await self.call_tool("n8n_get_workflow", {"workflowId": workflow_id})
|
||||
data = json.loads(content[0].text)
|
||||
return data
|
||||
|
||||
async def list_tags(self):
|
||||
"""列出所有 tags"""
|
||||
content = await self.call_tool("n8n_list_tags")
|
||||
data = json.loads(content[0].text)
|
||||
return data.get("data", [])
|
||||
|
||||
async def get_workflow_tags(self, workflow_id: str):
|
||||
"""取得 workflow 的 tags"""
|
||||
content = await self.call_tool(
|
||||
"n8n_get_workflow_tags", {"workflowId": workflow_id}
|
||||
)
|
||||
data = json.loads(content[0].text)
|
||||
return data
|
||||
|
||||
async def list_executions(self, workflow_id: str = None, limit: int = 50):
|
||||
"""列出執行記錄"""
|
||||
args = {"limit": limit}
|
||||
if workflow_id:
|
||||
args["workflowId"] = workflow_id
|
||||
|
||||
content = await self.call_tool("n8n_list_executions", args)
|
||||
data = json.loads(content[0].text)
|
||||
return data.get("data", [])
|
||||
|
||||
async def generate_audit(self):
|
||||
"""產生安全審計報告"""
|
||||
content = await self.call_tool("n8n_generate_audit", {})
|
||||
data = json.loads(content[0].text)
|
||||
return data
|
||||
|
||||
async def health_check(self):
|
||||
"""健康檢查"""
|
||||
content = await self.call_tool("n8n_health_check", {})
|
||||
return content[0].text if content else "Unknown"
|
||||
|
||||
|
||||
# ============================================
|
||||
# 備份功能
|
||||
# ============================================
|
||||
def compute_sha256(data: dict) -> str:
|
||||
"""計算 JSON 的 SHA256"""
|
||||
json_str = json.dumps(data, sort_keys=True, ensure_ascii=False)
|
||||
return hashlib.sha256(json_str.encode()).hexdigest()
|
||||
|
||||
|
||||
def save_backup(workflows: list, tags: list, backup_dir: Path, metadata: dict = None):
|
||||
"""保存備份到磁盤"""
|
||||
backup_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 主要備份文件
|
||||
backup_file = backup_dir / "workflows.json"
|
||||
with open(backup_file, "w", encoding="utf-8") as f:
|
||||
json.dump(workflows, f, indent=2, ensure_ascii=False)
|
||||
|
||||
# SHA256 校驗
|
||||
sha256_hash = compute_sha256(workflows)
|
||||
sha256_file = backup_dir / "workflows.json.sha256"
|
||||
with open(sha256_file, "w") as f:
|
||||
f.write(sha256_hash)
|
||||
|
||||
# Tags 文件
|
||||
if tags:
|
||||
tags_file = backup_dir / "tags.json"
|
||||
with open(tags_file, "w", encoding="utf-8") as f:
|
||||
json.dump(tags, f, indent=2, ensure_ascii=False)
|
||||
|
||||
# 元數據文件
|
||||
meta = metadata or {}
|
||||
meta.update(
|
||||
{
|
||||
"timestamp": TIMESTAMP,
|
||||
"workflow_count": len(workflows),
|
||||
"sha256": sha256_hash,
|
||||
}
|
||||
)
|
||||
meta_file = backup_dir / "manifest.json"
|
||||
with open(meta_file, "w", encoding="utf-8") as f:
|
||||
json.dump(meta, f, indent=2, ensure_ascii=False)
|
||||
|
||||
log_success(f"Backup saved: {backup_file} ({len(workflows)} workflows)")
|
||||
return backup_dir
|
||||
|
||||
|
||||
def load_latest_backup() -> tuple:
|
||||
"""載入最新備份"""
|
||||
if not BACKUP_ROOT.exists():
|
||||
return None, None, None
|
||||
|
||||
# 找到最新的備份目錄
|
||||
backup_dirs = sorted([d for d in BACKUP_ROOT.iterdir() if d.is_dir()], reverse=True)
|
||||
if not backup_dirs:
|
||||
return None, None, None
|
||||
|
||||
latest = backup_dirs[0]
|
||||
workflows_file = latest / "workflows.json"
|
||||
tags_file = latest / "tags.json"
|
||||
manifest_file = latest / "manifest.json"
|
||||
|
||||
workflows = None
|
||||
tags = None
|
||||
manifest = None
|
||||
|
||||
if workflows_file.exists():
|
||||
with open(workflows_file) as f:
|
||||
workflows = json.load(f)
|
||||
|
||||
if tags_file.exists():
|
||||
with open(tags_file) as f:
|
||||
tags = json.load(f)
|
||||
|
||||
if manifest_file.exists():
|
||||
with open(manifest_file) as f:
|
||||
manifest = json.load(f)
|
||||
|
||||
return workflows, tags, manifest
|
||||
|
||||
|
||||
def detect_changes(current: list, previous: list) -> dict:
|
||||
"""偵測 workflow 變更"""
|
||||
current_map = {wf["id"]: wf for wf in current}
|
||||
previous_map = {wf["id"]: wf for wf in previous} if previous else {}
|
||||
|
||||
changes = {
|
||||
"added": [],
|
||||
"modified": [],
|
||||
"deleted": [],
|
||||
}
|
||||
|
||||
# 新增或修改
|
||||
for wf_id, wf in current_map.items():
|
||||
if wf_id not in previous_map:
|
||||
changes["added"].append(wf["name"])
|
||||
else:
|
||||
current_hash = compute_sha256(wf)
|
||||
previous_hash = compute_sha256(previous_map[wf_id])
|
||||
if current_hash != previous_hash:
|
||||
changes["modified"].append(wf["name"])
|
||||
|
||||
# 刪除
|
||||
for wf_id in previous_map:
|
||||
if wf_id not in current_map:
|
||||
changes["deleted"].append(previous_map[wf_id]["name"])
|
||||
|
||||
return changes
|
||||
|
||||
|
||||
def print_changes(changes: dict):
|
||||
"""打印變更摘要"""
|
||||
print(f"\n{BLUE}{'=' * 50}{NC}")
|
||||
print(f"{BLUE}n8n Workflow 變更偵測{NC}")
|
||||
print(f"{BLUE}{'=' * 50}{NC}\n")
|
||||
|
||||
if changes["added"]:
|
||||
print(f"{GREEN}+ 新增 ({len(changes['added'])}):{NC}")
|
||||
for name in changes["added"]:
|
||||
print(f" - {name}")
|
||||
print()
|
||||
|
||||
if changes["modified"]:
|
||||
print(f"{YELLOW}~ 修改 ({len(changes['modified'])}):{NC}")
|
||||
for name in changes["modified"]:
|
||||
print(f" - {name}")
|
||||
print()
|
||||
|
||||
if changes["deleted"]:
|
||||
print(f"{RED}- 刪除 ({len(changes['deleted'])}):{NC}")
|
||||
for name in changes["deleted"]:
|
||||
print(f" - {name}")
|
||||
print()
|
||||
|
||||
if not any(changes.values()):
|
||||
print(f"{GREEN}✅ 無變更{NC}\n")
|
||||
|
||||
|
||||
# ============================================
|
||||
# 主程式
|
||||
# ============================================
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="n8n Workflow 備份 (MCP API)")
|
||||
parser.add_argument("--tags", nargs="*", help="只備份有特定 tags 的 workflows")
|
||||
parser.add_argument(
|
||||
"--active-only", action="store_true", help="只備份啟用的 workflows"
|
||||
)
|
||||
parser.add_argument("--diff", action="store_true", help="只顯示變更，不備份")
|
||||
parser.add_argument("--list", action="store_true", help="列出可用備份")
|
||||
parser.add_argument("--audit", action="store_true", help="產生安全審計報告")
|
||||
parser.add_argument("--executions", action="store_true", help="同時備份執行記錄")
|
||||
parser.add_argument("--dry-run", action="store_true", help="測試模式，不實際備份")
|
||||
args = parser.parse_args()
|
||||
|
||||
log_info("=" * 50)
|
||||
log_info("n8n Workflow 備份 (MCP API)")
|
||||
log_info("=" * 50)
|
||||
|
||||
# 列出可用備份
|
||||
if args.list:
|
||||
print(f"\n{BLUE}可用備份:{NC}\n")
|
||||
if BACKUP_ROOT.exists():
|
||||
for d in sorted(BACKUP_ROOT.iterdir(), reverse=True)[:10]:
|
||||
manifest_file = d / "manifest.json"
|
||||
if manifest_file.exists():
|
||||
with open(manifest_file) as f:
|
||||
meta = json.load(f)
|
||||
print(f" {d.name} - {meta.get('workflow_count', 0)} workflows")
|
||||
else:
|
||||
print(" 無可用備份")
|
||||
print()
|
||||
return
|
||||
|
||||
# 連接 MCP
|
||||
client = N8nMCPClient()
|
||||
try:
|
||||
await client.connect()
|
||||
except Exception as e:
|
||||
log_error(f"連接失敗: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# 健康檢查
|
||||
try:
|
||||
health = await client.health_check()
|
||||
log_info(f"健康狀態: {health}")
|
||||
except Exception as e:
|
||||
log_warn(f"健康檢查失敗: {e}")
|
||||
|
||||
# 安全審計
|
||||
if args.audit:
|
||||
log_info("產生安全審計報告...")
|
||||
try:
|
||||
audit = await client.generate_audit()
|
||||
audit_dir = BACKUP_ROOT / TIMESTAMP / "audit"
|
||||
audit_dir.mkdir(parents=True, exist_ok=True)
|
||||
with open(audit_dir / "audit.json", "w") as f:
|
||||
json.dump(audit, f, indent=2)
|
||||
log_success("審計報告已保存")
|
||||
except Exception as e:
|
||||
log_error(f"審計失敗: {e}")
|
||||
return
|
||||
|
||||
# 獲取 workflows
|
||||
log_info("獲取 workflows...")
|
||||
try:
|
||||
workflows = await client.list_workflows(
|
||||
tags=args.tags, active=True if args.active_only else None
|
||||
)
|
||||
log_info(f"找到 {len(workflows)} workflows")
|
||||
except Exception as e:
|
||||
log_error(f"獲取 workflows 失敗: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# 獲取 tags
|
||||
log_info("獲取 tags...")
|
||||
try:
|
||||
tags = await client.list_tags()
|
||||
log_info(f"找到 {len(tags)} tags")
|
||||
except Exception as e:
|
||||
log_warn(f"獲取 tags 失敗: {e}")
|
||||
tags = []
|
||||
|
||||
# 變更偵測
|
||||
log_info("檢查變更...")
|
||||
previous_wf, _, _ = load_latest_backup()
|
||||
changes = detect_changes(workflows, previous_wf)
|
||||
|
||||
if args.diff:
|
||||
print_changes(changes)
|
||||
return
|
||||
|
||||
# 顯示變更摘要
|
||||
total_changes = sum(len(v) for v in changes.values())
|
||||
if total_changes > 0:
|
||||
print_changes(changes)
|
||||
log_warn(f"發現 {total_changes} 個變更")
|
||||
else:
|
||||
log_info("✅ 無變更")
|
||||
|
||||
# 測試模式
|
||||
if args.dry_run:
|
||||
log_info("測試模式結束")
|
||||
return
|
||||
|
||||
# 執行備份
|
||||
backup_dir = BACKUP_ROOT / TIMESTAMP
|
||||
metadata = {
|
||||
"filter_tags": args.tags,
|
||||
"active_only": args.active_only,
|
||||
"changes": changes if total_changes > 0 else None,
|
||||
}
|
||||
|
||||
save_backup(workflows, tags, backup_dir, metadata)
|
||||
|
||||
# 執行記錄備份
|
||||
if args.executions:
|
||||
log_info("備份執行記錄...")
|
||||
try:
|
||||
executions_data = []
|
||||
for wf in workflows[:10]: # 只備份前 10 個 workflow
|
||||
execs = await client.list_executions(wf["id"], limit=20)
|
||||
executions_data.append(
|
||||
{
|
||||
"workflow_id": wf["id"],
|
||||
"workflow_name": wf["name"],
|
||||
"executions": execs[:10], # 每個只取 10 筆
|
||||
}
|
||||
)
|
||||
|
||||
exec_dir = backup_dir / "executions"
|
||||
exec_dir.mkdir(exist_ok=True)
|
||||
with open(exec_dir / "executions.json", "w") as f:
|
||||
json.dump(executions_data, f, indent=2)
|
||||
log_success(f"執行記錄已保存: {len(executions_data)} workflows")
|
||||
except Exception as e:
|
||||
log_warn(f"執行記錄備份失敗: {e}")
|
||||
|
||||
# 清理舊備份（保留 30 天）
|
||||
log_info("清理舊備份...")
|
||||
try:
|
||||
cutoff = datetime.now().timestamp() - (30 * 24 * 60 * 60)
|
||||
for d in BACKUP_ROOT.iterdir():
|
||||
if d.is_dir() and d.stat().st_mtime < cutoff:
|
||||
import shutil
|
||||
|
||||
shutil.rmtree(d)
|
||||
log_info(f"已刪除舊備份: {d.name}")
|
||||
except Exception as e:
|
||||
log_warn(f"清理失敗: {e}")
|
||||
|
||||
log_success("備份完成!")
|
||||
print(f"\n備份位置: {backup_dir}")
|
||||
print(f"SHA256: {compute_sha256(workflows)}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import asyncio
|
||||
|
||||
asyncio.run(main())
|
||||
376
v1.1/monitor_v1.11/workflow/backup_n8n_workflows_v1.11.sh
Executable file
376
v1.1/monitor_v1.11/workflow/backup_n8n_workflows_v1.11.sh
Executable file
@@ -0,0 +1,376 @@
|
||||
#!/bin/bash
|
||||
# Momentry n8n Workflow 備份腳本
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/workflow/backup_n8n_workflows.sh
|
||||
#
|
||||
# 功能:
|
||||
# - 導出所有 workflows 為 n8n 原生 JSON 格式
|
||||
# - 按用戶分組備份
|
||||
# - 版本化存儲
|
||||
#
|
||||
# 使用方式:
|
||||
# ./backup_n8n_workflows.sh # 備份所有 workflows
|
||||
# ./backup_n8n_workflows.sh daily # 每日增量備份
|
||||
# ./backup_n8n_workflows.sh full # 完整備份
|
||||
# ./backup_n8n_workflows.sh restore # 列出可恢復版本
|
||||
|
||||
set -e
|
||||
|
||||
# ============================================
|
||||
# 配置
|
||||
# ============================================
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
BACKUP_ROOT="/Users/accusys/momentry/backup/n8n_workflows"
|
||||
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
# 載入密碼配置
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
if [ -f "$MONITOR_DIR/common/load_credentials.sh" ]; then
|
||||
source "$MONITOR_DIR/common/load_credentials.sh"
|
||||
fi
|
||||
|
||||
# n8n 數據庫連接 (使用 n8n 用戶)
|
||||
N8N_PG_USER="${N8N_PG_USER:-n8n}"
|
||||
N8N_PG_PASSWORD="${N8N_PG_PASSWORD:-accusys}"
|
||||
N8N_PG_HOST="${N8N_PG_HOST:-localhost}"
|
||||
N8N_PG_DB="${N8N_PG_DB:-n8n}"
|
||||
|
||||
# 確保目錄存在
|
||||
mkdir -p "$LOG_DIR"
|
||||
mkdir -p "$BACKUP_ROOT"
|
||||
|
||||
# ============================================
|
||||
# 顏色
|
||||
# ============================================
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
# ============================================
|
||||
# 日誌函數
|
||||
# ============================================
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_DIR/workflow_backup.log"
|
||||
}
|
||||
|
||||
log_success() {
|
||||
echo -e "${GREEN}[$(date '+%Y-%m-%d %H:%M:%S')] ✅ $1${NC}" | tee -a "$LOG_DIR/workflow_backup.log"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[$(date '+%Y-%m-%d %H:%M:%S')] ❌ $1${NC}" | tee -a "$LOG_DIR/workflow_backup.log"
|
||||
}
|
||||
|
||||
log_warn() {
|
||||
echo -e "${YELLOW}[$(date '+%Y-%m-%d %H:%M:%S')] ⚠️ $1${NC}" | tee -a "$LOG_DIR/workflow_backup.log"
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 導出單個 Workflow (n8n 原生格式)
|
||||
# ============================================
|
||||
export_workflow() {
|
||||
local wf_id=$1
|
||||
local output_file=$2
|
||||
|
||||
# 查詢 workflow 數據 (包含所有 n8n 需要的欄位)
|
||||
PGPASSWORD="$N8N_PG_PASSWORD" psql -U "$N8N_PG_USER" -h "$N8N_PG_HOST" -d "$N8N_PG_DB" -t -A -c "
|
||||
SELECT json_build_object(
|
||||
'id', id,
|
||||
'name', name,
|
||||
'active', active,
|
||||
'nodes', nodes,
|
||||
'connections', connections,
|
||||
'settings', COALESCE(settings, '{}'),
|
||||
'staticData', COALESCE(\"staticData\", 'null'),
|
||||
'pinData', COALESCE(\"pinData\", 'null'),
|
||||
'versionId', \"versionId\",
|
||||
'triggerCount', \"triggerCount\",
|
||||
'createdAt', \"createdAt\",
|
||||
'updatedAt', \"updatedAt\",
|
||||
'isArchived', \"isArchived\",
|
||||
'versionCounter', \"versionCounter\",
|
||||
'description', COALESCE(description, ''),
|
||||
'meta', COALESCE(meta, '{}')
|
||||
)
|
||||
FROM workflow_entity
|
||||
WHERE id = '$wf_id';
|
||||
" > "$output_file" 2>&1
|
||||
|
||||
# 移除錯誤輸出
|
||||
if grep -q "ERROR:" "$output_file" 2>/dev/null; then
|
||||
sed -i '/ERROR:/d' "$output_file"
|
||||
fi
|
||||
|
||||
if [ -s "$output_file" ]; then
|
||||
return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 按用戶導出 Workflows
|
||||
# ============================================
|
||||
export_workflows_by_user() {
|
||||
local user_id=$1
|
||||
local user_email=$2
|
||||
local output_dir=$3
|
||||
|
||||
log "導出用戶 $user_email 的 workflows..."
|
||||
|
||||
mkdir -p "$output_dir"
|
||||
|
||||
local count=0
|
||||
|
||||
# 獲取該用戶的 workflows
|
||||
workflows=$(PGPASSWORD="$N8N_PG_PASSWORD" psql -U "$N8N_PG_USER" -h "$N8N_PG_HOST" -d "$N8N_PG_DB" -t -A -c "
|
||||
SELECT DISTINCT w.id, w.name
|
||||
FROM workflow_entity w
|
||||
LEFT JOIN shared_workflow sw ON w.id = sw.\"workflowId\"
|
||||
LEFT JOIN project p ON sw.\"projectId\" = p.id
|
||||
WHERE p.\"creatorId\" = '$user_id'
|
||||
OR w.id IN (
|
||||
SELECT \"workflowId\" FROM shared_workflow
|
||||
WHERE \"projectId\" IN (
|
||||
SELECT id FROM project WHERE \"creatorId\" = '$user_id'
|
||||
)
|
||||
)
|
||||
ORDER BY w.name;
|
||||
" 2>/dev/null)
|
||||
|
||||
while IFS='|' read -r wf_id wf_name; do
|
||||
if [ -n "$wf_id" ]; then
|
||||
# 清理文件名
|
||||
local safe_name=$(echo "$wf_name" | sed 's/[^a-zA-Z0-9_-]/_/g')
|
||||
local output_file="$output_dir/${safe_name}.json"
|
||||
|
||||
if export_workflow "$wf_id" "$output_file"; then
|
||||
log " ✅ $wf_name"
|
||||
((count++))
|
||||
else
|
||||
log_error " ❌ $wf_name (ID: $wf_id)"
|
||||
fi
|
||||
fi
|
||||
done <<< "$workflows"
|
||||
|
||||
echo "$count"
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 導出所有 Workflows (n8n 批量格式)
|
||||
# ============================================
|
||||
export_all_workflows() {
|
||||
local output_file="$1"
|
||||
|
||||
log "導出所有 workflows..."
|
||||
|
||||
# 導出 workflows
|
||||
PGPASSWORD="$N8N_PG_PASSWORD" psql -U "$N8N_PG_USER" -h "$N8N_PG_HOST" -d "$N8N_PG_DB" -t -A -c "
|
||||
SELECT json_agg(
|
||||
json_build_object(
|
||||
'id', id,
|
||||
'name', name,
|
||||
'active', active,
|
||||
'nodes', nodes,
|
||||
'connections', connections,
|
||||
'settings', COALESCE(settings, '{}'),
|
||||
'staticData', COALESCE(\"staticData\", 'null'),
|
||||
'pinData', COALESCE(\"pinData\", 'null'),
|
||||
'versionId', \"versionId\",
|
||||
'triggerCount', \"triggerCount\",
|
||||
'createdAt', \"createdAt\",
|
||||
'updatedAt', \"updatedAt\",
|
||||
'isArchived', \"isArchived\",
|
||||
'versionCounter', \"versionCounter\",
|
||||
'description', COALESCE(description, ''),
|
||||
'meta', COALESCE(meta, '{}')
|
||||
)
|
||||
)
|
||||
FROM workflow_entity;
|
||||
" > "$output_file" 2>&1
|
||||
|
||||
# 移除 psql 錯誤輸出
|
||||
if grep -q "ERROR:" "$output_file" 2>/dev/null; then
|
||||
sed -i '/ERROR:/d' "$output_file"
|
||||
fi
|
||||
|
||||
if [ $? -eq 0 ] && [ -s "$output_file" ]; then
|
||||
log_success "導出完成: $output_file"
|
||||
return 0
|
||||
else
|
||||
log_error "導出失敗"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 完整備份 (所有 workflows + 用戶)
|
||||
# ============================================
|
||||
backup_full() {
|
||||
log "=========================================="
|
||||
log "開始完整 n8n Workflow 備份"
|
||||
log "=========================================="
|
||||
|
||||
local backup_dir="$BACKUP_ROOT/full/$TIMESTAMP"
|
||||
mkdir -p "$backup_dir"
|
||||
|
||||
# 1. 導出所有 workflows 為單一檔案
|
||||
export_all_workflows "$backup_dir/all_workflows.json"
|
||||
|
||||
# 2. 導出執行歷史統計
|
||||
log ""
|
||||
log "導出執行統計..."
|
||||
PGPASSWORD="$N8N_PG_PASSWORD" psql -U "$N8N_PG_USER" -h "$N8N_PG_HOST" -d "$N8N_PG_DB" -t -A -c "
|
||||
SELECT json_agg(
|
||||
json_build_object(
|
||||
'workflow_id', w.id,
|
||||
'workflow_name', w.name,
|
||||
'total_executions', e.total_executions,
|
||||
'successful_executions', e.successful_executions,
|
||||
'failed_executions', e.failed_executions,
|
||||
'last_execution', e.last_execution
|
||||
)
|
||||
)
|
||||
FROM (
|
||||
SELECT
|
||||
\"workflowId\" as wid,
|
||||
COUNT(*) as total_executions,
|
||||
COUNT(CASE WHEN status = 'success' THEN 1 END) as successful_executions,
|
||||
COUNT(CASE WHEN status = 'error' THEN 1 END) as failed_executions,
|
||||
MAX(\"startedAt\") as last_execution
|
||||
FROM execution_entity
|
||||
GROUP BY \"workflowId\"
|
||||
) e
|
||||
JOIN workflow_entity w ON e.wid = w.id;
|
||||
" > "$backup_dir/execution_stats.json" 2>&1
|
||||
|
||||
# 移除錯誤輸出
|
||||
if grep -q "ERROR:" "$backup_dir/execution_stats.json" 2>/dev/null; then
|
||||
sed -i '/ERROR:/d' "$backup_dir/execution_stats.json"
|
||||
fi
|
||||
|
||||
# 3. 生成備份清單
|
||||
local workflow_count=$(cat "$backup_dir/all_workflows.json" 2>/dev/null | jq '. | length' 2>/dev/null || echo "0")
|
||||
cat > "$backup_dir/manifest.json" << EOF
|
||||
{
|
||||
"backup_time": "$(date -Iseconds)",
|
||||
"timestamp": "$TIMESTAMP",
|
||||
"workflow_count": $workflow_count,
|
||||
"version": "1.0"
|
||||
}
|
||||
EOF
|
||||
|
||||
# 5. 計算 SHA256
|
||||
sha256sum "$backup_dir/all_workflows.json" > "$backup_dir/all_workflows.json.sha256"
|
||||
|
||||
log ""
|
||||
log_success "備份完成!"
|
||||
log "備份目錄: $backup_dir"
|
||||
log "Workflow 總數: $workflow_count"
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 增量備份 (每日)
|
||||
# ============================================
|
||||
backup_daily() {
|
||||
local backup_dir="$BACKUP_ROOT/daily/$(date +%Y%m%d)"
|
||||
|
||||
log "=========================================="
|
||||
log "開始每日 n8n Workflow 增量備份"
|
||||
log "=========================================="
|
||||
|
||||
mkdir -p "$backup_dir"
|
||||
|
||||
# 導出所有 workflows
|
||||
export_all_workflows "$backup_dir/workflows.json"
|
||||
|
||||
# SHA256 校驗
|
||||
sha256sum "$backup_dir/workflows.json" > "$backup_dir/workflows.json.sha256"
|
||||
|
||||
log_success "每日備份完成: $backup_dir/workflows.json"
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 列出可恢復版本
|
||||
# ============================================
|
||||
list_backups() {
|
||||
log ""
|
||||
log "=========================================="
|
||||
log "可用備份版本"
|
||||
log "=========================================="
|
||||
log ""
|
||||
|
||||
echo "📁 完整備份:"
|
||||
for dir in "$BACKUP_ROOT"/full/*/; do
|
||||
if [ -d "$dir" ]; then
|
||||
local name=$(basename "$dir")
|
||||
local count=$(cat "$dir/all_workflows.json" 2>/dev/null | jq '. | length' 2>/dev/null || echo "?")
|
||||
echo " $name ($count workflows)"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "📁 每日備份:"
|
||||
for dir in "$BACKUP_ROOT"/daily/*/; do
|
||||
if [ -d "$dir" ]; then
|
||||
local name=$(basename "$dir")
|
||||
local size=$(du -sh "$dir" | cut -f1)
|
||||
echo " $name ($size)"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 恢復 Workflow
|
||||
# ============================================
|
||||
restore_workflow() {
|
||||
local backup_file=$1
|
||||
local wf_name=$2
|
||||
|
||||
if [ ! -f "$backup_file" ]; then
|
||||
log_error "備份文件不存在: $backup_file"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log "從 $backup_file 恢復 workflow..."
|
||||
|
||||
# 這需要 n8n API 或直接 SQL 插入
|
||||
# 這裡只是示範結構
|
||||
log_warn "請使用 n8n UI 或 API 進行恢復"
|
||||
log "備份文件格式:"
|
||||
jq 'keys' "$backup_file" 2>/dev/null
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 主程序
|
||||
# ============================================
|
||||
main() {
|
||||
case "${1:-full}" in
|
||||
full)
|
||||
backup_full
|
||||
;;
|
||||
daily)
|
||||
backup_daily
|
||||
;;
|
||||
list)
|
||||
list_backups
|
||||
;;
|
||||
restore)
|
||||
restore_workflow "$2" "$3"
|
||||
;;
|
||||
*)
|
||||
echo "用法: $0 [full|daily|list|restore]"
|
||||
echo ""
|
||||
echo "命令:"
|
||||
echo " full - 完整備份 (所有 workflows + 用戶分組)"
|
||||
echo " daily - 每日增量備份"
|
||||
echo " list - 列出可用備份"
|
||||
echo " restore - 恢復 workflow"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
265
v1.1/monitor_v1.11/workflow/n8n_workflow_monitor_v1.11.sh
Executable file
265
v1.1/monitor_v1.11/workflow/n8n_workflow_monitor_v1.11.sh
Executable file
@@ -0,0 +1,265 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Momentry n8n Workflow 監控 (Layer 3)
|
||||
# 路徑: /Users/accusys/momentry_core_0.1/monitor/workflow/n8n_workflow_monitor.sh
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MONITOR_DIR="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="/Users/accusys/momentry/log/monitor"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
LOG_FILE="$LOG_DIR/workflow_check.log"
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
|
||||
}
|
||||
|
||||
# n8n API 配置
|
||||
N8N_HOST="http://localhost:5678"
|
||||
N8N_API_KEY="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJlNjdiY2UzOS1iY2RkLTRjMjEtYmMwYy0yODNhYmI3ZjVjMjMiLCJpc3MiOiJuOG4iLCJhdWQiOiJwdWJsaWMtYXBpIiwiaWF0IjoxNzczNjM5ODU4fQ.QOmOju2jLy07GrgXYvylM5AyFINPC06crKEsLLC988I"
|
||||
|
||||
# 從資料庫獲取 workflow
|
||||
fetch_workflows_from_db() {
|
||||
PGPASSWORD=accusys psql -U n8n -h localhost -d n8n -t -A <<'EOF'
|
||||
SELECT json_agg(row_to_json(t)) FROM (
|
||||
SELECT w.id, w.name, w.active, w."createdAt", w."updatedAt",
|
||||
COALESCE(u.email, 'unknown') as owner_email
|
||||
FROM workflow_entity w
|
||||
LEFT JOIN shared_workflow sw ON w.id = sw."workflowId"
|
||||
LEFT JOIN project p ON sw."projectId" = p.id
|
||||
LEFT JOIN "user" u ON p."creatorId" = u.id
|
||||
) t
|
||||
EOF
|
||||
}
|
||||
|
||||
# 記錄 workflow
|
||||
record_workflow() {
|
||||
local wf_id=$1
|
||||
local wf_name=$2
|
||||
local active=$3
|
||||
local last_exec=$4
|
||||
local exec_count=$5
|
||||
local success=$6
|
||||
local failure=$7
|
||||
local avg_duration=$8
|
||||
local has_schedule=$9
|
||||
local has_webhook=${10}
|
||||
local idle_days=${11}
|
||||
local suggestion=${12}
|
||||
|
||||
psql -U accusys -h localhost -d momentry << EOF 2>/dev/null
|
||||
INSERT INTO monitor_workflows
|
||||
(workflow_id, workflow_name, is_active, last_executed_at, execution_count,
|
||||
success_count, failure_count, avg_duration_ms, has_schedule, has_webhook,
|
||||
idle_days, suggestion, checked_at)
|
||||
VALUES
|
||||
('$wf_id', '$wf_name', $active, $last_exec, $exec_count,
|
||||
$success, $failure, $avg_duration, $has_schedule, $has_webhook,
|
||||
$idle_days, '$suggestion', NOW())
|
||||
ON CONFLICT (workflow_id) DO UPDATE SET
|
||||
workflow_name = EXCLUDED.workflow_name,
|
||||
is_active = EXCLUDED.is_active,
|
||||
last_executed_at = EXCLUDED.last_executed_at,
|
||||
execution_count = EXCLUDED.execution_count,
|
||||
success_count = EXCLUDED.success_count,
|
||||
failure_count = EXCLUDED.failure_count,
|
||||
avg_duration_ms = EXCLUDED.avg_duration_ms,
|
||||
has_schedule = EXCLUDED.has_schedule,
|
||||
has_webhook = EXCLUDED.has_webhook,
|
||||
idle_days = EXCLUDED.idle_days,
|
||||
suggestion = EXCLUDED.suggestion,
|
||||
checked_at = NOW();
|
||||
EOF
|
||||
}
|
||||
|
||||
# 獲取 workflow 列表
|
||||
fetch_workflows() {
|
||||
curl -s -H "Accept: application/json" \
|
||||
-H "X-N8N-API-KEY: ${N8N_API_KEY}" \
|
||||
"${N8N_HOST}/rest/workflows" 2>/dev/null || echo "[]"
|
||||
}
|
||||
|
||||
# 獲取 workflow 執行統計
|
||||
fetch_executions() {
|
||||
local wf_id=$1
|
||||
curl -s -H "Accept: application/json" \
|
||||
-H "X-N8N-API-KEY: ${N8N_API_KEY}" \
|
||||
"${N8N_HOST}/rest/executions?workflowId=${wf_id}&limit=50" 2>/dev/null || echo "{\"data\":[]}"
|
||||
}
|
||||
|
||||
# 判斷是否有 schedule
|
||||
has_schedule() {
|
||||
local wf_data=$1
|
||||
echo "$wf_data" | grep -q '"type":"schedule"' && echo "true" || echo "false"
|
||||
}
|
||||
|
||||
# 判斷是否有 webhook
|
||||
has_webhook() {
|
||||
local wf_data=$1
|
||||
echo "$wf_data" | grep -q '"type":"webhook"' && echo "true" || echo "false"
|
||||
}
|
||||
|
||||
# 計算閒置天數
|
||||
calc_idle_days() {
|
||||
local last_exec=$1
|
||||
if [ "$last_exec" = "null" ] || [ -z "$last_exec" ]; then
|
||||
echo "999"
|
||||
else
|
||||
echo "0"
|
||||
fi
|
||||
}
|
||||
|
||||
# 生成建議
|
||||
generate_suggestion() {
|
||||
local has_schedule=$1
|
||||
local has_webhook=$2
|
||||
local idle_days=$3
|
||||
local failure_rate=$4
|
||||
|
||||
if [ "$idle_days" -ge 90 ]; then
|
||||
echo "建議刪除"
|
||||
elif [ "$idle_days" -ge 30 ] && [ "$has_schedule" = "false" ] && [ "$has_webhook" = "false" ]; then
|
||||
echo "建議停用"
|
||||
elif [ "$failure_rate" -gt 20 ]; then
|
||||
echo "建議優化"
|
||||
else
|
||||
echo ""
|
||||
fi
|
||||
}
|
||||
|
||||
# 主程序
|
||||
echo "========================================"
|
||||
echo "Layer 3: n8n Workflow Monitoring"
|
||||
echo "Time: $(date)"
|
||||
echo "========================================"
|
||||
echo ""
|
||||
|
||||
# 檢查 n8n 是否可用 (檢查 PostgreSQL 中的 n8n 資料庫)
|
||||
if ! PGPASSWORD=accusys psql -U n8n -h localhost -d n8n -c "SELECT 1" >/dev/null 2>&1; then
|
||||
echo "n8n 資料庫不可用"
|
||||
log "n8n database unavailable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 獲取 workflow 列表 (從資料庫)
|
||||
workflows=$(fetch_workflows_from_db)
|
||||
total_count=$(echo "$workflows" | jq 'length' 2>/dev/null || echo "0")
|
||||
active_count=$(echo "$workflows" | jq '[.[] | select(.active == true)] | length' 2>/dev/null || echo "0")
|
||||
|
||||
echo "總 Workflow: $total_count"
|
||||
echo "啟用中: $active_count"
|
||||
echo ""
|
||||
|
||||
# 閒置閾值
|
||||
IDLE_THRESHOLD=30
|
||||
|
||||
echo "Workflow 詳細:"
|
||||
echo "----------------------------------------"
|
||||
|
||||
total_idle=0
|
||||
|
||||
for wf in $(echo "$workflows" | jq -r '.[] | @base64' 2>/dev/null); do
|
||||
wf_decoded=$(echo "$wf" | base64 -d)
|
||||
|
||||
wf_id=$(echo "$wf_decoded" | jq -r '.id' 2>/dev/null)
|
||||
wf_name=$(echo "$wf_decoded" | jq -r '.name' 2>/dev/null)
|
||||
is_active=$(echo "$wf_decoded" | jq -r '.active' 2>/dev/null)
|
||||
wf_owner=$(echo "$wf_decoded" | jq -r '.owner_email' 2>/dev/null)
|
||||
|
||||
# 從資料庫獲取執行數據
|
||||
exec_data=$(PGPASSWORD=accusys psql -U n8n -h localhost -d n8n -t -A <<EOF
|
||||
SELECT json_agg(row_to_json(t)) FROM (
|
||||
SELECT status, "startedAt", "stoppedAt",
|
||||
EXTRACT(EPOCH FROM ("stoppedAt" - "startedAt")) * 1000 as execution_time
|
||||
FROM execution_entity
|
||||
WHERE "workflowId" = '$wf_id'
|
||||
ORDER BY "startedAt" DESC
|
||||
LIMIT 50
|
||||
) t
|
||||
EOF
|
||||
)
|
||||
|
||||
exec_count=$(echo "$exec_data" | jq '. | length' 2>/dev/null || echo "0")
|
||||
|
||||
# 計算成功/失敗
|
||||
success_count=$(echo "$exec_data" | jq '[.[] | select(.status == "success")] | length' 2>/dev/null || echo "0")
|
||||
failure_count=$(echo "$exec_data" | jq '[.[] | select(.status == "error")] | length' 2>/dev/null || echo "0")
|
||||
|
||||
# 平均執行時間
|
||||
avg_duration=$(echo "$exec_data" | jq '[.[] | .execution_time] | map(select(. != null)) | add / length | floor' 2>/dev/null || echo "0")
|
||||
|
||||
# 檢查是否有 webhook
|
||||
has_webh=$(PGPASSWORD=accusys psql -U n8n -h localhost -d n8n -t -A -c "
|
||||
SELECT COUNT(*) FROM webhook_entity WHERE workflow_id = '$wf_id'
|
||||
" 2>/dev/null || echo "0")
|
||||
[ "$has_webh" -gt 0 ] && has_webh="true" || has_webh="false"
|
||||
has_sched="false"
|
||||
|
||||
# 最後執行時間
|
||||
last_exec=$(echo "$exec_data" | jq -r '.[0].startedAt // "null"' 2>/dev/null | head -1)
|
||||
if [ "$last_exec" = "null" ] || [ -z "$last_exec" ]; then
|
||||
idle_days=999
|
||||
else
|
||||
idle_days=0
|
||||
fi
|
||||
|
||||
# 確保數值正確
|
||||
exec_count=$(echo "$exec_count" | tr -d '[:space:]' || echo "0")
|
||||
success_count=$(echo "$success_count" | tr -d '[:space:]' || echo "0")
|
||||
failure_count=$(echo "$failure_count" | tr -d '[:space:]' || echo "0")
|
||||
avg_duration=$(echo "$avg_duration" | tr -d '[:space:]' || echo "0")
|
||||
|
||||
# 計算失敗率
|
||||
if [ -n "$exec_count" ] && [ "$exec_count" -gt 0 ] 2>/dev/null; then
|
||||
failure_rate=$(( failure_count * 100 / exec_count ))
|
||||
else
|
||||
failure_rate=0
|
||||
fi
|
||||
|
||||
# 生成建議
|
||||
suggestion=$(generate_suggestion "$has_sched" "$has_webh" "$idle_days" "$failure_rate")
|
||||
|
||||
# 記錄到資料庫
|
||||
if [ "$last_exec" = "null" ] || [ -z "$last_exec" ]; then
|
||||
record_workflow "$wf_id" "$wf_name" "$is_active" "NULL" "$exec_count" "$success_count" "$failure_count" "$avg_duration" "$has_sched" "$has_webh" "$idle_days" "$suggestion"
|
||||
else
|
||||
record_workflow "$wf_id" "$wf_name" "$is_active" "'$last_exec'" "$exec_count" "$success_count" "$failure_count" "$avg_duration" "$has_sched" "$has_webh" "$idle_days" "$suggestion"
|
||||
fi
|
||||
|
||||
# 顯示
|
||||
status_icon="○"
|
||||
if [ "$is_active" = "true" ]; then
|
||||
status_icon="●"
|
||||
fi
|
||||
|
||||
idle_info=""
|
||||
if [ "$idle_days" -ge "$IDLE_THRESHOLD" ]; then
|
||||
idle_info=" [閒置 $idle_days 天]"
|
||||
total_idle=$((total_idle + 1))
|
||||
fi
|
||||
|
||||
suggestion_info=""
|
||||
if [ -n "$suggestion" ]; then
|
||||
suggestion_info=" [$suggestion]"
|
||||
fi
|
||||
|
||||
echo "$status_icon $wf_name (ID: $wf_id) [$wf_owner]$idle_info$suggestion_info"
|
||||
echo " 執行: $exec_count (成功: $success_count, 失敗: $failure_count) | 平均: ${avg_duration}ms"
|
||||
done
|
||||
|
||||
echo "----------------------------------------"
|
||||
echo "閒置 Workflow (> $IDLE_THRESHOLD 天): $total_idle"
|
||||
echo ""
|
||||
|
||||
log "Workflow check completed: $total_count total, $total_idle idle"
|
||||
|
||||
# 顯示閒置 workflow
|
||||
if [ $total_idle -gt 0 ]; then
|
||||
echo ""
|
||||
echo "閒置 Workflow 建議:"
|
||||
psql -U accusys -h localhost -d momentry -t -A -c "
|
||||
SELECT ' - ' || workflow_name || ': ' || suggestion
|
||||
FROM monitor_workflows
|
||||
WHERE idle_days >= $IDLE_THRESHOLD AND suggestion != '';
|
||||
" 2>/dev/null
|
||||
fi
|
||||
1
v1.1/release/phase1/latest
Symbolic link
1
v1.1/release/phase1/latest
Symbolic link
@@ -0,0 +1 @@
|
||||
v1.0.0_20260618_155213
|
||||
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233355
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:33:55.807345+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233355/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233355/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233355/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233355/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233355/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233355/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233401
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:01.814111+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233401/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233401/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233401/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233401/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233401/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233401/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233407
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:07.840966+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233407/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233407/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233407/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233407/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233407/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233407/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233413
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:13.764934+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233413/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233413/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233413/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233413/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233413/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233413/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233419
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:19.843331+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233419/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233419/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233419/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233419/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233419/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233419/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233425
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:25.768309+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233425/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233425/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233425/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233425/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233425/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233425/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233431
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:31.951221+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233431/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233431/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233431/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233431/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233431/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233431/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233437
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:38.051475+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233437/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233437/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233437/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233437/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233437/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233437/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233443
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:44.065778+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233443/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233443/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233443/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233443/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233443/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233443/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233450
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:50.226613+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233450/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233450/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233450/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233450/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233450/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233450/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233456
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:34:56.369677+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233456/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233456/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233456/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233456/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233456/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233456/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233502
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:35:02.458881+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233502/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233502/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233502/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233502/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233502/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233502/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233508
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:35:08.609206+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233508/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233508/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233508/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233508/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233508/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233508/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233514
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:35:14.932471+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233514/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233514/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233514/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233514/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233514/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233514/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233520
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:35:21.060246+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233520/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233520/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233520/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233520/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233520/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233520/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
@@ -0,0 +1,8 @@
|
||||
Model: 8703db167315e6d27de6687154169f58_v1
|
||||
Phase: 1
|
||||
Version: v1.0.0
|
||||
Timestamp: 20260616_233526
|
||||
File UUID: 8703db167315e6d27de6687154169f58
|
||||
Qdrant Collection: momentry_dev_rule1_v2
|
||||
Git Commit: 17e4e15860e059aeecee352d7d1a635d557d27ad
|
||||
Packaged at: 2026-06-16T23:35:27.055232+00:00
|
||||
24
v1.1/release/phase1/v1.0.0_20260616_233526/identities.csv
Normal file
24
v1.1/release/phase1/v1.0.0_20260616_233526/identities.csv
Normal file
File diff suppressed because one or more lines are too long
2191
v1.1/release/phase1/v1.0.0_20260616_233526/schema.sql
Normal file
2191
v1.1/release/phase1/v1.0.0_20260616_233526/schema.sql
Normal file
File diff suppressed because it is too large
Load Diff
1
v1.1/release/phase1/v1.0.0_20260616_233526/vectors.csv
Normal file
1
v1.1/release/phase1/v1.0.0_20260616_233526/vectors.csv
Normal file
@@ -0,0 +1 @@
|
||||
id,chunk_id,uuid,chunk_type,embedding,created_at
|
||||
|
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user