admin
8e5a449007
Some checks are pending
Copilot Setup Steps / copilot-setup-steps (push) Waiting to run
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Waiting to run
Python check requirements.txt / check-requirements (push) Waiting to run
Python Type-Check / python type-check (push) Waiting to run
Update Operations Documentation / update-ops-docs (push) Waiting to run
8 lines
249 B
Plaintext
8 lines
249 B
Plaintext
# A probably incorrect grammar for Japanese
|
|
root ::= jp-char+ ([ \t\n] jp-char+)*
|
|
jp-char ::= hiragana | katakana | punctuation | cjk
|
|
hiragana ::= [ぁ-ゟ]
|
|
katakana ::= [ァ-ヿ]
|
|
punctuation ::= [、-〾]
|
|
cjk ::= [一-鿿]
|