If tests/text_large.txt is given, I think 「その他 名詞,代名詞,一般,*,*,*,その他,ソノタ,ソノタ」 should be returned, but following tokens are returned. そ 名詞,特殊,助動詞語幹,*,*,*,そ,ソ,ソ の 助詞,連体化,*,*,*,*,の,ノ,ノ 他 名詞,非自立,副詞可能,*,*,*,他,ホカ,ホカ
If tests/text_large.txt is given, I think 「その他 名詞,代名詞,一般,,,*,その他,ソノタ,ソノタ」 should be returned, but following tokens are returned.
そ 名詞,特殊,助動詞語幹,,,,そ,ソ,ソ
の 助詞,連体化,,,,,の,ノ,ノ
他 名詞,非自立,副詞可能,,,,他,ホカ,ホカ