Results pending reproducible runs
No competitor scores are published yet. Results will appear only after each tool is run on the same version, settings, date, and rubric with raw output preserved.
Methodology
- Use a clean account and default settings unless the tool requires a documented setup.
- Submit the same 20 original Japanese lines in dataset order on the same day.
- Save the raw output before scoring and record unavailable capabilities as unavailable.
- Score each line on four one-point dimensions; do not infer credit from a different feature.
80-point scoring rubric
- Meaning (1) — The output communicates the line’s intended meaning without a material contradiction.
- Reading (1) — The output supplies or supports the correct reading of the kanji and kana in context.
- Grammar (1) — The output identifies the main grammatical or contraction point named in expectedInsights.
- Register and context (1) — The output explains the relevant formality, attitude, omission, or situational nuance.
Dataset
All lines are original KIKUGO examples. They are designed around recurring subtitle challenges and do not reproduce film or television dialogue.
| ID | Japanese line | Challenge | Expected meaning |
|---|---|---|---|
| jp-01 | 今日は学校に行かない。 | particle | I’m not going to school today. |
| jp-02 | メッセージ、もう読んだ？ | kana | Did you read the message already? |
| jp-03 | 雨が降ってるから、傘持ってきて。 | contraction | Since it’s raining, bring an umbrella. |
| jp-04 | そんなつもりじゃなかった。 | context | I didn’t mean it that way. |
| jp-05 | 先に食べてていいよ。 | contraction | You can start eating before me. |
| jp-06 | まさか本当に来るとは思わなかった。 | grammar | I never thought they would actually come. |
| jp-07 | これ、めっちゃうまい。 | register | This is really good. |
| jp-08 | 部長に確認しておきます。 | grammar | I’ll make sure to check with the department manager. |
| jp-09 | 彼が来るそうだ。 | grammar | I heard he is coming. |
| jp-10 | 雨が降りそうだ。 | grammar | It looks like it will rain. |
| jp-11 | 忘れないようにメモしておいた。 | grammar | I made a note so I wouldn’t forget. |
| jp-12 | もう少しゆっくり話してもらえますか。 | register | Could you speak a little more slowly? |
| jp-13 | 電車が遅れているみたい。 | context | It looks like the train is delayed. |
| jp-14 | せっかく来たんだから、楽しもう。 | context | Since we came all this way, let’s enjoy ourselves. |
| jp-15 | あいつ、またサボってる。 | register | That guy is slacking off again. |
| jp-16 | このアプリ、使いやすくない？ | context | Isn’t this app easy to use? |
| jp-17 | 何を言ってるのか全然わからない。 | contraction | I don’t understand at all what you’re saying. |
| jp-18 | 手伝ってくれて助かった。 | context | Your help really saved me. |
| jp-19 | 予約しておけばよかった。 | grammar | I should have made a reservation. |
| jp-20 | お客様、こちらで少々お待ちください。 | register | Please wait here for a moment. |
Reproduce before citing
A score is only comparable when the benchmark version, tool version, run date, settings, raw output, and scorer are disclosed. Cite the dataset separately from any future results.