Japanese Manga Machine Translation Experience (1) - Overview
This article is translated by AI.
It has been almost two months since I started exploring Japanese manga machine translation. I have accumulated some experience and would like to share it with you all, hoping it will be helpful.
Archive Directory
Japanese Manga Machine Translation Experience (0) - Table of Contents
Difficulties
Discussing Japanese manga alone might be a bit vague, so let's compare it with machine translation of Japanese light novels and Galgames.
Japanese light novels are basically pure text, which is the easiest. Basically, combining neavo/LinguaGacha with a powerful LLM can achieve good translation results.
For Galgames, besides pure text, operations like unpacking and repacking which require some computer skills are the most troublesome. Fortunately, Galgame engines are relatively fixed. Once you identify the engine, you can apply the appropriate solution.
Japanese manga, essentially, belongs to image translation. It has more spatial information than the pure text of light novels, which is where the difficulty of machine translation lies. In addition, typesetting and image editing are also required.
If it's just simple external subtitles, manga machine translation is actually not difficult, because current LLMs, such as gemini 2.5 pro, have strong enough visual capabilities to accurately recognize dialogues in Japanese manga bubbles, as well as onomatopoeia outside of them. Furthermore, it can sort each text block in the order of human reading.
For example, for the manga below.
之总述-1753679777101.jpeg)
The translation result of gemini 2.5 pro is as follows.
| No. (Left) | Type (Left) | Original (Left) | Translation (Left) | Note (Left) | No. (Right) | Type (Right) | Original (Right) | Translation (Right) | Note (Right) |
|---|---|---|---|---|---|---|---|---|---|
| 1.1 | Dialogue Box | そして エリザまで メイドになりたいと 言い出したのだ | Then, even Eliza said she wanted to be a maid. | 1.2 | Dialogue Box | 最近は 情勢も安定しており 皇帝としての仕事にも 余裕があるという その余暇を使って 民の暮らしを 体験したい ということだ | She said that the situation has been stable recently, and she has some spare time in her work as the emperor, so she wants to use that leisure time to experience the life of the common people. | This is relaying Eliza's reason. | |
| 1.3 | Dialogue Box | なんというか… エリザらしい | How should I put it... That's so like Eliza. | 1.4 | SFX | ガタ | Clatter (Gata) | Sound of an object shaking, possibly a carriage or train. | |
| 1.5 | SFX | ガタ | Clatter (Gata) | Same as above. | 2.1 | Dialogue Box | アメリア エリザは うまくやってる？ | Amelia, how is Eliza doing? | |
| 2.2 | Dialogue Box | はい それはもう | Yes, very well. | 2.3 | Dialogue Box | この春から アメリアは メイド長になった 私つきのメイドを 長にすることで 徐々に父上から家督を 譲り受ける準備をしている | Since this spring, Amelia has become the head maid. By making my personal maid the head, I am gradually preparing to inherit the family headship from my father. | ||
| 3.1 | Dialogue Box | エリザ様はひとたび メイド服をまとえば 完全にメイドとして 振る舞われて… ご身分をおくびにも 出しません | Once Lady Eliza puts on the maid uniform, she behaves completely like a maid... She doesn't show a hint of her status. | "おくびにも出さない" is an idiom meaning "not showing a hint of". | 3.2 | Dialogue Box | さすがは お姉さまです 知った時は びっくりしました | As expected of my older sister. I was surprised when I found out. | |
| 3.3 | SFX | ガタ | Clatter (Gata) | Sound of object shaking. | 4.1 | Thought Box | 元からうちにいた メイドたちは エリザが皇帝だと 知っているが 貴族の令嬢たちには 正体を明かすわけには いかない | The maids who were originally in our house know that Eliza is the emperor, but we cannot reveal her true identity to the noble young ladies. | This is an inner monologue. |
| 4.2 | Thought Box | どこから 噂が広まるか わからないからだ | Because we don't know where rumors might spread from. | 4.3 | Thought Box | 何事も 起きなければ いいけど… | I hope nothing happens... | The tone carries worry. |
The effect is quite good.
But reading like this is tiring... not comfortable... What we hope for is a manga after typesetting and image editing, not just external subtitles.
Current Status and Effect Display
I can only talk casually about the current status of Japanese manga machine translation based on my two months of experience.
I know very little about paid services, but for free methods, they can basically achieve smooth reading. Of course, there is still a big gap compared to professional translation groups, but it is basically no problem for personal use.
For example, for manga with uncomplicated scenes.
之总述-1753680270998.jpeg)
It can basically translate "perfectly".
之总述-1753680285546.jpeg)
If the scene is slightly more complex, involving rotated text bubbles.
之总述-1753680434732.jpg)
It also has good results.
之总述-1753680413826.jpg)
For Furigana.
之总述-1753680499146.jpg)
It can basically recognize and remove them without causing confusion.
之总述-1753680512122.jpg)
Basic Process
Detection
Simply put, it detects where there is text. Of course, some detection models will deliberately ignore onomatopoeia that is difficult to OCR.
The blue boxes below are the detection results.
之总述-1753681274713.png)
OCR
After detection, naturally, we need to "see" what text is in the places with text.
Below are the OCR results.
之总述-1753681411951.png)
Translation
Hand it over to LLM for translation.
Below are the translation results.
之总述-1753681518215.png)
You can see that the original text on the left overlaps with the translation, so image inpainting is needed at this time.
Inpainting
Equivalent to image editing, including but not limited to erasing the original text.
The effect is as follows, the original text has been erased.
之总述-1753681626110.png)
Other Steps
In addition to the four steps listed above, there can be more detailed steps such as proofreading and color picking. But these four steps are the foundation and also the common process of the translation tools introduced next.
Translation Tools
There are mainly five translation tools, four free and one paid.
Free ones:
- zyddnys/manga-image-translator
- dmMaze/BallonsTranslator
- MashiroSaber03/Saber-Translator
- hgmzhn/manga-translator-ui
Paid one:
I made a table based on subjective judgment, you can choose a suitable translator according to your needs.
| Saber-Translator | manga-image-translator | BallonsTranslator | ImageTrans | manga-translator-ui | |
|---|---|---|---|---|---|
| Batch Translation | 0 | 3 | 3 | 2 | 3 |
| Export Original | 1 | 0.5 | 1 | 3 | 3 |
| Import Translation | 1 | 0 | 1 | 3 | 3 |
| Model Richness | 1 | 3 | 2 | 3 | 2 |
| CLI Support | 0 | 3 | 1 | 1 | 3 |
| AI OCR | 2 | 0.5 | 1 | 2 | 2 |
| AI Integration | 3 | 1 | 2 | 2 | 2 |
| Ease of Use | 3 | 1 | 2 | 1 | 3 |
| Customizability | 1 | 3 | 2 | 5 | 3 |
| UI Interface | 1 | 0.5 | 2 | 2 | 2 |
| Plugin Management | 1 | 0 | 0 | 2 | 0 |
Note: 0 means this function is not available. The larger the number, the better the function support.
Among the above five tools, you can choose one. I personally recommend manga-translator-ui the most.
There are also some auxiliary tools, which will be discussed in detail later.