Japanese Manga Machine Translation Experience (7) - manga-translator-ui
This article is translated by AI.
If Saber-Translator and BallonsTranslator are not good enough in translation effect, manga-image-translator is difficult to install, and ImageTrans is paid, then the recently developed manga-translator-ui basically makes up for all the shortcomings of the previous four translators, and combines the strengths of the three, reaching a new high in translation effect.
Archive Directory
Japanese Manga Machine Translation Experience (0) - Table of Contents
Project Address
Horizontal Comparison
Installation
manga-translator-ui provides a packaged version (does not support updates) and a one-click installation script (supports updates). If the network environment is suitable, it will almost not get stuck in the installation step. Even if the model download is slow, you can use the network disk link sent above to get it.
manga-image-translator and BallonsTranslator are more complicated to install, for which I also wrote "Installation Suggestions", see Japanese Manga Machine Translation Experience (4) - manga-image-translator.
Detection
It integrates the default detection model detector of manga-image-translator, which currently has the best detection effect (please refer to the "Translation Quality" section of Japanese Manga Machine Translation Experience (4) - manga-image-translator), and combines my suggestion to use hybrid detection and mangaocr for secondary OCR to avoid omission of single characters or sentences with few characters such as modal particles.
Specifically
之manga-translator-ui-1766365968817.webp)
之manga-translator-ui-1766368630864.png)
The manga-translator-ui one actually translated "... ツ" into "...".
OCR
For detailed differences in OCR models, please refer to "OCR" in "Usage Suggestions" of Japanese Manga Machine Translation Experience (3) - BallonsTranslator.
manga-translator-ui integrates the mit 48 px model, which currently has the best OCR effect for Japanese manga, and also supports AI visual OCR.
Translation
First of all, it supports high-quality translation, which is equivalent to letting AI translate while looking at the picture, capable of distinguishing characters and situations, and the effect is better than traditional pure text translation. This was also a featured function of Saber-Translator, see Japanese Manga Machine Translation Experience (2) - Saber-Translator.
Secondly, it supports setting the number of context pages carried, so that translation can be combined with context, unifying terminology and continuing the context of the previous text.
Finally, it supports large-scale export of original text and import of translation. For specific functions, please refer to the article Japanese Manga Machine Translation Experience (6) - Comprehensive Guide I wrote before.
BUG of manga-image-translator
When translating the following picture, manga-image-translator will report an error.
之manga-translator-ui-1766331083407.jpg)
ERROR: [local] Error during rendering:
Traceback (most recent call last):
File "D:\Tools\manga-image-translator\manga_translator\manga_translator.py", line 612, in _translate
ctx.img_rendered = await self._run_text_rendering(config, ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Tools\manga-image-translator\manga_translator\manga_translator.py", line 1376, in _run_text_rendering
output = await dispatch_rendering(ctx.img_inpainted, ctx.text_regions, self.font_path, config.render.font_size,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Tools\manga-image-translator\manga_translator\rendering\__init__.py", line 261, in dispatch
img = render(img, region, dst_points, hyphenate, line_spacing, disable_font_border)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Tools\manga-image-translator\manga_translator\rendering\__init__.py", line 312, in render
temp_box = text_render.put_text_vertical(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Tools\manga-image-translator\manga_translator\rendering\text_render.py", line 550, in put_text_vertical
num_char_x = len(text) // num_char_y + 1
~~~~~~~~~~^^~~~~~~~~~~~
ZeroDivisionError: integer division or modulo by zero
ERROR: [local] ZeroDivisionError: integer division or modulo by zeroAlthough manga-translator-ui integrates the algorithm of the repair process of manga-image-translator, it does not integrate this BUG.
BUG of BallonsTranslator
In the issue Bug Feedback: ctd+mit48px bubble frame edge text not OCRed, a BUG of BallonsTranslator is described, but this does not happen in manga-translator-ui.
Batch Export Original Text and Import Translation
See "Precautions" in Japanese Manga Machine Translation Experience (4) - manga-image-translator, this function of manga-image-translator is no longer available.
Although BallonsTranslator supports this function, it does not support output as json files and other files with relatively strong formats, which is easy to cause confusion during the translation process, and is incompatible with LinguaGacha and KeywordGacha these two translators.
manga-translator-ui supports this function, and the speed of skipping files has been greatly improved.
Integration and Features
Manga Super-Resolution
Simply put, manga super-resolution is to improve the clarity of manga.
manga-translator-ui integrates five super-resolution models
- Waifu 2 x
- ESRGAN
- 4 x UltraSharp
- Real-CUGAN
- MangaJaNai
Real-CUGAN is not recommended because this model handles text poorly and will remove shadows without authorization. It is more recommended to use MangaJaNai, just adjust the magnification to 2.
之manga-translator-ui-1766368824552.png)
之manga-translator-ui-1766368859775.png)
之manga-translator-ui-1766368941024.jpeg)
Manga Colorization
The integrated colorization model is the qweasdd/manga-colorization-v2 model mentioned in Manga Reading and Management Experience, the effect is very poor and not recommended.
Concurrent Threads
OCR and repair will be performed during translation, and will not wait for the api to return.
Automatic Glossary Extraction
To be tested. In fact, it is also recommended to use KeywordGacha for glossary extraction in the next article I wrote.
AI Sentence Segmentation
Never used.
Usage Suggestions
Hybrid OCR
Hybrid OCR can reduce the missed detection rate, see above for details. Japanese manga recommends mit 48 px+mangaocr, English manga recommends mit 48 px+paddleocr, mit 48 px as the first choice, mangaocr as auxiliary OCR is mainly to OCR modal particles and other single characters or sentences with few characters, which is easily missed by mit 48 px.
yolo Auxiliary Detection
This is another piece of the puzzle to solve the problem of missing modal particles and other single characters or sentences with few characters. It is recommended to set yolo confidence threshold to 0.3.
Image Editable
It is recommended to enable it, otherwise there will be cases where the file is not overwritten and the new translation result does not take effect.
Usage Tips
Translation Separation
It is recommended to refer to the next article I wrote.
Export Original Text
Use config_save_text.json configuration file.
{
"app": {
"last_open_dir": "",
"last_output_path": "",
"favorite_folders": [
"J:/漫画/mangas",
"J:/漫画/RAW"
],
"theme": "dark",
"ui_language": "auto",
"current_preset": "Default"
},
"filter_text": null,
"kernel_size": 3,
"mask_dilation_offset": 70,
"translator": {
"translator": "openai",
"target_lang": "CHS",
"no_text_lang_skip": false,
"gpt_config": "examples/my_gpt_config.yaml",
"high_quality_prompt_path": "dict/prompt_example.json",
"max_requests_per_minute": 0,
"attempts": -1
},
"ocr": {
"use_mocr_merge": false,
"ocr": "48px",
"use_hybrid_ocr": true,
"secondary_ocr": "mocr",
"min_text_length": 0,
"ignore_bubble": 0,
"prob": 0.1,
"merge_gamma": 0.8,
"merge_sigma": 2.5,
"merge_edge_ratio_threshold": 0.0
},
"detector": {
"detector": "default",
"detection_size": 2048,
"text_threshold": 0.5,
"det_rotate": false,
"det_auto_rotate": false,
"det_invert": false,
"det_gamma_correct": false,
"box_threshold": 0.5,
"unclip_ratio": 2.5,
"use_yolo_obb": true,
"yolo_obb_conf": 0.3,
"yolo_obb_iou": 0.6,
"yolo_obb_overlap_threshold": 0.1,
"min_box_area_ratio": 0
},
"inpainter": {
"inpainter": "lama_large",
"inpainting_size": 2048.0,
"inpainting_precision": "fp32",
"inpainting_split_ratio": 3.0
},
"render": {
"renderer": "default",
"alignment": "auto",
"disable_font_border": false,
"disable_auto_wrap": false,
"font_size_offset": 0,
"font_size_minimum": 0,
"direction": "auto",
"uppercase": false,
"lowercase": false,
"font_path": "fonts/Arial-Unicode-Regular.ttf",
"no_hyphenation": false,
"font_color": null,
"line_spacing": null,
"font_size": null,
"auto_rotate_symbols": false,
"rtl": true,
"layout_mode": "smart_scaling",
"max_font_size": 0,
"font_scale_ratio": 1.0,
"center_text_in_bubble": false,
"optimize_line_breaks": false,
"check_br_and_retry": false,
"strict_smart_scaling": false,
"stroke_width": 0.07
},
"upscale": {
"upscaler": "realcugan",
"upscale_ratio": null,
"realcugan_model": null,
"tile_size": 600,
"revert_upscaling": false
},
"colorizer": {
"colorization_size": 2048,
"denoise_sigma": 30,
"colorizer": "none"
},
"cli": {
"verbose": false,
"attempts": -1,
"ignore_errors": false,
"use_gpu": true,
"use_gpu_limited": false,
"context_size": 60,
"format": "png",
"overwrite": false,
"skip_no_text": false,
"save_text": true,
"load_text": false,
"template": true,
"save_quality": 100,
"batch_size": 1,
"batch_concurrent": false,
"generate_and_export": false,
"colorize_only": false,
"upscale_only": false,
"inpaint_only": false,
"save_to_source_dir": false
}
}For English manga, just change "secondary_ocr": "mocr" to "secondary_ocr": "paddleocr".
Import Translation
Use config_load_text.json configuration file.
{
"app": {
"last_open_dir": "",
"last_output_path": "",
"favorite_folders": [
"J:/漫画/mangas",
"J:/漫画/RAW"
],
"theme": "dark",
"ui_language": "auto",
"current_preset": "Default"
},
"filter_text": null,
"kernel_size": 3,
"mask_dilation_offset": 70,
"translator": {
"translator": "openai",
"target_lang": "CHS",
"no_text_lang_skip": false,
"gpt_config": "examples/my_gpt_config.yaml",
"high_quality_prompt_path": "dict/prompt_example.json",
"max_requests_per_minute": 0,
"attempts": -1
},
"ocr": {
"use_mocr_merge": false,
"ocr": "48px",
"use_hybrid_ocr": true,
"secondary_ocr": "mocr",
"min_text_length": 0,
"ignore_bubble": 0,
"prob": 0.1,
"merge_gamma": 0.8,
"merge_sigma": 2.5,
"merge_edge_ratio_threshold": 0.0
},
"detector": {
"detector": "default",
"detection_size": 2048,
"text_threshold": 0.5,
"det_rotate": false,
"det_auto_rotate": false,
"det_invert": false,
"det_gamma_correct": false,
"box_threshold": 0.5,
"unclip_ratio": 2.5,
"use_yolo_obb": true,
"yolo_obb_conf": 0.3,
"yolo_obb_iou": 0.6,
"yolo_obb_overlap_threshold": 0.1,
"min_box_area_ratio": 0
},
"inpainter": {
"inpainter": "lama_large",
"inpainting_size": 1024,
"inpainting_precision": "fp32",
"inpainting_split_ratio": 3.0
},
"render": {
"renderer": "default",
"alignment": "auto",
"disable_font_border": false,
"disable_auto_wrap": false,
"font_size_offset": 0,
"font_size_minimum": 0,
"direction": "auto",
"uppercase": false,
"lowercase": false,
"font_path": "fonts/Arial-Unicode-Regular.ttf",
"no_hyphenation": false,
"font_color": null,
"line_spacing": null,
"font_size": null,
"auto_rotate_symbols": false,
"rtl": true,
"layout_mode": "smart_scaling",
"max_font_size": 0,
"font_scale_ratio": 1.0,
"center_text_in_bubble": false,
"optimize_line_breaks": false,
"check_br_and_retry": false,
"strict_smart_scaling": false,
"stroke_width": 0.07
},
"upscale": {
"upscaler": "realcugan",
"upscale_ratio": null,
"realcugan_model": null,
"tile_size": 600,
"revert_upscaling": false
},
"colorizer": {
"colorization_size": 2048,
"denoise_sigma": 30,
"colorizer": "none"
},
"cli": {
"verbose": false,
"attempts": -1,
"ignore_errors": false,
"use_gpu": true,
"use_gpu_limited": false,
"context_size": 60,
"format": "png",
"overwrite": false,
"skip_no_text": false,
"save_text": false,
"load_text": true,
"template": true,
"save_quality": 100,
"batch_size": 1,
"batch_concurrent": false,
"generate_and_export": false,
"colorize_only": false,
"upscale_only": false,
"inpaint_only": false,
"save_to_source_dir": false
}
}Usage Experience
This project started development in August this year and was basically usable by November, but there are still some small BUGs now, but don't worry, just report them to the developer in time, and they can usually be solved within a day or two.
For translating Japanese manga, the effect is slightly better than manga-image-translator, but without the manga-image-translator specific BUG mentioned above, so it will not be interrupted from time to time. And it also provides command line mode and batch export original text and import translation functions, while the latter is not supported by manga-image-translator.
However, manga-translator-ui has high memory usage (my computer has 16 G memory). Although memory usage can be limited by the --memory-percent parameter, there are still cases of freezing, such as chrome page freezing, program crash, and vscode crash, and the most serious one even caused a blue screen of death. This is similar to the problem described in "Usage Experience" in Japanese Manga Machine Translation Experience (5) - ImageTrans.
And I am now also switching from manga-image-translator to manga-translator-ui. After translating more than 10,000 pictures, I can confirm that this is indeed a good translator.
FAQ
Should gemini model use Google gemini type translator?
Depends on the interface type of the api. Official channel gemini chooses Google gemini type translator, other transit channel api, generally choose openai type translator.
Why can't I edit the translated picture?
Click Edit Original Image.
之manga-translator-ui-1766371658153.png)
Installation script failed to run
Choose a foreign network environment and enable global proxy.
Translation failed
Click the "Test" button first to ensure that there is no problem with the api.
之manga-translator-ui-1766371786600.png)
Promotion End Page
之manga-translator-ui-1766373827980.png)
