This article is translated by AI.

If Saber-Translator and BallonsTranslator are not good enough in translation effect, manga-image-translator is difficult to install, and ImageTrans is paid, then the recently developed manga-translator-ui basically makes up for all the shortcomings of the previous four translators, and combines the strengths of the three, reaching a new high in translation effect.

Archive Directory

Japanese Manga Machine Translation Experience (0) - Table of Contents

Project Address

hgmzhn/manga-translator-ui

Horizontal Comparison

Installation

manga-translator-ui provides a packaged version (does not support updates) and a one-click installation script (supports updates). If the network environment is suitable, it will almost not get stuck in the installation step. Even if the model download is slow, you can use the network disk link sent above to get it.

manga-image-translator and BallonsTranslator are more complicated to install, for which I also wrote "Installation Suggestions", see Japanese Manga Machine Translation Experience (4) - manga-image-translator.

Detection

It integrates the default detection model detector of manga-image-translator, which currently has the best detection effect (please refer to the "Translation Quality" section of Japanese Manga Machine Translation Experience (4) - manga-image-translator), and combines my suggestion to use hybrid detection and mangaocr for secondary OCR to avoid omission of single characters or sentences with few characters such as modal particles.

Specifically

manga-image-translator

manga-translator-ui

The manga-translator-ui one actually translated "... ツ" into "...".

OCR

For detailed differences in OCR models, please refer to "OCR" in "Usage Suggestions" of Japanese Manga Machine Translation Experience (3) - BallonsTranslator.

manga-translator-ui integrates the mit 48 px model, which currently has the best OCR effect for Japanese manga, and also supports AI visual OCR.

Translation

First of all, it supports high-quality translation, which is equivalent to letting AI translate while looking at the picture, capable of distinguishing characters and situations, and the effect is better than traditional pure text translation. This was also a featured function of Saber-Translator, see Japanese Manga Machine Translation Experience (2) - Saber-Translator.

Secondly, it supports setting the number of context pages carried, so that translation can be combined with context, unifying terminology and continuing the context of the previous text.

Finally, it supports large-scale export of original text and import of translation. For specific functions, please refer to the article Japanese Manga Machine Translation Experience (6) - Comprehensive Guide I wrote before.

BUG of manga-image-translator

When translating the following picture, manga-image-translator will report an error.

ERROR: [local] Error during rendering:
Traceback (most recent call last):
  File "D:\Tools\manga-image-translator\manga_translator\manga_translator.py", line 612, in _translate
    ctx.img_rendered = await self._run_text_rendering(config, ctx)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Tools\manga-image-translator\manga_translator\manga_translator.py", line 1376, in _run_text_rendering
    output = await dispatch_rendering(ctx.img_inpainted, ctx.text_regions, self.font_path, config.render.font_size,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Tools\manga-image-translator\manga_translator\rendering\__init__.py", line 261, in dispatch
    img = render(img, region, dst_points, hyphenate, line_spacing, disable_font_border)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Tools\manga-image-translator\manga_translator\rendering\__init__.py", line 312, in render
    temp_box = text_render.put_text_vertical(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Tools\manga-image-translator\manga_translator\rendering\text_render.py", line 550, in put_text_vertical
    num_char_x = len(text) // num_char_y + 1
                 ~~~~~~~~~~^^~~~~~~~~~~~
ZeroDivisionError: integer division or modulo by zero

ERROR: [local] ZeroDivisionError: integer division or modulo by zero

Although manga-translator-ui integrates the algorithm of the repair process of manga-image-translator, it does not integrate this BUG.

BUG of BallonsTranslator

In the issue Bug Feedback: ctd+mit48px bubble frame edge text not OCRed, a BUG of BallonsTranslator is described, but this does not happen in manga-translator-ui.

Batch Export Original Text and Import Translation

See "Precautions" in Japanese Manga Machine Translation Experience (4) - manga-image-translator, this function of manga-image-translator is no longer available.

Although BallonsTranslator supports this function, it does not support output as json files and other files with relatively strong formats, which is easy to cause confusion during the translation process, and is incompatible with LinguaGacha and KeywordGacha these two translators.

manga-translator-ui supports this function, and the speed of skipping files has been greatly improved.

Integration and Features

Manga Super-Resolution

Simply put, manga super-resolution is to improve the clarity of manga.

manga-translator-ui integrates five super-resolution models

• Waifu 2 x
• ESRGAN
• 4 x UltraSharp
• Real-CUGAN
• MangaJaNai

Real-CUGAN is not recommended because this model handles text poorly and will remove shadows without authorization. It is more recommended to use MangaJaNai, just adjust the magnification to 2.

Original image

real-cugan-2x-conservative pro, character hair shadow removed, text deformed

MangaJaNai 2x super-resolution, retains character hair shadow, text normal

Manga Colorization

The integrated colorization model is the qweasdd/manga-colorization-v2 model mentioned in Manga Reading and Management Experience, the effect is very poor and not recommended.

Concurrent Threads

OCR and repair will be performed during translation, and will not wait for the api to return.

Automatic Glossary Extraction

To be tested. In fact, it is also recommended to use KeywordGacha for glossary extraction in the next article I wrote.

AI Sentence Segmentation

Never used.

Usage Suggestions

Hybrid OCR

Hybrid OCR can reduce the missed detection rate, see above for details. Japanese manga recommends mit 48 px+mangaocr, English manga recommends mit 48 px+paddleocr, mit 48 px as the first choice, mangaocr as auxiliary OCR is mainly to OCR modal particles and other single characters or sentences with few characters, which is easily missed by mit 48 px.

yolo Auxiliary Detection

This is another piece of the puzzle to solve the problem of missing modal particles and other single characters or sentences with few characters. It is recommended to set yolo confidence threshold to 0.3.

Image Editable

It is recommended to enable it, otherwise there will be cases where the file is not overwritten and the new translation result does not take effect.

Usage Tips

Translation Separation

It is recommended to refer to the next article I wrote.

Export Original Text

Use config_save_text.json configuration file.

{
  "app": {
    "last_open_dir": "",
    "last_output_path": "",
    "favorite_folders": [
      "J:/漫画/mangas",
      "J:/漫画/RAW"
    ],
    "theme": "dark",
    "ui_language": "auto",
    "current_preset": "Default"
  },
  "filter_text": null,
  "kernel_size": 3,
  "mask_dilation_offset": 70,
  "translator": {
    "translator": "openai",
    "target_lang": "CHS",
    "no_text_lang_skip": false,
    "gpt_config": "examples/my_gpt_config.yaml",
    "high_quality_prompt_path": "dict/prompt_example.json",
    "max_requests_per_minute": 0,
    "attempts": -1
  },
  "ocr": {
    "use_mocr_merge": false,
    "ocr": "48px",
    "use_hybrid_ocr": true,
    "secondary_ocr": "mocr",
    "min_text_length": 0,
    "ignore_bubble": 0,
    "prob": 0.1,
    "merge_gamma": 0.8,
    "merge_sigma": 2.5,
    "merge_edge_ratio_threshold": 0.0
  },
  "detector": {
    "detector": "default",
    "detection_size": 2048,
    "text_threshold": 0.5,
    "det_rotate": false,
    "det_auto_rotate": false,
    "det_invert": false,
    "det_gamma_correct": false,
    "box_threshold": 0.5,
    "unclip_ratio": 2.5,
    "use_yolo_obb": true,
    "yolo_obb_conf": 0.3,
    "yolo_obb_iou": 0.6,
    "yolo_obb_overlap_threshold": 0.1,
    "min_box_area_ratio": 0
  },
  "inpainter": {
    "inpainter": "lama_large",
    "inpainting_size": 2048.0,
    "inpainting_precision": "fp32",
    "inpainting_split_ratio": 3.0
  },
  "render": {
    "renderer": "default",
    "alignment": "auto",
    "disable_font_border": false,
    "disable_auto_wrap": false,
    "font_size_offset": 0,
    "font_size_minimum": 0,
    "direction": "auto",
    "uppercase": false,
    "lowercase": false,
    "font_path": "fonts/Arial-Unicode-Regular.ttf",
    "no_hyphenation": false,
    "font_color": null,
    "line_spacing": null,
    "font_size": null,
    "auto_rotate_symbols": false,
    "rtl": true,
    "layout_mode": "smart_scaling",
    "max_font_size": 0,
    "font_scale_ratio": 1.0,
    "center_text_in_bubble": false,
    "optimize_line_breaks": false,
    "check_br_and_retry": false,
    "strict_smart_scaling": false,
    "stroke_width": 0.07
  },
  "upscale": {
    "upscaler": "realcugan",
    "upscale_ratio": null,
    "realcugan_model": null,
    "tile_size": 600,
    "revert_upscaling": false
  },
  "colorizer": {
    "colorization_size": 2048,
    "denoise_sigma": 30,
    "colorizer": "none"
  },
  "cli": {
    "verbose": false,
    "attempts": -1,
    "ignore_errors": false,
    "use_gpu": true,
    "use_gpu_limited": false,
    "context_size": 60,
    "format": "png",
    "overwrite": false,
    "skip_no_text": false,
    "save_text": true,
    "load_text": false,
    "template": true,
    "save_quality": 100,
    "batch_size": 1,
    "batch_concurrent": false,
    "generate_and_export": false,
    "colorize_only": false,
    "upscale_only": false,
    "inpaint_only": false,
    "save_to_source_dir": false
  }
}

For English manga, just change "secondary_ocr": "mocr" to "secondary_ocr": "paddleocr".

Import Translation

Use config_load_text.json configuration file.

{
  "app": {
    "last_open_dir": "",
    "last_output_path": "",
    "favorite_folders": [
      "J:/漫画/mangas",
      "J:/漫画/RAW"
    ],
    "theme": "dark",
    "ui_language": "auto",
    "current_preset": "Default"
  },
  "filter_text": null,
  "kernel_size": 3,
  "mask_dilation_offset": 70,
  "translator": {
    "translator": "openai",
    "target_lang": "CHS",
    "no_text_lang_skip": false,
    "gpt_config": "examples/my_gpt_config.yaml",
    "high_quality_prompt_path": "dict/prompt_example.json",
    "max_requests_per_minute": 0,
    "attempts": -1
  },
  "ocr": {
    "use_mocr_merge": false,
    "ocr": "48px",
    "use_hybrid_ocr": true,
    "secondary_ocr": "mocr",
    "min_text_length": 0,
    "ignore_bubble": 0,
    "prob": 0.1,
    "merge_gamma": 0.8,
    "merge_sigma": 2.5,
    "merge_edge_ratio_threshold": 0.0
  },
  "detector": {
    "detector": "default",
    "detection_size": 2048,
    "text_threshold": 0.5,
    "det_rotate": false,
    "det_auto_rotate": false,
    "det_invert": false,
    "det_gamma_correct": false,
    "box_threshold": 0.5,
    "unclip_ratio": 2.5,
    "use_yolo_obb": true,
    "yolo_obb_conf": 0.3,
    "yolo_obb_iou": 0.6,
    "yolo_obb_overlap_threshold": 0.1,
    "min_box_area_ratio": 0
  },
  "inpainter": {
    "inpainter": "lama_large",
    "inpainting_size": 1024,
    "inpainting_precision": "fp32",
    "inpainting_split_ratio": 3.0
  },
  "render": {
    "renderer": "default",
    "alignment": "auto",
    "disable_font_border": false,
    "disable_auto_wrap": false,
    "font_size_offset": 0,
    "font_size_minimum": 0,
    "direction": "auto",
    "uppercase": false,
    "lowercase": false,
    "font_path": "fonts/Arial-Unicode-Regular.ttf",
    "no_hyphenation": false,
    "font_color": null,
    "line_spacing": null,
    "font_size": null,
    "auto_rotate_symbols": false,
    "rtl": true,
    "layout_mode": "smart_scaling",
    "max_font_size": 0,
    "font_scale_ratio": 1.0,
    "center_text_in_bubble": false,
    "optimize_line_breaks": false,
    "check_br_and_retry": false,
    "strict_smart_scaling": false,
    "stroke_width": 0.07
  },
  "upscale": {
    "upscaler": "realcugan",
    "upscale_ratio": null,
    "realcugan_model": null,
    "tile_size": 600,
    "revert_upscaling": false
  },
  "colorizer": {
    "colorization_size": 2048,
    "denoise_sigma": 30,
    "colorizer": "none"
  },
  "cli": {
    "verbose": false,
    "attempts": -1,
    "ignore_errors": false,
    "use_gpu": true,
    "use_gpu_limited": false,
    "context_size": 60,
    "format": "png",
    "overwrite": false,
    "skip_no_text": false,
    "save_text": false,
    "load_text": true,
    "template": true,
    "save_quality": 100,
    "batch_size": 1,
    "batch_concurrent": false,
    "generate_and_export": false,
    "colorize_only": false,
    "upscale_only": false,
    "inpaint_only": false,
    "save_to_source_dir": false
  }
}

Usage Experience

This project started development in August this year and was basically usable by November, but there are still some small BUGs now, but don't worry, just report them to the developer in time, and they can usually be solved within a day or two.

For translating Japanese manga, the effect is slightly better than manga-image-translator, but without the manga-image-translator specific BUG mentioned above, so it will not be interrupted from time to time. And it also provides command line mode and batch export original text and import translation functions, while the latter is not supported by manga-image-translator.

However, manga-translator-ui has high memory usage (my computer has 16 G memory). Although memory usage can be limited by the --memory-percent parameter, there are still cases of freezing, such as chrome page freezing, program crash, and vscode crash, and the most serious one even caused a blue screen of death. This is similar to the problem described in "Usage Experience" in Japanese Manga Machine Translation Experience (5) - ImageTrans.

And I am now also switching from manga-image-translator to manga-translator-ui. After translating more than 10,000 pictures, I can confirm that this is indeed a good translator.

FAQ

Should gemini model use Google gemini type translator?

Depends on the interface type of the api. Official channel gemini chooses Google gemini type translator, other transit channel api, generally choose openai type translator.

Why can't I edit the translated picture?

Click Edit Original Image.

Installation script failed to run

Choose a foreign network environment and enable global proxy.

Translation failed

Click the "Test" button first to ensure that there is no problem with the api.

Promotion End Page

Contributors: Ming Wei

Previous pageJapanese Manga Machine Translation Experience (8) - Comprehensive Guide Part 2

Next pagePDF Optimization: Errata, Directory, Sharpening, Blank Page, Deskew, De-shadow, OCR, Cover Back Cover, Split Double Page into Single Page