RMMV Another localization plugin - translate your game with PO files!

[Solar_Flare]

Translate your Game!​

I realize that there already exist at least two other localization plugins (one originally by Iavra, another by DK). However, I figure this is still worth posting because the approach is totally different from both of those.

The plugin itself has detailed documentation in the plugin itself and some additional information in the demo project. However, I will give a summary here as well.

Spoiler: How to Use & How it Works

Once you have installed SFG_Utils, jsonpath, and my localization plugin, a new scene is added to the game allowing you to manage your translations. At the title screen, press F9 to bring up the language menu. (This menu will automatically be disabled in a deployed build.) You can choose to export your source language or import any languages that you defined in the plugin's parameters.

The plugin works by scanning your game for information that would need to be translated and building up a database which specifies what the string is and where it exists in your game. By default, it scans the entire database and all your maps. It does not scan plugins by default, as there is no way for it to know what parameters in a plugin would need to be translated, but you can easily configure what plugin data it gathers in its parameters.

It also supports translating assets (sounds, images, and movies). By default it marks the Game Over, Loading, and Made with MV pictures as translatable assets. You can configure whatever other assets you choose in the plugin parameters. You will need to create translated versions of the assets and link them to the source asset in the parameters.

Gathering strings from plugins is difficult. Many plugins do strange things that make it difficult or impossible for this plugin to gather its strings and translate them. However, I have done my best to provide as many ways as possible to configure the plugin to find them. It directly supports standard note tags (the ones that automatically parse to "meta" keys), and offers an advanced configuration mode which allows gathering from arbitrary global JavaScript objects.

When you export your language, a .POT file is produced in a translations folder in your project's root directory. This file contains all the strings the plugin knows about in a standard format. There are a wide variety of editors available for editing .POT files, so whoever is doing the actual work of translating can choose the tool they like best. When you open a .POT file in one of these editors, it will create a .PO file to hold the translated strings. If you place the .PO file in the translations folder, you can then import those translations back into the game using the title F9 menu.

The plugin automatically adds an option to select the language from the Options screen. If you use an options manager however (such as Yanfly's or SRD's), you will probably need to add the option back manually.

Note: Before exporting or importing, be sure to switch to the default language in the Options screen. Otherwise, the export/import may not work correctly.

I have a work-in-progress demo to showcase the plugin at work, featuring a rather amateurish French translation. Though this contains translations of all default database strings (such as terms and messages), don't use these translations in a game. I didn't put much effort into them, and it would probably just make both you and me look bad. The demo also includes altered versions of some Kadokawa plugins which make them compatible with this plugin. (NB: The demo contains a slightly older version of the plugin, so I recommend also downloading the latest version and substituting that into the demo project.)

I'm posting this here instead of in releases because there are some issues I haven't worked out. Some of them are problems with RPGMaker itself - the way it builds messages is not very translation-friendly. All the core messages editable in Terms are impossible to correctly translate into any language with grammatical gender, for example (including French). In fact, the plugin currently has no means of dealing with grammatical gender or plurality. (There's a note tag to specify gender, but it's just a stub, not used at this time.)

I'm open to suggestions on how to handle things like plurality and gender. I'm not sure if the problem is generically solvable, but I'm sure there has to be a way that gets close.

Spoiler: Screenshots

Import or export language catalogs from within your test game


Open the POT file in any PO editor to produce a PO file that can be imported back into the game

This screenshot shows PoEdit running on a Mac, but there are many other PO editors available online for Windows, Mac, and Linux.

Switch languages at runtime in the Options menu


Easily configure available languages in the plugin parameters


Configure translatable assets in the plugin parameters


With some configuration, this plugin can even gather strings from other plugins



Note: Advanced mode, above, may be able to support plugin parameters or note tags that aren't supported by the other two modes.

TERMS

• If you're making a non-commercial game, feel free to make use of this plugin as long as you give credit. I would prefer if you don't modify the code without permission, however - if you find a bug, or need a feature I didn't think of, I'd like to hear of it so I can release the bugfix or (assuming I like the idea) the new feature to everyone.
• Don't post the file anywhere else. You can link to this thread instead.
• Don't use anything from the demo in your game. (Exception: any included plugin that's a modification of a Kadokawa plugin can be used, as can SFG_ExtraMaps which is a modification of Zeriab_ExtraMaps.)
• Regarding commercial games, I haven't yet decided on terms. Feel free to make suggestions.

 

[Solar_Flare]

It's so quiet in here... no-one has anything to say at all?

Anyway, I uploaded a new version that fixes several bugs and also has special support for SFG_SwitchCommand and ALOE_ConditionalChoices.

All changes in this latest version:

• Some parameters gathered from plugins were added to the localization catalog with a non-normalized path.
• Add a function to manually look up the translation of a string from code.
• Using the arrow keys on the options menu option produced an invalid language; this is fixed.
• After switching languages, bugs would be introduced into the language catalog. Restarting the game (ie, quit and launch again) mostly fixed them, but now they're no longer introduced in the first place.
• If using SFG_SwitchCommand, the choices won't be considered translatable (unless they're a straight string without an operator).
• If using ALOE_ConditionalChoices, the condition part of the choices will be excluded from the language catalog, meaning there should be no chance of translators accidentally breaking your conditions.

I didn't update the demo, so if anyone wants to check it out they should also download the plugin separately and substitute that into the demo project.

 

[Trihan]

I also made one called Dialogue From File which can handle branching possibilities if you want to check it out for ideas. It's for VX Ace though, haven't ported it to MV yet.

 

[Solar_Flare]

What do you mean by branching possibilities? It doesn't sound like something relevant to a localization plugin but I may just be misinterpreting your meaning...

 

[Solar_Flare]

So has anyone tried this out? The forum says the updated plugin had three views...

 

[Solar_Flare]

I'm not sure how I missed it when I first posted this thread, but I've now added the other file that's needed to get this to run in the opening post. The demo project should have that file already, but I'm sure there are people who don't want to download the demo just to get a working product.

 

[Kronicide]

I'm surprised that no one has commented on this at all, it sounds like a really competent piece of software. Very cool. Just out of curiosity, do you have a game in development that's using this? It looks like functionality for pretty ambitious projects.

 

[Solar_Flare]

Yeah, I'm using it in a game, with an amateurish French translation for testing purposes. Still trying to decide how to deal with grammatical gender and plurals, though.

 

[Tsukihime]

I didn't look at the demo since I'm browsing on a phone, but I was looking for screenshots hoping to get an idea what the scene looks like, how the pot file looks, and a recommended pot editor.

I would also consider hosting it as a web demo.

Might help with the feedback. Any tool that helps streamline the localization process is always nice.

 

[Solar_Flare]

I think the main reason I didn't make screenshots is because I couldn't think of anything I could screenshot that really shows off what the plugin does. I guess I could take a shot of the language management scene or the options scene, or give a side-by-side view of a message window in English and in French?

I don't have any particular recommendation for po editors. I was using PoEdit (which is free but with a paid upgrade), and I've used Virtaal in the past (which I believe may be open source), but po is an open format, so there's quite a few options available.

I will consider putting the demo online as a web demo, as well as the manual document I wrote (which is included in the demo but would probably benefit from being more visible).

 

[Tsukihime]

As a user, I'm interested in the following

1. How easy is it to use it for my project (which might be released already, or half-way done, etc)
2. Which tools can I use (free, paid, doesn't matter)

I searched for poedit and see they have this screenshot, which looks pretty nice



So if the workflow looks something like

1. I add the plugins
2. I run it and get a PO file
3. I open the PO file in POedit, do my changes
4. Take the result and give it to the plugin

That would make translation efforts 10000x easier and without knowing how other localization plugins currently work, would use it immediately since it doesn't require me to go and mess around with my events, database, etc.

 

[Solar_Flare]

1. I add the plugins
2. I run it and get a PO file
3. I open the PO file in POedit, do my changes
4. Take the result and give it to the plugin

The workflow is pretty similar to that.

1. Add the plugin.
2. Configure it if necessary. Note: this includes letting my plugin know about any plugins you use that offer strings to be translated.
3. Run the game; at the title screen, press F9 and export a POT file
4. Open the POT file in POedit, make your changes, save as a PO file. The filename should be the langage code, eg fr.po.
5. Run the game; at the title screen, press F9 and import the PO file.

For a game with no plugins, step 2 is as simple as ticking or unticking a few boxes. With plugins, it could be as simple as adding a reference to the plugin, but I've found that most plugins require a bit more work to register them for localization. Some (at least the Kadokawa ones, I haven't seen any others yet that I can recall) even require you to modify the plugin itself. Most fall into a middle ground. All that said, localization will still work if you don't configure plugins - there just might be some strings missing from the POT file. So you could do it bit by bit, rather than right up front.

I've edited some screenshots into the opening post. I'm not sure how good they are for getting across how to use this plugin, but hopefully they will be helpful.

 

[Tsukihime]

The workflow is pretty similar to that.

1. Add the plugin.
2. Configure it if necessary. Note: this includes letting my plugin know about any plugins you use that offer strings to be translated.
3. Run the game; at the title screen, press F9 and export a POT file
4. Open the POT file in POedit, make your changes, save as a PO file. The filename should be the langage code, eg fr.po.
5. Run the game; at the title screen, press F9 and import the PO file.

For a game with no plugins, step 2 is as simple as ticking or unticking a few boxes. With plugins, it could be as simple as adding a reference to the plugin, but I've found that most plugins require a bit more work to register them for localization. Some (at least the Kadokawa ones, I haven't seen any others yet that I can recall) even require you to modify the plugin itself. Most fall into a middle ground. All that said, localization will still work if you don't configure plugins - there just might be some strings missing from the POT file. So you could do it bit by bit, rather than right up front.

I've edited some screenshots into the opening post. I'm not sure how good they are for getting across how to use this plugin, but hopefully they will be helpful.

Screenshots could be better, but ok lol

I downloaded the demo. Deleted the game.pot and fr.po that you had, then I started the game
Export English, created the Game.pot file

I opened in POedit, went and changed the "New Game" title string to "NEW GAME FR" and also changed actor "Harold" to "Harold FR", then saved as fr.po

Started the game, hit F9, import french.
But the changes weren't there. I didn't touch any configuration options though, I was thinking it would just work

 

[Solar_Flare]

I downloaded the demo. Deleted the game.pot and fr.po that you had, then I started the game
Export English, created the Game.pot file

I opened in POedit, went and changed the "New Game" title string to "NEW GAME FR" and also changed actor "Harold" to "Harold FR", then saved as fr.po

Started the game, hit F9, import french.
But the changes weren't there. I didn't touch any configuration options though, I was thinking it would just work

Okay, so there are two possible things I can think of that might have gone wrong.

1. The PO file needs to be saved in the same directory as the POT file.
2. You need to switch languages in the Options menu to see the imported strings.

I guess I should add some sort of notification if the first point is missed...

 

[Tsukihime]

It was the second step lol
In the options menu I just see a ????

And then I use that one, and go in the game, try to load menu, it crashes cause it couldn't find what "Harold" key was.

I think the problem is related to the ????, not reading the file properly?

 

[Solar_Flare]

It was the second step lol

I added another screenshot in the opening post to make it clearer that that's how you switch languages.

In the options menu I just see a ????

And then I use that one, and go in the game, try to load menu, it crashes cause it couldn't find what "Harold" key was.

I think the problem is related to the ????, not reading the file properly?

I'm not quite sure what you mean with the ????, but there was a bug in the version of the plugin bundled with the demo that resulted in the language being set to question marks if you used the arrow keys to switch languages. Pressing the OK key (space/enter) works, or you could substitute in the latest version of the plugin from the opening post.

If that's not what it was, and the issue is reproduceable with the latest version of the plugin, then could I see a screenshot of the issue and/or a stack trace of the error?

 

[Tsukihime]

I updated the JS files and it works now.

Looks good. That's how I would have approached localization as well: I didn't have to know anything about the project and I could translate stuff.

How does it handle changes to project data? Example scenario:

1. I generate a PO file today, send it to my translator
2. I continue to work on the game. I add some new items. Change some names. Update some quest descriptions. Add new events/maps. Delete some events maybe.
3. Now I want to get my translator an updated file.

What would be the workflow here? How can this plugin facilitate collaboration?

Another more simpler scenario

1. I generate a PO file
2. I go and do some translations
3. Then I add a new item, delete a couple items

The PO file will likely be not-sync'd. When I generate a new PO file, how does it merge?

 

[Solar_Flare]

The PO file will likely be not-sync'd. When I generate a new PO file, how does it merge?

That depends on the editor. For example, PoEdit has a function to merge an updated POT file into an existing PO file. Someone using the official gettext tools (which I believe are command-line tools) would run msgmerge to perform the merge. I don't know if other editors also provide the merge function but I would assume that at least some of them do.

So, if you've updated some strings, you just need to generate a new POT file and send it to the translator. They should be able to take it from there, especially if they're used to working with gettext.

 

[Tsukihime]

That depends on the editor. For example, PoEdit has a function to merge an updated POT file into an existing PO file. Someone using the official gettext tools (which I believe are command-line tools) would run msgmerge to perform the merge. I don't know if other editors also provide the merge function but I would assume that at least some of them do.

So, if you've updated some strings, you just need to generate a new POT file and send it to the translator. They should be able to take it from there, especially if they're used to working with gettext.

For the merges I'm thinking more about the POT itself.

Case 1: new item

So for example I generate POT that contains 5 weapons. I do my translations.
Then I go and create item 6 in my database.
Now I want to generate a POT again. The plugin should recognize that item 6 is new, and then update the POT with the new entry.

Case 2: edited item

I go and edit weapon 1's name and description.
Then I generate a POT again. Now, it should know that weapon 1's name and description has changed.

I would also like to be able to know that weapon 6 is new, and weapon 1 has changed, so that I can send the changelog to the translator.

 

[Solar_Flare]

So, perhaps I'll start by explaining in a bit more detail how the gettext system actually works (and how my plugin mimics it).

• The generated POT file contains all strings that the plugin was able to identify as probably needing to be translated. The T in POT stands for "template", so it's a "PO template file".
• When you open the POT in PoEdit, it makes a copy of it and renames it to fr.po (or whatever language you choose).
• Every time you export the POT from the game, it overwrites the POT file, but that shouldn't be an issue, because the POT file doesn't contain any actual translations, only the source strings.
• If you have an outdated PO file and an up-to-date POT file, then in PoEdit you can basically merge the two together. If there are new strings (a new item was added, for example), then it will add that new string to the PO file. If a string was removed, it marks it "fuzzy" (which causes PoEdit to show it in red).
• If a string was changed, that's no different from deleting the old one and adding the new one. However, if it detects that a new string is similar to an old one, then it'll automatically substitute the old string's translation for the new string (still marking it "fuzzy"). So, if you just corrected a typo or made some small change, then it automatically detects it; and if you completely rewrote the description, it would need a completely new translation anyway, so not detecting the change probably isn't a major issue.

I believe fuzzy strings are supposed to be only for translator reference and not actually used by the program; my importer currently doesn't respect that however.

Does all of that help?

I would also like to be able to know that weapon 6 is new, and weapon 1 has changed, so that I can send the changelog to the translator.

I think the best way to get a log of what changed that you can send to the translator would be to actually run a diff between the old and new POT files. (You could also diff the Language.json, but that contains both source and translated strings, so it might have a lot of spurious changes.) I'm not sure if such a log is really needed though, since the PO editor should have some way of prioritizing new or changed strings (ie, untranslated or fuzzy strings) for the translator.