Documentation/Programming Tutorials/Adding a new localization file format

From NeoAxis 3D Engine Wiki

Jump to: navigation, search
Go to higher level
Adding a new localization file format

Introduction

When dealing with project localization one usually faces problems with translating the interface text into various languages.

NeoAxis engine supports translating applications to another languages by means built-in functionality, which includes special file format for translations (.language files) and tools for editing. Files with translated texts are located in Data\Languages. Translation files for each language are located in corresponding sub directories, which names match the name of the language: Russian, French, and so on. Each separate translation file is designed for an appropriate application. For instance, Engine.language is a translation file for Game.exe, while ResourceEditor.language contains translated text for the Resource Editor.

The default language of the engine is English. Therefore, the translation files for the English language are not used. All text displayed in the user interface elements or elsewhere is taken either from GUI files, or directly from the code. Additionally, English text works as an identifier that allows to look up translation text for a different language.

Sometimes, however one may need to be able to download translation text from an alternative file format, since those who deal with localization prefer to prepare translation files in a different format. For those purposes there is an option to integrate custom translation files available. This can be performed by generating a translation TextBlock – in order to do this, the AlternativeLanguageFileLoaderHandler method of the LanguageManager class should be overridden and the text block should be filled with the translation text so, as if it was read from the usual .language translation file.

In the tutorial below you can see the example of a custom translation system based on XML.

Translation file format

The translation file format has the following structure:

Translation file format structure

In this case language it is the main block that keeps the translated data.

toolsUICulture - is the attribute that contains a CultureInfo identifier CultureInfo taken from the settings of OS Windows. This identifier is used for the proper toolkit performance.

textTranslations - block for the translated text. It contains data as a group of translations.

Translation Group - is a block for a group of translation. Let us look into the UISystem, translation group for instance, which contains translations of all elements of the user interface – this translation group block contains the attributes of the "name" = "text" form, where name - is some initial text in English, while text is its translation. Users can also add new translation groups for their own needs.

fileRedirections – is a block that remaps paths to corresponding translation files. It consists of the attributes of the form "source" = "destination", where source - is the path to the file used when the default language set, while destination is the path to the file used when any language of translation is selected.

Here is the sample of translation file for French:

language
{
	toolsUICulture = fr-FR
	textTranslations
	{
		UISystem
		{
			About ="A propos SDK"    
			"Engine loading..." ="Chargement du moteur..."
			Exit ="Quitter"
			"Gui Test" ="Test GUI"
			"Load/Save" ="Sauvegardes"
			Maps ="Cartes"
			"Movement controls and buttons for interaction can be located in the Options menu." =
"Les touches de contrôle peuvent être trouvées dans le menu des options."
			Multiplayer ="Multijoueur"
			Options ="Options"
			Profile" =" Profiler "	
			"Run Demo" ="Lancer démo"
			"Welcome to the" ="Bienvenue sur le"
		}
	}
	fileRedirections
	{
		"Sounds\\ButtonClick.ogg" = "Sounds\\Explode.ogg"
	}
}

Tutorial - Adding a new translation file format

This tutorial is going to show how to add support of a new translation file format.

The source of the loader for a new translation file format can be downloaded here.

The text of translation will be stored in an XML file that repeats the structure of a standard translation file format.

Example of an XML with engine translated into French::

<?xml version="1.0" encoding="UTF-8"?>
<language>
  <toolsUICulture>fr-FR</toolsUICulture>
  <group name="textTranslations">
    <group name="UISystem">  
      <phrase name="About" text="About" text="A propos SDK" />    
      <phrase name="Engine loading..." text="Chargement du moteur..." />
      <phrase name="Exit" text=" Quitter " />
      <phrase name="Gui Test" text=" Test GUI " />
      <phrase name="Load/Save" text=" Sauvegardes " />
      <phrase name="Maps" text=" Cartes " />
      <phrase name="Movement controls and buttons for interaction can be located in the Options menu." text="Movement controls and buttons for interaction can be located in the Options menu." text="Les touches de contrôle peuvent être trouvées dans le menu des options."/>
      <phrase name="Multiplayer" text=" Multijoueur " />
      <phrase name="Options" text=" Options " />
      <phrase name="Profiler" text=" Profiler " />
      <phrase name="Run Demo" text="Lancer démo" />
      <phrase name="Welcome to the" text=" Bienvenue sur le " />
    </group>
  </group>
  <group name="fileRedirections">
    <redirection name="Sounds\ButtonClick.ogg" text="Sounds\Explode.ogg" />
  </group>
</language>

Let us start adding the new translation file format to the engine by changing the "Main2" method in Game.exe.

static void Main2()
{
	if( !VirtualFileSystem.Init( "user:Logs/Game.log", true, null, null, null ) )
		return;
 
	EngineApp.ConfigName = "user:Configs/Game.config";
 
	//The line to add
	LanguageManager.AlternativeLanguageFileLoaderHandler +=
		LanguageManager_AlternativeLanguageFileLoaderHandler;
 
	...
}

The LanguageManager.AlternativeLanguageFileLoaderHandler delegate is designed specially for remapping, in case the user needs to create their own translation file format. In this case we simply let the LanguageManager_AlternativeLanguageFileLoaderHandler method be aquired by this new format. Below goes the code it contains:

static void LanguageManager_AlternativeLanguageFileLoaderHandler(ref TextBlock generatedBlock, ref bool error)
{
	string language = LanguageManager.Instance.Language;
	string fileName = Path.Combine(LanguageManager.LanguagesDirectory,
		Path.Combine(language, "Engine.xml"));
 
	XmlDocument xmlDocument = new XmlDocument();
	xmlDocument.Load(VirtualFileSystem.GetRealPathByVirtual(fileName));
 
	generatedBlock = new TextBlock();
	TextBlock languageBlock = generatedBlock.AddChild("language");
 
	ParseLanguageBlockXMLNode(languageBlock, xmlDocument, fileName, ref error);
}

Our method accepts two input parameters:

generatedBlock - is a text blockthat should be filled in according to the standard translation file format.

error - a flag that alerts errors. If an error occurs during the process of translation file loading, this parameter should have the value True assigned to it.

Let us now look at the code in the method body. At first, there is a path to the translation file (fileName) required to be formed by joining the path to the directory with translations (LanguageManager.LanguagesDirectory), the name of the selected language (LanguageManager.Instance.Language) and the name of file with translations (Engine.xml).

string language = LanguageManager.Instance.Language;
string fileName = Path.Combine(LanguageManager.LanguagesDirectory, Path.Combine(language, "Engine.xml"));

At second, our XML-file should be loaded.

XmlDocument xmlDocument = new XmlDocument();
xmlDocument.Load(VirtualFileSystem.GetRealPathByVirtual(fileName));

At third, a new text block should be created with a new translation block - language - added to it.

generatedBlock = new TextBlock();
TextBlock languageBlock = generatedBlock.AddChild("language");

Finally we call the XML-file reading method - ParseLanguageBlockXMLNode.

ParseLanguageBlockXMLNode(languageBlock, xmlDocument, fileName, ref error);

Let us proceed to the ParseLanguageBlockXMLNode method overview.

static void ParseLanguageBlockXMLNode(TextBlock languageBlock, XmlDocument xmlDocument, 
	string fileName, ref bool error)
{
	foreach (XmlNode node in xmlDocument.DocumentElement.ChildNodes)
	{
		if (node.Name == "toolsUICulture")
		{
			languageBlock.SetAttribute("toolsUICulture", node.InnerText);
		}
 
		if (node.Name == "group" && node.Attributes["name"].Value == "textTranslations")
		{
			TextBlock textTranslationsBlock = languageBlock.AddChild("textTranslations");
			ParseTextTranslationsNodeRecursive(node, textTranslationsBlock);
		}
 
		if (node.Name == "group" && node.Attributes["name"].Value == "fileRedirections")
		{
			TextBlock fileRedirectionsBlock = languageBlock.AddChild("fileRedirections");
			ParseFileRedirectionsNode(node, fileRedirectionsBlock);
		}
	}
 
	if (!languageBlock.IsAttributeExist("toolsUICulture"))
	{
		Log.Error("\"toolsUICulture\" node is not founded in {0}.", fileName);
		error = true;
	}
}

The method starts from looping all the children of the main node through the XML-file - <language>.

foreach (XmlNode node in xmlDocument.DocumentElement.ChildNodes)
{
	...
}

The method starts from looping all the children of the main node through the XML-file toolsUICulture, then the node value acquires the same name attribute of the text block.

if (node.Name == "toolsUICulture")
{
	languageBlock.SetAttribute("toolsUICulture", node.InnerText);
}

If the node name is group, while the value of attribute name is textTranslations, that means the node contains translated text. Thus, in order to load translation, we call the ParseTextTranslationsNodeRecursive method.

if (node.Name == "group" && node.Attributes["name"].Value == "textTranslations")
{
	TextBlock textTranslationsBlock = languageBlock.AddChild("textTranslations");
	ParseTextTranslationsNodeRecursive(node, textTranslationsBlock);
}

Finally, if the node name is group, while the value of attribute name is fileRedirections, then this node contains redirected file paths. Those are loaded with the help of method ParseFileRedirectionsNode.

if (node.Name == "group" && node.Attributes["name"].Value == "fileRedirections")
{
	TextBlock fileRedirectionsBlock = languageBlock.AddChild("fileRedirections");
	ParseFileRedirectionsNode(node, fileRedirectionsBlock);
}

After completing the loop, the attribute toolsUICulture should be checked for being set for the given text block. Otherwise, it has to be reported as an error.

if (!languageBlock.IsAttributeExist("toolsUICulture"))
{
	Log.Error("\"toolsUICulture\" node is not founded in {0}.", fileName);
	error = true;
}

Now let us take a look at the method that reads the text of translation.

static void ParseTextTranslationsNodeRecursive(XmlNode textTranslationsNode, TextBlock textBlock)
{
	if (textTranslationsNode.ChildNodes != null)
	{
		foreach (XmlNode childNode in textTranslationsNode.ChildNodes)
		{
			if (childNode.Name == "group")
			{
				string groupName = childNode.Attributes["name"].Value;
				TextBlock groupBlock = textBlock.AddChild(groupName);
				ParseTextTranslationsNodeRecursive(childNode, groupBlock);
			}
 
			if (childNode.Name == "phrase")
			{
				string phraseName = childNode.Attributes["name"].Value;
				string text = childNode.Attributes["text"].Value;
				textBlock.SetAttribute(phraseName, text);
			}
		}
	}
}

First, we need to make sure that the current node has children.

if (textTranslationsNode.ChildNodes != null)
{
	...
}

Next, we loop through all the node's children .

foreach (XmlNode childNode in textTranslationsNode.ChildNodes)
{
	...
}

And analyze names of each of them. If the node name is group, that node contains the translation group. In this case we extract its name from the name node attribute. Then a text block should be created for the translation group and the translation group should be loaded by a recursive call of the ParseTextTranslationsNodeRecursive method.

if (childNode.Name == "group")
{
	string groupName = childNode.Attributes["name"].Value;
	TextBlock groupBlock = textBlock.AddChild(groupName);
	ParseTextTranslationsNodeRecursive(childNode, groupBlock);
}

If the node name is phrase, then the node contains the text of translation. The source then is extracted from the name attribute of the node, while the translation is taken from the attribute text. After that we set the translation as an attribute of the text block.

if (childNode.Name == "phrase")
{
	string phraseName = childNode.Attributes["name"].Value;
	string text = childNode.Attributes["text"].Value;
	textBlock.SetAttribute(phraseName, text);
}

Now let us look at the ParseFileRedirectionsNode method, that loads file paths redirections.

static void ParseFileRedirectionsNode(XmlNode redirectionNode, TextBlock textBlock)
{
	if (redirectionNode.ChildNodes != null)
	{
		foreach (XmlNode childNode in redirectionNode.ChildNodes)
		{
			string name = childNode.Attributes["name"].Value;
			string text = childNode.Attributes["text"].Value;
			textBlock.SetAttribute(name, text);
		}
	}
}

At first we check whether the node in question owns any children.

if (redirectionNode.ChildNodes != null)
{
	...
}

Next, we get all the node's children looped through.

foreach (XmlNode childNode in redirectionNode.ChildNodes)
{
	...
}

The attribute name is exctracted from each node, since its value is the path to the source file. Remapped paths are extracted from the attribute text. Then both those parameters get written into the text block attributes.

string name = childNode.Attributes["name"].Value;
string text = childNode.Attributes["text"].Value;
textBlock.SetAttribute(name, text);

We are now done with the code for adding the new translation file format so all we need to do is just to copy the Engine.xml file to a folder Data\Languages\French and set language to French in the Configurator. In order to test the translation let us compile and run Game.exe.

NewLocalizationFileFormat02.jpg

This brings our tutorial to the end. It helped us learn how to add a new engine file format translations.

New translation format source loader can be downloaded here.