i18n: move the scripts to the top-level 'i18n' directory
Also add a README.md to python3-flightgear explaining how to use the PYTHONPATH environment variable or a .pth file in order to run the Python scripts in FGMeta, and pointing to the top-level directories 'catalog' and 'i18n'.
This commit is contained in:
Florent Rougon
@@ -1,138 +0,0 @@
|
||||
-*- coding: utf-8 -*-
|
||||
|
||||
Quick start for the localization (l10n) scripts
|
||||
===============================================
|
||||
|
||||
The following assumes that all of these are in present in
|
||||
$FG_ROOT/Translations:
|
||||
- the default translation (default/*.xml);
|
||||
- the legacy FlightGear XML localization files (<language_code>/*.xml);
|
||||
- except for 'fg-convert-translation-files' which creates them, existing
|
||||
XLIFF 1.2 files (<language_code>/FlightGear-nonQt.xlf).
|
||||
|
||||
Note: the legacy FlightGear XML localization files are only needed by
|
||||
'fg-convert-translation-files' when migrating to the XLIFF format. The
|
||||
other scripts only need the default translation and obviously, for
|
||||
'fg-update-translation-files', the current XLIFF files[1].
|
||||
|
||||
|
||||
Creating XLIFF files from existing FlightGear legacy XML translation files
|
||||
--------------------------------------------------------------------------
|
||||
|
||||
To get the initial XLIFF files (generated from the default translation in
|
||||
$FG_ROOT/Translations/default as well as the legacy FlightGear XML
|
||||
localization files in $FG_ROOT/Translations/<language_code>):
|
||||
|
||||
languages="de en_US es fr it nl pl pt zh_CN"
|
||||
|
||||
# Your shell must expand $languages as several words for the following
|
||||
# commands to work. POSIX shell does that, Bash too apparently, but not Zsh
|
||||
# (by default). In Zsh, you can use $=languages or ${=languages} to ensure
|
||||
# the expansion uses word splitting.
|
||||
fg-convert-translation-files --transl-dir="$FG_ROOT/Translations" $languages
|
||||
|
||||
# Add strings found in the default translation but missing in the legacy FG
|
||||
# XML l10n files
|
||||
fg-update-translation-files --transl-dir="$FG_ROOT/Translations" \
|
||||
merge-new-master $languages
|
||||
|
||||
Note: you may omit $languages in the fg-update-translation-files command if
|
||||
you want to autodetect the FlightGear-nonQt.xlf files present in
|
||||
$FG_ROOT/Translations.
|
||||
|
||||
Updating XLIFF files to reflect changes in the default translation
|
||||
------------------------------------------------------------------
|
||||
|
||||
When master strings[2] have changed (in a large sense, i.e.: strings added,
|
||||
modified or removed, or categories added or removed[3]):
|
||||
|
||||
fg-update-translation-files --transl-dir="$FG_ROOT/Translations" \
|
||||
merge-new-master $languages
|
||||
|
||||
Note: you may omit $languages in this command if you want to autodetect the
|
||||
FlightGear-nonQt.xlf files present in $FG_ROOT/Translations.
|
||||
|
||||
Updating XLIFF files to mark or remove obsolete translated strings
|
||||
------------------------------------------------------------------
|
||||
|
||||
To remove unused translated strings (not to be done too often in my opinion):
|
||||
|
||||
fg-update-translation-files --transl-dir="$FG_ROOT/Translations" \
|
||||
remove-unused $languages
|
||||
|
||||
Notes:
|
||||
|
||||
- You may omit $languages in this command if you want to autodetect the
|
||||
FlightGear-nonQt.xlf files present in $FG_ROOT/Translations.
|
||||
|
||||
- It is possible to replace 'remove-unused' with 'mark-unused' to just mark
|
||||
the strings as not-to-be-translated; however, 'merge-new-master' presented
|
||||
above already does that.
|
||||
|
||||
Merging contents from an XLIFF file into another one
|
||||
----------------------------------------------------
|
||||
|
||||
Suppose a translator has been working on a particular translation file, and
|
||||
meanwhile the official XLIFF file for this translation has been updated in
|
||||
FGData (new translatable strings added, obsolete strings marked or removed,
|
||||
etc.). In such a case, 'fg-merge-xliff-into-xliff' can be used to merge the
|
||||
translator's work into the project file. Essentially, this means that for all
|
||||
strings that have the same source text, plural status, number of plural forms
|
||||
and of course target language, the target texts, “approved” status and
|
||||
translator comments will be taken from the first file passed in the following
|
||||
command:
|
||||
|
||||
fg-merge-xliff-into-xliff TRANSLATOR_FILE PROJECT_FILE
|
||||
|
||||
Used like this, PROJECT_FILE will be updated with data from TRANSLATOR_FILE.
|
||||
If you don't want to modify PROJECT_FILE, use the -o (--output) option. If '-'
|
||||
is passed as argument to this option, then the result is written to the
|
||||
standard output.
|
||||
|
||||
Creating skeleton XLIFF files for new translations
|
||||
--------------------------------------------------
|
||||
|
||||
To create skeleton translations for new languages (e.g., for fr_BE, en_AU and
|
||||
ca):
|
||||
|
||||
1) Check (add if necessary) that flightgear/meta/i18n.py knows the plural
|
||||
forms used in the new languages. This is done by editing PLURAL_FORMS
|
||||
towards the top of this i18n.py file (very easy). If the existing entry
|
||||
for, e.g., "zh" is sufficient for zh_TW or zh_HK, just let "zh" handle
|
||||
them: it will be tried as fallback if there is no perfect match on
|
||||
language and territory.
|
||||
|
||||
2) Run a command such as:
|
||||
|
||||
fg-new-translations --transl-dir="$FG_ROOT/Translations" fr_BE en_AU ca
|
||||
|
||||
(if you do this for only one language at a time, you can use the -o
|
||||
option to precisely control where the output goes, otherwise
|
||||
fg-new-translations chooses an appropriate place based on the value
|
||||
specified for --transl-dir)
|
||||
|
||||
Getting more information on the scripts
|
||||
---------------------------------------
|
||||
|
||||
fg-convert-translation-files, fg-update-translation-files,
|
||||
fg-merge-xliff-into-xliff and fg-new-translations all support the --help
|
||||
option for more detailed information.
|
||||
|
||||
|
||||
Footnotes
|
||||
---------
|
||||
|
||||
[1] Except for the fg-merge-xliff-into-xliff script, which doesn't have any
|
||||
of these requirements.
|
||||
|
||||
[2] Strings in the default translation.
|
||||
|
||||
[3] Only empty categories are removed by this command. An obsolete category
|
||||
can be made empty by manual editing (easy, just locate the right
|
||||
<group>) or this way:
|
||||
|
||||
fg-update-translation-files --transl-dir=... mark-unused
|
||||
fg-update-translation-files --transl-dir=... remove-unused
|
||||
|
||||
(note that this will remove *all* strings marked as unused in the first
|
||||
step, not only those in some particular category!)
|
||||
48
python3-flightgear/README.md
Normal file
48
python3-flightgear/README.md
Normal file
@@ -0,0 +1,48 @@
|
||||
Python code for FlightGear “meta” work
|
||||
======================================
|
||||
|
||||
The `flightgear` directory contains FlightGear-specific Python 3 modules.
|
||||
These modules are mostly of interest to FlightGear developers.
|
||||
|
||||
|
||||
Telling your Python interpreter how to access the modules
|
||||
---------------------------------------------------------
|
||||
|
||||
In order to run most of the Python scripts in FGMeta, your Python 3
|
||||
installation must have the `/path/to/fgmeta/python3-flightgear` directory in
|
||||
its `sys.path`. One way to do this is to use something like the following in
|
||||
your shell setup:
|
||||
|
||||
export PYTHONPATH="/path/to/fgmeta/python3-flightgear"
|
||||
|
||||
This example uses Bourne-style syntax; adjust for your particular shell.
|
||||
Several directories may be added this way using a colon separator on Unix, and
|
||||
presumably a semicolon on Windows.
|
||||
|
||||
An alternative to setting `PYTHONPATH` is to add .pth files in special
|
||||
directories of your Python installation(s). For instance, you can create a
|
||||
file, say, `FlightGear-FGMeta.pth`, containing a single line (with no space at
|
||||
the beginning):
|
||||
|
||||
/path/to/fgmeta/python3-flightgear
|
||||
|
||||
If you want the modules present in `/path/to/fgmeta/python3-flightgear` to be
|
||||
accessible to a particular Python interpreter (say, a Python 3.8), simply put
|
||||
the `.pth` file in `/path/to/python-install-dir/lib/python3.8/site-packages/`.
|
||||
This can even be a virtual environment if you want. For the system Python
|
||||
interpreters on Debian, you can put the `.pth` file in, e.g,
|
||||
`/usr/local/lib/python3.8/dist-packages/`. Note that you may add more lines to
|
||||
a `.pth` file in case you want to add other paths to the Python interpreter's
|
||||
`sys.path`.
|
||||
|
||||
|
||||
The scripts
|
||||
-----------
|
||||
|
||||
Once you've done the above setup, the Python 3 scripts in FGMeta should run
|
||||
fine. This concerns in particular scripts located in the following top-level
|
||||
directories of FGMeta:
|
||||
|
||||
catalog Generation of aircraft catalogs
|
||||
i18n Management of translations in FlightGear (i18n stands for
|
||||
“internationalization”)
|
||||
@@ -1,181 +0,0 @@
|
||||
#! /usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# fg-convert-translation-files --- Convert FlightGear's translation files
|
||||
# Copyright (C) 2017 Florent Rougon
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation; either version 2 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along
|
||||
# with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import locale
|
||||
import os
|
||||
import sys
|
||||
|
||||
import flightgear.meta.logging
|
||||
import flightgear.meta.i18n as fg_i18n
|
||||
|
||||
|
||||
PROGNAME = os.path.basename(sys.argv[0])
|
||||
|
||||
# Only messages with severity >= info will be printed to the terminal (it's
|
||||
# possible to also log all messages to a file regardless of their level, see
|
||||
# the Logger class). Of course, there is also the standard logging module...
|
||||
logger = flightgear.meta.logging.Logger(
|
||||
progname=PROGNAME,
|
||||
logLevel=flightgear.meta.logging.LogLevel.info,
|
||||
defaultOutputStream=sys.stderr)
|
||||
|
||||
debug = logger.debug
|
||||
info = logger.info
|
||||
notice = logger.notice
|
||||
warning = logger.warning
|
||||
error = logger.error
|
||||
critical = logger.critical
|
||||
|
||||
|
||||
# We could use Translation.__str__(): not as readable (for now) but more
|
||||
# accurate on metadata
|
||||
def printPlainText(l10nResPoolMgr, translations):
|
||||
"""Print output suitable for a quick review (by the programmer)."""
|
||||
firstLang = True
|
||||
|
||||
for langCode, (transl, nbWhitespacePbs) in translations.items():
|
||||
# 'transl' is a Translation instance
|
||||
if firstLang:
|
||||
firstLang = False
|
||||
else:
|
||||
print()
|
||||
|
||||
print("-" * 78 + "\n" + langCode + "\n" + "-" * 78)
|
||||
print("\nNumber of leading and/or trailing whitespace problems: {}"
|
||||
.format(nbWhitespacePbs))
|
||||
|
||||
for cat in transl:
|
||||
print("\nCategory: {cat}\n{underline}".format(
|
||||
cat=cat, underline="~"*(len("Category: ") + len(cat))))
|
||||
t = transl[cat]
|
||||
|
||||
for tid, translUnit in sorted(t.items()):
|
||||
# - Using '{master!r}' and '{transl!r}' prints stuff such as
|
||||
# \xa0 for nobreak spaces, which can lead to the erroneous
|
||||
# conclusion that there was an encoding problem.
|
||||
# - Only printing the first target text here (no plural forms)
|
||||
print("\n{id}\n '{sourceText}'\n '{targetText}'"
|
||||
.format(id=tid.id(), sourceText=translUnit.sourceText,
|
||||
targetText=translUnit.targetTexts[0]))
|
||||
|
||||
|
||||
def writeXliff(l10nResPoolMgr, translations):
|
||||
formatHandler = fg_i18n.XliffFormatHandler()
|
||||
|
||||
for langCode, translData in translations.items():
|
||||
translation = translData.transl # Translation instance
|
||||
|
||||
if params.output_dir is None:
|
||||
# Use default locations for the written xliff files
|
||||
l10nResPoolMgr.writeTranslation(formatHandler, translation)
|
||||
else:
|
||||
basename = "{}-{}.{}".format(
|
||||
formatHandler.defaultFileStem(langCode),
|
||||
langCode,
|
||||
formatHandler.standardExtension)
|
||||
filePath = os.path.join(params.output_dir, basename)
|
||||
formatHandler.writeTranslation(translation, filePath)
|
||||
|
||||
|
||||
def processCommandLine():
|
||||
params = argparse.Namespace()
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
usage="""\
|
||||
%(prog)s [OPTION ...] LANGUAGE_CODE...
|
||||
Convert FlightGear's old XML translation files into other formats.""",
|
||||
description="""\
|
||||
Most notably, XLIFF format can be chosen for output. The script performs
|
||||
a few automated checks on the input files too.""",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
# I want --help but not -h (it might be useful for something else)
|
||||
add_help=False)
|
||||
|
||||
parser.add_argument("-t", "--transl-dir",
|
||||
help="""\
|
||||
directory containing all translation subdirs (such as
|
||||
{default!r}, 'en_GB', 'fr_FR', 'de', 'it'...). This
|
||||
"option" MUST be specified.""".format(
|
||||
default=fg_i18n.DEFAULT_LANG_DIR))
|
||||
parser.add_argument("lang_code", metavar="LANGUAGE_CODE", nargs="+",
|
||||
help="""\
|
||||
codes of languages to read translations for (don't
|
||||
specify {default!r} this way, it is special and not a
|
||||
language code)"""
|
||||
.format(default=fg_i18n.DEFAULT_LANG_DIR))
|
||||
parser.add_argument("-o", "--output-dir",
|
||||
help="""\
|
||||
output directory for written XLIFF files
|
||||
(default: for each output file, use a suitable location
|
||||
under TRANSL_DIR)""")
|
||||
parser.add_argument("-f", "--output-format", default="xliff",
|
||||
choices=("xliff", "text"), help="""\
|
||||
format to use for the output files""")
|
||||
parser.add_argument("--help", action="help",
|
||||
help="display this message and exit")
|
||||
|
||||
params = parser.parse_args(namespace=params)
|
||||
|
||||
if params.transl_dir is None:
|
||||
error("--transl-dir must be given, aborting")
|
||||
sys.exit(1)
|
||||
|
||||
return params
|
||||
|
||||
|
||||
def main():
|
||||
global params
|
||||
|
||||
locale.setlocale(locale.LC_ALL, '')
|
||||
params = processCommandLine()
|
||||
|
||||
l10nResPoolMgr = fg_i18n.L10NResourcePoolManager(params.transl_dir, logger)
|
||||
# English version of all translatable strings
|
||||
masterTransl, nbWhitespaceProblemsInMaster = \
|
||||
l10nResPoolMgr.readFgMasterTranslation()
|
||||
translations = collections.OrderedDict()
|
||||
|
||||
# Sort elements of 'translations' according to language code (= the keys)
|
||||
for langCode in sorted(params.lang_code):
|
||||
translationData = l10nResPoolMgr.readFgTranslation(masterTransl,
|
||||
langCode)
|
||||
translations[translationData.transl.targetLanguage] = translationData
|
||||
|
||||
if params.output_format == "xliff":
|
||||
writeFunc = writeXliff # write to files
|
||||
elif params.output_format == "text":
|
||||
writeFunc = printPlainText # print to stdout
|
||||
else:
|
||||
assert False, \
|
||||
"Unexpected output format: '{}'".format(params.output_format)
|
||||
|
||||
writeFunc(l10nResPoolMgr, translations)
|
||||
|
||||
nbWhitespaceProblemsInTransl = sum(
|
||||
(translData.nbWhitespacePbs for translData in translations.values() ))
|
||||
info("total number of leading and/or trailing whitespace problems: {}"
|
||||
.format(nbWhitespaceProblemsInMaster + nbWhitespaceProblemsInTransl))
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
@@ -1,129 +0,0 @@
|
||||
#! /usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright (C) 2020 James Turner
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation; either version 2 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along
|
||||
# with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
import argparse
|
||||
import locale
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import textwrap
|
||||
import lxml.etree as ET
|
||||
|
||||
import flightgear.meta.strutils as strutils
|
||||
from flightgear.meta import sgprops
|
||||
|
||||
PROGNAME = os.path.basename(sys.argv[0])
|
||||
|
||||
|
||||
def processCommandLine():
|
||||
params = argparse.Namespace()
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
usage="""\
|
||||
%(prog)s [OPTION ...] FGDATA
|
||||
Copy weather scenario descriptions to the default translation XML""",
|
||||
description="""\
|
||||
""",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
# I want --help but not -h (it might be useful for something else)
|
||||
add_help=False)
|
||||
|
||||
parser.add_argument("fgdata", metavar="FGDATA",
|
||||
help="""\
|
||||
location of FGData""")
|
||||
parser.add_argument("--help", action="help",
|
||||
help="display this message and exit")
|
||||
|
||||
return parser.parse_args(namespace=params)
|
||||
|
||||
|
||||
def insertInitialComment(root_elt, rel_input_path):
|
||||
"""Insert an XML comment before element 'root_elt'."""
|
||||
s = textwrap.dedent("""\
|
||||
This file was automatically generated from {input_file} using the
|
||||
{progname} script from FGMeta. Modifications should be done either in
|
||||
{input_file} or in that script.""".format(
|
||||
progname=PROGNAME,
|
||||
input_file=os.path.join("$FG_ROOT", rel_input_path)))
|
||||
filled_paragraph = textwrap.fill(s, width=79)
|
||||
comment_pseudo_element = ET.Comment(
|
||||
" !!! Don't modify this file manually. !!!\n" + filled_paragraph + " ")
|
||||
root_elt.addprevious(comment_pseudo_element)
|
||||
|
||||
|
||||
def stringifyChildValue(node, child):
|
||||
# The 'or ""' is needed because an empty node is returned as None!
|
||||
return strutils.simplify(node.getValue(child, "") or "")
|
||||
|
||||
|
||||
def makeXmlLeaf(name, text):
|
||||
"""Create an XML element with text contents."""
|
||||
leaf = ET.Element(name)
|
||||
leaf.text = '' if text is None else str(text)
|
||||
return leaf
|
||||
|
||||
|
||||
def copyWeatherScenarios(fgdata):
|
||||
rel_input_path = os.path.join("Environment", "environment.xml")
|
||||
environment_node = sgprops.readProps(os.path.join(fgdata, rel_input_path))
|
||||
scenarios = environment_node.getChild('weather-scenarios')
|
||||
|
||||
root = ET.Element("PropertyList")
|
||||
insertInitialComment(root, rel_input_path)
|
||||
|
||||
for scen_idx, scen_node in enumerate(scenarios.getChildren("scenario")):
|
||||
scenarioId = scen_node.getValue("id", None)
|
||||
if (not scenarioId) or scenarioId != strutils.simplify(scenarioId):
|
||||
sys.exit(
|
||||
"{prg}: 'scenario' element number {i} has a missing, empty "
|
||||
"or suspiciously-formatted 'id' child; aborting.".format(
|
||||
prg=PROGNAME, i=scen_idx+1))
|
||||
|
||||
name = stringifyChildValue(scen_node, "name")
|
||||
desc = stringifyChildValue(scen_node, "description")
|
||||
|
||||
if not (name and desc):
|
||||
sys.exit(
|
||||
"{prg}: scenario '{scen}' has an empty or missing name or "
|
||||
"description after string simplification; aborting.".format(
|
||||
prg=PROGNAME, scen=scenarioId))
|
||||
|
||||
root.append(makeXmlLeaf(scenarioId + "-name", name))
|
||||
root.append(makeXmlLeaf(scenarioId + "-desc", desc))
|
||||
|
||||
default_trans_file = os.path.join(fgdata, "Translations", "default",
|
||||
"weather-scenarios.xml")
|
||||
|
||||
doc = ET.ElementTree(root)
|
||||
doc.write(default_trans_file, encoding='utf-8',
|
||||
xml_declaration=True, pretty_print=True)
|
||||
|
||||
|
||||
def main():
|
||||
global params
|
||||
|
||||
locale.setlocale(locale.LC_ALL, '')
|
||||
params = processCommandLine()
|
||||
|
||||
copyWeatherScenarios(params.fgdata)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
@@ -1,123 +0,0 @@
|
||||
#! /usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# fg-merge-xliff-into-xliff --- Merge translations from one XLIFF file into
|
||||
# another one
|
||||
# Copyright (C) 2017 Florent Rougon
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation; either version 2 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along
|
||||
# with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
import argparse
|
||||
import locale
|
||||
import os
|
||||
import sys
|
||||
|
||||
import flightgear.meta.logging
|
||||
import flightgear.meta.i18n as fg_i18n
|
||||
|
||||
|
||||
PROGNAME = os.path.basename(sys.argv[0])
|
||||
|
||||
# Only messages with severity >= info will be printed to the terminal (it's
|
||||
# possible to also log all messages to a file regardless of their level, see
|
||||
# the Logger class). Of course, there is also the standard logging module...
|
||||
logger = flightgear.meta.logging.Logger(
|
||||
progname=PROGNAME,
|
||||
logLevel=flightgear.meta.logging.LogLevel.info,
|
||||
defaultOutputStream=sys.stderr)
|
||||
|
||||
|
||||
def processCommandLine():
|
||||
params = argparse.Namespace()
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
usage="""\
|
||||
%(prog)s [OPTION ...] SOURCE INTO
|
||||
Merge strings from a FlightGear XLIFF localization file into another one.""",
|
||||
description="""\
|
||||
This program merges a FlightGear XLIFF localization file into another one.
|
||||
This means that every translatable string that:
|
||||
|
||||
(1) exists in both SOURCE and INTO;
|
||||
|
||||
(2) has the same target language, source text, plural status and number of
|
||||
plural forms in SOURCE and in INTO;
|
||||
|
||||
is updated from SOURCE, i.e.: the target texts, 'approved' status and
|
||||
translator comments are copied from SOURCE.
|
||||
|
||||
The result is written to INTO unless the -o (--output) option is given.
|
||||
|
||||
Note that this program is different from fg-update-translation-files's
|
||||
'merge-new-master' command, which is for updating an XLIFF file according to
|
||||
the default translation ("master").
|
||||
|
||||
Expected use case: suppose that a translator is working on a translation
|
||||
file, and meanwhile the official XLIFF file for this translation is updated
|
||||
in the project repository (new translatable strings added, obsolete strings
|
||||
marked or removed, etc.). This program can then be used to merge the
|
||||
translator work into the project file for all strings for which it makes
|
||||
sense (source text unchanged, same plural status, etc.).""",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
# I want --help but not -h (it might be useful for something else)
|
||||
add_help=False)
|
||||
|
||||
parser.add_argument("source", metavar="SOURCE",
|
||||
help="""\
|
||||
input XLIFF file; read updated translated strings
|
||||
from this file""")
|
||||
parser.add_argument("into", metavar="INTO",
|
||||
help="""\
|
||||
XLIFF file to compare to SOURCE in order to decide
|
||||
which translated strings to update; unless the -o
|
||||
option is used, updated strings are written to this
|
||||
file""")
|
||||
parser.add_argument("-o", "--output",
|
||||
help="""\
|
||||
write the XLIFF merged output to OUTPUT instead of
|
||||
INTO. When this option is used, INTO is read but not
|
||||
modified. If OUTPUT is '-', write the XLIFF merged
|
||||
output to the standard output.""")
|
||||
parser.add_argument("--help", action="help",
|
||||
help="display this message and exit")
|
||||
|
||||
return parser.parse_args(namespace=params)
|
||||
|
||||
|
||||
def mergeXliffIntoXliff(source, into, output):
|
||||
formatHandler = fg_i18n.XliffFormatHandler()
|
||||
|
||||
srcTransl = formatHandler.readTranslation(source)
|
||||
transl = formatHandler.readTranslation(into)
|
||||
# Merge 'srcTransl' into 'transl'
|
||||
transl.mergeNonMasterTransl(srcTransl, logger=logger)
|
||||
|
||||
# File path, or '-' for the standard output
|
||||
outputFile = into if output is None else output
|
||||
formatHandler.writeTranslation(transl, outputFile)
|
||||
|
||||
|
||||
def main():
|
||||
global params
|
||||
|
||||
locale.setlocale(locale.LC_ALL, '')
|
||||
params = processCommandLine()
|
||||
|
||||
mergeXliffIntoXliff(params.source, params.into, params.output)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
@@ -1,120 +0,0 @@
|
||||
#! /usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# fg-new-translations --- Create new translations for FlightGear
|
||||
# Copyright (C) 2017 Florent Rougon
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation; either version 2 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along
|
||||
# with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
import argparse
|
||||
import collections
|
||||
import locale
|
||||
import os
|
||||
import sys
|
||||
|
||||
import flightgear.meta.logging
|
||||
import flightgear.meta.i18n as fg_i18n
|
||||
|
||||
|
||||
PROGNAME = os.path.basename(sys.argv[0])
|
||||
|
||||
# Only messages with severity >= info will be printed to the terminal (it's
|
||||
# possible to also log all messages to a file regardless of their level, see
|
||||
# the Logger class). Of course, there is also the standard logging module...
|
||||
logger = flightgear.meta.logging.Logger(
|
||||
progname=PROGNAME,
|
||||
logLevel=flightgear.meta.logging.LogLevel.info,
|
||||
defaultOutputStream=sys.stderr)
|
||||
|
||||
|
||||
def processCommandLine():
|
||||
params = argparse.Namespace()
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
usage="""\
|
||||
%(prog)s [OPTION ...] LANGUAGE_CODE...
|
||||
Write the skeleton of XLIFF translation files.""",
|
||||
description="""\
|
||||
This program writes XLIFF translation files with the strings to translate
|
||||
for the specified languages (target strings are empty). This is what you need
|
||||
to start a translation for a new language.""",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
# I want --help but not -h (it might be useful for something else)
|
||||
add_help=False)
|
||||
|
||||
parser.add_argument("-t", "--transl-dir",
|
||||
help="""\
|
||||
directory containing all translation subdirs (such as
|
||||
{default!r}, 'en_GB', 'fr_FR', 'de', 'it'...). This
|
||||
"option" MUST be specified.""".format(
|
||||
default=fg_i18n.DEFAULT_LANG_DIR))
|
||||
parser.add_argument("lang_code", metavar="LANGUAGE_CODE", nargs="+",
|
||||
help="""\
|
||||
codes of languages to create translations for (e.g., fr,
|
||||
fr_BE, en_GB, it, es_ES...)""")
|
||||
parser.add_argument("-o", "--output-file",
|
||||
help="""\
|
||||
where to write the output to (use '-' for standard
|
||||
output); if not specified, a suitable file under
|
||||
TRANSL_DIR will be chosen for each LANGUAGE_CODE.
|
||||
Note: this option can only be given when exactly one
|
||||
LANGUAGE_CODE has been specified on the command
|
||||
line (it doesn't make sense otherwise).""")
|
||||
parser.add_argument("--output-format", default="xliff",
|
||||
choices=fg_i18n.FORMAT_HANDLERS_NAMES,
|
||||
help="format to use for the output files")
|
||||
parser.add_argument("--help", action="help",
|
||||
help="display this message and exit")
|
||||
|
||||
params = parser.parse_args(namespace=params)
|
||||
|
||||
if params.transl_dir is None:
|
||||
logger.error("--transl-dir must be given, aborting")
|
||||
sys.exit(1)
|
||||
|
||||
if params.output_file is not None and len(params.lang_code) > 1:
|
||||
logger.error("--output-file can only be given when exactly one "
|
||||
"LANGUAGE_CODE has been specified on the command line "
|
||||
"(it doesn't make sense otherwise)")
|
||||
sys.exit(1)
|
||||
|
||||
return params
|
||||
|
||||
|
||||
def main():
|
||||
global params
|
||||
|
||||
locale.setlocale(locale.LC_ALL, '')
|
||||
params = processCommandLine()
|
||||
|
||||
l10nResPoolMgr = fg_i18n.L10NResourcePoolManager(params.transl_dir, logger)
|
||||
xliffFormatHandler = fg_i18n.FORMAT_HANDLERS_MAP[params.output_format]()
|
||||
|
||||
if params.output_file is not None:
|
||||
assert len(params.lang_code) == 1, params.lang_code
|
||||
# Output to one file or to stdout
|
||||
l10nResPoolMgr.writeSkeletonTranslation(
|
||||
xliffFormatHandler, params.lang_code[0],
|
||||
filePath=params.output_file)
|
||||
else:
|
||||
# Output to several files
|
||||
for langCode in params.lang_code:
|
||||
l10nResPoolMgr.writeSkeletonTranslation(xliffFormatHandler,
|
||||
langCode)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
@@ -1,195 +0,0 @@
|
||||
#! /usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# fg-update-translation-files --- Merge new default translation,
|
||||
# remove obsolete strings from a translation
|
||||
# Copyright (C) 2017 Florent Rougon
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation; either version 2 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along
|
||||
# with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
import argparse
|
||||
import enum
|
||||
import locale
|
||||
import os
|
||||
import sys
|
||||
|
||||
import flightgear.meta.logging
|
||||
import flightgear.meta.i18n as fg_i18n
|
||||
from flightgear.meta.i18n import XliffFormatHandler
|
||||
|
||||
|
||||
PROGNAME = os.path.basename(sys.argv[0])
|
||||
|
||||
# Only messages with severity >= info will be printed to the terminal (it's
|
||||
# possible to also log all messages to a file regardless of their level, see
|
||||
# the Logger class). Of course, there is also the standard logging module...
|
||||
logger = flightgear.meta.logging.Logger(
|
||||
progname=PROGNAME,
|
||||
logLevel=flightgear.meta.logging.LogLevel.info,
|
||||
defaultOutputStream=sys.stderr)
|
||||
|
||||
|
||||
def processCommandLine():
|
||||
params = argparse.Namespace()
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
usage="""\
|
||||
%(prog)s [OPTION ...] ACTION [LANGUAGE_CODE]...
|
||||
Update FlightGear XLIFF localization files.""",
|
||||
description="""\
|
||||
This program performs the following operations (actions) on FlightGear XLIFF
|
||||
translation files (*.xlf):
|
||||
|
||||
- [merge-new-master]
|
||||
Read the default translation[1], add new translated strings it contains to
|
||||
the XLIFF localization files corresponding to the specified language(s),
|
||||
mark the translated strings in said files that need review (modified in
|
||||
the default translation) as well as those that are not used anymore
|
||||
(disappeared in the default translation, or marked in a way that says they
|
||||
don't need to be translated);
|
||||
|
||||
- [mark-unused]
|
||||
Read the default translation and mark translated strings (in the XLIFF
|
||||
localization files corresponding to the specified language(s)) that are
|
||||
not used anymore;
|
||||
|
||||
- [remove-unused]
|
||||
In the XLIFF localization files corresponding to the specified
|
||||
language(s), remove all translated strings that are marked as unused.
|
||||
|
||||
If no LANGUAGE_CODE is provided as an argument, then assuming $transl_dir
|
||||
represents the value passed to --transl-dir, all directories $d such that a
|
||||
file named FlightGear-nonQt.xlf is found in $transl_dir/$d will be acted on as
|
||||
if they had been passed as LANGUAGE_CODE arguments (actually, the directory
|
||||
$transl_dir/default is not considered as a candidate; it is simply skipped).
|
||||
Typically, $transl_dir is /path/to/FGData/Translations.
|
||||
|
||||
A translated string that is marked as unused is still present in the XLIFF
|
||||
localization file; it is just presented in a way that tells translators they
|
||||
don't need to worry about it. On the other hand, when a translated string is
|
||||
removed, translators don't see it anymore and the translation is lost, except
|
||||
if rescued by external means such as backups or version control systems (Git,
|
||||
Subversion, etc.)
|
||||
|
||||
Note that the 'remove-unused' action does *not* imply 'mark-unused'. It only
|
||||
removes translation units that are already marked as unused (i.e., with
|
||||
translate="no"). Thus, it makes sense to do 'mark-unused' followed by
|
||||
'remove-unused' if you really want to get rid of old translations (you need to
|
||||
invoke the program twice, or make a small change for this). Leaving unused
|
||||
translated strings marked as such in XLIFF files shouldn't harm much in
|
||||
general on the short or mid-term: they only take some space.
|
||||
|
||||
[1] FlightGear XML files in $FG_ROOT/Translations/default containing strings
|
||||
used for the default locale (English).""",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
# I want --help but not -h (it might be useful for something else)
|
||||
add_help=False)
|
||||
|
||||
parser.add_argument("-t", "--transl-dir",
|
||||
help="""\
|
||||
directory containing all translation subdirs (such as
|
||||
{default!r}, 'en_GB', 'fr_FR', 'de', 'it'...). This
|
||||
"option" MUST be specified.""".format(
|
||||
default=fg_i18n.DEFAULT_LANG_DIR))
|
||||
parser.add_argument("action", metavar="ACTION",
|
||||
choices=("merge-new-master",
|
||||
"mark-unused",
|
||||
"remove-unused"),
|
||||
help="""\
|
||||
what to do: merge a new default (= master)
|
||||
translation, or mark unused translation units, or
|
||||
remove those already marked as unused from the XLIFF
|
||||
files corresponding to each given LANGUAGE_CODE (i.e.,
|
||||
those that are not in the default translation)""")
|
||||
parser.add_argument("lang_code", metavar="LANGUAGE_CODE", nargs="*",
|
||||
help="""\
|
||||
codes of languages to operate on (e.g., fr, en_GB, it,
|
||||
es_ES...)""")
|
||||
parser.add_argument("--help", action="help",
|
||||
help="display this message and exit")
|
||||
|
||||
params = parser.parse_args(namespace=params)
|
||||
|
||||
if params.transl_dir is None:
|
||||
logger.error("--transl-dir must be given, aborting")
|
||||
sys.exit(1)
|
||||
|
||||
return params
|
||||
|
||||
|
||||
class MarkOrRemoveUnusedAction(enum.Enum):
|
||||
mark, remove = range(2)
|
||||
|
||||
|
||||
def langCodesToActOn():
|
||||
"""Return an iterable of all language codes we were told to work on."""
|
||||
if params.lang_code:
|
||||
return params.lang_code
|
||||
else:
|
||||
return XliffFormatHandler.availableTranslations(params.transl_dir)
|
||||
|
||||
|
||||
def markOrRemoveUnused(l10nResPoolMgr, action):
|
||||
formatHandler = fg_i18n.XliffFormatHandler()
|
||||
masterTransl = l10nResPoolMgr.readFgMasterTranslation().transl
|
||||
|
||||
for langCode in langCodesToActOn():
|
||||
xliffPath = formatHandler.defaultFilePath(params.transl_dir, langCode)
|
||||
transl = formatHandler.readTranslation(xliffPath)
|
||||
|
||||
if action == MarkOrRemoveUnusedAction.mark:
|
||||
transl.markObsoleteOrVanished(masterTransl, logger=logger)
|
||||
elif action == MarkOrRemoveUnusedAction.remove:
|
||||
transl.removeObsoleteOrVanished(logger=logger)
|
||||
else:
|
||||
assert False, "unexpected action: {!r}".format(action)
|
||||
|
||||
l10nResPoolMgr.writeTranslation(formatHandler, transl,
|
||||
filePath=xliffPath)
|
||||
|
||||
|
||||
def mergeNewMaster(l10nResPoolMgr):
|
||||
formatHandler = fg_i18n.XliffFormatHandler()
|
||||
masterTransl = l10nResPoolMgr.readFgMasterTranslation().transl
|
||||
|
||||
for langCode in langCodesToActOn():
|
||||
xliffPath = formatHandler.defaultFilePath(params.transl_dir, langCode)
|
||||
transl = formatHandler.readTranslation(xliffPath)
|
||||
transl.mergeMasterTranslation(masterTransl, logger=logger)
|
||||
l10nResPoolMgr.writeTranslation(formatHandler, transl,
|
||||
filePath=xliffPath)
|
||||
|
||||
|
||||
def main():
|
||||
global params
|
||||
|
||||
locale.setlocale(locale.LC_ALL, '')
|
||||
params = processCommandLine()
|
||||
|
||||
l10nResPoolMgr = fg_i18n.L10NResourcePoolManager(params.transl_dir, logger)
|
||||
|
||||
if params.action == "mark-unused":
|
||||
markOrRemoveUnused(l10nResPoolMgr, MarkOrRemoveUnusedAction.mark)
|
||||
elif params.action == "remove-unused":
|
||||
markOrRemoveUnused(l10nResPoolMgr, MarkOrRemoveUnusedAction.remove)
|
||||
elif params.action == "merge-new-master":
|
||||
mergeNewMaster(l10nResPoolMgr)
|
||||
else:
|
||||
assert False, "Bug: unexpected action: {!r}".format(params.action)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
Reference in New Issue
Block a user