Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Babel / Large Contribution Import Process"

 
(7 intermediate revisions by 3 users not shown)
Line 2: Line 2:
 
This document describes the process the Babel committer uses to import large contributions into the Babel server.
 
This document describes the process the Babel committer uses to import large contributions into the Babel server.
  
Submitting such a contribution is documented here: http://www.eclipse.org/babel/development/large_contributions.php
+
Submitting such a contribution is documented here: https://eclipse.dev/babel/development/large_contributions.php
  
== Preparation ==
+
== Preparation ==
# Make sure the contribution follows the spec, and that is has been approved by Eclipse Legal by submitting it to IPZilla
+
 
# Unzip the file to babel.eclipse.org:/tmp/tmp-babel/import
+
#Make sure the contribution follows the spec, and that is has been approved by Eclipse Legal by submitting it to IPZilla  
# Inspect some files in each language to ensure they are properly encoded. Run/hack convertiso.sh as needed.
+
#Unzip the file to babel.eclipse.org:/tmp/tmp-babel/import  
 +
##To copy the files to the import directory you will need to su as genie
 +
#Inspect some files in each language to ensure they are properly encoded as Unicode. Run/hack convertiso.sh as needed to convert the files from their current encoding to Unicode.
 +
##The unix command 'find' can be used to help determine the encoding.
 +
##example: ''find /tmp/tmp-import/ -type f -exec file {} \;''
 +
#Determine the accuracy of the contribution.
 +
##Reviewed: Translations were done by professionals, and were reviewed and tested '''in context''' by loading them up in Eclipse
 +
##Fuzzy: Translations were not done by professional translators; translations were done using software and dictionaries; translations were done by professional translators, but were not reviewed and tested '''in context'''.
 +
##If unsure, ask the contributor. If unsure, choose Fuzzy.
  
 
== Import on Staging ==
 
== Import on Staging ==
Line 13: Line 21:
 
# copy the CVS version of import_translation_zip.php to the STAGING web root: cp /home/data/httpd/babel.eclipse.org/staging/classes/export/import_translation_zip.php /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php  
 
# copy the CVS version of import_translation_zip.php to the STAGING web root: cp /home/data/httpd/babel.eclipse.org/staging/classes/export/import_translation_zip.php /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php  
 
# Edit /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php to ensure the target project_id and version are set (see note, below)
 
# Edit /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php to ensure the target project_id and version are set (see note, below)
 +
# Set the Fuzzy factor according to the accuracy of the contribution.
 
# 'run' the above file using wget: wget http://babel.eclipse.org/staging/import_XXXX_zip.php  <-- file name fudged to avoid crawlers
 
# 'run' the above file using wget: wget http://babel.eclipse.org/staging/import_XXXX_zip.php  <-- file name fudged to avoid crawlers
 
# Check the translations using the http://babel.eclipse.org/staging/recent.php
 
# Check the translations using the http://babel.eclipse.org/staging/recent.php
 
+
# If something went wrong, you can still delete whatever you contributed by logging into the Mysql database and running this query for example (it deletes all the records contributed during the current day):
 +
<code lang="sql">delete from translations WHERE UNIX_TIMESTAMP(created_on) BETWEEN UNIX_TIMESTAMP(CURDATE()) AND UNIX_TIMESTAMP(DATE_ADD(CURDATE(), INTERVAL + 86399 SECOND));
 +
</code>
 +
Or if you need to delete all the records for a specific project:
 +
<code>delete from translations where string_id IN (select string_id from strings, files where strings.file_id = files.file_id and files.project_id = "birt");</code>
 +
Manipulate those expressions with care and make sure you use them on the staging database exclusively.
  
 
== Import on Live ==
 
== Import on Live ==

Latest revision as of 09:57, 11 April 2023

This document describes the process the Babel committer uses to import large contributions into the Babel server.

Submitting such a contribution is documented here: https://eclipse.dev/babel/development/large_contributions.php

Preparation

  1. Make sure the contribution follows the spec, and that is has been approved by Eclipse Legal by submitting it to IPZilla
  2. Unzip the file to babel.eclipse.org:/tmp/tmp-babel/import
    1. To copy the files to the import directory you will need to su as genie
  3. Inspect some files in each language to ensure they are properly encoded as Unicode. Run/hack convertiso.sh as needed to convert the files from their current encoding to Unicode.
    1. The unix command 'find' can be used to help determine the encoding.
    2. example: find /tmp/tmp-import/ -type f -exec file {} \;
  4. Determine the accuracy of the contribution.
    1. Reviewed: Translations were done by professionals, and were reviewed and tested in context by loading them up in Eclipse
    2. Fuzzy: Translations were not done by professional translators; translations were done using software and dictionaries; translations were done by professional translators, but were not reviewed and tested in context.
    3. If unsure, ask the contributor. If unsure, choose Fuzzy.

Import on Staging

  1. su - genie
  2. copy the CVS version of import_translation_zip.php to the STAGING web root: cp /home/data/httpd/babel.eclipse.org/staging/classes/export/import_translation_zip.php /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php
  3. Edit /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php to ensure the target project_id and version are set (see note, below)
  4. Set the Fuzzy factor according to the accuracy of the contribution.
  5. 'run' the above file using wget: wget http://babel.eclipse.org/staging/import_XXXX_zip.php <-- file name fudged to avoid crawlers
  6. Check the translations using the http://babel.eclipse.org/staging/recent.php
  7. If something went wrong, you can still delete whatever you contributed by logging into the Mysql database and running this query for example (it deletes all the records contributed during the current day):

delete from translations WHERE UNIX_TIMESTAMP(created_on) BETWEEN UNIX_TIMESTAMP(CURDATE()) AND UNIX_TIMESTAMP(DATE_ADD(CURDATE(), INTERVAL + 86399 SECOND)); Or if you need to delete all the records for a specific project: delete from translations where string_id IN (select string_id from strings, files where strings.file_id = files.file_id and files.project_id = "birt"); Manipulate those expressions with care and make sure you use them on the staging database exclusively.

Import on Live

  1. su - genie
  2. copy the STAGING version of import_translation_zip.php to the LIVE web root: cp /home/data/httpd/babel.eclipse.org/staging/html/import_translation_zip.php /home/data/httpd/babel.eclipse.org/babel/html/import_translation_zip.php
  3. 'run' the above file using wget: wget http://babel.eclipse.org/babel/import_XXXX_zip.php <-- file name fudged to avoid crawlers
  4. Check the translations using the http://babel.eclipse.org/babel/recent.php
  5. Delete the import script: rm /home/data/httpd/babel.eclipse.org/babel/html/import_translation_zip.php
  6. Delete the translation files.

Note: If a contributions crosses many projects, such as those from the JapanWG, simply delete the following line in the SQL statements (there are currently two occurrences of this line) :

   AND F.project_id = '" . $PROJECT_ID . "' AND F.version = '" . $VERSION . "'

Back to the top