Online Again

Open-Tran.eu is back online. I am working on configuring the FTP server. Please let me know, if you are experiencing any issues with it.

Server problems

There have been issues with the server recently and I had to take the service down temporarily. I will try to restore it as soon as possible, but I'm not sure if I will be able to activate it back within the next 24 hours. I will post an update to the blog as soon as the problems have been resolved. Please excuse me all the inconvenience.

Downtime

The machine running Open-Tran has been restarted four days ago. Unfortunately, I was on vacation and received the notification about the server being down while relaxing in the Dead Sea :) The service was unavailable for about 48 hours until I got back home. I apologize for any inconvenience caused and promise that this will not happen again.

In order to avoid such situations in the future, I created a cron job that periodically tries to start the web server. So in case the server ever makes a restart like this again, the service should be back up within a couple of minutes.

More Projects

I am proudly announcing the following changes in the list of projects imported to the Open-Tran:

• I have switched Mozilla localization to Mercurial and imported phrases from 1.9.1 branch (which, as far as I can tell, is an equivalent for Firefox 3.5).
• Fedora (master) and Mandriva translations have been added.

Furthermore, I have changed the site's rendering so that the invisible elements do not show up while the site is loading.

JSON

Yesterday, I added the next feature. Now, you can use JSON in order to access translation suggestions from Open-Tran. As an example consider the following request: http://en.id.open-tran.eu/json/suggest/save as. It will return a list of results encoded as JavaScript objects, but with exactly the same fields as if you retrieved the output of an XML-RPC call (see our API documentation) of suggest2('en', 'id', 'save as'). Refer to a commented excerpt of an output for more details.

Words statistics

I've got one more feature for you. The words link now directs to a statistics of the most common words. The language depends on the URL-based language selection. That is: http://nl.open-tran.eu/words.html shows the most popular words for Dutch and http://af.open-tran.eu/words.html shows the most popular words for Afrikaans. You can also set 2 languages in order to have the links directing you to the appropriate comparison pages. For example: http://en.pt_br.open-tran.eu/words.html will show you the most popular English words with the links to their comparisons in Portuguese. If you have any questions regarding this feature, just let me know.

Specific phrases

Here it comes - the next most requested feature has been implemented. Now, you can require that the original phrase looks exactly (or almost exactly) like you specify. In a discussion with Leandro Regueiro, Alexander Dupuy suggested a way to implement this feature in a very nice, effective way. And so now, if you enclose your phrase (or its part) in double-quotes, open-tran will only show the results that contain that phrase. As you can see in the screenshot, I requested the phrase "_save as". The server interpreted the quotes as a request to return only those results, where the original phrase included this phrase literally. So now you can go ahead and try it yourself: http://en.pt_br.open-tran.eu/suggest/"_save as".

This is not the end of news. Did you know that:

• Virtaal supports Open-Tran as its Memory Translation?
• Open-Tran website has been translated into twenty languages in 5 alphabets?

Periodic Updates

I'd like to announce a long-awaited feature - periodic updates of the database. Me and Leonardo, have been working on this feature for more than 2 months and yesterday, the service has successfully updated itself automatically for the first time. In order to circumvent the limitations of the open-tran.eu server, we have split the importing process and distributed it between 2 servers. We are running the update twice a week, but not every execution succeeds.

There are still couple of details to be worked out: the statistics of words, phrases and projects are not updated yet and the switch still needs to be performed manually.

Search Engine Plugin

Some time ago, while implementing one of the new features I had to turn off the advertising of search engines. They existed, but were not really easy to add to the browser if you didn't know the exact URL. Today, I updated the service so that the plugin is easier to add.

The combination of languages for the advertised search engine is inferred from the URL and your settings. So the search engine advertised at http://pl.en.open-tran.eu adds an engine for suggestions between Polish and English, but the one at http://de.fr.open-tran.eu adds an engine for suggestions between German and French.

Fresh import

Fresh import of the translations has been uploaded to the server. By the way, I introduced the following changes to the service:

• After several long outages in September, I had to adopt additional measures to prevent the server from going down with every peak of usage. I am proud to announce to you that open-tran has been working constantly for 58 days without even a single outage (see uptime statistics) despite the growth of the number of visits.
• Frisian and Galician versions of the pages were updated.
• Kannada was added to the list of supported languages.
• Yves Savourel contributed a Java example for the use of open-tran's API. He also suggested introduction of suggest3 method for limiting the number of returned records.

New Layout: Consistency Matters

I've just uploaded the prototype of the new layout to the server. You can see it by visiting the following URL: http://open-tran.eu/front.shtml.

I'd like this page to become the new front page of open-tran.eu. It is based on this set of ideas presented in the Google Redux. You may call it a shameless plagiarism, but (as far as I know) it has never been realized. I haven't asked the author yet, but I hope that he will not object.

I had the following objectives while implementing the page:

1. Due to the new compare functionality, the page needs to have an unlimited width, such that it might be scrolled horizontally without breaking the layout.
2. The new front page has to be very simple - just the search box - like... you know... Google, or something.
3. The contents of the current front pages will be moved to "help" (but I haven't done that, yet).
4. And finally... I like blue :)

I need your help again. I had to introduce some new phrases and am not 100% sure if I translated them well, so I'd like you to verify if they are correct. Obviously, I hope for more feedback and suggestions.

Known Issues
I'm not an HTML guru - it took me a lot of time to learn it well enough to bring the page to the current state. Any help will be appreciated. Furthermore, I might have done some fundamental mistakes. If you have better ideas on how to improve the site, just let me know.

1. You cannot select the mode yet. The search box doesn't work either. I am working on the form and the accompanying JavaScript.
2. For some reason, the search box border has a proper padding in Opera and IE, but not in Firefox.. I've got no idea, why.
3. Internet Explorer doesn't render the blue box at the top and has problems with transparent backgrounds, so the language choice looks stupid.
4. In some setups, the list of language choices renders incorrectly - the longest line (Biełaruskaja) breaks.
5. I was not able to position all the elements exactly where I wanted. The arrow (↔) is not vertically aligned, the drop down lists are not in the right positions, the submit button should probably be lower.
6. The logo is terrible, there is no real favicon.

I will be mailing those, who translated the page, but I know that some of them read this blog, so beware ;) If you have other questions or ideas, post a comment or send an e-mail to the mailing list.

Articles in various languages

I need your advice again. In an effort to further reduce the database and provide more accurate results I am trying to exclude various types of "words" (lexemes or tokens to be precise). First, I removed all format specifiers, like %s. Next, I decided to reduce the number of words that don't carry any meaning. An obvious example from English language is the article "the". Looking up "the" in open-tran won't display any results, because the engine considers this phrase empty (no words). However, I speak only English and German. Together with my girlfriend and Wikipedia we prepared the following list of articles in several European languages:

Code	Language	List of ignored lexemes
de	German	das, dem, den, der, deren, des, dessen, die, ein, eine, einem, einen
en	English	a, an, the
es	Spanish	el, la, las, los, una, uno, unas, unos
fr	French	la, le, les, un, une
it	Italian	i, il, lo, gli, la, le, un, uno, una
pl	Polish	by
pt	Portuguese	o, os, a, as, um, uns, uma, umas

I am planning to add Dutch articles (de, een, het) in the future, too. As you can see, my table covers only 7 languages and open-tran supports more than 90. So I hope that maybe you could help me assembling similar lists for the remaining languages.

I am aware that suffixes may be used as articles in some languages (e.g. Romanian) and if you have any idea on how to tackle this issue without integration of expensive, language-specific dictionaries, then let me know.

If you have other ideas on how to improve the accuracy and/or limit the number of records stored in the database - I will appreciate your feedback. Leave a comment here, or send an e-mail to open-tran@groups.google.com. Thanks!

Compare

I added new functionality to the service. Now you can compare the translations of phrases between the projects. Right now, the only way to see the comparison table is to enter an appropriate URL (or use the API). Here is an example URL:

• http://pt_br.open-tran.eu/compare/zoom in

If you follow it, you will see the table with suggestions grouped by the projects, so that you can see the differences between their translations.

Do you have any idea, how I could put in on the page? Any suggestions for improvements?

Module names and fuzzy translations

I have just finished the implementation, testing and processing of the newest release and am currently deploying it on the server. Below is the screenshot that depicts the new features: module names and fuzzy translation indication.

I am writing about it today, but you won't be able to see any of the results before tomorrow, when the databases are copied to the server (I've got an ADSL link at home and will need around 10 hours to send 3GB over it). I will try to restart the service before leaving to work.
UPDATE: The new version has already been deployed.

Off-line mode

The off-line mode is ready for download. Here is what you need to take advantage of it:

1. Download the latest release of the scripts from this location.
2. Create .open-tran subfolder in your home directory.
3. Download nine-en.db (85MB) and place it in the .open-tran folder.
4. Download a database for your language. To do this, you need to replace en with your language code (e.g.: nine-pt_br.db for Brazilian Portuguese or nine-es.db for Spanish) and place it under .open-tran.
5. Extract the scripts and enjoy :)

You will find 2 executable scripts in the tarball:

1. suggest.py is a central part of the web service. You can use it with other Python scripts (the TranDB class has the methods described in our developers' corner). But at the same time, it is a command-line tool for retrieving suggestions. You should run it like this: suggest.py "do you really want to quit" pt_br.
2. open-tran.py is a GTK tool that opens a po file and provides you suggestions for them. It is a very old tool that I wrote more than year ago. I decided not to develop it any further once I learned that PyGTK is one-threaded. Yesterday I tweaked it to support the off-line mode, but I'm not going to maintain it - it was always meant to provide an example.

It is too late today, I will try to write more in the next days. Anyway, if you have any problems with it, if something crashes, if you need some improvements or don't understand how it works, if you need assistance, send an e-mail to open-tran@googlegroups.com.