Objective
This article aims to describe some steps to follow when you encounter a translation freeze. It also shows what can be done when the SYSTRAN application is not translating anymore.
Introduction
When trying to translate a text or a file, it can happen that the translation job freezes: the text does not translate or the file can be stuck in "translating", even after hours. It can be a frustrating situation, but below some steps are described to show you what can be done.
Steps
- Check of the number of instance(s) and profile status
- Check of the services' statu
- Relaunch of the Translation Resource Monitor service
- Check the logs
- Check the memory and the disk space
1. Check of the number of instance(s) and profile status
- Check of instances
Go to the menu, then click on Admin panel > Translation Resources. You need to be sure that the translation resource(s) is/are green, with the "running" status. The filter also needs to have the same status. The minimum number of instance of each TR used needs to be 1. For the filter, the minimum is 2. According to the number of active TR, it can be more: the more you have TR, the more you need to add filter instances.
If you add more instances to the filter, you can check if translation works after.
Regarding the profiles, go to the menu, click on Linguistic configuration > Translation profiles. The status color has to be green with "activated" and "running". If the status color is orange or red with "deactivated" and/or "not running", the translation will not be functional.
2. Check of the services' status
Check the status of all services on the web interface. The SYSTRAN services which affect the translation are in particular: Systran Translation Resource Monitor, Systran File Translation Consumer (for the file translation), Systran Translation Gateway (for the translation by API).
The third party services that can impact the translation are RabbitMQ and MongoDB.
To check the status of all services, go to the menu, then to Admin panel > Service management > Service status. If a service is up, it will be in green. If it is down, it will appear in red. A service in trouble will be in orange.
From the server terminal, the commands to check are:
systemctl status systran*
systemctl status rabbitmq-server mongod
3. Relaunch of the Translation Resource Monitor service
When typing a text in the translation box, if dots appear in the right box instead of translated text, the translation is unfunctional. You can do the following commands in the terminal:
systemctl stop systran-translation-resource-monitor.service
systemctl start systran-translation-resource-monitor.service
4. Check the logs
You can check the logs of SYSTRAN services with the command below:
journalctl -fu systran-\* | grep -v poller
Pay particularly attention to the error logs.
If, by checking the logs, you see that the issue is with one SYSTRAN service, such as the file translation consumer service or the dispatcher, you can restart the concerned service:
systemctl restart systran-ses-file-translation-consumer
systemctl restart systran-translation-dispatcher
You can go to this article to see in details how to check the status and logs of services.
5. Check the memory and the disk space
Usually when there is a lack of memory on the server, you can have this error log:
ERROR requestId=5127fd6f-baaa-4e86-999c-e7d09e04d423 | Not enough memory after filter import
In that case, you can do the following commands to check the memory on your server:
free -m
To solve this issue you can:
- increase the memory of your server
- decrease the number of instance(s) used by Translation Resource(s) and by the filter
- check the service that consumes a lot of memory and restart this service:
top
systemctl restart <name_of_service>
You can check the disk space with this command:
df -h
If your disk space exceeds in /, you can identify large folders (specifically in /var and /opt):
du -h --max-depth=1
ls -lShr
You can check with Systran support if those files can be safely deleted. If your server is online, it means TRSL is set to disk, so you can safely remove tar.gz files located here:
rm -rf /opt/systran/apps-node/translation-resource-store/file-backend-storage/*
If none of these steps helps you to solve your issue, you can contact the SYSTRAN support.