Struggling with GitHub sync and 'has no metadata to indicate the GitHub file path'

Hi folks!

I am trying to set up the automated sync of translations from our Read the Docs project up to Transifex.

I have followed these instructions to get the files created and pushed them up to Transifex manually, as described in the docs: Manage Translations for Sphinx projects — Read the Docs user documentation 8.7.0 documentation

I have then made a 100% translation to en_GB on Transifex, but when I try to sync to GitHub I am getting (replicated for each translation):

o:mautic:p:mautic-end-user-documentation:r:docs_translations_authentication:l:en_GB

Transifex resource with project slug `mautic-end-user-documentation` and resource slug `docs_translations_authentication` has no metadata to indicate the GitHub file path

I have checked that the files are present in the GitHub repo, per this thread.

I also realised the .pot files had not been pushed due to a .gitignore rule, but even after fixing that, still have the problem.

Repo is here: GitHub - mautic/user-documentation and transifex.yml file is here:
user-documentation/transifex.yml at main · mautic/user-documentation · GitHub.

I don’t understand what is meant by this thread, so I don’t know how to check if it applies to this situation or not.

Here is what my slug looks like (as generated when doing the bulk mapping with the Transifex client) - matches what we have in GitHub for the filenames, preceded by the folder name.

Would really value a nudge in the right direction!

@Panagiotis_Kavrakis this was the post I was referring to which got hidden.

Hello @RCheesley

Thank you for your message.

Well, let me please describe what I understood so far from your last message.

Based on this post

I understood that your Tx project is integrated with the following GitHub repository
However, do you have any related questions to this post? If yes, please reply to the specific post, so that we don’t mess up the posts.

Now, you have another Tx Project here which is related to the following repository
As I can see, all the resources are PO file formats, and the slug of one of these resources for example is not as we described here

Also, I don’t see any path locale/en/LC_MESSAGES/ to exist under the following repository so this will redirect you again to the issue of this post.

Having said that, I suggest you first to ensure that you have the source files in your Repository here under the folder locale/en/LC_MESSAGES/

Once you ensure this, please Unlink the Github integration from the Tx project here
by clicking on the 3 dots and select Unlink.

Once you do this, then you can Sync again the same Tx project with the previous settings, and I hope that your PO files should be SYNCED with the existing files you have in Transifex.

In order to confirm this, the example of resource slug l mentioned previously should have been changed.

Looking forward to your feedback.

Kind regards,
Panagiotis
.

This isn’t to do with my project. It was the issue that we were not getting any PRs appearing in our repository after the translation had been created and I was getting the same error message that was posted there that I was referring to. That project/repo is nothing to do with Mautic.

Yes, this is the project we are having the difficulties with.

I’ll look into the suggestions, it has been quite confusing trying to follow the docs from Read the Docs (using the Transifex client) which don’t seem to align with the instructions from Transifex. Maybe something has changed which means there are some missing steps. Hopefully I can help to document the steps to get this working.

We need to maintain the sync when we push new changes to our docs, so the PO files are going to need to be re-created with the Transifex client. Therefore I might need to fix the root cause of the problem (the slugs being incorrectly named by the process we are following to generate and map the files) because otherwise this will just recur every time we regenerate the files?

Hey @RCheesley ,

Thank you for your response.
Regarding the following post, I know that it’s unrelated to the Mautic project,

However, do you still have an issue, or we can consider this also Resolved? If you still have an issue, please just add any further questions in the post here.

As far as your following paragraph

Can you please clarify what are the instructions from Transifex you are referring to?

Last but not least, if I understood it correctly you add PO files from your “Read the Docs” and use the CLI tool in Transifex, but at the same time you want these files to get SYNCED with a GitHub repository.
By default, the Resource’s slug created through CLI is different than the Resource’s slug created through the GitHub Integration.

Having said that, I would suggest you, add your new PO files to the GitHub repository, and then the Transifex-Github Integration will work smoothly.

Looking forward to your response.

Kind regards,
Panagiotis

This is how I created the files, specifically following these instructions using the CLI tool. I then pushed that up to Transifex and made a translation into en_GB.

I built into /docs_translations and then used the expression:

--expression 'locale/<lang>/LC_MESSAGES/{filename}.po' \

This is slightly different as:

  • _build/gettext folder would be confusing - docs_translations makes more sense to folks
  • We don’t need to have the folder path in the name of the resource

Translations are here: https://github.com/mautic/user-documentation/tree/main/docs/docs_translations

Locale folder is here: https://github.com/mautic/user-documentation/tree/main/docs/locale

The PO files are already on GitHub (see above) and I am trying to sync them from Transifex now a language is at 100%, so that we can build the translated version on Read the Docs and make sure that part works. This is where I hit the error I mentioned.

Indeed I did the first creation of the files as explained by Read the Docs, however they do not explain how to keep them in sync. That is where I came across the GitHub app which seems to do what we need.

Are you suggesting that we should not bother at all with the GitHub app and maybe automate the whole process with (for example) a GitHub Action which polls for language completions and then runs a sync?

I did come across GitHub - liri-infra/translation-action: ➕ GitHub action to update translations which might be something we can fork and adjust for our needs (eg the TX file is not a json file so would have to do a bit of work to make it work).

Hello @RCheesley ,

Thank you for your response.

Regarding the links you shared I understand that the source files that need to be translated are under the following Github folder.

Also, the translations for each one of the above source files are under the following path

Is that correct?

Moreover, can you please try to create in Tx a new file-based project and create a transifex.yml file in your repository with the following configuration?

git:
  filters:
  - filter_type: dir
    file_format: PO
    source_file_extension: po
    source_language: en
    source_file_dir: tree/main/docs/docs_translations
    translation_files_expression: 'docs/locale/<lang>/LC_MESSAGES'

I hope that this will SYNC your source files from Github to Transifex, and during the first SYNC, translations will be triggered from Github to Transifex. After that, only translations that follow your SYNC setup settings will be pushed from Transifex to Github.

As far as the link GitHub - liri-infra/translation-action: GitHub action to update translations you shared, unfortunately, I am not familiar with this and I cannot provide any further information for this.

Looking forward to your response.

Kind regards,
Panagiotis

1 Like

Thanks - I’ve kicked that off and it says ‘new syncing started’ so fingers crossed!

Once it completes, is there any way to move across the existing translations from the original Transifex project to the new project? We have a 100% translation on en_GB and a small amount on Hungarian (translators getting overly keen, it would seem!)

Hey @RCheesley ,

Thanks for the quick response. Let’s wait to see your results.

Regarding the existing translations that exist in the other project, you can create a TM Group as described here so that you don’t need to do any manual work. I have to mention here that this feature is only available on the Premium plan and up.

Alternatively, you can download the TMX file from the other project following the instructions here.

Then you can upload the TMX file under the other project following the instructions here.

Looking forward to your response.

Kind regards,
Panagiotis

OK good to know there is an option!

Seems the translation sync has not worked:

There do not seem to be any resources: Mautic End-User Documentation localization | Transifex

Transifex.yml updated as requested:

Hello @RCheesley ,

We are looking into this. I will update you as soon as possible.

Kind regards
Panagiotis

1 Like

Hello @RCheesley ,

I hope you are fine!

I have tested in a test project of mine the following YAML config file and works fine for both source and translations during the initial SYNC, and the translations after I added them manually on the Transifex side.

So, I would like to ask you once again to Unlink the integration of the existing new project, and also try to remove the tree/main/ from the source_file_dir parameter so it will look like
source_file_dir: docs/docs_translations

Looking forward to your response.

Kind regards,
Panagiotis

I have updated the transifex.yml per your suggestions and have unlinked/re-linked but still no sync seems to be happening :frowning_face: .

Hello @RCheesley ,

Allow me to chime in here since @Panagiotis_Kavrakis is out of the office.
I went over your project and noticed that there was a successful sync earlier today, could you please confirm if the problem persists?

Thanks in advance
Cesar

Aha! Yes it looks like it worked but took some time!

Can I now pull over the translations that were already made in the other project (the one above this in the left hand side) into this one? As it looks like the resources are named differently, so guessing we will have to do a find/replace to update?

It was only about 50 strings so I just copy pasted.

Hello @RCheesley ,

I’m glad to read it worked :slight_smile:

Please let us know if there is anything else comes up

Best Regards
Cesar

1 Like

Hi @Panagiotis_Kavrakis and @Cesar_Garcia sorry to re-open an old thread but we do have a small issue with this which I am hoping might be a quick fix.

The PRs are coming over to our GitHub repo when they are 100% translated but we just noticed that rather than retaining the source folder structure, the translated files are all landing in the root of the LC_MESSAGES folder which means that they are not getting picked up as translations.

Here for example: user-documentation/docs/locale/es/LC_MESSAGES at main · mautic/user-documentation · GitHub you see all the files are in the root of this folder where for example stages.po should be in /stages within that folder.

I think it is due to the translation_files_expression setting but I am not sure how to tweak that to retain the existing structure dynamically? If the file was in /stages before translating it must retain that folder location to be seen as a translated file.

Any ideas?

Thanks in advance!

Hello @RCheesley ,

Let me try to understand the scenario; You have the following structure under LC_MESSAGES:

FILES:
authentication.po
builders.po
index.po
stages.po

FOLDERS:
authentications
builders
stages

And you wish to have these files placed inside these folders? If that’s the case, then I’m afraid this is not possible, as the only variable is your language code name.

Current config:
translation_files_expression: ‘docs/locale//LC_MESSAGES’

I assume you are looking for something like this :
translation_files_expression: ‘docs/locale//LC_MESSAGES/authentication/’
translation_files_expression: ‘docs/locale//LC_MESSAGES/stages/’

Please let me know if I understood well your question or If I missed something
Thanks in advance

Hi there,

This is the process we are following to use Transifex with Read the Docs:

Here are the docs on localisation with Read the Docs:

The only part that is different is that we’re using your GitHub integration to push the translations when complete over to GitHub which was the recommended approach by Transifex.

The problem is that when the PR is being made from Transifex, it is not retaining the correct file system structure. Without the file system structure, the whole translation process will completely fail. The docs have to be like for like, in order for the multilingual part to work.

As it stands currently, I’d have to manually edit every single PR that comes from Transifex to move the file to the correct location which is simply not practical.

If we know the original folder location, is there some way that can be captured as a variable and appended to the translation_files_expression?

For example the original path is in the files themselves: Updates for docs/docs_translations/builders.pot in es by transifex-integration · Pull Request #163 · mautic/user-documentation · GitHub

I’m not fully sure how it works but it seems that the structure is also represented in the doctrees:

I’m wondering if anyone has actually got RTD working with Transifex?

I’m also wondering if part of the issue is that perhaps we need to have the folder name in the file path when we generate the .pot files - would that resolve this issue, perhaps?

If that’s the case, what’s the best way to address that issue as I’ll need to somehow push the corrected strings to Transifex and hopefully pull over the existing translations so they aren’t lost.