Bidi CCSID Support via Extension Settings

[yuvalnn]

Changes

This PR adds configurable Bidi (bi-directional text) support for source members encoded in CCSIDs like 424, 420, and others.

Features

• New extension settings under "Source Code":

  1. Enable Bidi support for member editing (checkbox).
  2. Bidi-compatible CCSID (number input).

• Conversion logic only runs when enabled in the settings.

• Encoding is handled via IFS-based copy to avoid corruption.

• The implementation is cleanly isolated to avoid side effects.

Note: Bidi support is currently not compatible with "Source Dates" mode.

How to test this PR

1. Open the extension settings → "Source Code" tab.
2. Enable Enable Bidi support for member editing.
3. Set Bidi-compatible CCSID.
4. Open a source member originally encoded with a Bidi CCSID.
5. Verify the content is displayed correctly and saved as expected.
6. Disable the setting and confirm fallback to normal behavior.

Checklist

• have tested my change
• have created one or more test cases
• updated relevant documentation
• Remove any/all console.logs I added
• have added myself to the contributors' list in CONTRIBUTING.md

[worksofliam]

@yuvalnn This is look like a good start, but we're going to need some granular tests in encoding.test.ts to make sure this works long term. Can you look into adding some? Let me know if you have any questions. Thanks for your work so far!

[yuvalnn]

@worksofliam I’ve added a new synchronous test suite in encoding.test.ts to cover the BiDi changes, thanks to the work from @Itsanexpriment

The suite includes:
Valid/invalid conversions for CCSID 420 (Arabic) and 424 (Hebrew)
Centralized BidiContents test data object for easier extension
Helper convertToUTF8WithCCSID (similar to runCommandsWithCCSID)

We kept this in a separate suite to avoid interfering with existing async tests.
All tests are passing on my side

[yuvalnn]

Hi @sebjulliand , could you please review this PR?

[sebjulliand]

Hi @yuvalnn , I finally have time to look at this, sorry about the delay.
Quick question after looking the code: I like the fact that it's been well isolated, but do you think we could simply automate the detection of the need of Bidi conversion, based on the source file's CCSID?

Something like: if target source file's CCSID is 420 or 424, then convert using the special Bidi method.
Could we use a hardcoded map th know which CCSSID to use to convert based on the source file CCSID? (424 => 62211, 420 => 8612 , etc).

What do you think?

[yuvalnn]

Hi @sebjulliand, thanks for the review.
That was actually my initial thought as well, but I wanted to avoid always checking the CCSID unless the user enabled it.
Thinking about it again, your approach is simpler and with no user involvement required, which seems better.
If needed, we can extend the CCSID map later.

I'll make the change and update.

[yuvalnn]

Hi @sebjulliand after further checking there are situations where automation is not enough.
Hebrew or Arabic with CCSID 62211/8612 usually works best, but not always.

• If the file CCSID is 65535, we might need to look at an override
• Support for more languages may be needed in the future (I just found out Persian is needed as well)
• Doesn’t cover complex RTL/LTR cases.

By the way, other IBM open-source projects, like JTOpen, follow the same approach: the user explicitly enables Bidi and chooses the algorithm (similar to CCSID).
So I think it’s better to keep it like this.