Kashida option incompatible with Ligatures=Rare (+dlig)

[hirmanpour]

Hi,

Through a lot of trial and error, I seem to have found out that settextfont does not pass Ligatures=Rare (or RawFeature=+dlig) to fontspec when package class option Kashida is used.

Is this a bug or a feature ?

In the latter case, then the documentation should specify that using Kashida excludes using discretionary ligatures.

Many thanks for your help.

Arach Hirmanpour

[u-fischer]

At first glance I see nothing in the code that would actively suppress a ligature. Show a small, complete example with a freely available font that can be used for a test.

[hirmanpour]

Upon further inspection, the issue is not related to package option Kashida per se, but with Kashida activation within text which disables ligatures as a side effect.

Here is a working example :

%%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode
\documentclass[12pt,a4paper]{article}
\usepackage{polyglossia}
\setdefaultlanguage{persian}
\usepackage[Kashida]{xepersian}
\settextfont{Noto Sans Arabic}[Ligatures=Rare]
\begin{document}
\begin{latin}
With KashidaOn:
\end{latin}

\KashidaOn
فرج‌الله نیک‌نژاد.


\begin{latin}
With KashidaOff:
\end{latin}

\KashidaOff
فرج‌الله نیک‌نژاد.
\end{document}

Note: The source code does not contain the kasra that is shown on Github site (where ligatures are activated !!).

Here is the output :

Same issue appears with other fonts such as Adobe Arabic.

Take care,

Arach

[u-fischer]

Yes, this is expected. If you add \showoutput to your code and then look in the log, you can see that with kashida off the words is in one piece:

 TU/NotoSansArabic(0)/m/n/12 نیک‌نژاد.

But with kashida on, rules and penalties have been inserted:

\penalty 10000
....\leaders 0.0 plus 6.0
.....\rule(0.924+0.06)x*
....\TU/NotoSansArabic(0)/m/n/12 �ی�
....\penalty 10000
....\leaders 0.0 plus 6.0
.....\rule(0.924+0.06)x*
....\TU/NotoSansArabic(0)/m/n/12 �ک‌ن�
....\penalty 10000
....\leaders 0.0 plus 6.0
.....\rule(0.924+0.06)x*
....\TU/NotoSansArabic(0)/m/n/12 �ژاد.
....\penalty 10000

and this quite naturally disturbs ligatures. Sorry there is imho nothing that can be done here.

[hirmanpour]

I understand that this is "expected" behavior due to the way Kashida was implemented in xepersian. I do believe that it would be useful to specify in the documentation that Kashida is not compatible with ligatures, which, to the end user, is quite unexpected.

Many thanks for your help !

Arach

[davidcarlisle]

You might also try

https://ctan.org/pkg/xepersian-hm

which is an extension of the package here, the only documented feature is that it "fixes some problems with Kashida" I would be interested to know how it works with your test cases.

[hirmanpour]

Hi David,

xepersian-hm seems to exhibit the exact same behaviour, using the following code:

%%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode
\documentclass[12pt,a4paper]{article}
\usepackage{graphicx}
\usepackage{polyglossia}
\setdefaultlanguage{persian}
\usepackage[Kashida=hrule]{xepersian-hm}
\settextfont{Noto Sans Arabic}[Ligatures=Rare]
\begin{document}
\begin{latin}
With KashidaOn:
\end{latin}

\KashidaOn
فرج‌الله نیک‌نژاد.

\begin{latin}
With KashidaOff:
\end{latin}

Output is :

As a side note I notice that with xepersian-hm Kashida is being used on a paragraph that is not justified. I thought that the purpose of Kashida was to justify paragraphs, so really I am at a loss here...

In contrast with xepersian-hm, xepersian did not visibly use any Kashida on the paragraph, but still exhibited the side effect of disabling ligatures.

[dma8hm1334]

Hi,

You are right. Kashida should justify the paragraph except the last line. Please note that with Kashida=hrule, where a horizontal rule is used to stretch the text, the bahavior of xepersian-hm is expected to be the same as xepersian. However, with Kashida=glyph, where a kashida character is used to stretch the text, the situation is different. Would you please provide a MWE which shows the problem.

Yours
Hossein Movahhedian

[hirmanpour]

Hi Hossein,

As you can see from the previous examples, the behaviour of xepersian and xepersian-hm is not the same : you can observe that a Kashida was inserted by xepersian-hm after the N of Niknejad even though this was a single line paragraph.

For completeness, the following example is with xepersian-hm, Kashida=glyph and Adobe Arabic Font (I didn't find any nonfree font with a kashida glyph and ligatures). The incompatibility between KashidaOn and ligatures can again be observed as well as the insertion of a Kashida after the first letter of the second word.

 %%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode
\documentclass[12pt,a4paper]{article}
\usepackage{graphicx}
\usepackage{polyglossia}
\setdefaultlanguage{persian}
\usepackage[Kashida=glyph]{xepersian-hm}
\settextfont{Adobe Arabic}[Ligatures=Rare]
\begin{document}
\begin{latin}
With KashidaOn:
\end{latin}

\KashidaOn
فرج‌الله نیک‌نژاد.

\begin{latin}
With KashidaOff:
\end{latin}

\KashidaOff
فرج‌الله نیک‌نژاد.
\end{document}

The output is:

Take care,
Arach

[dma8hm1334]

Hi Arach,

Please note that in xepersian the value of the glue is "0 pt plus 0.5 em" while in xepersian-hm the default value of the glue is "0.14 em plus 0.5 em". So
if you want to compare the outputs of these packages please try:

\usepackage[Kashida]{xepersian} % in xepersian

\usepackage[Kashida=hrule,kashidastretch=noskip]{xepersian-hm} % in xepersian-hm

You can also use:

\usepackage[Kashida=hrule,kashidastretch=0 pt plus 0.5 em]{xepersian-hm} % in xepersian-hm

Please let me know if the outputs are not the same on your system.

Thanks
Hossein Movahhedian

[hirmanpour]

Hi,

With kashidastretch=0, I can indeed verify that xepersian and xepersian-hm give the same output on my system.

Now back to the reason why ligatures do not work when Kashida is On. My understanding is that ligatures are broken because rules are inserted between characters. I wonder whether xepersian could be amended to avoid inserting rules between characters forming a ligature. Here is a link discussing a similar issue in Indesign and Xpress (space breaking ligatures in latin text) and stating that "In XPress there is an option to not break ligatures unless they are over a certain tracking amount."

Until such a feature is implemented, IMHO the documentation should specify the incompatibility between Ligatures and Kashida.

Take care,

Arach

[dma8hm1334]

Hi David,

In connection with your comment, I agree that the earlier versions of xepersian-hm had fairly poor documentation. However, in the latest version of the xepersian-hm package its features are more documented. For your information the fixes in xepersian-hm, when Kashida feature is applied to the text, are briefly:

• in xepersian overlapping of glyphs occurs in letter combinations such as "‏ت‍ی‏" ,"‏پ‍ی‏" ,"‏ب‍ی‏" because the character U+FEF0 (ARABIC LETTER Farsi YEH FINAL FORM) is replaced with another character. This is fixed by modifying the fixed part of the glue inserted between the consecutive characters. Furthermore, both, the fixed and the stretchable parts of the glue can be changed by the user.

• in letter combination "‏ه‍ی‏", in xepersian, the character U+FEEB (ARABIC LETTER HEH INITIAL FORM) is replaced with U+06BE (ARABIC LETTER HEH DOACHASHEMEE). This is fixed by inserting the character U+200E before the combination "U+200D + hrule + U+200D" which is used as stretching item.

• in xepersian when outline or shadow fonts are used, the gaps filled by horizontal rule appears as a solid box. This is fixed by using the kashida glyph instead of the horizontal rule in xepersian-hm.

• if italic or oblique style of some fonts such as X Series 2 is used in xepersian, an extra gap is left between the stretched characters. This is fixed in HM Series fonts.

The following figures are the outputs of xepersian with Kashida=on, xepersian-hm with Kashida=hrule and xepersian-hm with Kashida=glyph.

Thanks
Hossein Movahhedian

[hirmanpour]

Hi Hossein,

I do believe that the documentation for xepersian-hm can be improved.

Specifically, the example that is shown on the user manual in CTAN dated 2020-05-01 is as follows:

\PassOptionsToPackage{Kashida=off,RTLdocument=on}{xepersian}
\documentclass{report}
\usepackage{xcolor}
\usepackage[Kashida,kashidastretch=0.14 em plus 0.5 em]{xepersian-hm}

On my system this triggers an error until I change Kashida option to Kashida=glyph or Kashida=hrule.

It would be great also if the documentation showed what xepersian-hm corrects -- as you described in your last email -- so that one can appreciate better the added value of the package.

Thanks for your work,

Arach

[dma8hm1334]

Hi Arach,

Many thanks for reporting the error. That was my mistake. In the next version of xepersian-hm I will try to describe all of the features of the package in the "User Documentation" as much as I can.

Thanks
Hossein Movahhedian

[dma8hm1334]

This issue is resolved in xepersian-hm v0.5e uploaded on CTAN (https://ctan.org/pkg/xepersian-hm) today.
It is not the best solution, but it works.

With the following code in xepersian:

\documentclass{article}
\usepackage[Kashida=on]{xepersian}
\settextfont[Scale=1.1,Ligatures=Rare]{Noto Sans Arabic}

\pagestyle{empty}
\begin{document}
\KashidaOn
 الله، جل‌جلاله و اکبر ‎\hfill\lr{With KashidaOn:}‎ \\

\KashidaOff
 الله، جل‌جلاله و اکبر ‎\hfill\lr{With KashidaOff:}‎
\end{document}

the output is:

and with the following code in xepersian-hm:

\documentclass{article}
\usepackage[Kashida=glyph,ligatures=dlig]{xepersian-hm}
\settextfont[Scale=1.1,Ligatures=Rare]{Noto Sans Arabic}

\pagestyle{empty}
\begin{document}
\KashidaOn
\noindent
\الله\unskip، \جلجلاله و \اکبر به‌ترتیب خروجی\ \verb|\|الله, \verb|\|جلجلاله و \verb|\|اکبر ‎\hfill\lr{With KashidaOn:}‎ \\

\KashidaOff
\noindent
\الله\unskip، \جلجلاله و \اکبر به‌ترتیب خروجی\ \verb|\|ا‌لله، \verb|\|جلجلاله و \verb|\|اکبر ‎\hfill\lr{With KashidaOff:}‎

\end{document}

the output is:

Thanks
Hossein Movahhedian