Supporting additional fonts per-document

[ronaldtse]

Some documents, such as ISO 24229, required support of additional fonts not available in the normal ISO repertoire.

For example, a font for Canadian Syllabics is necessary for rendering those symbols in ISO 24229 (metanorma/mn-native-pdf#181).

The font is available in Fontist as https://github.com/fontist/formulas/blob/v3/Formulas/euphemia.yml (name: Euphemia, formula name: euphemia)

A mechanism for additional font usage can be implemented in Metanorma:

= My doc with Canadian Syllabics
:mn-document-class: iso
:fonts: {fontist font names delimited by semicolon, or fontist formula names prefixed with `formula:xxx`}
:font-license-agreement: {default: no-install-fonts. alternatives: agree-to-terms, continue-without-fonts}

e.g.,

= My doc with Canadian Syllabics
:mn-document-class: iso
:fonts: Euphemia
:font-license-agreement: agree-to-terms

Then Metanorma will ask Fontist to install that font with the appropriate options, and pass the font information to mn2pdf.

This is possibly a global (cross-flavour) option rather than an ISO specific one.

[Intelligent2013]

@opoudjis could you pass the value from :fonts: Euphemia into presentation xml also? I have to pass this parameter into the xslt, otherwise xslt doesn't know about new font, even if it presents in the fontist's manifest. Thanks.

[opoudjis]

I think this is an ad hoc solution to the problem of font coverage of scripts, but it's going to be a requirement inevitably.

I'm going to shove the fonts values into Presentation XML as

<presentation-metadata>
  <name>fonts</name><value>...</value>
  <name>font-license-agreement</name><value>...</value>
</presentation-metadata>

[opoudjis]

These are metanorma processing options; so the attribute name shall be prefixed with mn-, consistent with other such options: mn-fonts, mn-font-license-agreement. This is indeed generic metanorma functionality, not merely ISO.

[ronaldtse]

@opoudjis we probably don't need to prefix mn- in front of the parameters. We already have many attributes that do not use mn- but are in any case only understandable by Metanorma.

[opoudjis]

I do not understand at all the distinction between font font names and font formula names that is being made by @ronaldtse and, more importantly, the https://github.com/fontist/fontist README doesn't. I'm therefore treating them the same.

[ronaldtse]

@opoudjis maybe only do the fonts. Ignore "font family".

[ronaldtse]

It was a brain fart on my part.

[opoudjis]

Because I don't actually understand the confirmation options, please tell me whether this code does what is requested or not:

    def font_install(opt)
      FontistUtils.install_fonts(@processor, opt) unless @fontist_installed
      @fontist_installed = true
      return if !opt[:fonts] ||
        opt[:fontlicenseagreement] == "continue-without-fonts"
      
      confirm = opt[:fontlicenseagreement] == "no-install-fonts" ? "no" : "yes"
      CSV.parse_line(opt[:fonts], col_sep: ";").map(&:strip).each do |f| 
        Fontist::Font.install(f, confirmation: confirm)
      end
    end

[ronaldtse]

@opoudjis can you put this into a PR so @CAMOBAP can test/review? Thanks.

[opoudjis]

As far as I can tell, mn2pdf does not take a fonts parameter; metanorma gem ensures that the requested fonts are installed, but I have no idea if the foregoing is what is needed for mn2pdf to be aware of them.

@CAMOBAP @Intelligent2013 please review

[Intelligent2013]

@opoudjis no need to pass a new info about additional fonts as separated parameter.

Just FYI, how Apache FOP (xsl-fo) works:

• in xslt there is parameter font-family, for instance Cambria, Times New Roman, Cambria Math, Source Han Sans.
• when xsl-fo processor (Apache FOP) tries to find glyph for some character, it opens first font and use glyph from it if found, otherwise opens 2nd font and use glyph from it, etc.
• to extend the font's list we have to add the new font at the end, for example:
  Cambria, Times New Roman, Cambria Math, Source Han Sans, Euphemia
• I can analyze fontist manifest file for a new fonts, but if source adoc contains the attribute :fonts: Euphemia, then it would be simple to include it into presentation xml. Then xslt just will read this value and extend font's list.

[ronaldtse]

I think a font manifest should technically go into Presentation XML. Before the Presentational XML is rendered into other formats, Fontist can use the font manifest to ensure the fonts are installed.

Right now, mn2pdf takes the font manifest file directly which already contains the full locations of the fonts.

[opoudjis]

I have no idea what the desired format is, or how to generate it from a flavour. Someone is going to have to tell me, given that this is not documented in fontist.

[Intelligent2013]

@opoudjis FYI, I've found some strange behavior.

1. Test case 1:

:fonts: Euphemia
:font-license-agreement: agree-to-terms
:toclevels: 2

Resulted presentation xml contains misc-container:

<misc-container>
	<presentation-metadata>
		<name>TOC Heading Levels</name>
		<value>2</value>
	</presentation-metadata>
	<presentation-metadata>
		<name>font-license-agreement</name>
		<value>agree-to-terms</value>
	</presentation-metadata>
	<presentation-metadata>
		<name>fonts</name>
		<value>Euphemia</value>
	</presentation-metadata>
	<presentation-metadata>
		<name>TOC Heading Levels</name>
		<value>2</value>
	</presentation-metadata>
	<presentation-metadata>
		<name>TOC Heading Levels</name>
		<value>2</value>
	</presentation-metadata>
</misc-container>

2. Test case 2 - there isn't another metadata options except 'fonts' features:

:fonts: Euphemia
:font-license-agreement: agree-to-terms

Resulted presentation xml doesn't contain misc-container:

<presentation-metadata>
	<name>font-license-agreement</name>
	<value>agree-to-terms</value>
</presentation-metadata>
<presentation-metadata>
	<name>fonts</name>
	<value>Euphemia</value>
</presentation-metadata>

[Intelligent2013]

@opoudjis you can ignore my previous comment:

FYI, I've found some strange behavior.

I've made patch in xslt already in metanorma/mn-native-pdf#361.

[opoudjis]

As I've said, I don't know what I'm supposed to be doing here. If @ronaldtse and/or @Intelligent2013 you are happy with the implementation, please close the ticket; if not, please tell me what I need to do.

[ronaldtse]

@opoudjis this functionality is not completed -- i.e. Fontist is not installing or using the specified fonts.

@CAMOBAP can you please help?

[ronaldtse]

For the record (cc: @ribose-jeffreylau ), I found out that "Noto Sans Canadian Aboriginal" provides Canadian Syllabics. This is probably better than using a font only free for personal use.

This font is already available in Fontist: https://github.com/fontist/formulas/blob/v3/Formulas/google/noto_sans_canadian_aboriginal.yml

[opoudjis]

Unassigning myself. If someone can identify something for me to do, they can reassign me.

[ronaldtse]

@CAMOBAP can you please help here? Thanks.

[CAMOBAP]

Summarizing

1. additional fonts will be obtained from adoc
2. those fonts will be mixed with font_manifest hardcoded in metanorma-* processor to metanorma manifest XML file
3. this manifest file will be passed to mn2pdf as usual

@Intelligent2013 @opoudjis I'm a bit out of context about presentation.xml. As far as I understand metanorma don't work with presentation.xml directly, correct me if I'm wrong

**also please check proposed plan

[Intelligent2013]

also please check proposed plan

@CAMOBAP agree.

As far as I understand metanorma don't work with presentation.xml directly, correct me if I'm wrong

@CAMOBAP I can say only that mn2pdf reads presentation.xml (generated by metanorma) directly.

[opoudjis]

Not so. The original semantic XML is converted by Metanorma into Presentation XML, which does things like resolve cross-references and i18n. All of PDF, DOC, HTML are generated from the Presentation XML. It is only semantics-specific formats, like STS, which are generated from the semantic XML. So any font directives are appropriate to insert in Presentation XML, not semantic XML.

[ronaldtse]

This issue is also happening here:

• https://github.com/metanorma/pdfa-iso-32000-2/issues/5#issuecomment-1619455887

[opoudjis]

@Intelligent2013 @opoudjis I'm a bit out of context about presentation.xml. As far as I understand metanorma don't work with presentation.xml directly, correct me if I'm wrong

Metanorma Asciidoctor is converted into Metanorma Semantic XML. Then, Metanorma Semantic XML is converted to Presentation XML : this includes internationalisation, resolving cross-references to printable content, and other such render-ready preprocessing.

All of HTML, DOC, and PDF are generated from Presentation XML, rendering it with minimal duplication of preprocessing. Occasionally it turns out PDF needs access to the semantic original information, so the Semantic XML is embedded inside the Presentation XML.

Inasmuch as I understand your plan, it looks fine to me :) Various rendering directives are passed on as metadata from Semantic XML through to Presentation XML, and the Semantic XML by definition does nothing with them; that includes choices of font. So the font is named in Metanorma Asciidoctor, but is not used until the Presentation XML is processed for PDF generation.