Skip to content

LibreOffice goes BCP 47

This week I accomplished an important milestone of the major rewrite that – apart from the daily work such as fixing bugs, coding small enhancements and reviewing patches – I'm working on since 9 months or so. In current master LibreOffice finally is able to transparently handle arbitrary (if valid) BCP 47 language tags and fully support the fo:script and *:rfc-language-tag attributes defined in ODF 1.2.

So what does this mean? It means that you'll be able to get your language in.

It means that already supported languages or writing scripts that so far used a kludge to squeeze them into ISO 639 language codes and ISO 3166 country codes only, are finally supported using the proper language tags registered with IANA. For example:

ca-ES-valencia Catalan Valencian
The Valencian variant of Catalan previously used the ca-XV kludge where XV is a reserved for private use ISO 3166 code, which meant it could be used for UI translation purposes but not for document content. This is now stored in ODF as style:rfc-language-tag='ca-ES-valencia' attributes.
sr-Latn Serbian Latin
Previously the deprecated sh kludge was used to differentiate between Serbian Latin and sr Serbian Cyrillic. Serbian Latin in Serbia sr-Latn-RS is now stored in ODF as fo:language='sr' fo:script='Latn' fo:country='RS' attributes.

It also means that a tag en-GB-oed can be and now is already supported, including the corresponding language list entry already being added to the list. This is English, Oxford English Dictionary spelling, which is mandatory for UN documents and as it seems also used for EU documents. LibreOffice will be the first free office suite to support spell-checkers with Oxford English Dictionary spelling along with en-GB and en-US spelling at the same time.

Transparently handle arbitrary tags means that when a document is read that contains language attribution not specifically known to LibreOffice (i.e. does not have an entry in the language list), when positioning the cursor on or selecting such text the language tag is shown in the status bar and in the language list of the character attribution so you will not see Unknown or, even worse, nothing or the system locale's language. If a dictionary was installed that handled such tag then it could be used for spell-checking. Transparently of course also means that the tag will be stored again to ODF when saving the document so the attribution is not lost.

The following screenshot shows an example of a document that uses the tag de-DE-1901 to designate German, German variant, traditional orthography:

Screenshot of LibreOffice displaying a BCP 47 language tag.
Screenshot of LibreOffice displaying a BCP 47 language tag.

 

I'm extremely glad to have this step ready just in time and of course I'll talk about it at the LibreOffice Conference 2013 at Milano, so to get all the details please join me and attend Getting you language in on Thursday, 26 September at 15:30 in Sala Alfa.

LibreOffice Milano Conference 2013 logo

If you are interested in the technical details of BCP 47 language tags I recommend my bookmarks as a starting point.

 

Editable Date Acceptance Patterns in LibreOffice

The introduction of more restrictive date acceptance patterns in LibreOffice 3.6 (see earlier blog entries here and here) generated quite some discussion whether the change was good or bad. The fact that not all locales had patterns for incomplete (only day and month) date input added to their data also added some angry voices.

Independent from that there was one thing overlooked: users want to be able to input dates using the numeric keypad and in locales with a '.' dot date separator that was not possible anymore because usually then there is no dot on the keypad due to the decimal separator being different. That certainly needed to be addressed.

There is no way to satisfy everyone with a default set of patterns, I therefore implemented a Date acceptance patterns edit field in the Tools→Options→LanguageSettings→Languages dialogue that follows the selected locale and enables users to add, edit and remove patterns.

Date acceptance patterns edit field.
Date acceptance patterns edit field.

The change is currently in master and pending review as a late feature for inclusion to the 3.6.2 release.

Example for the German de-DE locale:

  • default patterns: D.M.Y;D.M.
  • to enable additional input on numeric keypad: D.M.Y;D.M.;D-M-Y;D-M
    • if 3-4 shall not result in a date, D-M- could be used instead of D-M
    • note that to enter an ISO 8601 Y-M-D date with a D-M-Y pattern active one needs to enter a year >31 or with at least 3 digits, e.g. 011
  • instead of D-M-Y;D-M also D/M/Y;D/M could be used

Changes to the patterns become effective immediately after having confirmed and closed the dialog.

LibreOffice date acceptance patterns

Update 2012-08-31T23:08+0200 : Editable Date Acceptance Patterns in LibreOffice

Abstract: Calc's (and in Writer table) cell input now needs to match locale dependent date acceptance patterns before it is recognized as a valid date.

Previously the number formatter's input scanner was very lax in what it accepted as a "valid" date. All combinations of 2-3 numbers separated by '.' '/' '-' or the locale's date separator even with blanks in between that somehow could be interpreted as a date was accepted as such, which was especially confusing with incomplete dates containing only 2 numbers that in many cases were meant as textual input instead. For example

  • In en-US locale, M/D is a valid date input to be interpreted as day of month of current year. However, M/D/ and M.D. were accepted as well.
  • In de-DE locale, D.M. is a valid date input to be interpreted as day of month of current year. However, D.M and D/M and D/M/ were accepted as well.

In case of an input like 1.2 in a de-DE locale or others using '.' separator, meant as some sort of textual numbering, this was extremely annoying, it was interpreted as 1st of February of current year and the user had to prepend a single quote / apostrophe to suppress date recognition. Similar for 1.2.3 in locales that do not use the '.' date separator.

Now, during build time for each locale one full date acceptance pattern is generated from the existing locale data's number format FormatElement with formatindex="21" that is also used to edit dates, taking the DMY order and the defined DateSeparator. For example, in the en-US locale this generates M/D/Y from the MM/DD/YYYY FormatCode, and in the de-DE locale D.M.Y from the DD.MM.YYYY code. For this to work correctly the separator used in the FormatCode must match the DateSeparator element defined in Separators. As for all rules there's one exception though ;) if the format code uses a different separator and that is one of the known '-' '.' '/' separators, a second pattern is generated using the format's separator. This as a generalized case for locales that for example may use an ISO 8601 edit format, as hu-HU does, regardless what the date separator is defined to.

Additionally to the date acceptance pattern every locale of course still accepts input in an ISO 8601 Y-M-D pattern, and since LibreOffice 3.5 that also leads to the YYYY-MM-DD format being applied.

Localizers, HEADS UP please

If in your locale incomplete dates should be accepted or additional patterns that vary from the generated full date pattern are needed, those are to be defined in the locale data LC_FORMAT element for which a new DateAcceptancePattern element exists, of which zero or more can occur before the FormatElement elements. Currently only the following patterns are defined as they are the only ones I knew were plausible:

  • bg-BG, a trailing breaking or non-breaking space followed by lower case or upper case Cyrillic letter GHE and a dot, as defined in the edit format
    • D.M.Y г.
    • D.M.Y г.
    • D.M.Y Г.
    • D.M.Y Г.
  • de-DE, incomplete date
    • D.M.
  • en-US, incomplete date
    • M/D
  • sl-SI, date separator dot plus space
    • D. M. Y

For example see i18npool/source/localedata/data/en_US.xml

Happy date accepting :-)

Update: an updated list of locales and patterns is available in a newer blog post.

LibreOffice possessive genitive case and partitive case month names

Poss...what? you may ask.. yes, the month that owns the day-of-month.

In some languages (and thus locales) a month name has different cases, depending on the context the name is used in. That's for example the case in Slavic languages, Greek, Russian, Finnish, Gaelic, ... and probably a few more I didn't hear of. Totally unknown to native English speakers ;-) (which I'm not)

  • a standalone month name is the nominative case, the noun, as in November
  • a possessive genitive case month name can be described as "the month's day", as in November's 17th
  • a partitive case month name can be described as "day of month", as in 17 of November

This feature in number formatting was requested for quite some time. Recently I found time to implement it. To achieve this, I added optional elements to LibreOffice's internal locale data <LC_CALENDAR><Calendar> element and implemented the general rules in the number formatter.

Locale data (submitted by localizers)
  • <MonthsOfYear> element, nominative (nouns) month names
    • always specified
    • includes <Month>, <MonthID>, <DefaultAbbrvName> and <DefautFullName> elements
  • <GenitiveMonths> element, genitive case month names
    • optional
    • follows the <MonthsOfYear> element
    • consists of same elements as <MonthsOfYear> element
    • if <GenitiveMonths> are not specified then <MonthsOfYear> names are used in the context of the number formatter's genitive case
  • <PartitiveMonths> element, partitive case month names
    • optional
    • follows the <GenitiveMonths> element, or follows the <MonthsOfYear> element if the <GenitiveMonths> element is not specified
    • consists of same elements as <MonthsOfYear> element
    • if <PartitiveMonths> are not specified then <GenitiveMonths> names are used, if that is not specified then <MonthsOfYear> names are used

Rules for use of nominative / genitive / partitive case month names in number formatter when encountering MMM or MMMM
  • MMM or MMMM immediately preceded or followed by a literal character other than space ⇒ nominative month name (noun), for Excel and backwards compatibility such as Finnish MMMM"ta"
  • no day of month (D or DD) present in format code ⇒ nominative name
  • day of month (D or DD) after MMM or MMMM ⇒ genitive name
    • no genitive names defined ⇒ nominative names
  • day of month (D or DD) before MMM or MMMM ⇒ partitive name
    • no partitive names defined ⇒ genitive names
      • no genitive names defined ⇒ nominative names
NOTE:

If only <MonthsOfYear> and <PartitiveMonths> are specified but not <GenitiveMonths>, then for MMM(M) D(D) formats the <MonthsOfYear> nominative name is displayed. Only for D(D) MMM(M) formats the <PartitiveMonths> name is displayed.

If only for MMM(M) D(D) formats the <GenitiveMonths> are to be displayed but nominative names for D(D) MMM(M), then specify <PartitiveMonths> identical to <MonthsOfYear>, do not omit it as otherwise it would inherit from <GenitiveMonths> again.

Screenshot

To illustrate, here's a screenshot using the Finnish fi-FI locale, Finnish is an extraordinary case that uses all three, nominative, genitive and partitive case month names.

Screenshot of nominative, genitive and partitive month names in Finnish.
Screenshot of nominative, genitive and partitive month names in Finnish.
Locales featuring month name cases
Currently for the following locales genitive and/or partitive case month names were contributed:
  • [an-ES] Aragonese, Spain
  • [ast-ES] Asturian, Spain
  • [be-BY] Belarusian, Belarus
  • [fi-FI] Finnish, Finland
  • [gd-GB] Gaelic (Scottish), United Kingdom
  • [la-VA] Latin, State of the Vatican City
  • [lt-LT] Lithuanian, Lithuania
  • [ru-RU] Russian, Russia

As you can see, that's only a few locales and not all that should be covered. So if you're working on localization of LibreOffice and your language uses month name cases, please contribute the locale data additions as lined out above. Best send a patch of your locale's .xml data file as attachment to the developer mailing list and I'll pick it up. If uncertain how to do that just ask and we'll help. For an example how the data looks like see i18npool/source/localedata/data/lt_LT.xml and search for GenitiveMonths. If you're interested in technical details of locale data files see i18npool/source/localedata/data/locale.dtd

Happy month casing :-)