Skip to content

OPENNLP-1782: Add tagging examples to verify French POS model #863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

OPENNLP-1782: Add tagging examples to verify French POS model #863

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0b7a278
french+arabic+maghrebi
meriam2303 Sep 1, 2025
56ccbbe
arabic+maghrebi_arabic_french rauskommentiert
meriam2303 Sep 29, 2025
b69b920
added french constant and fixed the syntax error
meriam2303 Sep 30, 2025
6fe239a
Fix checkstyle
rzo1 Oct 14, 2025
72c7965
Init FRENCH resources for test context
rzo1 Oct 14, 2025
814ab2d
changed faisait to verb
meriam2303 Nov 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/postag/POSTaggerMEIT.java
View file
Open in desktop
Copy link
Author

@meriam2303 meriam2303 Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to the dictionary of the french academy AUX is “a verb that functions as a grammatical tool used to build the compound tenses of other verbs.” and faire is considered a semi-auxiliare. Because this option is not available and because "faire" doesn't build a compound tense of the verb "souffrir" it is safer and more correct to consider faire a verb NOT an auxiliare.

Copy link
Contributor

@rzo1 rzo1 Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: current CI failure is unrelated to this change (403, sourceforge)

Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ public class POSTaggerMEIT {
private static final String GERMAN = "de";
private static final String POLISH = "pl";
private static final String PORTUGUESE = "pt";
private static final String FRENCH = "fr";

private static final Map<String, Tokenizer> TOKENIZERS = new HashMap<>();
private static final Map<String, POSTagger> TAGGERS = new HashMap<>();
Expand All @@ -51,7 +52,7 @@ public class POSTaggerMEIT {

@BeforeAll
public static void initResources() throws IOException {
List<String> langs = List.of(CATALAN, ENGLISH, GERMAN, POLISH, PORTUGUESE);
final List<String> langs = List.of(CATALAN, ENGLISH, FRENCH, GERMAN, POLISH, PORTUGUESE);
for (String langCode: langs) {
TOKENIZERS.put(langCode, new ThreadSafeTokenizerME(langCode));
TAGGERS.put(langCode, new ThreadSafePOSTaggerME(langCode));
Expand Down Expand Up @@ -142,7 +143,7 @@ private static Stream<Arguments> provideData() {
"Un gran embossament d'aire fred es comença a despenjar cap al centre d'Europa.",
// OpenNLP, different at: idx pos 2, 3, 5, and 13(+14) -> however, only pos 5 is "wrong" (ref)
new String[]{"DET", "ADJ", "NOUN", "ADP", "NOUN", "ADJ", "PRON", "VERB", "ADP", "VERB", "NOUN",
"ADP+DET", "NOUN", "ADP", "PROPN", "PUNCT"})
"ADP+DET", "NOUN", "ADP", "PROPN", "PUNCT"}),
// REFERENCE ("gold"):
// "DET", "ADJ", "NOUN", "ADP", "NOUN", "ADJ", "PRON", "VERB", "ADP", "VERB", "NOUN", "ADP+DET",
// "NOUN", "ADP", "PROPN", "PUNCT"})
Expand All @@ -152,6 +153,22 @@ private static Stream<Arguments> provideData() {
// "NOUN", "PROPN", "PROPN", "PUNCT"
// ok! , ok! , ??? , ??? , ok! , ok! , ok! , ok! , ok! , ok! , ok! , ok! + ok! ,
// ok! , ??? , ok! , ok!
// via: @meriam2303 , original by Guillaume Musso:
// La jeune fille et la nuit, S.469
Arguments.of(FRENCH, 0,
"Vivre avec elle me faisait souffrir, mais vivre sans elle m'aurait tué.",
new String[] {"VERB", "ADP", "PRON", "PRON", "VERB", "VERB", "PUNCT", "CCONJ", "VERB",
"ADP", "PRON", "PRON", "AUX", "VERB", "PUNCT"})
// via @meriam2303, original by Hind Choueykh Ben Salah
// التجريد في الشّعر العربي , S. 42
//Arguments.of(ARABIC,0,
//"عشق أبو نواس جارية تدعى جنان",
//new String[]{"VERB","PROPN","NOUN","VERB","PROPN"})
// via @meriam2303, original by Mohamed Laarousi Elmetoui
// التوت المر , S.7
//Arguments.of(MARGHREBI_ARABIC_FRENCH,0,
//"Wassa3 belek ya baba...",
//new String[]{"VERB","NOUN","ITNJ","NOUN","PUNCT"})
);
}
}
Loading