Laboratory of Computer and Information Science / Neural Networks Research Centre CIS Lab Helsinki University of Technology

Competition 1

English


Table: The submitted morpheme analysis compared to the Gold Standard in English (Competition 1). Methods without authors are reference methods by the organizers. The types are: S = semi-supervised algorithm, P = unsupervised algorithm with supervised parameter tuning, U = fully unsupervised algorithm.
AUTHOR METHOD PRECISION RECALL F-MEASURE TYPE
KohonenMorfessor S+W65.62%69.28%67.40%S
KohonenMorfessor S+W+L67.87%66.43%67.14%S
LignosBase Inference80.77%53.76%64.55%U
LignosIterative Compounding80.27%52.76%63.67%U
SpieglerDEAP MDL-NOCAT51.44%80.95%62.91%S
SpieglerDEAP PROB-NOCAT58.52%63.06%60.70%S
LignosAggressive Compounding71.45%52.31%60.40%U
KohonenMorfessor U+W60.33%59.55%59.94%P
SpieglerDEAP MDL-CAT49.83%75.08%59.90%S
NicolasMorphAcq67.83%53.43%59.78%U
SpieglerDEAP PROB-CAT55.00%56.61%55.79%S
-Morfessor Baseline81.39%41.70%55.14%U
SpieglerPromodes39.59%64.72%49.13%P
SpieglerPromodes-E49.69%46.22%47.90%P
-Morfessor CatMAP86.84%30.03%44.63%U
GoleniaMAGIP29.88%70.65%42.00%S
SpieglerPromodes-H26.84%63.69%37.77%P
-Letters4.41%99.88%8.44%-

Finnish


Table: The submitted morpheme analysis compared to the Gold Standard in Finnish (Competition 1). Methods without authors are reference methods by the organizers. The types are: S = semi-supervised algorithm, P = unsupervised algorithm with supervised parameter tuning, U = fully unsupervised algorithm.
AUTHOR METHOD PRECISION RECALL F-MEASURE TYPE
SpieglerDEAP MDL-NOCAT56.03%70.71%62.52%S
SpieglerDEAP MDL-CAT57.43%66.59%61.67%S
KohonenMorfessor S+W+L58.38%63.35%60.76%S
KohonenMorfessor S+W57.59%55.21%56.38%S
SpieglerDEAP PROB-NOCAT56.66%44.79%50.03%S
KohonenMorfessor U+W56.97%42.98%49.00%P
LignosAggressive Compounding63.07%39.98%48.94%U
SpieglerDEAP PROB-CAT51.93%42.41%46.69%S
SpieglerPromodes-E43.26%45.73%44.46%P
SpieglerPromodes41.59%47.41%44.31%P
GoleniaMAGIP36.25%53.73%43.29%S
-Morfessor CatMAP80.31%29.51%43.16%U
SpieglerPromodes-H35.08%53.45%42.36%P
LignosIterative Compounding75.32%26.70%39.43%U
LignosBase Inference78.98%24.46%37.35%U
-Morfessor Baseline90.60%14.39%24.83%U
-Letters5.04%99.89%9.60%-

German


Table: The submitted morpheme analysis compared to the Gold Standard in German (Competition 1). Methods without authors are reference methods by the organizers. The types are: S = semi-supervised algorithm, P = unsupervised algorithm with supervised parameter tuning, U = fully unsupervised algorithm.
AUTHOR METHOD PRECISION RECALL F-MEASURE TYPE
KohonenMorfessor U+W58.55%44.94%50.85%P
-Morfessor CatMAP72.70%35.43%47.64%U
LignosBase Inference66.38%35.36%46.14%U
LignosAggressive Compounding59.41%37.21%45.76%U
LignosIterative Compounding62.13%34.70%44.53%U
NicolasMorphAcq69.97%25.29%37.15%U
-Morfessor Baseline82.80%19.77%31.92%U
-Letters2.80%99.92%5.45%-

Turkish


Table: The submitted morpheme analysis compared to the Gold Standard in Turkish (Competition 1). Methods without authors are reference methods by the organizers. The types are: S = semi-supervised algorithm, P = unsupervised algorithm with supervised parameter tuning, U = fully unsupervised algorithm.
AUTHOR METHOD PRECISION RECALL F-MEASURE TYPE
KohonenMorfessor S+W+L71.69%59.97%65.31%S
SpieglerDEAP MDL-NOCAT68.93%56.12%61.87%S
SpieglerDEAP MDL-CAT72.75%53.49%61.65%S
SpieglerDEAP PROB-CAT68.86%55.81%61.65%S
SpieglerDEAP PROB-NOCAT65.71%58.04%61.63%S
KohonenMorfessor S+W65.71%47.15%54.90%S
SpieglerPromodes46.59%51.67%49.00%P
SpieglerPromodes-E40.75%52.39%45.84%P
-Morfessor CatMAP79.38%31.88%45.49%U
KohonenMorfessor U+W40.71%46.76%43.52%P
SpieglerPromodes-H47.88%39.37%43.21%P
GoleniaMAGIP32.00%65.80%43.06%S
LignosAggressive Compounding55.51%34.36%42.45%U
LignosIterative Compounding68.69%21.44%32.68%U
NicolasMorphAcq79.02%19.78%31.64%U
-Morfessor Baseline89.68%17.78%29.67%U
LignosBase Inference72.81%16.11%26.38%U
-Letters8.66%99.13%15.93%-

Return to the result page

HOME | RULES | SCHEDULE | DATASETS | EVALUATION | WORKSHOP | FAQ | CONTACT

You are at: CIS → Unsupervised Morpheme Analysis -- Morpho Challenge 2010

Page maintained by webmaster at cis.hut.fi, last updated Tuesday, 24-Aug-2010 16:07:46 EEST