Merge pull request #326 from korpling/fix/conllu-dropped-column-shoul…

…d-be-an-error

Fix/conllu dropped column should be an error

CHANGELOG.md

@@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 
 - import meta csv tables iterates over all lines and completely imports the entire table unlike before
+- import `conllu` did not fail on missing column, but rather dropped the remaining content without notification
 
 ## [0.18.0] - 2024-10-15
 

Cargo.toml

@@ -35,7 +35,7 @@ normpath = "1.1"
 ordered-float = {version = "4.1", default-features = false}
 pathdiff = "0.2"
 percent-encoding = "2.3.1"
-pest = "2.7"
+pest = "2.7.14"
 pest_derive = "2.0"
 quick-xml = "0.34"
 rayon = "1.1"

src/exporter/snapshots/annatto__exporter__table__tests__custom.snap

@@ -4,14 +4,15 @@ expression: export.unwrap()
 ---
 ---- website_example.csv:
 'annis::tok';'id_annis::tok';'lemma';'id_lemma';'upos';'id_upos';'xpos';'id_xpos';'Case';'id_Case';'Number';'id_Number';'in_Pointing__dep_deprel';'in_Pointing__dep';'sent_id';'id_sent_id';'text';'id_text';'Person';'id_Person';'Tense';'id_Tense';'out_Pointing__dep_deprel';'out_Pointing__dep';'SpaceAfter';'id_SpaceAfter';'PronType';'id_PronType'
-'They';'valid/website_example#t5_1';'they';'valid/website_example#t5_1';'PRON';'valid/website_example#t5_1';'PRP';'valid/website_example#t5_1';'Nom';'valid/website_example#t5_1';'Plur';'valid/website_example#t5_1';'nsubj';'valid/website_example#t5_68';'1';'valid/website_example#s1_1';'They buy and sell books.';'valid/website_example#s1_1';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
-'buy';'valid/website_example#t5_68';'buy';'valid/website_example#t5_68';'VERB';'valid/website_example#t5_68';'VBP';'valid/website_example#t5_68';'n/a';'n/a';'Plur';'valid/website_example#t5_68';'n/a';'n/a';'1';'valid/website_example#s1_1';'They buy and sell books.';'valid/website_example#s1_1';'3';'valid/website_example#t5_68';'Pres';'valid/website_example#t5_68';'punct;obj;conj;nsubj';'valid/website_example#t9_66;valid/website_example#t8_76;valid/website_example#t7_32;valid/website_example#t5_1';'n/a';'n/a';'n/a';'n/a'
-'and';'valid/website_example#t6_67';'and';'valid/website_example#t6_67';'CONJ';'valid/website_example#t6_67';'CC';'valid/website_example#t6_67';'n/a';'n/a';'n/a';'n/a';'cc';'valid/website_example#t7_32';'1';'valid/website_example#s1_1';'They buy and sell books.';'valid/website_example#s1_1';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
-'sell';'valid/website_example#t7_32';'sell';'valid/website_example#t7_32';'VERB';'valid/website_example#t7_32';'VBP';'valid/website_example#t7_32';'n/a';'n/a';'Plur';'valid/website_example#t7_32';'conj';'valid/website_example#t5_68';'1';'valid/website_example#s1_1';'They buy and sell books.';'valid/website_example#s1_1';'3';'valid/website_example#t7_32';'Pres';'valid/website_example#t7_32';'cc';'valid/website_example#t6_67';'n/a';'n/a';'n/a';'n/a'
-'books';'valid/website_example#t8_76';'book';'valid/website_example#t8_76';'NOUN';'valid/website_example#t8_76';'NNS';'valid/website_example#t8_76';'n/a';'n/a';'Plur';'valid/website_example#t8_76';'obj';'valid/website_example#t5_68';'1';'valid/website_example#s1_1';'They buy and sell books.';'valid/website_example#s1_1';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'No';'valid/website_example#t8_76';'n/a';'n/a'
-'.';'valid/website_example#t9_66';'.';'valid/website_example#t9_66';'PUNCT';'valid/website_example#t9_66';'.';'valid/website_example#t9_66';'n/a';'n/a';'n/a';'n/a';'punct';'valid/website_example#t5_68';'1';'valid/website_example#s1_1';'They buy and sell books.';'valid/website_example#s1_1';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
-'I';'valid/website_example#t14_1';'I';'valid/website_example#t14_1';'PRON';'valid/website_example#t14_1';'PRP';'valid/website_example#t14_1';'Nom';'valid/website_example#t14_1';'Sing';'valid/website_example#t14_1';'nsubj';'valid/website_example#t14_57';'2';'valid/website_example#s10_34';'I have no clue.';'valid/website_example#s10_34';'1';'valid/website_example#t14_1';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
-'have';'valid/website_example#t14_57';'have';'valid/website_example#t14_57';'VERB';'valid/website_example#t14_57';'VBP';'valid/website_example#t14_57';'n/a';'n/a';'Sing';'valid/website_example#t14_57';'n/a';'n/a';'2';'valid/website_example#s10_34';'I have no clue.';'valid/website_example#s10_34';'1';'valid/website_example#t14_57';'Pres';'valid/website_example#t14_57';'punct;obj;nsubj';'valid/website_example#t17_54;valid/website_example#t16_38;valid/website_example#t14_1';'n/a';'n/a';'n/a';'n/a'
-'no';'valid/website_example#t15_64';'no';'valid/website_example#t15_64';'DET';'valid/website_example#t15_64';'DT';'valid/website_example#t15_64';'n/a';'n/a';'n/a';'n/a';'det';'valid/website_example#t16_38';'2';'valid/website_example#s10_34';'I have no clue.';'valid/website_example#s10_34';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'Neg';'valid/website_example#t15_64'
-'clue';'valid/website_example#t16_38';'clue';'valid/website_example#t16_38';'NOUN';'valid/website_example#t16_38';'NN';'valid/website_example#t16_38';'n/a';'n/a';'Sing';'valid/website_example#t16_38';'obj';'valid/website_example#t14_57';'2';'valid/website_example#s10_34';'I have no clue.';'valid/website_example#s10_34';'n/a';'n/a';'n/a';'n/a';'det';'valid/website_example#t15_64';'No';'valid/website_example#t16_38';'n/a';'n/a'
-'.';'valid/website_example#t17_54';'.';'valid/website_example#t17_54';'PUNCT';'valid/website_example#t17_54';'.';'valid/website_example#t17_54';'n/a';'n/a';'n/a';'n/a';'punct';'valid/website_example#t14_57';'2';'valid/website_example#s10_34';'I have no clue.';'valid/website_example#s10_34';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
+'They';'valid/website_example#t5';'they';'valid/website_example#t5';'PRON';'valid/website_example#t5';'PRP';'valid/website_example#t5';'Nom';'valid/website_example#t5';'Plur';'valid/website_example#t5';'nsubj';'valid/website_example#t6';'1';'valid/website_example#s3';'They buy and sell books.';'valid/website_example#s3';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
+'buy';'valid/website_example#t6';'buy';'valid/website_example#t6';'VERB';'valid/website_example#t6';'VBP';'valid/website_example#t6';'n/a';'n/a';'Plur';'valid/website_example#t6';'n/a';'n/a';'1';'valid/website_example#s3';'They buy and sell books.';'valid/website_example#s3';'3';'valid/website_example#t6';'Pres';'valid/website_example#t6';'punct;obj;conj;nsubj';'valid/website_example#t10;valid/website_example#t9;valid/website_example#t8;valid/website_example#t5';'n/a';'n/a';'n/a';'n/a'
+'and';'valid/website_example#t7';'and';'valid/website_example#t7';'CONJ';'valid/website_example#t7';'CC';'valid/website_example#t7';'n/a';'n/a';'n/a';'n/a';'cc';'valid/website_example#t8';'1';'valid/website_example#s3';'They buy and sell books.';'valid/website_example#s3';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
+'sell';'valid/website_example#t8';'sell';'valid/website_example#t8';'VERB';'valid/website_example#t8';'VBP';'valid/website_example#t8';'n/a';'n/a';'Plur';'valid/website_example#t8';'conj';'valid/website_example#t6';'1';'valid/website_example#s3';'They buy and sell books.';'valid/website_example#s3';'3';'valid/website_example#t8';'Pres';'valid/website_example#t8';'cc';'valid/website_example#t7';'n/a';'n/a';'n/a';'n/a'
+'books';'valid/website_example#t9';'book';'valid/website_example#t9';'NOUN';'valid/website_example#t9';'NNS';'valid/website_example#t9';'n/a';'n/a';'Plur';'valid/website_example#t9';'obj';'valid/website_example#t6';'1';'valid/website_example#s3';'They buy and sell books.';'valid/website_example#s3';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'No';'valid/website_example#t9';'n/a';'n/a'
+'.';'valid/website_example#t10';'.';'valid/website_example#t10';'PUNCT';'valid/website_example#t10';'.';'valid/website_example#t10';'n/a';'n/a';'n/a';'n/a';'punct';'valid/website_example#t6';'1';'valid/website_example#s3';'They buy and sell books.';'valid/website_example#s3';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
+'I';'valid/website_example#t14';'I';'valid/website_example#t14';'PRON';'valid/website_example#t14';'PRP';'valid/website_example#t14';'Nom';'valid/website_example#t14';'Sing';'valid/website_example#t14';'nsubj';'valid/website_example#t15';'2';'valid/website_example#s12';'I have no clue.';'valid/website_example#s12';'1';'valid/website_example#t14';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
+'have';'valid/website_example#t15';'have';'valid/website_example#t15';'VERB';'valid/website_example#t15';'VBP';'valid/website_example#t15';'n/a';'n/a';'Sing';'valid/website_example#t15';'n/a';'n/a';'2';'valid/website_example#s12';'I have no clue.';'valid/website_example#s12';'1';'valid/website_example#t15';'Pres';'valid/website_example#t15';'punct;obj;nsubj';'valid/website_example#t18;valid/website_example#t17;valid/website_example#t14';'n/a';'n/a';'n/a';'n/a'
+'no';'valid/website_example#t16';'no';'valid/website_example#t16';'DET';'valid/website_example#t16';'DT';'valid/website_example#t16';'n/a';'n/a';'n/a';'n/a';'det';'valid/website_example#t17';'2';'valid/website_example#s12';'I have no clue.';'valid/website_example#s12';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'Neg';'valid/website_example#t16'
+'clue';'valid/website_example#t17';'clue';'valid/website_example#t17';'NOUN';'valid/website_example#t17';'NN';'valid/website_example#t17';'n/a';'n/a';'Sing';'valid/website_example#t17';'obj';'valid/website_example#t15';'2';'valid/website_example#s12';'I have no clue.';'valid/website_example#s12';'n/a';'n/a';'n/a';'n/a';'det';'valid/website_example#t16';'No';'valid/website_example#t17';'n/a';'n/a'
+'.';'valid/website_example#t18';'.';'valid/website_example#t18';'PUNCT';'valid/website_example#t18';'.';'valid/website_example#t18';'n/a';'n/a';'n/a';'n/a';'punct';'valid/website_example#t15';'2';'valid/website_example#s12';'I have no clue.';'valid/website_example#s12';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a';'n/a'
+

src/exporter/snapshots/annatto__exporter__table__tests__edge_features.snap

@@ -4,14 +4,15 @@ expression: export.unwrap()
 ---
 ---- website_example.csv:
 annis::tok	id_annis::tok	lemma	id_lemma	upos	id_upos	xpos	id_xpos	Case	id_Case	Number	id_Number	in_Pointing__dep_deprel	in_Pointing__dep	sent_id	id_sent_id	text	id_text	Person	id_Person	Tense	id_Tense	SpaceAfter	id_SpaceAfter	PronType	id_PronType
-They	valid/website_example#t5_1	they	valid/website_example#t5_1	PRON	valid/website_example#t5_1	PRP	valid/website_example#t5_1	Nom	valid/website_example#t5_1	Plur	valid/website_example#t5_1	nsubj	valid/website_example#t5_68	1	valid/website_example#s1_1	They buy and sell books.	valid/website_example#s1_1								
-buy	valid/website_example#t5_68	buy	valid/website_example#t5_68	VERB	valid/website_example#t5_68	VBP	valid/website_example#t5_68			Plur	valid/website_example#t5_68			1	valid/website_example#s1_1	They buy and sell books.	valid/website_example#s1_1	3	valid/website_example#t5_68	Pres	valid/website_example#t5_68				
-and	valid/website_example#t6_67	and	valid/website_example#t6_67	CONJ	valid/website_example#t6_67	CC	valid/website_example#t6_67					cc	valid/website_example#t7_32	1	valid/website_example#s1_1	They buy and sell books.	valid/website_example#s1_1								
-sell	valid/website_example#t7_32	sell	valid/website_example#t7_32	VERB	valid/website_example#t7_32	VBP	valid/website_example#t7_32			Plur	valid/website_example#t7_32	conj	valid/website_example#t5_68	1	valid/website_example#s1_1	They buy and sell books.	valid/website_example#s1_1	3	valid/website_example#t7_32	Pres	valid/website_example#t7_32				
-books	valid/website_example#t8_76	book	valid/website_example#t8_76	NOUN	valid/website_example#t8_76	NNS	valid/website_example#t8_76			Plur	valid/website_example#t8_76	obj	valid/website_example#t5_68	1	valid/website_example#s1_1	They buy and sell books.	valid/website_example#s1_1					No	valid/website_example#t8_76		
-.	valid/website_example#t9_66	.	valid/website_example#t9_66	PUNCT	valid/website_example#t9_66	.	valid/website_example#t9_66					punct	valid/website_example#t5_68	1	valid/website_example#s1_1	They buy and sell books.	valid/website_example#s1_1								
-I	valid/website_example#t14_1	I	valid/website_example#t14_1	PRON	valid/website_example#t14_1	PRP	valid/website_example#t14_1	Nom	valid/website_example#t14_1	Sing	valid/website_example#t14_1	nsubj	valid/website_example#t14_57	2	valid/website_example#s10_34	I have no clue.	valid/website_example#s10_34	1	valid/website_example#t14_1						
-have	valid/website_example#t14_57	have	valid/website_example#t14_57	VERB	valid/website_example#t14_57	VBP	valid/website_example#t14_57			Sing	valid/website_example#t14_57			2	valid/website_example#s10_34	I have no clue.	valid/website_example#s10_34	1	valid/website_example#t14_57	Pres	valid/website_example#t14_57				
-no	valid/website_example#t15_64	no	valid/website_example#t15_64	DET	valid/website_example#t15_64	DT	valid/website_example#t15_64					det	valid/website_example#t16_38	2	valid/website_example#s10_34	I have no clue.	valid/website_example#s10_34							Neg	valid/website_example#t15_64
-clue	valid/website_example#t16_38	clue	valid/website_example#t16_38	NOUN	valid/website_example#t16_38	NN	valid/website_example#t16_38			Sing	valid/website_example#t16_38	obj	valid/website_example#t14_57	2	valid/website_example#s10_34	I have no clue.	valid/website_example#s10_34					No	valid/website_example#t16_38		
-.	valid/website_example#t17_54	.	valid/website_example#t17_54	PUNCT	valid/website_example#t17_54	.	valid/website_example#t17_54					punct	valid/website_example#t14_57	2	valid/website_example#s10_34	I have no clue.	valid/website_example#s10_34
+They	valid/website_example#t5	they	valid/website_example#t5	PRON	valid/website_example#t5	PRP	valid/website_example#t5	Nom	valid/website_example#t5	Plur	valid/website_example#t5	nsubj	valid/website_example#t6	1	valid/website_example#s3	They buy and sell books.	valid/website_example#s3								
+buy	valid/website_example#t6	buy	valid/website_example#t6	VERB	valid/website_example#t6	VBP	valid/website_example#t6			Plur	valid/website_example#t6			1	valid/website_example#s3	They buy and sell books.	valid/website_example#s3	3	valid/website_example#t6	Pres	valid/website_example#t6				
+and	valid/website_example#t7	and	valid/website_example#t7	CONJ	valid/website_example#t7	CC	valid/website_example#t7					cc	valid/website_example#t8	1	valid/website_example#s3	They buy and sell books.	valid/website_example#s3								
+sell	valid/website_example#t8	sell	valid/website_example#t8	VERB	valid/website_example#t8	VBP	valid/website_example#t8			Plur	valid/website_example#t8	conj	valid/website_example#t6	1	valid/website_example#s3	They buy and sell books.	valid/website_example#s3	3	valid/website_example#t8	Pres	valid/website_example#t8				
+books	valid/website_example#t9	book	valid/website_example#t9	NOUN	valid/website_example#t9	NNS	valid/website_example#t9			Plur	valid/website_example#t9	obj	valid/website_example#t6	1	valid/website_example#s3	They buy and sell books.	valid/website_example#s3					No	valid/website_example#t9		
+.	valid/website_example#t10	.	valid/website_example#t10	PUNCT	valid/website_example#t10	.	valid/website_example#t10					punct	valid/website_example#t6	1	valid/website_example#s3	They buy and sell books.	valid/website_example#s3								
+I	valid/website_example#t14	I	valid/website_example#t14	PRON	valid/website_example#t14	PRP	valid/website_example#t14	Nom	valid/website_example#t14	Sing	valid/website_example#t14	nsubj	valid/website_example#t15	2	valid/website_example#s12	I have no clue.	valid/website_example#s12	1	valid/website_example#t14						
+have	valid/website_example#t15	have	valid/website_example#t15	VERB	valid/website_example#t15	VBP	valid/website_example#t15			Sing	valid/website_example#t15			2	valid/website_example#s12	I have no clue.	valid/website_example#s12	1	valid/website_example#t15	Pres	valid/website_example#t15				
+no	valid/website_example#t16	no	valid/website_example#t16	DET	valid/website_example#t16	DT	valid/website_example#t16					det	valid/website_example#t17	2	valid/website_example#s12	I have no clue.	valid/website_example#s12							Neg	valid/website_example#t16
+clue	valid/website_example#t17	clue	valid/website_example#t17	NOUN	valid/website_example#t17	NN	valid/website_example#t17			Sing	valid/website_example#t17	obj	valid/website_example#t15	2	valid/website_example#s12	I have no clue.	valid/website_example#s12					No	valid/website_example#t17		
+.	valid/website_example#t18	.	valid/website_example#t18	PUNCT	valid/website_example#t18	.	valid/website_example#t18					punct	valid/website_example#t15	2	valid/website_example#s12	I have no clue.	valid/website_example#s12								
+