Run baselines

[kasnerz]

Now that the web app can display pre-generated outputs, it would be useful to have some outputs to display.

Goals:

• prepare suitable baselines
• prepare the inputs (tabgenie export should come in useful here)
• run the baselines
• import the outputs in the app

Baselines = any existing models which can be applied on table-to-text generation.

It is not necessary that the baselines are applicable on each dataset (some work only for triples, some work only with highlighted cells, some work with logical forms, etc.), but it would be cool to run them on as many datasets as possible.

It is also not necessary to actually run the baselines - if they provide the outputs exactly for our datasets, we can just use the outputs (or, if the datasets are similar, we can try to add them / adapt them).

Proposals:

• BART / T5 finetuned for seq2seq generation
• prompted LLMs
• [@kasnerz] https://github.com/luka-group/Lattice
• [@kategerasimenko] https://github.com/yxuansu/PlanGen/
• https://github.com/microsoft/PLOG
  • [@kategerasimenko] models not available, only output for test sets for LogicNLG and Logic2Text
• https://github.com/XiangLi1999/PrefixTuning
• https://github.com/syw1996/TableGPT
• https://github.com/ErnestGong/data2text-duv
• [@oplatek-DONE] https://github.com/ReneeYe/VariationalTemplateMachine
  • "Runable" & paper readable - progress tracked in [oplatek's fork]
  • See VAE outputs - with "high" temperature and five different hypotheses /lnet/work/people/oplatek/VariationalTemplateMachine/exp/Wiki.20230112_095301/generate/temp-5hyp.html
  • See VAE with beamsearch with minimal temperature and 1 best hypotheses TODO
  • (reproducing - fixing the Pytorch version and documenting env setup oplatek/VariationalTemplateMachine#1)
• https://github.com/h-shahidi/2birds-gen/tree/master/table-to-text
• https://github.com/ernestgong/data2text-three-dimensions
• https://github.com/lancopku/Pivot
• https://github.com/anindyasarkarIITH/Structure_data_to_summary
• https://github.com/wentinghome/amg
• [@oplatek] https://github.com/GXimingLu/a_star_neurologic
• https://github.com/yxuansu/few-shot-table-to-text-generation
• https://github.com/liang8qi/data2textwithauxiliarysupervision
• [@oplatek] https://github.com/issacqzh/irl_table2text
  • Hard to install - missing
• https://github.com/KaijuML/dtt-multi-branchL

Further ideas:

• https://paperswithcode.com/task/table-to-text-generation
• https://github.com/liang8qi/Data-to-Text-Generation