Skip to content
This repository has been archived by the owner on Oct 18, 2021. It is now read-only.

Preprocessing #5

Open
bartvm opened this issue Dec 2, 2015 · 0 comments
Open

Preprocessing #5

bartvm opened this issue Dec 2, 2015 · 0 comments
Labels

Comments

@bartvm
Copy link
Owner

bartvm commented Dec 2, 2015

For our models so far we've always done minimal amounts of preprocessing (anything OOV is mapped to UNK, that's about it). Preprocessing could help (and perhaps more so than with SMT, considering we have bigger issues with rare words), and Henry mentioned gaining 3 BLEU points by playing around with preprocessing, making it worthwhile to consider making this part of our pipeline.

@bartvm bartvm added the research label Dec 2, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant