11.5기의 beyondBERT의 토론 내용을 정리하는 repository입니다.
- 논문 당 한명의 호스트가 모임을 진행합니다.
- 호스트는 배정된 논문의 idea 위주로 정리합니다.
- 사전학습으로 참가자는 주차 별 논문에 question을 한개 이상 준비하고, 논문 issue의 thread에 question을 등록합니다.
- thread에 달린 question에 thumb up을 하고, thumb up의 개수가 높은 question 위주로 모임시간에 토론하고 해결합니다.
- ice breaking
- 진행 방식 결정
- The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
- How multilingual is Multilingual BERT?
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- Data Augmentation using Pre-trained Transformer Models
- Mask-Predict: Parallel Decoding of Conditional Masked Language Models
- Unsupervised Data Augmentation for Consistency Training
- You Impress Me: Dialogue Generation via Mutual Persona Perception
- Recipes for building an open-domain chatbot
- ToD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogues
- A Simple Language Model for Task-Oriented Dialogue
- ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation
- FastBERT: a Self-distilling BERT with Adaptive Inference Time