【Dr. MIT Thesis】 Control neural language generation

Author:Data School Thu Time:2022.09.07

Source: Specialty

This article is introduced in the paper. It is recommended to read for 5 minutes

This article explores the flexible sequence model and weak supervision methods to perform various control generation tasks.

Large -scale neural language models have made impressive progress in natural language generation. However, the typical model runs in a way from left to right, unrestrained, and has limited control of generating content. This article explores the flexible sequence model and weak supervision methods to perform various control generation tasks. We expect these technologies to be widely used in other fields, such as the generation of images, molecules and biological sequences.

We first introduced a sequence model called blank language model (BLMS), which generates sequences by dynamically creation and filling gap. Given a specified text with one or more spaces, BLM will fill these spaces with a label with the consistent variable amount of context. Our model is very suitable for various text editors and rewriting tasks, and proves effectiveness in text filling, ancient text recovery, and emotional migration.

Next, we study the use of text automatic encoders and its use of potential space operations. We have established a theory how to shape a meaningful potential space geometric discrete text data. On this basis, we have developed a series of automatic encoders of noise, which shows the potential of attribute modification (such as, tense, emotion, etc.) through a simple vector algorithm.

The last two chapters discussed the migration of language style in the absence of supervision data. We first form the task of migration of non -combined style and discuss the feasibility of learning problems. We have proposed a method of using potential distribution to perform style transmission. Then, we studied the mixed factor and stated that by dividing the data into two different styles, the set in each group shows that we do not want to change the change. Delivery the text.

https://dspace.mit.edu/handle/1721.1/144561