SuperGLUE is a challenge that evaluates the performance of AI technologies for a variety of natural language understanding tasks. It is characterized by consisting of tasks that are relatively more difficult than the existing GLUE, and the DeBERTa model recently announced by Microsoft achieved SOTA (state-of-the-arts), and the evaluation result is said to exceed human performance. In particular, while the existing SOTA model, Google's T5-based model, used about 11 billion parameters, DeBERTa showed better performance with only 1.5 billion parameters. Here are some related articles and links to github:
It is implemented based on Pytorch 1.3, and the trained model and source code are also released on github. Of course, even if SuperGLUE surpasses human performance, it does not change that the result is limited to a specific task, but it is considered a significant step forward to equip a more human-like conversational ability.
