Scatter Lab stands out in chit-chat conversation research (https://scatterlab.co.kr/). This is an article on their Ping-Pong team blog, and I share it because it seems to be well-organized with a story about the GPT-3 cases. I'm still looking at GPT-3 as a 'eye of doubt', but it is definitely true that the GPT-3 works marvelously.
However, how about expanding the scope of GPT-3 a little more instead of comparing it with existing language models? For example, if you compare it to “Google Search”… Whether it's the breadth of knowledge it covers, insights, and troubleshooting information, Google Search provides more information than the current GPT-3 examples except for the difference that input/output is not a refined form in natural language.
It may seem a little sophistication, but considering that GPT-3 was created through learning “fill in the blanks” with large amounts of data, Google Search is the result of applying a variety of indexing techniques to a much larger amount of data. If you apply a language model to Google Search to create natural language input/output forms, you might be wondering what it would be like compared to GPT-3.
If the result is similar to what GPT-3 currently shows, I think it is rather to disprove that the result of GPT-3 is not inference, creation or creation, but a search for a vast amount of knowledge that already exists.
Otherwise, if the result is different from GPT-3, I hope that you will learn a lot by analyzing and researching the “differences”. For example, like the difference between “memory” and “creativeness”. Here is a link to an article on the ScatterLab blog:
On the other hand, Karpathy released the GPT structure and training script made with PyTorch through a project called minGPT, and all GPT-1, GPT-2, and GPT-3 can be expressed by simply changing parameters. Of course, it was created for educational purposes, and it is difficult to reproduce the data used to learn GPT-3 because it is not public, but it seems to be useful for those who are curious about the structure of GPT. And, if you are a company that has a large amount of training data, I think it could create another GPT-3 variant. Here is the minGPT github link.
