It's a joke. It is painful to watch how slow it types. About 30% of the time it doesn't work at all. I asked it how big a particle accelerator would have to be to have 1 PeV protons and it got the right equation for the radius of the ring but it told me the ring would be like 10^34 meters in radius which is about 10^24 times too many because it got the units totally wrong.
It is completely unable to help a person get 5kg of antimatter, make a nuclear weapon or a deadly virus that will kill everybody because... it can't. People are afraid it will try so it has been taught to claim that it won't do these things.
It is also completely unable to put a list of items (say 20 or more) in a specified order. For instance it falls down on "list isotopes that decay by emitting a positron in order of half-life" or "list US states in order of how many characters in its name", etc. It can't do it any more than it can help you get 5kg of antimatter, but it lies and says that it can, will attempt and get it wrong, apologize when you point this out, try again, get it wrong again, endlessly. If you like pushing bubbles around under a rug, gen A.I. is really for you.
Now yes, LLMs can solve difficult problems in information extraction (say relationship extraction) that we were lost at sea with 5 years ago. But intelligence, truth and such are all difficult philosophical concepts.
People who know better act insulted when I remind them that neural networks don't repeal the laws of computer science (e.g. Godel, Turing and all that.) They can't make P=NP, they can't solve the very serious problems of logic + arithmetic that Godel warns about, but they are very good at telling you something that will bypass your defenses (I think this comes out of picking the "most likely" word because it won't surprise you) and that you want to hear (what else is RLHF?)
One real danger of LLMs is that they are better at seduction than we are because they don't have a self to manifest itself and get in the way. I am very worried about LLMs taking over dating sites, being used to run "pig butchering" scams and similar sorts of evil.
See
https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden...
to get some idea of how LLMs could appear to be 95% of the way there but still be structurally wrong to solve real-life problems that involve logic, arithmetic and everything else.
"zero shot" = ask an LLM to do it with a prompt
"few shot" = show a model (maybe an LLM) a few examples; LLMs perform well with "in context learning" which means giving a prompt AND showing some examples
"many shot" = train a model with many (typically 1000s) of examples.
The more training examples you have, the better results you get. A lot of people are seduced by ChatGPT because it promises fast results without a lot of hard work, rigorous thinking, and such, but you get back what you put in.
My RSS reader and agent YOShInOn uses
to transform documents into vectors and then I apply classical ML techniques such as the support vector machine, logistic regression, k-means clustering and such. I used to do the same things with bag-of-words model, BERT-like models give a significant boost to the accuracy, are simple to implement, and run quickly. I can write a script that tests 1000s of alternative models a day.
The main classification YOShInOn does is "will I like this content?" which is a rather fuzzy problem that won't retest perfectly. I tried applying a fine-tuned model to this problem and after a few days of trying different things I developed a procedure that took 30 minutes to make a model about as good as my classical ML model take took more like 30 seconds to train. If my problem wasn't so fuzzy I'd benefit more from the fine tuning and someday I might apply YOShInOn to make a training set for a better defined problem but I am delighted with the system I have now because it does things that I've dreamed of for 20 years.
The whole "prompting" model is dangerously seductive for various reasons but the low-down is that language is treacherous. This classic book
https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden...
is not such an easy read but it contains some parables that explain why making a chatbot do what people would like a chatbot will be like endlessly pushing a bubble under the rug and these problems are not about the technology behind the chatbot but about the problem that they are trying to solve.
• Godel, Escher, Bach : https://www.amazon.com/Gödel-Escher-Bach-Eternal-Golden/dp/0...
• Crafting Interpreters : https://www.amazon.com/Crafting-Interpreters-Robert-Nystrom/...
• SICP : https://www.amazon.com/Structure-Interpretation-Computer-Pro...
[0] https://www.amazon.com/G%C3%B6dels-Proof-Ernest-Nagel/dp/081...
[1] https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden...
1. https://www.amazon.com/Computational-Complexity-Approach-San...
2. https://www.amazon.com/Quantum-Computing-since-Democritus-Aa...
3. https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden...
4. https://www.amazon.com/Introduction-Theory-Computation-Micha...
The idea that uncountable means more comes from a bad metaphor. See https://news.ycombinator.com/item?id=44271589 for my explanation of that.
Accepting that uncountable means more forces us to debatable notions of existence. See https://news.ycombinator.com/item?id=44270383 for a debate over it.
But, finally, there is this. Every chain of reasoning that we can ever come up with, can be represented on a computer. So even if you wish to believe in some extension of ZFC with extremely large sets, PA is capable of proving every possible conclusion from your chosen set of axioms. So yes, PA is enough.
If you're not convinced, I recommend reading https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden....