https://twitter.com/blader/status/1670578014508433410?s=46&t=9xanL2tZoKj22erGoTuL4A
Terence Tao:
The 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process.
My sister dated Terry Tao’s brother for some years. That guy is among a handful of true geniuses in the world, a freak of nature. I’ll see if I can pull some strings and ask him for his opinion on the contribution of ants to mathematics.
I can think of a number of substantive hints and promising leads ants and other insects, such as bees, can offer mathematics, particularly in geometry. Hell, their broader behavior suggests the hypothesis that a straight line may be the shortest distance between two points.
[ ] “Are you being sarcastic and/or abusive?”
This is a joke right
Bing: For example, the electron is a very stable particle with a very long lifetime, because it has no electric charge and no lighter particles to decay into.
This should be useful.
This is a basic intro into how LLM work. I was following it for a bit but eventually lost me due to tired head.
Logically I’m not surprised by this.
My understanding is that the LLM is trained by repeatedly trying to convince a different machine learning model that it is fact a human not an AI.
Seems weird to think you could create another AI that does a better job of telling the difference that couldn’t just be used to improve the LLM
One thing I still don’t understand (and I think part of the problem is that it’s different for different versions) is the extent to which the models are able to “research” versus “remember”. Actually, I’m specifically talking about GPT4, I think (or maybe 3? Which was the one whose “knowledge” was cut off at 2020 or so?).
As I was trying to explain/demonstrate these things to my dad a few weeks ago, I got a little confused. My dad’s a pretty well-known musician in his genre (and especially locally) but by no means famous. When I asked GPT to tell me about him, it accurately described him in terms of his genre and instrument and reputation, but it named three famous musicians who had played with, and those names were made up (however, they were comparable to the ones he has played with).
When I asked the same question to Bing (and whatever model they use), it gave a similar answer, but since it could look up info on the web (and cite its sources), it correctly named some of the famous musicians he’s played with.
So, for the first model, I understand why it “hallucinated” the second part of its answer (and thereby helpfully demonstrated why less sophisticated models are so dangerous), but I don’t know where it turned for the accurate first part of its answer. Does it have something like a database (or “memory”) of what it has “read” in the past? If so, why does it leave that information and start inventing things?
Even as a computer programmer I was still pretty lost on this.
GPT4 base relies on “memory” for all of its knowledge. There is a version (currently disabled) that can browse the web for information.
The hallucination/confabulation issue is that GPT doesn’t have a clean distinction between memory and the rest of its “brain.” Its neurons encode patterns that help it anticipate how to continue a text. But in most cases, those facts are likely stored in some “dehydrated” fashion to save space, and the model has to fill in the blanks when it wants to use the fact. For simple memories or ones that are important enough to get a lot of space, it’s not a problem. But if the memory is complicated and it’s being asked a more detailed question than it knows the answer to, it will just fill in the blanks in a plausible way.
GPT isn’t trained like this. You’re probably thinking of GANs.
More importantly though, the last step in GPT’s training is fine tuning for instruction following and safety. It’s completely plausible that the fine tuning step would introduce detectable signatures that you wouldn’t want to eliminate. Indeed, the “As an AI language model” lead-in is a well known example of such a signature, it’s just easy for cheaters to suppress.
Chat GPT wasn’t adversarial?
Correct. Masked token prediction followed by instruction-following/safety fine tuning.
