The same lmao that part is dumb. As are the final moments of the movie showing the cold zone greenhouse. It’s a horrifying ending presented as a victory for true love or something.
1 Like
This is old, but it discusses the use of RLHF to improve truthfulness:
It’s absolutely true that hallucinations are a big problem for current-gen AI, but it’s not true that the models “just predict the next word” or “don’t care if they are being truthful.” A huge part of the recent AI advance has been teaching the pre trained models to care about stuff other than token prediction.
1 Like
I think Deustch is largely spouting BS here (the turing stuff is 70s cognitive science), but I do think his second criterion gets at something. The capacity to understand/explain/infer is what is most essential to human cognition, which is related to categorization/analogy, using base-level similarity recognition to bootstrap into comprehension of objects/relationships/causes. It seems to me that current AI is all in on brute force similarity computation but without anything that I would regard as categorization (which is maybe necessitated by the limited capacity of brains and the paucity of information encoded into language, turned into a virtue). I suspect that once you have categorization you can get logic, because logic can be modeled by inclusion/exclusion in venn diagrams, ie, category membership.
Basically, I think current AI is all in on brute forcing step one, but we need step 2 and 3 to get intelligence, which is a version of what Gary Marcus argues in many places.
One of the most famous works of philosophy is Hume’s Enquiry Concerning Human Understanding. It seems like not more than 5% of people in AI have read it or grappled with Hume’s effort to characterize thought and understanding. They are just using their hammer to hit everything and hope it’s a nail.
This is to say nothing of goals. Goals regulate behavior of every organism. For things like viruses, bacteria, or plants, the “goals” are externally articulated, because the virus/bacteria is essentially playing out an elaborate chemical dance underwritten by evolution. For things like “higher” mammals, we can have long-term “rational” goals and act on subgoals dictated by a rational plan based on our model of the world. This is a complex issue in AI, because things like a chess or go-playing AI has goals, at least from an external perspective, and these goals look something like the “goals” of viruses, but I am not aware of any neural networks that formulate subgoals or anything likely externally directed goals. This is in part because they have no interests that drive behavior. I suspect that if we can give AI something like interests it will generate goals, and then it can maybe bootstrap itself toward intelligence by building up a representation of reality that allows it to work toward its goals. At least I suspect that would be a necessary step for general intelligence.
2 Likes
Agreed.
I would say something like “self motivated goals.”
But I need to read Hume. I don’t remember if I ever read it.
Targeted learning and filtering prompts and other methods reduce the amount of bad results getting through but the AI doesn’t know truth. Anyway this is a bit nitpicking and I agree my original post was too strong but too many people and news outlets have completely wrong ideas what you can and can’t do with a LLM.
We live in a world where sophisticated nation-state actors can disrupt the US power grid. This statement seems a bit rosy.
1 Like
https://www.nature.com/articles/s41586-023-06647-8
This is not good, but it is kinda funny that it’s basically a ripoff of a LessWrong post that got published in Nature.
I need to go find the article but they did a productivity study on programmers and found chat GPT boosted low skilled programmers’ productivity while not really helping highly skilled programmers
1 Like
I would imagine that this may something that may be more complicated than that. With a bunch of confounding factors.
E.g. this could be “young programmers improve more with ChatGPT than older programmers”
Or
“Solving new problems is helped more with chatGPT than well known problems”
Or there’s an incentive hump. Using ChatGPT gets you a faster return for utility if you are new to something.
That makes sense to me.
Like in storytelling, a lot of people would be well served by being handed plot templates and suggested outlines for each chapter, even elements throughout each chapter that then need to be fleshed out by the author.
But high level writers using ChatGPT for storytelling are like Peter Frampton trying to play Guitar Hero. I don’t do it because there’s way too many elements I’m playing around with that need me to be the one to see where they go and why I put them there from the beginning idea. Obv not referring to myself as any version of a guitar legend in my field just an analogy lol.
Yeah. But maybe there’s also an additional level of chatGPT which involves finding and developing mental tools that improve storytelling and creativity that could aid writers. But that takes a lot more effort, and you won’t be getting any utility of that effort upfront. Unlike a less accomplished story teller
1 Like
I worked very briefly with a company called Authors AI/Inkers Con, which utilizes an AI-based editor for those wishing to purchase feedback from it rather than or in addition to a human option.
As the human part of the equation, I was really impressed. I think there is a lot of sense to what you say and your point that once the tech operates in the manner I would need, absolutely I’d utilize it as a tool for creativity. Just nowhere near what I would need.
Soon tho.
2 Likes
I’ve become mildly obsessed with the idea that the missing link for AGI is integrating causal modeling (GPT-style next token prediction) with acausal modeling (Stable Diffusion-style modeling where you work with the whole picture all at once). You can train an LLM on a fill-in-the-blank task, which is acausal, but then it doesn’t learn about cause and effect or build a useful world model, so it’s not that smart. But producing coherent long-form text one token at a time is just so insanely hard that it shouldn’t work as well as it does. Especially when you consider that you often want the model to be creative, which means it needs to be a bit unpredictable. So you have thousands of near-totally independent executions of the model each trying to do their own thing and contribute their own token, like the worlds largest game of exquisite corpse.
But yeah, I think this is the root of GPT’s weakness at serious thinking.
1 Like
Potentially important paper on how transformers work. Basically, they designed an architecture that was narrowly focused on compressing data and found that it achieved similar performance to transformers, suggesting that transformers are basically doing some sort of compression task.
Suggests that hallucination and GPT’s ability to perform tasks that aren’t in the training set are two sides of the same coin. In the latter case, the prompt is converted to decompression instructions for data that was never originally compressed, and those instructions produce a novel correct answer. In the hallucination case, they produce a made-up fact or strategy. I don’t see imagination used in a literal sense very much for LLMs, but that seems like a really good word to capture what’s going on (according to the paper at least). The LLM asks itself what an answer would look like if it existed, and it believes that whatever it imagines is true. Its imagination is trained by reality, but it doesn’t really have a distinction between remembering and imagining. Humans are similar in that memories are largely
hallucinated/imagined/confabulated, but (normally) from a kernel of remember truth.
1 Like