I wonder if you could have one AI that specializes in coming up with arguments, and another that specializes in verifying everything put out by the first AI.
It’s like all these image-generating AIs that garble letters and numbers. Can’t someone teach it basic letters and numbers and install that as a parameter that can’t be violated? Like how hard is it to teach it “These are the countries of the world. If someone asks you to create a map, unless they specifically ask to make up country names, use these names”? I’m guessing it’s a much harder problem than I imagine, because no one’s done it yet.
Once we get AI + basic 1st-grader knowledge about the world, we’re fucked.
Drafting a basic will or property transaction I’d give you, but I’m skeptical when it comes to a merger agreement. Not that I don’t think computers can draft well (there’s a reason why templates and form filings exist), but because so much of the negotiation and drafting revolves around around strange indiosyncratic demands of the parties rather than legal requirements. A lot of stuff that may initially seem legally suboptimal may have some deeper logic behind it. I guess if you gave the AI super detailed instructions about deal terms and rationale it could work, and it would probably be cleaner in terms of typos and version control, but you’d still need a lot of oversight to make sure that all of the deal objectives were being met by whatever agreement it spit out.
[Eta: I guess my point in all this is that AI is going to be a powerful assistant especially at some tasks but humans are going to be very reluctant to set up a black box system where the AI handles a matter from soup to nuts. And that human impulse is going to slow AI adoption even once the technology is good enough]
You’re describing symbolic AI, which is not a new thing. The currently hot models use a connectionist paradigm. I assume people are trying to marry the two approaches, but Idk how successful they are. GPT proponents might claim it’s not necessary, that these AIs are already generating useful models of the world. I don’t know enough about it to have a strong opinion.
I would just like a cartoonish black and white map of Mexico and Central America that shows non-hallucinated country names. Is that something AI can do for me at this point in time?
Beats me. When we were talking about NFL coinflips, I tried to get an explanation from AIs and they gave BS responses based on made up rules. When I read the rules and asked based on my improved understanding, they gave good answers though.
I think that’s where a lot of the progress will come. I.e. you’ve got a bunch of broad capabilities in the new wave of models.
To make that really useful in narrow fields going to require some building of solutions using combinations of different AI, human thinking and design, and traditional programming.
AI is actually really strong at handling weird outliers. Converting arbitrary requirements into reasonable legal language is probably the part of drafting an LLM would be best at. The problem will be cross-references and consistent defined terms. Also subtle issues with very nuanced details.
More importantly though, there’ll be major dislocations once AI is just a better, cheaper associate. Law firms will replace their associates with cheaper AIs in a heartbeat.
I have no idea what exactly was used to get that answer from chatGPT as you should be able to restrict it to only cite actual cases. You would have to tell it what cases it can use. That still doesn’t mean the argument makes sense legally though.
I expect a model trained for legal usage can already do this without hallucinating. Basically the research now being done by junior lawyers. Just don’t ask it to come up with something new and you still need to check its work.
It’s not really that complicated. LLMs are auto-regressive, which means that they build up long outputs one step at a time. Each word that is generated is added to the end of the input to form the input used to generate the next word. One consequence is that it can generate itself into a dead-end, where it doesn’t know how to continue. If it inserts a footnote reference, it knows it has to complete the reference, but if it doesn’t know what citation is correct, the best it can do is create something that structurally looks like a citation, even if it isn’t real.
In all likelihood, the model “realizes” that it’s in trouble (i.e., it doesn’t have a strong preference for what comes next), but it doesn’t have the option to go back and skip the citation.
8/ Limitations: A typical 2-year-old’s vocabulary and word learning skills are still out of reach for the current version of CVCL. What else is missing? Note that the modeling, and the data, are inherently limited compared to children’s actual experiences and capabilities.
CVCL has no taste, touch, smell.
CVCL learns passively compared to a child’s fundamentally active and embodied experiences.
CVCL has no social cognition: it doesn’t explicitly perceive desires, goals,or social cues from others, nor is it motivated by wants and doesn’t realize language is a means of achieving wants (like cheerios!).
This is where it gets fascinating to me and feels like the first baby step (hah) to some kind of real human-like AI. It would need to have a massive embedded codebase just for facial expressions. Smell, touch and taste would be awesome. And you’d need to artificially create wants and desires. And then Skynet here we come!
Also, internal states like balance, hunger, pain, temperature. Action/reaction from manipulation of the environment. Still, this is a genuinely interesting experiment.
Marcus is pro-AI, he just believes that deep learning needs to be united with logic-based “traditional” AI systems. (His views are much more informed and nuanced than this.) His substack is quite good and I believe free.
Though I’m unfortunately out of my depth these days (despite having studied with some neural net pioneers in the 90s), I would think that logic should be capturable in neural net representations, not hard coded. I suspect that if you can implement neural nets that do hierarchical categorization and inference with venn diagrams, you can get standard inferential logic from such a system.
