https://futureofcoding.org/ logo
#of-ai
Title
# of-ai
d

Denny Vrandečić

08/31/2023, 9:45 PM
Doug Lenat, inventor of Cyc, has passed away
k

Kartik Agaram

09/01/2023, 1:16 AM
Where are you seeing this?
d

Denny Vrandečić

09/01/2023, 1:17 AM
One of his former students and colleagues is a friend of mine, he broke the news on his Facebook page.
k

Kartik Agaram

09/01/2023, 1:19 AM
Thanks. Lenat (along with Hofstadter) inspired me to come to the US for grad school in hopes of doing AI research. I ended up not going into AI, but still bear a soft spot for Eurisko and Copycat.
a

Alex Cruise

09/01/2023, 9:19 PM
I’ve wanted someone to try to build bridges between Cyc, HTM (Numenta) and the FARGitechture for so many years
speaking of which… it’d be great to find a community of more-than-me who can’t stop thinking about FCCA. The book is most famous for being the first product ever sold by Amazon, but I think it deserves a lot more attention
d

Denny Vrandečić

09/01/2023, 9:27 PM
what's FCCA?
a

Alex Cruise

09/01/2023, 9:27 PM
Fluid Concepts and Creative Analogies
Hofstadter’s least famous book I guess
published after GEB
d

Denny Vrandečić

09/01/2023, 9:28 PM
Ah! I bought it and read it. And then forgot about it. And then I heard about it and was like, Oh that sounds interesting. So I wanted to buy it again. And Amazon told me "You already bought that book before, are you sure?", and I was like "Oh yes I did, and I read it before, thanks!"
a

Alex Cruise

09/01/2023, 9:30 PM
I’ve had a strong hunch for years that the copycat/etc. architecture could generalize to so many more use cases, but I don’t think anyone’s tried very hard to turn it into a framework rather than a collection of one-offs
If you compare it to SHRDLU, which was famously a pile of spaghetti hacks, it definitely deserves to be further along the spectrum towards “generally applicable to multiple problem domains”
d

Denny Vrandečić

09/01/2023, 9:32 PM
I think that's a good hunch. Most of the techniques in FCCA (here, I am using the acronym!) seemed to me, at the time of reading, basically unimplementable. Today, thanks to LLMs, this might have changed. If one were to use an LLM as the API to the natural language text, and implement some copycat processes on top of that, the result might be interesting.
a

Alex Cruise

09/01/2023, 9:33 PM
e.g. first-class modeling of concepts and their relationships (i.e. the “slipnet”)… the conceptual relationships are part of how the system works, not just a bunch of tags
Cyc has higher-order reasoning, so the concepts really do have semantic functions
With ANNs you hope they’ll learn the concepts but good luck explaining anything 🙂
@Denny Vrandečić could you say more about the association you’re seeing between FCCA and LLMs? My recollection is the FCCA problem domains were mainly about simpler symbolic systems
d

Denny Vrandečić

09/01/2023, 9:43 PM
IIRC (which may not be the case, as said, I previously forgot I have even read the book!), a lot of the work in FCCA is in the space of concepts. The best way we can get to concepts is through natural language texts. But until recently, our ability to write code against natural language text was limited. This has dramatically changed with LLMs, as we can now use text like a database (it's expensive, but we can!). So it should be possible to implement the reasoning algorithms from FCCA and have them run on a text corpus. Again, I might misremember everything.
a

Alex Cruise

09/01/2023, 9:47 PM
There are two ways I can interpret what you’re saying, probably both wrong: 1. use an LLM to help populate nodes and edges in the slipnet — basically semantic autocomplete? 2. build an FCCA system that understands/learns grammar?
d

Denny Vrandečić

09/01/2023, 10:19 PM
I think 1. Not just help, but do it automatically.
a

alltom

09/04/2023, 8:17 PM
I’ve read FCCA several times, and Analogy-Making as Perception, and Copycat’s source, and can confirm that it’s unimplementable. 😂 I did get a nub of it working in Wolfram Language the other day, though. And I have ideas for how LLMs may be used to break out of the toy domain. I’d love to talk (a lot) more with other people who have thought deeply about it, because I share the belief that there are useful techniques that would be useful in programming generally, in the way Sussman had in mind when he said, “This piece of the problem wants to be a functional program. This piece of the program wants to be imperative. This piece wants to be object-oriented. And guess what, this piece wants to be logic-based.” If I ever have the time to devlog, it will be about this. :)
a

Alex Cruise

09/07/2023, 10:28 PM
@alltom I strongly agree about different parts of non-trivial programs wanting different implementation languages/DSLs, that's part of what makes Tar Pit so endlessly fascinating to me... Yet another holy grail that's unlikely to ever be implemented well 😉