has anyone seen this? http://www.try-alf.org/blog/2013-10-21-relations-as-first-class-citizen I think it's an interesting idea... I think almost all high-level programming can be done with only types, relations, and pattern-matching/destructuring. (of course, we will always need people or Machine Learning to find how to execute the high level code in efficient ways)

I haven't, but thanks, I love it! I'm curious to hear other opinions, though, because I hashed out a related API in here a while back without being able to explain my motivation nearly as well, and seemed to get mostly confused looks.

"Almost all high-level programming" sounds like a huge claim for something that looks specific to databases. Am I overlooking something that makes this idea much more general?

This idea is more general than relational databases. 1. The author references "Out of the Tar Pit", which persuasively argues that relational programming should be used for general purpose programming. http://curtclifton.net/papers/MoseleyMarks06a.pdf 2. Every general purpose language has hierarchical data structures, and means for querying and updating these data structures. For my Curv language, I started by copying these features from popular languages. So, I have records and arrays; R.foo selects field 'foo' from R; A[i] selects element 'i' from A; etc. And I have multi-dimensional array slices. But I discovered you can't abstract properly over these language features, and they aren't properly composable. So I've broken down these features into orthogonal, composable query and update operators, and I have rediscovered the "Lens", popularized by Haskell, but Lens packages are popping up in every language. And I discovered that Lenses are good for building pure functional GUIs that are modular and composable (you use them to connect the model with the view and controller). And I noticed that my new query / update operators resemble relational algebra without the "join". So I'm looking at Tarpit and this paper, and wondering if I should just add a join.

I don't understand this at all. The low level relational algebrea is handled by the SQL engine itself. What is any of this work buying you?

@Samuel Squire The relational algebra is a more suitable abstraction for programming than SQL (aka the relational calculus). Quote: • SQL has not been designed with composition and separation of concerns in mind, • Avoiding strong coupling between subqueries tends to be very difficult in practice, • Coupling hurts separation of concerns and software design. This shows that embedding SQL in a programming language is not a great idea. It is a poor API, and this paper demonstrates a better alternative (the relational algebra).

awesome, I like this direction a lot, I’ve also had a nagging hunch that all program data can (and maybe should) be stored as relations

LINQ I understand and it is good. I'm not seeing the connection between this and LINQ

@Andy F I don't agree with the Tarpit paper that all program data should be stored as relational tables. For my domain (computer graphics), you need multi-dimensional arrays and linear algebra. Storing a 4x4 transformation matrix as 16 individual cells in a relational database seems crazy. ADTs are heavily overused in OOP programming, and I'm not a fan of the style, but I still think we need abstract data types for at least some use cases. The ability to perform relational queries on compound data encapsulated in ADTs is cool.

@Doug Moen Something I’m optimistic about (and I have a baby side project in this area) is that a compiler could code-generate a fast path that doesn’t run full database queries at runtime. The programmer would need to declare all data access patterns upfront, and maybe give some implementation hints too. Then the compiler converts those access patterns into efficient in-memory operations and data structures. So the final program ends up doing the same stuff

If a database stores queries themselves, it can make data layout and storage decisions that are fast

@Doug Moen Interesting, thanks for the explanations! Are there any examples of non-database code using relations and relational algebra? I remember being rather disappointed by the tarpit paper recommending this style but not giving any concrete examples.

@Konrad Hinsen Eve puts almost everything into a database. IIRC, it didn't put much of a spotlight on the acts of projecting and joining, since they were so basic to the execution model.

This sounds cool, but I don't quite "get" the new algebra he is introducing. For one thing, in what sense are relations "first-class citizens"? Maybe I don't know the definition of a "relation". Also, why does the join work?

project(
  join(suppliers, cities),
  [:sid, :name, :city, :country])

does it choose implicitly to join on "city" just because the two tables both have a city column? But what if the column names don't match, and how is this going to work if two tables happen to share a column name but it is just a coincidence and not intended for joining? I have always found relational algebra unnatural. In non-DB code, if you want to know what city a person lives in, it's just

person.address.city

... DBs are so clumsy in comparison, constantly making you think about how to join tables together while including and excluding the right data. You might say "well what if you want to know what people are associated with an address... not so easy then, huh?" Yeah, then I'd have to run a query like

people.Select(p => p.address).Where(a => a == address)

, but by golly, simple paths like

person.address.city

are both very convenient and maximally efficient computationally.

@David Piepgrass AFAICT, it's just the natural join from relational algebra, with formal definition here: https://en.wikipedia.org/wiki/Relational_algebra#Natural_join_(%E2%8B%88)

@David Piepgrass A relation is just what you probably think: a table, or a set of tuples. Relations are first class in Alf because relations can be directly manipulated as first class objects (of class "Relation") in the host language, Ruby. This is in contrast with the software he is comparing Alf to, which allow you to run SQL queries against an external relational database, but don't model the relations as objects. In 2013, maybe this was a bigger deal than it is today.

The analogue of implementing an interface for a struct in Go in order to call a function like sort.Sort would be to project/join your way to having the right fields for applying another, first class relation. Like a SQL statement with parameters, such as

SELECT a FROM $tbl

.

Okay, so relations are not relationships (which certainly feel second-class in SQL). But what does it mean for tables to be first class? Couldn't we say, for example, that that a growable list of records (objects with properties) in C# is basically the same as a table, and that it is first-class in some sense? It seems like maybe the goal is to make "query composition" first class, or something like that.

i've come around to this idea now. It's a good idea.