Is anyone aware of a data structure that supports ...
# linking-togethera
alltom
09/10/2019, 7:50 PMIs anyone aware of a data structure that supports SQL-like operations on bags such as projection, selection, and join? I wrote something in Java patterned vaguely on Guava's Table interface, but my first crack at the API was too unreadable to actually submit. 😅 The important part for me was that it had a join that was implemented as the application of an arbitrary mapping function (input row → 0 or more values for the new column).
🤔 1
alltom
09/10/2019, 8:06 PMFor example, you might want a table like <country, account, agent, agent-is-active> with restrictions like the agent is located in the country, and whether an agent is active is account-specific. So it doesn't compress nicely into countries.map().map().map(), unless you either make a record type for each intermediate result so you can refer back to the country later in the computation, or one big record type where each field's validity depends on the current level of nesting. You can alternatively use for-loops if you're willing to nest a bunch of for-loops.
d
Don Abrams
09/10/2019, 9:28 PMI’m confused; is the data structure a bag?
a
alltom
09/10/2019, 9:48 PMI wanted to make sure people knew that I wasn't talking about sets even though I was using relational algebra terms... From the way the Wikipedia page is worded, it seems that relational algebra fans are generally sad that SQL operates on bags instead of sets: "Even the query language of SQL is loosely based on a relational algebra, though the operands in SQL (tables) are not exactly relations ... (arguably to the detriment of optimisers and/or users). The SQL table model is a bag (multiset), rather than a set." https://en.wikipedia.org/wiki/Relational_algebra#Implementations
alltom
09/10/2019, 9:50 PMBut the container of rows is only half the data structure, since you also need to describe the columns...
alltom
09/10/2019, 9:51 PMEh, maybe I need to publish a draft of the library for it to be concrete enough for anyone to know what I'm talking about. 😛
alltom
09/10/2019, 11:22 PMalltom
09/10/2019, 11:27 PMflatMap but you get to keep the left side of the map for reference in future maps.
d
Don Abrams
09/11/2019, 6:24 AMLooks an awful lot like CQL
a
alltom
09/11/2019, 1:57 PMThanks, I haven’t heard of that! I also realized that it’s probably just a regular flatMap that feels very natural to write in languages where it’s easy to add and remove fields from records, and I just haven’t needed it in a language with syntax for that yet.