I just downloaded K from <Shakti.com>. The executa...
# thinking-together
d
I just downloaded K from Shakti.com. The executable is 84728 bytes on Linux (after stripping the symbols), depending only on libc. That is pretty compact for what you get: a powerful array-oriented programming language plus a high performance column-oriented database, and a REPL. If I slim Curv down to just the interpreter and compiler (no REPL, no UI, no OpenGL or mesh export) then the executable is 3.5MB without symbols: 42x larger. K is arguably a more powerful language than Curv, and it is definitely much faster.
๐Ÿ‘ 4
w
I'm guessing K is entirely self-contained, no libraries used
d
K is entirely self-contained. The implementation is < 1000 lines of C, depending only on libc. Not open source, unfortunately. Curv is written in C++, and template expansion is partly responsible for the code bloat. Plus, there is just a lot more code.
w
is it self contained beyond C++ STL and such?
ahh sorry, just noticed it probably pulls in a lot of 3D stuff
Windows lobster.exe is 4MB but 90% of that is the 3D engine and its dependencies
when I first wrote cubeengine.com, I was able to keep it under 64K for a while.. this being an OpenGL 3D engine with a multiplayer FPS.. in 2000
the latest version (from 2005) is still 149K ๐Ÿ™‚
d
One reason to care about executable size: if you are compiling your language implementation into WASM to run in a web browser. Download time of WASM modules matters.
Another reason to care about executable size: According to the K people, one reason K runs so fast is that the entire system fits in the CPU cache.
๐Ÿ˜ฎ 1
w
yup and yup
d
The 3.5MB curv executable I mentioned is statically linked, depending only on Boost::filesystem, Boost::system, and double-conversion. It contains a Curv parser, interpreter, and compiler that outputs GLSL. No actual 3D stuff. It's a consequence of using C++, and years of code accretion without worrying about code size. The core library is now 19,000 lines of C++, and I'm not sure how it got that big.
w
thats some pretty expansive C++ yes ๐Ÿ™‚
k
Provocative claim: All our high-level languages are only good for building prototypes. Since all software is built in HLLs, all mainstream software is prototypes. We've forgotten to throw the first one away on a grand scale. There's holes to poke in this argument, but the number of rewrites K has been through lead me to think it's more right than wrong. A healthy life cycle should lead to many more rewrites, and rewrites should successively trim dependencies. All the way down to machine code. (We may eventually end up with HLLs suitable for non-prototype software, but we can't do that while HLL implementations are all prototypes.)
๐Ÿค” 3
๐Ÿ‘ 4
w
I get a sense you're right about the benefit we would have from more ground up rewrites.. I don't what the relationship with HLLs is though
in some sense rewrites happen a lot.. just by different authors ๐Ÿ™‚
k
I don't what the relationship with HLLs is though
HLL implementations don't have enough ground-up rewrites.
i
This sounds like it might be a good candidate for Agaram's 1st Law (in the spirit of https://futureofcoding.slack.com/archives/C5T9GPWFL/p1580718026447200)
๐Ÿ˜„ 2
w
oh, those too
i
We certainly got a ton of value out of constantly rewriting. It's the only way I'm aware of to really combat the exponentiating wrongness problem (innocuous "bad" decisions cause further bad decisions as things get built on top over time).
As Joe Armstrong used to say, you're at your most ignorant the first time you write something, it isn't until the 5th or 6th that you start to know something. ๐Ÿ™‚
๐Ÿ‘ 4
We found this to be pretty tough for people though. If you're a craftsperson, you tend to place pride in the artifact and this requires what we ended up calling "artifact zen" - you just have to let the thing you produced go.
We struggled with that at several points during Eve.
๐Ÿ‘ 2
k
My comment also owes something to this description of the process Rob Pike went through in rewriting Jim to Sam: http://texteditors.org/cgi-bin/wiki.pl?Jim
Hack for getting to "artifact zen": take pride in the input space you have designed, rather than how you serve it. If you find yourself attached to the code, maybe you haven't written enough tests?
๐Ÿ‘ 2
i
Yeah, we talked about it as the outcome is the only thing that ultimately matters - did you solve the problem or not? If not, it doesn't matter how great the thing you produced is. Understanding and accepting the distinction between outcome vs output is a serious force multiplier in life ๐Ÿ™‚
โค๏ธ 1
k
@Wouter:
in some sense rewrites happen a lot.. just by different authors ๐Ÿ™‚
This might be the hidden variable in the debate over whether rewrites are good: whether learning accumulates. K's rewrites required Arthur Whitney's continuity on the project.
๐Ÿ’ฏ 1
w
yup exactly: the original author rewrites too little.. because they put too much value in their existing code.. anyone else rewrites too casually, since they can't see the value in someone elses code
โค๏ธ 2
maybe if people wrote more post-mortems ("If I'd do this project again, I'd change...") so we can get some leverage out of all this effort duplicating
oh wait, that would require people to read about people's previous efforts ๐Ÿ™‚
k
What we need is some way to scale up the review of "project X tried this, but ran into Y and Z." I don't know how to make tests work across projects.
e
The Author of the K language is named Whitney who worked with Iverson on the J language which was Iverson's sequel to the APL language, which is famous in language history because APL was super brief, super powerful, but used its own invented alphabet which make it cryptography to use. The K language is reminiscent of APL and J, but has its own twist to things. You can find the 1998 language reference manual here: http://www.nsl.com/k/k2/k295/kreflite.pdf Nowadays K has evolved into being part of a high performance analytics platform that included a database Kdb+. Evidently Whitney the inventor has been bought out and that product is with another firm, and recently he started a new company called Shakti, staffed with many former members of the KDB+ project, and they are offering a high performance analytics platform. Shakti has no documentation or materials on what their current language or product looks like. I think in this case if you have to ask you can't afford it. APL, J, and K are not suitable for normal mortals. They require too much mathematics training. Both Whitney and Iverson had higher degrees in Math and what they think is a reasonable level of abstraction is beyond the average mortal. I will take Prof. Wirth's more logic based approach in Pascal/Modula-2/Oberon any day of the week over array processing languages that love to force you to memory operators with tricky left-to-right or right-to-left application. But certainly array languages have validity.
i
K is about as far from OOP as I can imagine, and is used heavily in the financial sector - they've made many millions of dollars more than any of the languages the rest of us have ever made ๐Ÿ™‚
e
Having spent my early days making reports in FORTRAN for the first mainframe portfolio management system i am quite familiar with financial computing. And my mainframe buddies are still using terminal emulators, Assembler and COBOL. You would be shocked at how old some of the big banks' systems are. When i refinanced my mortgage a few years back at JP Morgan Chase, the officer was running on a PC a terminal emulator that was running DOS. One thing about mainframes that people don't realize is the amount of I/O those machines can handle. China Mobile has the fastest mainframe IBM makes, because they 700 million monthly bills to process. I think that is the largest number of paying customers any firm has in the world. Anyway i am sure that the algorithmic traders, and "quants" on wall street like to use array languages like K, but if you are building PacMan or making anything user centric like a web app or mobile app, it is useless. Very little computation is needed by the average person in their normal course of life. So the advantages of being able to spread a 1000x1000 cell matrix inversion across multiple cores is just not a factor, and these high performance array languages can do that.
d
The K language from Shakti.com is K version 9, and it is a development snapshot of a from-scratch rewrite and redesign of the K language. I haven't found any public documentation for K 9 other than the output of the help command "\" in the REPL, but this version of the language isn't finished, and the K 9 forum appears to be a private google group that I don't have visibility into. Maybe public documentation will be made available for the release version. It's not that different from any other version of K, at least at first glance. It has a better syntax for dictionary literals, and better support for database tables, and some of the operator syntax has been rearranged. For example, dyadic ^ is now "cut" instead of exponentiation, which breaks some of the examples on Wikipedia. It certainly is not an OOP language, since it has no concept of objects, classes, inheritance, or any of that. It certainly does have date and time values. The help mentions "exp" in a list of trancendental functions, so I assume that is exponentiation, although it isn't implemented. Other commenters on the web say that K9 is simplified from its predecessors.
k
@Kartik Agaram I mostly agree with your analysis, but I donโ€™t blame languages specifically. In computing technology, everything is a prototype these days. All these prototypes look unsatisfying to a large part of stakeholders, and therefore never get implemented carefully. Only stabilizing forces such as standards or conservative clients (finance) can make a difference. Implementations of languages such as Fortran or Common Lisp have continued to improve over decades.
Relevant in this context: this Twitter thread by software environmentalist @Tudor Girba: https://twitter.com/girba/status/1246750717538373632
Note: I really like the term "software environmentalist". I wish more people would adopt that attitude.
๐Ÿ‘ 4
d
Er... I thought I agreed, but you lost me at "machine code".