https://futureofcoding.org/ logo
Join Slack
Powered by
But I think that's waaaaayyyyyyyyyyyyyyyyyy too co...
# thinking-together
l
But I think that's waaaaayyyyyyyyyyyyyyyyyy too complicated
s
thread! start a thread (that’s what people do in this slack)
l
How do I start a thread? ;x
s
you’re already in one
😄
they tend to keep the “main” channel clean, and start threads like this one for extended discussion
l
Ah yeah alright, there's a button to create threads when you hover over messages
didn't know
And makes sense to keep them clean 😛
In addition to this weird mutation finder approach I thought of, I did have an alternative, somewhat similar to what @Cole suggested, but a bit more powerful
I made a program to collect execution traces in an SQL database, so the execution can be queried
s
gave you a link in the other thread
l
Ah, I'll take a look
So, take the ui, maybe you see the text 
wellcome
and you want to fix the typo
you can query the execution for 
wellcome
and it'd give you back the source location for the invocations
s
I’m putting Clojure function inputs and outputs in a database for this very purpose
but I’m using a graph database
it’s a bit more appropriate for this sort of thing
l
Yeah, I found that SQL is a bit limiting
I have never used a graph db though
Speaking of putting traces in a db
there's a lot of great potential there
s
graph databases (or graph representations) are used both for this sort of provenance work, but also for static analysis of code (which is what would give you the connections between different parts of the code)
l
Ah yeah, I am somewhat familiar with call graphs
Although I hadn't considered them for this yet
I'm reading the wiki article on provenance atm
s
yeah sorry for the link bombardment!
l
npnp 😛
Yeah, the idea of data lineage is exactly what I was thinking of
it's great when you find a word you had a vague feeling for in your head for a while 😛
s
same thing happened to me, I thought about this, then discovered the term
the thing is, this is what we do manually anyway, we reconstruct the data lineage within the code painstakingly by reading the code, adding logging etc
l
Yeah exactly, I feel like 80% of my work is just figuring out lineage
I get a ticket, there's a bug, it needs fixing
I find out that three files are relevant
5 functions are relevant
and through those functions, I don't know, there are 20+ parameters
and like 40 constants referenced
I then either modify random stuff to see what influences the bug, or if the code is neat enough I'll try and see if I can find the exact issue by reasoning
And then in the end, only like 2 lines and 1 parameter ended up being relevant
It would save soooooooooo much time and frustration if some of that work could just be automated
Like a 1000-line function with 80 variables, I only care about one specific variable, I want my iDE to show what lines of code are influenced by that variable
so reduce the function from 1000 lines to like 6
Also I type a lot, sorry xD
s
so, the thing I find limiting about IDEs is that you have a limited set of questions you can “ask” about the code
you can say “show me what functions call this”
or “stop when the program gets to this point”
but you can’t say “show me all the private functions that call this function and whose names all start with ‘data-’ ”
the information is there
l
Yep yep yep
I've been thinking along the same line
s
but the UI can only cater for certain types of questions
l
"show me if this function has ever been called before with True as a second parameter"
I have the data to do that now, with the SQL-db
I've even added a Sublime text plugin to show some information about a selected function call with a keybinding
s
that’s amazing
l
But there's soooo much more
Like I'm barely scratching the surface of what's possible with this kind of data
I added a feature to infer whether a function is deterministic or not
(ie, this function has been called 6793 times, and there were 0 times when it returned a different value for the same input)
it records run-time performance too, because the php trace thingy just gives me that for free
s
my approach is this: I have Clojure code (which is my main lang) and I “instrument” functions so I can collect their inputs/outputs in Datomic. I’m also gathering “static” information about the code (how it’s structured, what the function signatures are etc). Datomic is a graph database which has a full-featured query language, so you can ask anything you like about these two aspects
l
so there's a lot to play around with there
Yeah I've heard about datomic before
s
“deterministic” also known as pure function
l
Yeah, pure function, you're right
Datomic's something I've been interested in for a while
I just love the idea of being able to like go back in history without a major headache 😮
s
the other thing you can do with your setup is infer the types of each parameter
from examples of actual calls
l
Like even today, during lunch, a colleague working support was complaining about a customer who did something extremely stupid on 200 documents of his, so he had to like actually manually fix them
With datomic you could just scrub back to before when the damage was done :S
Oh yeah I'm doing that already
It infers the return type as well
s
well let me show you another project of mine: https://github.com/stathissideris/spec-provider
l
The major pain point with this setup at the moment is that collecting the data slows down the system to a crawl
like a request that normally takes 200ms now takes 26 seconds
s
yeah, I can imagine
l
.. and the db grows to like 20GB after two days of usage 😅
Actually, I really like your perspective of "asking the ide questions"
s
if you can get your system to understand the code a bit better, you could be more targeted in what data you collect. Say you pinpoint a specific value that is null when it shouldn’t be, you don’t need to trace your whole codebase, just the parts that “contribute” to it
l
I initially started with this trace recording idea to make it easier for me to see if a change introduced a regression
Yeah true
Ie, I have a function addTwo(a, b) => a + b;
with some recordings
[2, 2] => 4 [4, 5] => 9
if I change it to addtwo(a, b) => a - b;
it could tell me there's a regression
s
so you’re actually using this already for development, right?
l
Yeah a little bit
It's still somewhat inconvenient to use for various reasons
but yeah
s
did you see spec-provider above?
l
Not yet
I'm not super familiar with clojure specs
i read the introduction once, but I haven't ever done anything with clojure
(introduction to specs, not intro to clojure :p)
s
specs are Clojure’s declarative way of saying what type something is
but it does beyond that, it can describe whether something can be nil for example
or if it’s a hashmap, you can say which keys are required, and which are optional
l
Ah yeah, so you could have a spec for even and odd numbers for example
"this function takes a string, but the string must be at least of length 1"
s
yeah, and it would simply be 
odd?
in clojure, because it already has that predicate function
exactly, so it’s types but not really 🙂
l
yep
s
anyway spec provider takes data and tries to infer the spec (at least the basic specs) for it
l
Yeah, so similar to what I'm doing, except a little more formal
s
the nice thing about specs is that they are a form of documentation, they can be used for validation of course, but get this: since it’s a declarative description, it can also used for generating data
l
And so you can use that to generate test cases, db values,
Yepp
Useful, feels like PHP is 20 years behind Clojure U_u
s
I guess PHP hasn’t been getting much attention lately!
out of fashion maybe?
l
Out of fashion for sure
PHP is practical, but not really cutting edge in any way
There have been some good developments on the core language
But right now, we've only barely reached a point where the core language is "acceptable"
And that's seen as progress
We finally have classes, type hints, interfaces, etc...
But even today, type hints don't work everywhere and need a lot of work still
Like I recall, we couldn't give hints for class properties or something like that
it's coming in a next version of php
it's a slightly depressing thing to look at to be honest :x
s
so, our chat reminded me of this part of a FoC episode: https://futureofcoding.org/episodes/030#159
l
He saw their talk, tried it at work, and after four weeks he said that he had gotten a deep enough understanding of the Excel codebase that he thinks would have taken him two years without the tool.
Haha wow that's inspiring
Thoougghh for real, it's surreal how much of an impact Bret Victor has had on my perspective on programming
Like I would never have thought of building any of these side-tools if it wasn't for him
He didn't give me the tools themselves, he showed me that they are possible
s
Agreed, I think Bret Victor may be responsible for this whole slack! 😂
l
And planted a very, very deeply rooted seed of dissatisfaction with the current state of the tooling, (and industry as a whole, :x)
Yeaapp
s
so if I piqued your interest about Clojure a bit, let me point out that there are parts of the experience of working with Clojure (and most Lisps) that are hard to describe, but definitely worth having
for example, REPL-driven development
l
Yeah, I'm familiar enough with the experience
s
oh you are! ok
with a Lisp or something else?
l
It's mostly just, I'd like to program in clojure, of all languages I'd like to look at, clojure I'm most interested in
Yeah I've done some common lisp
And a tiny bit of clojure
s
oh great 🙂
l
But uhh, unfortunately, nobody employs lisp programmers
And I'm quite happy with Python/js for my personal projects so far 😛
s
depends, there are some clojure jobs around and more are appearing
l
especially python
yeah true, but still
s
just today I got an email from a London-based Clojure consultancy
(who are looking for people)
l
Ahh yeah haha, if you have a fairly niche skill on your cv stated publicly somewhere
You'll get those mails
s
it was on a mailing list in fact (I used to work for these guys and they seem to be expanding lately)
l
But I mean mostly like, I don't want to have to change my job to change my tools
For me, programming is about the service I can provide to end-users
So focusing on clojure would be very limiting in my view
Not against it, but I won't look for a clojure (or haskell, or lisp, or ....) job specifically
s
I know what you’re saying… but I’m not like that, I did quit specifically at some point so I could move to a job because I would get the chance to use Clojure 😂
l
Like I worked on an xray scanner
I tran delphi
I like xray scanners
I don't think I'm going to find a clojure-based xray scanner 😅
s
as in medical equipment?
l
yeah
whoops :x
s
I’d be too nervous to have to program medical equipment… I remember reading about a bug that resulted in patients getting an excessive amount of radiation
so you have my respect for having worked in this field
l
It was an internship, I never got to control the machine itself luckily
Or at least, change code that affects control
In the end, I worked on the UI
which is a lot safer to work on 😛
postprocessing of the images, mostly
but that was definitely a very confrontational feeling
s
still, pretty nice area to be into
l
When I saw that the code responsible for the radiation dose was literally
Copy code
max_radiation = 1200; // don't change
s
ouch
l
Interesting field
then again, this machine wasn't used for serious purposes, it was a low-radiation scanner mostly used in airports etc, the machine itself couldn't output very high doses of radiation anyway
So the potential damage was never super high
But in like medical x-ray machines, there's a looott of testing and speccing and verification etc going on
s
I can imagine
they probably use formal techniques too
so, great to talk to you and to discover someone with very similar ideas! It’s getting a bit late over here so I’m heading to bed, I hope to run into you again sometime
l
Yep, same!
Still really like the framework of asking the ide questions 😛
It's a helpful way of thinking about how I interact with my tools
And what kind of new insights my extensions may provide
good night!
s
good night!
2 Views
Open in Slack
PreviousNext
Powered by