It’s important because it’s a way to conceptualize why neural networks are in a way better than classical ML algorithms. Because this lack of leaks means that anyone can play around with them without breaking the whole thing and being thrown one level down. Want to change the shape ? Sure. Want to change the activation function ? Sure. Want to add a state function to certain elements ? Go ahead. Want to add random connections between various elements ? Don’t see why not… etc.
I truly think that they are among the first of a "new" type of mathematical abstractions. They allow people that don't have the dozen+ years background of learning applied mathematics, to do applied mathematics.