I'm not sure how to ask this but ... how often does simple ideas run into complex reality? I'm not sure I can explain myself well but I'm trying to think of examples. Let's say you have a configuration file stored (pick a format, JSON, yaml, xml, ...). You start with something simple

config = JSON.parse(readAsText('configfile'))
print(config.setting1);
print(config.setting2);

Looks great and it works. But then at some point you need to print an error message if the config file is wrong. First thing. the JSON/Yaml/XML is wrong and the message the library prints is cryptic. For JSON in JavaScript the message would be like "unexpected } at character 2647" That's entirely not useful for the end user (let's skip the part whether or not JSON is useful for the end user for now). So you go end up having to find a parser that will give a line number at least 'error on line 537' You also maybe add a schema or something to check the structure of the file is correct. It will tell you there is no 'foobaz' setting or that 'size' must be a number not a string but in general that stuff happens after parsing so if you'll get an error "size must be a number not a string" when the error you want is "size at line 537 is an error not a string". Now your schema checker either needs to do it's own parsing making it vastly more complex than it was OR your schema checker needs to ingest a much more complex format of data than just the parsed data where every piece of data is annotated from where it came from. You might argue such configuration should have a UI where you can't make those mistakes but that's just part of same problem. What started as a effectively 1 line of code is suddenly 1000s of lines of code. I feel like this pattern repeats itself in many other ways. Maybe this is like the text editor example where someone rants that a text editor can function in a few hundred lines of code but then someone else points out that it's not a few hundred lines of code when you have to support CJK, Thai, Arabic, and emoji

I think the complexity here stems from our historical love of serializing data in order to persist and distribute it. We wouldn't have these problems if we weren't working with (text and binary) strings all the time. The "correct solution" is to have a software universe where data is always structured (and typed), and serialization is left as an infrastructural concern.

My point wasn't about serialization. that's just an example. My point was "it's never simple". Handling the real world, regardless of what it is, is almost always several orders of magnitude more complicated than we first expect.

Some days my whole freaking job is checking the consistency of CSV uploaded by Excel jockeys to give them real nice error messages. Let's see in the background here I have 150 lines check a super simple file (three columns) and 150 lines of tests to identify 12 ways it could reasonably be messed up.

I think this is a general case of “as the number of entities increase linearly, the number of potential relations between them increases exponentially”. What may seem like a small addition, can turn out to increase complexity significantly.

The vocabulary that comes to mind for me for the problems you are describing break down into two parts: incidental complexity: problems you have to solve that have nothing to do with the problem you are actually trying to solve, but are necessary because of the inefficiencies of our current technology. To play off your example, you want to parse some configuration, and suddenly you need to become an expert in parser theory. You don't care about parsers, it's incidental to the problem, but necessary because of the limits of our current tools. "*software requirements gathering*" and "*scope creep*". Humans, in general, are really bad at communicating what they actually want or need. Partially this is because we are really bad at thinking of unforeseen consequences. "Out of sight out of mind" so to speak. Often, you start out thinking you need one thing, and then it turns out that you needed something else. Requirements also change! You maybe start out with a little script you hacked together for yourself. You have a very clear idea of the inputs and outputs, so you don't add much error checking. Then, after a while, you share your script, and suddenly lots of people start using it. Now you need to add a lot more error checking than you had before. This happens all the time. Python3 and Unicode, IPV4 to IPV6... the list goes on. As a field, we have not solved either of these problems very well. Our state of the art is just iteration (Agile Development to use a crappy buzz word.) I recommend reading Mythical Man Month by Fred Brooks.

Spot on @Ray Imber! I am usually figuring out some core incidental algorithm and experiencing that familiar feeling of having to do 'the thing that gets me to the thing' - the incidental complexity. If not that, I'm adding some new feature that I never anticipated. Having written my first text editor recently, I can attest to it being a perfect example. It's really easy to display text, parse input and change it. A day or 2 of work. Then the problem becomes the gap between your simple implementation and all the professional editors you use.... Pretty soon you are working on transactional operations, syntax coloring, performance, etc.....