Question: How did REGEX jaibreak from Compiler Technology and become popular with non-compilerists? First guess: (1) grep, sed, awk, etc. broke the ice, and, (2) perl brought REGEX into the mainstream. I guess that perl solved a latent problem, which made it very popular. What was that problem?

I think part of the answer is that is that it’s representation as strings meant that it was supported by a near-ubiquitous infrastructure of plaintext which made it much easier to share, store, and use in everyday systems.

The real question for me is why isn’t grep/regex search more commonly used by non-programmers?

It's funny how all declarative syntaxes (syntaxii?) acquire a reputation for fiendishness! Take CSS - coulda been a non-programmer language but it's notoriously hard to do simple things like line stuff up; take SQL - a team at my work had the "SQL expert" who everyone went to cos it was so bloody hard, and of course, the most notorious of all: regexes - if only there were a non-brain-twisting way of doing them!

@User There are better ways of writing them that date back to the 1960s. This rant from Sussman captures the problems, and the implementation that follows shows one way of improving matters: https://github.com/bzinberg/regex-combinator/blob/c49d6aba03d2a42c33b1bdafc7d5e5ded9d60eb8/ps.txt#L33

One part of that seems a bit off. The nice formalism is for regular expressions, which don’t support backreferences (one of the ways that modern “regexes” drifted from the original concept—towards more power, but more obscurity).

Parsing is an essential part of any IO more complicated than accepting and echoing byte strings. Regex solves or helps solve parsing across a wide variety of problems. I think that answers why it "jailbroke" from "compilerists": it was never just a compiler thing, everyone needs parsing. Once someone figured out the math, something like regex was inevitable. The question of Perl's adoption, and by extension PCRE, is IMO a separate one, more historical than theoretical. I assume it's the usual right-place-right-time/path-dependent/worse-is-better type of story. Certainly Perl is more than PCRE, and was responsive to more problems than just parsing.