Parsing is an essential part of any IO more complicated than accepting and echoing byte strings. Regex solves or helps solve parsing across a wide variety of problems. I think that answers why it "jailbroke" from "compilerists": it was never just a compiler thing, everyone needs parsing. Once someone figured out the math, something like regex was inevitable.
The question of Perl's adoption, and by extension PCRE, is IMO a separate one, more historical than theoretical. I assume it's the usual right-place-right-time/path-dependent/worse-is-better type of story. Certainly Perl is more than PCRE, and was responsive to more problems than just parsing.