Hi everyone! 👋 I'm doing some user research on building reliable products at scale. If you work in development, testing, or product and have experiences with QA (or 🐛 where there shouldn't be), I'd love to hear your story! Even a short 15-min call would be super helpful. Let me know if you're open to a convo and we'll set up a time. Thanks so much in advance 😊

Do you have any more specific questions or topics you'd like to hear experiences about? Feels like the whole career of most people here would fall under "🐛 where there shouldn't be" and it'd difficult to put a whole career into a 15 minute story 😅

Great point! Here are some I'd like to hear more about: 1. What testing strategies have you used and if stopped using any, why? 2. What types of testing, practices, tools, and metrics have you used to measure the readiness of a product for release? 3. What were the most significant challenges you faced during the testing process, coverage, maintenance, or automation? How were these addressed? 4. Who was responsible for what portion of testing in your teams?

Okay, well I can try to say something about 1. So over the years I've only worked in one project that did automated testing by the book: unit testing, integration testing automated UI testing. There was a dedicated person working on automated UI testing and couple people doing manual testing part-time. Probably about half of the effort in the project went into testing in different forms. It was cool, but I didn't really feel the payback was necessary there for a system on which no ones life depends on. Code quality of the tests was bit of an issue and maintaining them was certainly a pain at times. Now days I suppose LLMs could be useful here. Anyway, I do vouch for integration tests much more than for unit testing. When you're testing bigger pieces of your system it's usually possible to work with APIs that very very rarely change. Like testing that your rest API gives the correct response for a query and database state. You'll inject test data in you DB schema (which seldom changes) and read the results from the API response (which seldom changes, or often has to be backwards compatible anyway). I've also built solutions for generating the test data out of a schema and then assert against the generated data. Some would say that it's too complicated code for tests so you can have bugs in your tests, but I see the changes for false positives very low. What I like about tests like this is that you can refactor everything between your database and the API without having to rework tests while still getting valuable results from tests. Additionally these tests serve as a nice debugging setup: set a breakpoint, run a test in degugger and you can start walking your code with your system in a somewhat realistic state. Another approach I've found giving good payback for the effort is using quite a bit of asserts throughout your code. So checks on value ranges etc that aren't necessarily run on production but let you know if something that's obviously wrong is happening in your system.

Wow, I appreciate the detailed response! I've got some follow-up questions I'll post once I've explored what you've mentioned.