Hey all! I need a tool / stack suggestion. We'd like to track internal metrics for development like build performance stats, build size, source control size, etc. I'd like to be able to see these trends in a graph by date/commit.
• CI tools can do some of this, but they only surface a small, inconfigurable set of metrics. Usually just test failures and build times.
• There are monitoring solutions like Grafana Cloud, Datadog, but these all seem geared towards monitoring servers and cloud systems. They also assume you're ingesting massive amounts of data.
What I really just want is a POST-able URL that I can spit metrics to, and a dashboard with some basic alerting. Is there something like this that exists?
My understanding was that Prometheus doesn't have persistent storage? Essentially I want to retain this data forever. It's on the order of a few hundred data points per day.
Ionuț G. Stan
06/07/2023, 8:11 PM
Yeah, it might not be the best choice. Are you looking for an existent service or is it something that you can set up yourself? One possibly very stupid idea might be to just Google Sheets API and push data into a spreadsheet there. A small script could handle the notifications too.
06/07/2023, 8:13 PM
I guess I was hoping for an existing solution. 😐 But if that fails I'm not above spinning up a couple of docker containers. What I'm trying to avoid is spending time doing sysadmin stuff down the road.
So far it seems like maybe I could use influxdb + grafana and maybe that would be persistent. Not sure...
06/09/2023, 3:49 AM
I haven't tried this, but maybe writing to postgrest wrapping a postgres db, with Grafana providing the graphs? Postgrest is a smidge intimidating, especially the auth situation, but maybe still easier than handrolling something. Otoh, a very small single-endpoint web server that writes to postgres, which is read from grafana, might be a good compromise... (Thinking out loud at this point)
06/09/2023, 3:50 AM
I ended up trying a lot of stuff. But ended with hosted InfluxDB + Grafana. Works well enough. Influx has a POST Rest API, and it's all pretty easy to query in Grafana.
Luckily Influx has a pay-as-you-go permanent storage option, too.
Hadn't tried postgrest, though! I considered TimescaleDB which is on Postgres, but it requires you to map out the schemas in advance.
The other contender was Graphite, which is essentially built just for this purpose, but it's pretty old and kludgy.