WAL functions cannot be executed in standby servers
```
ERROR: recovery is in progress
HINT: WAL control functions cannot be executed during recovery.
```
In PostgreSQL 16 replication slots persist in the standby servers.
This is also the case when using extensions like pg_failover_slots that transfer the slot information to the standby.
This condition will make those queries to return values only in the primary node avoiding the errors.
Co-authored-by: Jiří Čtvrtka <62988319+JiriCtvrtka@users.noreply.github.com>
Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>
* Dashboard linting improvements for mixin
Signed-off-by: Ryan J. Geyer <me@ryangeyer.com>
* WIP: Add prelim multi-target support
- Remove multi server support from new collector package
- Add http handler for multi-target support
Signed-off-by: Joe Adams <github@joeadams.io>
* Add config module
The config module supports adding configuration to the exporter via a config file. This supports adding authentication details in a config file so that /probe requests can specify authentication for endpoints
Signed-off-by: Joe Adams <github@joeadams.io>
* cleanup and README
Signed-off-by: Joe Adams <github@joeadams.io>
Co-authored-by: Ben Kochie <superq@gmail.com>
* Update cmd/postgres_exporter/main.go
Signed-off-by: Joe Adams <github@joeadams.io>
* fix for exporter issue 633
fix for exporter issue 633: https://github.com/prometheus-community/postgres_exporter/issues/633
"Scan error on column index 2, name \"checkpoint_write_time\": converting driver.Value type float64 (\"6.594096e+06\") to a int: invalid syntax #633"
Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
* Fix checkpoint_sync_time value type
Error:
sql: Scan error on column index 3, name \"checkpoint_sync_time\": converting driver.Value type float64 (\"1.876469e+06\") to a int: invalid syntax
See also:
https://github.com/prometheus-community/postgres_exporter/issues/633https://github.com/prometheus-community/postgres_exporter/pull/666
Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>
* Bump github.com/prometheus/common from 0.35.0 to 0.37.0
Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.35.0 to 0.37.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](https://github.com/prometheus/common/compare/v0.35.0...v0.37.0)
---
updated-dependencies:
- dependency-name: github.com/prometheus/common
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Correct minor typos in README.md
Signed-off-by: Luckz <224748+Luckz@users.noreply.github.com>
* Release 0.11.1
* [BUGFIX] Fix checkpoint_write_time value type #666
* [BUGFIX] Fix checkpoint_sync_time value type #667
Signed-off-by: SuperQ <superq@gmail.com>
* Add dsn type for handling datasources
dsn is designed to replace the other uses of dsn as a string in the long term. dsn is designed to be safe to log, properly redacting passwords. The goal is eventually always parse datasource information into a dsn type object which can safely be passed around and logged without worrying about wrapping calls in a redaction function (today this function is loggableDSN().
This should solve the root issue in #648, #677, and #643, although the full fix will require more changes to update all code references over to use the dsn type.
Signed-off-by: Joe Adams <github@joeadams.io>
* Release 0.12.0-rc.0
BREAKING CHANGES:
This release changes support for multiple postgres servers to use the
multi-target exporter pattern. This makes it much easier to monitor multiple
PostgreSQL servers from a single exporter by passing the target via URL
params. See the Multi-Target Support section of the README.
* [CHANGE] Add multi-target support #618
* [BUGFIX] Add dsn type for handling datasources #678
Signed-off-by: SuperQ <superq@gmail.com>
* fix: typo
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
* Update multi-target handler to use new DSN type
- Moves new dsn type to config.DSN. This will prevent circular dependencies.
- Change DSN.query to be url.Values. This allows the multi-target functionality to merge values without re-parsing the query string
- Change NewProbeCollector to use the new config.DSN type
- Add DSN.GetConnectionString to return a string formatted for the sql driver to use during connection
Signed-off-by: Joe Adams <github@joeadams.io>
* Add missing license header
Signed-off-by: Joe Adams <github@joeadams.io>
* Convert pg_stat_database to new collector model
Signed-off-by: Joe Adams <github@joeadams.io>
* extended /probe path metrics
Signed-off-by: Ildar Valiullin <preved.911@gmail.com>
* Bump github.com/lib/pq from 1.10.6 to 1.10.7
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.6 to 1.10.7.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.6...v1.10.7)
---
updated-dependencies:
- dependency-name: github.com/lib/pq
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Capture usename and application_name for pg_stat_activity
It is necessary to be able to exclude backups from long-running
transaction alerts, as they are to be expected. With the current
pg_stat_activity metric there is no ability to filter out
specific users or application names.
Resolves#668
Signed-off-by: cezmunsta <github@incoming-email.co.uk>
* Fixed formatting
Signed-off-by: cezmunsta <github@incoming-email.co.uk>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* 4kB size added for postgres with 4kB block_size
Signed-off-by: Sergey Morozov <38383507+ken3122@users.noreply.github.com>
* Correct additional typo in README.md
Signed-off-by: Luckz <224748+Luckz@users.noreply.github.com>
* Set gauge to 1 when collector is successful
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* probe: clean-up database connection after probe to prevent connection leak
Signed-off-by: Kurtis Bass <kurtis.bass@hinge.co>
* Set gauge to 1 when collector is successful
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
Signed-off-by: Khiem Doan <doankhiem.crazy@gmail.com>
* Add postgres 15 for CI test
Signed-off-by: Khiem Doan <doankhiem.crazy@gmail.com>
* Add postgres 15 for CI test
Signed-off-by: Khiem Doan <doankhiem.crazy@gmail.com>
* New unit value 64kB
Signed-off-by: Oleksandr Mysyura <olexandr.mysyura@pragmaticplay.com>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Update exporter-toolkit
Update to the latest exporter-toolkit
* Enables multi-listener and systemd socket activation.
* Bump Go to 1.19.
* Remove `PG_EXPORTER_WEB_LISTEN_ADDRESS` env var because this is now a
repeatable flag.
Signed-off-by: SuperQ <superq@gmail.com>
* go fmt
Signed-off-by: SuperQ <superq@gmail.com>
* adding codified functionality for logical replication metrics
Signed-off-by: Zachary Caldarola <zachary.caldarola@reddit.com>
* Bump github.com/prometheus/client_golang from 1.13.0 to 1.14.0
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.13.0 to 1.14.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.13.0...v1.14.0)
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump github.com/prometheus/common from 0.37.0 to 0.39.0
Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.37.0 to 0.39.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](https://github.com/prometheus/common/compare/v0.37.0...v0.39.0)
---
updated-dependencies:
- dependency-name: github.com/prometheus/common
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* addressing comments
Signed-off-by: Zachary Caldarola <zachary.caldarola@reddit.com>
* more comments
Signed-off-by: Zachary Caldarola <zmc2005@gmail.com>
* fmt
Signed-off-by: Zachary Caldarola <zmc2005@gmail.com>
* typing
Signed-off-by: Zachary Caldarola <zmc2005@gmail.com>
* fmt
Signed-off-by: Zachary Caldarola <zmc2005@gmail.com>
* send stdout/stderr to syslog
Signed-off-by: Mike <gavrikster@gmail.com>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Fix exclude-databases for collector package
The pg_database collector was not respecting the --exclude-databases flag and causing problems where databases were not accessible. This now respects the list of databases to exclude.
- Adjusts the Collector create func to take a config struct instead of a logger. This allows more changes like this in the future. I figured we would need to do this at some point but I wasn't sure if we could hold off.
- Split the database size collection to a separate query when database is not excluded.
- Comment some probe code that was not useful/accurate
Signed-off-by: Joe Adams <github@joeadams.io>
* Remove commented code
Signed-off-by: Joe Adams <github@joeadams.io>
* Remove more dead code
Signed-off-by: Joe Adams <github@joeadams.io>
* Update build
* Update Go to 1.20.
* Update golanci-lint.
* Bump modules.
* Update CI orb.
* Fix up use of deprecated ioutil.
Signed-off-by: SuperQ <superq@gmail.com>
* Reduce cardinality of pg_stat_statements
Make the example queries.yaml `pg_stat_statements` query safer.
* Select the top 10% of queries by total query time.
* Only expose the top 100 queries by total query time.
* Keep only the most useful metrics.
* Comment out the example by default.
Fixes: https://github.com/prometheus-community/postgres_exporter/issues/549
Signed-off-by: SuperQ <superq@gmail.com>
* Update changelog and version for v0.12.0 release
Signed-off-by: Joe Adams <github@joeadams.io>
* Update exporter-toolkit
Updates the exporter-toolkit to the latest version
* Adds new landing page feature.
* Allow metrics path to be on `/`.
Signed-off-by: SuperQ <superq@gmail.com>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Fix column type for pg_replication_slots
Change the data type of `active` from int64 to bool. The documentation confirms that this is a boolean field.
https://www.postgresql.org/docs/current/view-pg-replication-slots.htmlfixes#769
Signed-off-by: Joe Adams <github@joeadams.io>
* Update versions listed in the README
Update the supported versions based on what we actually test in CI.
Signed-off-by: SuperQ <superq@gmail.com>
* Update README cli flags
These have not been kept up to date.
Signed-off-by: Joe Adams <github@joeadams.io>
* Adjust log level for collector startup
Since we support both multi-target and typical direct scrapes, either of these can fail and it is no longer an error.
Signed-off-by: Joe Adams <github@joeadams.io>
* Fix pg_setting different help values
Signed-off-by: GitHub <noreply@github.com>
* Supports alternate postgres:// prefix in URLs
Adds support for the alternate postgres:// prefix in URLs. It's maybe
not the cleanest approach, but works. Hoping I can either get some
pointers on a more appropriate patch, or that we could use this in the
interim to unblock this use-case.
Signed-off-by: Jack Wink <57678801+mothershipper@users.noreply.github.com>
* Bump github.com/lib/pq from 1.10.7 to 1.10.9
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.7 to 1.10.9.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.7...v1.10.9)
---
updated-dependencies:
- dependency-name: github.com/lib/pq
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Refactor collector descriptors
Use individual collector metric descriptor vars to help avoid
miss-mapped or unused metrics.
Signed-off-by: SuperQ <superq@gmail.com>
* Bump github.com/prometheus/common from 0.42.0 to 0.44.0
Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.42.0 to 0.44.0.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](https://github.com/prometheus/common/compare/v0.42.0...v0.44.0)
---
updated-dependencies:
- dependency-name: github.com/prometheus/common
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update linting
* Move errcheck exclude list to config file.
* Enable revive linter
* Fix up revive linting issues.
Signed-off-by: SuperQ <superq@gmail.com>
* Bump github.com/prometheus/exporter-toolkit from 0.9.1 to 0.10.0
Bumps [github.com/prometheus/exporter-toolkit](https://github.com/prometheus/exporter-toolkit) from 0.9.1 to 0.10.0.
- [Release notes](https://github.com/prometheus/exporter-toolkit/releases)
- [Changelog](https://github.com/prometheus/exporter-toolkit/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prometheus/exporter-toolkit/compare/v0.9.1...v0.10.0)
---
updated-dependencies:
- dependency-name: github.com/prometheus/exporter-toolkit
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Move queries from queries.yaml to collectors (#801)
Signed-off-by: Ben Kochie <superq@gmail.com>
* Fix pg_stat_database collector
The signature for creating a collector changed and CI didn't retrigger. Move metrics out of map and into individual vars.
Signed-off-by: Joe Adams <github@joeadams.io>
* Fix up collector registration (#812)
Use const definitions to make collector registration consistent.
* Use collector subsystem name consistently.
* Fix up replication metric name unit.
Signed-off-by: SuperQ <superq@gmail.com>
* Update release info for v0.12.1
Signed-off-by: Joe Adams <github@joeadams.io>
* Deprecate extend queries feature (#811)
Mark the extend queries feature as deprecated in favor of recommending
the sql_exporter.
Signed-off-by: SuperQ <superq@gmail.com>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Deprecate additional database features
Now that we have deprecated extended queries we can deprecate related
database features.
* Deprecate flags/functions around auto discover databases.
* Deprecate flags/functions for additional constant labels.
Signed-off-by: SuperQ <superq@gmail.com>
* Release v0.13.0
BREAKING CHANGES:
Please note, the following features are deprecated and may be removed in a future release:
- `auto-discover-databases`
- `extend.query-path`
- `constantLabels`
- `exclude-databases`
- `include-databases`
This exporter is meant to monitor PostgresSQL servers, not the user data/databases. If
you need a generic SQL report exporter https://github.com/burningalchemist/sql_exporter
is recommended.
* [CHANGE] Adjust log level for collector startup #784
* [CHANGE] Move queries from queries.yaml to collectors #801
* [CHANGE] Deprecate extend queries feature #811
* [CHANGE] Deprecate additional database features #815
* [CHANGE] Convert pg_stat_database to new collector #685
* [ENHANCEMENT] Supports alternate postgres:// prefix in URLs #787
* [BUGFIX] Fix pg_setting different help values #771
* [BUGFIX] Fix column type for pg_replication_slots #777
* [BUGFIX] Fix pg_stat_database collector #809
Signed-off-by: SuperQ <superq@gmail.com>
* Add the instance struct to handle connections
The intent is to use the instance struct to hold the connection to the database as well as metadata about the instance. Currently this metadata only includes the version of postgres for the instance which can be used in the collectors to decide what query to run. In the future this could hold more metadata but for now it keeps the Collector interface arguments to a reasonable number.
Signed-off-by: Joe Adams <github@joeadams.io>
* chore: fix a few typos
Signed-off-by: Alex Tymchuk <alexander.tymchuk@percona.com>
* Bug fix: Make collector not fail on null values (#823)
* Make all values nullable
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
* Release 0.13.1 (#824)
* [BUGFIX] Make collectors not fail on null values #823
Signed-off-by: SuperQ <superq@gmail.com>
* Fixed replication pgReplicationSlotQuery - now it's working correctly for replica and primary (#825)
Signed-off-by: Vadim Voitenko <vadim.voitenko@exness.com>
Co-authored-by: Vadim Voitenko <vadim.voitenko@exness.com>
* Migrate pg_locks to collector package (#817)
Migrate the `pg_locks_count` query from `main` to the `collector`
package.
Signed-off-by: SuperQ <superq@gmail.com>
* Cleanup collectors (#826)
Fix up `replication` and `process_idle` Update input params to match
the rest of the collectors.
Signed-off-by: SuperQ <superq@gmail.com>
* Bug Fix: Fix lingering type issues (#828)
* Fix postmaster type issue
* Disable postmaster collector by default
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update common Prometheus files (#829)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Fix replication collector
Signed-off-by: Tom Hughes <tom@compton.nu>
* Add some more escapes to the query sanitizer
Signed-off-by: Tom Hughes <tom@compton.nu>
* Add a collector to gather metrics on WAL size
Signed-off-by: Tom Hughes <tom@compton.nu>
* Bump github.com/prometheus/client_golang from 1.15.1 to 1.16.0 (#853)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.15.1...v1.16.0)
---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Fix untyped integer overflows on 32-bit archs (#857)
go-sqlmock's Rows.AddRow() takes values which have a type alias of
"any", and appear to default to untyped ints if not explicitly cast.
When large values are passed which would overflow int32, tests fail.
Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
* Bump github.com/smartystreets/goconvey from 1.8.0 to 1.8.1 (#852)
Bumps [github.com/smartystreets/goconvey](https://github.com/smartystreets/goconvey) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/smartystreets/goconvey/releases)
- [Commits](https://github.com/smartystreets/goconvey/compare/v1.8.0...v1.8.1)
---
updated-dependencies:
- dependency-name: github.com/smartystreets/goconvey
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Unpack postgres arrays for process idle times correctly (#855)
Signed-off-by: Ben Kochie <superq@gmail.com>
* Include all idle processes in the process idle metrics
Signed-off-by: Tom Hughes <tom@compton.nu>
* Improve linting (#861)
* Disable unused-parameter check due to false positives on Collect()
calls.
* Enable misspell.
* Simplify error returns.
Signed-off-by: SuperQ <superq@gmail.com>
* Update common Prometheus files (#860)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
* Update common Prometheus files
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Gitlab collector: Database wraparound collector and test (#834)
* Database wraparound collector and test
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Joe Adams <github@joeadams.io>
* Add a logger to stat_database collector to get better handle on error
(also clean up some metric validity checks)
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update changelog for release 0.13.2 (#872)
Signed-off-by: Joe Adams <github@joeadams.io>
* Gitlab Collector: Autovacuum collector and test (#840)
* Autovacuum collector and test
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update collector/pg_stat_activity_autovacuum.go
Co-authored-by: Joe Adams <github@joeadams.io>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update collector/pg_stat_activity_autovacuum.go
Co-authored-by: Joe Adams <github@joeadams.io>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Use timestamp seconds
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* query formating
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* SQL format
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Loosen autovacuum query
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Joe Adams <github@joeadams.io>
* Gitlab Collector: Wal Receiver Collector and Test (#844)
* Wal Receiver Collector and Test
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Add more escapes
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Corrections to wal_receiver
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Continue on null labels
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Skip nulls and log a message
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Redundant breaks
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Fix up walreceiver
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Remove extra label
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update collector/pg_stat_walreceiver.go
Co-authored-by: Ben Kochie <superq@gmail.com>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Clean up the extra assignments
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update collector/pg_stat_walreceiver.go
Co-authored-by: Joe Adams <github@joeadams.io>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
Co-authored-by: Joe Adams <github@joeadams.io>
* Gitlab collector: Xlog location collector and test (#849)
* Xlog location collector and test
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Add more escapes
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Change to Gauge
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Handle new pg_stat_statements column names (#874)
Update pg_stat_statements collector to handle the new column names in
PostgreSQL 13.
Fixes: https://github.com/prometheus-community/postgres_exporter/issues/502
Signed-off-by: SuperQ <superq@gmail.com>
* Fixup new pg_stats_statements query (#876)
Fix all renames of `total_time` to `total_exec_time`.
Fixes: https://github.com/prometheus-community/postgres_exporter/issues/502
Signed-off-by: SuperQ <superq@gmail.com>
* Add a multi-target example config (#890)
Add an example Prometheus scrape config, similar to the
blackbox_exporter's example config.
Fixes: https://github.com/prometheus-community/postgres_exporter/issues/888
Signed-off-by: SuperQ <superq@gmail.com>
* Delay database connection until scrape (#882)
This no longer returns an error when creating a collector.instance when the database cannot be reached for the version query. This will resolve the entire postgresCollector not being registered for metrics collection when a database is not available. If the version query fails, the scrape will fail.
Resolves#880
Signed-off-by: Joe Adams <github@joeadams.io>
* Bugfix: Make statsreset nullable (#877)
* Stats_reset as null seems to actually be legitimate for new databases,
so don't fail for it
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
* Gitlab Collector: User Index io stats collector and test (#845)
* User Index io stats collector and test
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
* Update README to reflect changes made in #828 (#894)
Signed-off-by: Mathis Raguin <mathis.raguin@gitguardian.com>
* Gitlab Collector: Long running transactions collector and test (#836)
* Long running transactions collector and test
---------
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Ben Kochie <superq@gmail.com>
* Update common Prometheus files (#900)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Fix a connection leak (#902)
The leak was introduced in PR#882
Signed-off-by: Christian Albrecht <cal@albix.de>
Co-authored-by: Christian Albrecht <christian.albrecht@akquinet.de>
* Fix cross-compilation command in README.md (#903)
Signed-off-by: David Cook <dcook@divviup.org>
* fix pg_replication_lag_seconds (#895)
Signed-off-by: Vladimir Luksha <waldemarluksha@gmail.com>
Co-authored-by: Vladimir Luksha <luksha@limcore.io>
* stat_user_tables: Add total size metric (#904)
Signed-off-by: David Cook <dcook@divviup.org>
* Fix bugs mentioned in #908 (#910)
* Fix bugs mentioned in #908
These collectors are disabled by default, so unless enabled, they are not tested regularly.
Signed-off-by: Joe Adams <github@joeadams.io>
---------
Signed-off-by: Joe Adams <github@joeadams.io>
* Update common Prometheus files (#913)
Signed-off-by: prombot <prometheus-team@googlegroups.com>
* Add changelog for v0.14 (#906)
* Add changelog for v0.14
- Add changelog entries since v0.13.2
- Update README with new options
- Bump version file
Signed-off-by: Joe Adams <github@joeadams.io>
* Add changelog entry for #904
Signed-off-by: Joe Adams <github@joeadams.io>
---------
Signed-off-by: Joe Adams <github@joeadams.io>
* PMM-12154 pull upstream changes.
* PMM-12154 Fix go mod.
* PMM-12154 Remove some built-in queries, they were moved to collectors.
* PMM-12154 compatibility improvements.
* PMM-12154 compatibility improvements.
* PMM-12154 performance improvement.
* revert pg_lock_conflicts
---------
Signed-off-by: Ryan J. Geyer <me@ryangeyer.com>
Signed-off-by: Joe Adams <github@joeadams.io>
Signed-off-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Signed-off-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Luckz <224748+Luckz@users.noreply.github.com>
Signed-off-by: SuperQ <superq@gmail.com>
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
Signed-off-by: Ildar Valiullin <preved.911@gmail.com>
Signed-off-by: cezmunsta <github@incoming-email.co.uk>
Signed-off-by: prombot <prometheus-team@googlegroups.com>
Signed-off-by: Sergey Morozov <38383507+ken3122@users.noreply.github.com>
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
Signed-off-by: Kurtis Bass <kurtis.bass@hinge.co>
Signed-off-by: Khiem Doan <doankhiem.crazy@gmail.com>
Signed-off-by: Oleksandr Mysyura <olexandr.mysyura@pragmaticplay.com>
Signed-off-by: Zachary Caldarola <zachary.caldarola@reddit.com>
Signed-off-by: Zachary Caldarola <zmc2005@gmail.com>
Signed-off-by: Mike <gavrikster@gmail.com>
Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Jack Wink <57678801+mothershipper@users.noreply.github.com>
Signed-off-by: Ben Kochie <superq@gmail.com>
Signed-off-by: Alex Tymchuk <alexander.tymchuk@percona.com>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Signed-off-by: Vadim Voitenko <vadim.voitenko@exness.com>
Signed-off-by: Tom Hughes <tom@compton.nu>
Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
Signed-off-by: Mathis Raguin <mathis.raguin@gitguardian.com>
Signed-off-by: Christian Albrecht <cal@albix.de>
Signed-off-by: David Cook <dcook@divviup.org>
Signed-off-by: Vladimir Luksha <waldemarluksha@gmail.com>
Co-authored-by: Ryan J. Geyer <me@ryangeyer.com>
Co-authored-by: Joe Adams <github@joeadams.io>
Co-authored-by: Ben Kochie <superq@gmail.com>
Co-authored-by: Joe Adams <adams10301@gmail.com>
Co-authored-by: bravosierrasierra <bravosierrasierra@users.noreply.github.com>
Co-authored-by: Nicolas Rodriguez <nico@nicoladmin.fr>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Luckz <224748+Luckz@users.noreply.github.com>
Co-authored-by: Yoan Blanc <yoan@dosimple.ch>
Co-authored-by: Ildar Valiullin <preved.911@gmail.com>
Co-authored-by: cezmunsta <github@incoming-email.co.uk>
Co-authored-by: prombot <prometheus-team@googlegroups.com>
Co-authored-by: Sergey Morozov <38383507+ken3122@users.noreply.github.com>
Co-authored-by: Julien Pivotto <roidelapluie@o11y.eu>
Co-authored-by: Kurtis Bass <kurtis.bass@hinge.co>
Co-authored-by: Khiem Doan <doankhiem.crazy@gmail.com>
Co-authored-by: Oleksandr Mysyura <olexandr.mysyura@pragmaticplay.com>
Co-authored-by: Zachary Caldarola <zachary.caldarola@reddit.com>
Co-authored-by: Zachary Caldarola <zmc2005@gmail.com>
Co-authored-by: Mike <gavrikster@gmail.com>
Co-authored-by: Khaled Khalifa <33331600+khkhalifa@users.noreply.github.com>
Co-authored-by: Jack Wink <57678801+mothershipper@users.noreply.github.com>
Co-authored-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Alex Tymchuk <alexander.tymchuk@percona.com>
Co-authored-by: Vadim Voitenko <74241416+wwoytenko@users.noreply.github.com>
Co-authored-by: Vadim Voitenko <vadim.voitenko@exness.com>
Co-authored-by: Tom Hughes <tom@compton.nu>
Co-authored-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
Co-authored-by: Mathis Raguin <evaelis.market@gmail.com>
Co-authored-by: Christian Albrecht <cal@albix.de>
Co-authored-by: Christian Albrecht <christian.albrecht@akquinet.de>
Co-authored-by: David Cook <divergentdave@gmail.com>
Co-authored-by: Vladimir Luksha <waldemarluksha@gmail.com>
Co-authored-by: Vladimir Luksha <luksha@limcore.io>
Co-authored-by: David Cook <dcook@divviup.org>
This moves the metrics that are queried from pg_stat_bgwriter into a dedicated collector instead of dynamically generating queries and metrics from a map. It renames some metrics including adding the `_total` suffix on all of the counters to match prometheus standards. This implementation uses QueryRowContext to allow for later addition of context for cancellation. From the Postgres documentation, it states that there is one line per WAL sender process, but it is unclear how to differentiate between them in any meaningful way. When querying the table, there is no column to identify the row, only metrics about bgwriter.
Signed-off-by: Joe Adams <github@joeadams.io>
Converts the pg_database metrics from queries.yaml to a built in collector. This is enabled by default because it is not likely to be a performance problem and likely very useful data.
Signed-off-by: Joe Adams <github@joeadams.io>
Moves code into more manageable, logical files to group behavior together. This should help improve a developer's ability to navigate the code.
Signed-off-by: Joe Adams <github@joeadams.io>
* Run the query for specific database version if provided from yml file.
By default query will run on all the databases if "runonserver" is not provided.
If user want the query to be run on multiple database versions, use below string.
runonserver: "9.5, 9.6"
Example yml file as below. ( e.g. below query will run only on database version 9.5 )
pg_replication:
query: "SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp())) as lag"
master: true
runonserver: "9.5"
metrics:
- lag:
usage: "GAUGE"
description: "Replication lag behind master in seconds"
* Fixed the below review comments given by Ashesh Vashi
Instead of having db version string from yml file, user can define the range of
database server version where query is to be executed.
If user want to run the query on database version greater than 10.0.0, use below format.
runonserver: ">=10.0.0"
Below are the example of db version range user can defined in yml file.
<=10.1.0
>=12.1.0
=11.0.0
<9.6.0 || >=11.0.0
* Remove the call from unused places where 'runOnServer' is not required.
Only Server type hold that value.
* Fix compilation issues.
* Fix the issue with Debugln to print the database server version
* Support connstring syntax when discovering databases
Support connstring DSNs (`host=... user=... password=... dbname=...`) in
addition to URIs (`postgresql://user:pass@host/dbname`) for purposes of
database discovery.
Connstring syntax is needed to support accessing PostgreSQL via Unix
domain sockets (`host=/run/postgres`), which is not really possible with
URI syntax.
* Appease gometalinter, don't shadow namespace
When the connection to the PostgreSQL instance cannot be established straight
at startup, a race condition can happen when autoDiscoverDatabases is true. If
discoverDatabaseDSNs fails, no dsn is set as the master database, and, if
scrapeDSN succeeds, checkMapVersions will have omitted the default metrics in
the server metric map. The metric map won't be updated unless the version
returned by the PostgreSQL instance changes. With this patch, scrapeDSN won't
be run unless discoverDatabaseDSNs succeeded and thus the race condition is
eliminated.
Signed-off-by: Yann Soubeyrand <yann.soubeyrand@camptocamp.com>
Since we cannot know in advance the metrics which the exporter will generate,
the workaround is to run a Collect and return the metric descriptors. This is
problematic when the connection to the PostgreSQL instance cannot be
established straight from the start. This patch makes Describe return no
descriptors, effectively turning the collector in an unchecked one, which we're
in the typical use case here:
https://pkg.go.dev/github.com/prometheus/client_golang/prometheus?tab=doc#hdr-Custom_Collectors_and_constant_Metrics.
Signed-off-by: Yann Soubeyrand <yann.soubeyrand@camptocamp.com>
