Commit Graph

126 Commits

Author SHA1 Message Date
Krasi Georgiev b75428ec19 rename package retrieve to scrape
no fucnctinal changes just renaming retrieval to scrape
2018-02-01 09:55:07 +00:00
Frederic Branczyk d3ae1ac40e
Merge pull request #3741 from krasi-georgiev/discovery-race
read/write race for the  context field in the discovery package
2018-01-30 18:17:09 +01:00
pasquier-s bde64cf5a6 Fix Kubernetes endpoints SD for empty subsets (#3660)
* Fix Kubernetes endpoints SD for empty subsets

When an endpoints object has no associated pods (replica scaled to zero
for instance), the endpoints SD should return a target group with no
targets so that the SD manager propagates this information to the scrape
manager.

Fixes #3659

* Don't send nil target groups from the Kubernetes SD

This is to be consistent with the endpoints SD part.
2018-01-30 15:00:33 +00:00
Krasi Georgiev 818dda72db updated the sd tests 2018-01-29 15:19:15 +00:00
Krasi Georgiev acc4197098 remove dicovery race for the context field 2018-01-29 15:18:07 +00:00
Frederic Branczyk 47538cf6ce
Merge pull request #3747 from prometheus/sched-update-throttle
Update throttle & tsdb update
2018-01-29 16:05:05 +01:00
Frederic Branczyk 73e829137b
discovery: Cleanup ticker 2018-01-29 13:51:04 +01:00
Ganesh Vernekar 66b0aa3b45 Fixed race condition in map iteration and map write in Discovery (#3735) (#3738)
* Fixed concurrent map iteration and map write in Discovery (#3735)

* discovery: Changed Lock to RLock in Collect
2018-01-28 22:24:31 +05:30
Krasi Georgiev fe926e7829 update the discover tests
the discovery test is now only testing update and get groups.
It doesn't do an e2e test but just a unit test of setting and receiving
target groups
2018-01-27 12:03:06 +00:00
Callum Styan 7dc05538f7 docs: SD implementations do not have to only send new/changed target groups (#3713) 2018-01-26 22:03:11 +00:00
Frederic Branczyk cfa0253ed8
discovery: Schedule updates to throttle 2018-01-26 16:24:44 +01:00
zemek 8a01a0fbed Set consul server default to localhost:8500 (#3703) 2018-01-24 12:14:32 +00:00
Julius Volz 09e460a647
discovery: Rename file SD mtime metric (#3723)
- "timestamp" -> "mtime" to be in line with node exporter and clearer.
- add unit suffix
2018-01-22 14:02:24 +01:00
Krasi Georgiev ec26751fd2 use mutexes for the discovery manager instead of a loop as this was a stupid idea 2018-01-17 18:12:58 +00:00
Krasi Georgiev 767faa44b6 fixed the tests
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-15 13:39:47 +00:00
Krasi Georgiev d12e6f29fc discovery manager ApplyConfig now takes a direct ServiceDiscoveryConfig so that it can be used for the notify manager
reimplement the service discovery for the notify manager

Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-15 13:39:44 +00:00
Goutham Veeramachaneni b20a1b1b1b
Merge pull request #3654 from krasi-georgiev/discovery-handle-discoverer-updates
discovery - handle Discoverers that send only target Group updates.
2018-01-15 18:53:22 +05:30
Krasi Georgiev 790cf30fcb remove uneeded check 2018-01-15 11:52:20 +00:00
Krasi Georgiev 38938ba493 comment nits 2018-01-15 11:47:36 +00:00
Krasi Georgiev febebcd49a more comments for the future ME, and reverted the Discovery manager execution changes as these were correct in the first place 2018-01-12 22:07:21 +00:00
Krasi Georgiev 78ba5e62a6 few mote usefull comments 2018-01-12 13:58:23 +00:00
Krasi Georgiev cabce21b70 delete empty targets sets to avoid memory leaks 2018-01-12 13:10:59 +00:00
Krasi Georgiev abfd9f1920 nits 2018-01-12 12:19:52 +00:00
Shubheksha Jalan 0471e64ad1 Use shared types from the `common` repo (#3674)
* refactor: use shared types from common repo, remove util/config

* vendor: add common/config

* fix nit
2018-01-11 16:10:25 +01:00
Krasi Georgiev 546c29af5b return early for nil target groups 2018-01-09 16:34:23 +00:00
Callum Styan 97464236c7 comments with TargetProvider should read Discoverer instead (#3667) 2018-01-08 23:59:18 +00:00
Krasi Georgiev 77bf6bece0 discovery-manager comment update 2018-01-04 21:57:28 +00:00
Krasi Georgiev 135ea0f793 discovery manager - doesn't need sorting of the target groups so move it in the discovery manager tests as we only need it there.
discovery manager - refactor the discovery tests.

Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-04 21:41:54 +00:00
Krasi Georgiev 638818a974 some Discoverers send nil targetgroup so need to check for it when updating a group 2018-01-04 13:57:34 +00:00
Krasi Georgiev 7e28397a2c discovery - handle Discoverers that send only target Group updates rather than all Targets on every update.
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2018-01-04 13:28:37 +00:00
Shubheksha Jalan ec94df49d4 Refactor SD configuration to remove `config` dependency (#3629)
* refactor: move targetGroup struct and CheckOverflow() to their own package

* refactor: move auth and security related structs to a utility package, fix import error in utility package

* refactor: Azure SD, remove SD struct from config

* refactor: DNS SD, remove SD struct from config into dns package

* refactor: ec2 SD, move SD struct from config into the ec2 package

* refactor: file SD, move SD struct from config to file discovery package

* refactor: gce, move SD struct from config to gce discovery package

* refactor: move HTTPClientConfig and URL into util/config, fix import error in httputil

* refactor: consul, move SD struct from config into consul discovery package

* refactor: marathon, move SD struct from config into marathon discovery package

* refactor: triton, move SD struct from config to triton discovery package, fix test

* refactor: zookeeper, move SD structs from config to zookeeper discovery package

* refactor: openstack, remove SD struct from config, move into openstack discovery package

* refactor: kubernetes, move SD struct from config into kubernetes discovery package

* refactor: notifier, use targetgroup package instead of config

* refactor: tests for file, marathon, triton SD - use targetgroup package instead of config.TargetGroup

* refactor: retrieval, use targetgroup package instead of config.TargetGroup

* refactor: storage, use config util package

* refactor: discovery manager, use targetgroup package instead of config.TargetGroup

* refactor: use HTTPClient and TLS config from configUtil instead of config

* refactor: tests, use targetgroup package instead of config.TargetGroup

* refactor: fix tagetgroup.Group pointers that were removed by mistake

* refactor: openstack, kubernetes: drop prefixes

* refactor: remove import aliases forced due to vscode bug

* refactor: move main SD struct out of config into discovery/config

* refactor: rename configUtil to config_util

* refactor: rename yamlUtil to yaml_config

* refactor: kubernetes, remove prefixes

* refactor: move the TargetGroup package to discovery/

* refactor: fix order of imports
2017-12-29 21:01:34 +01:00
Callum Styan d76d5de66f refactor to make timestamp collector work for multiple file_sd's 2017-12-23 10:13:11 +00:00
KalivarapuReshma a00fc883c3 Add metric for timestamp of the files file_sd is using. 2017-12-23 10:13:11 +00:00
pasquier-s 78625f85a7 Fix race condition on file SD (#3468)
The file discovery should only stop the watcher if it has been created
otherwise it may trigger a segmentation fault.
2017-12-21 10:07:43 +00:00
Krasi Georgiev 587dec9eb9 rebased and resolved conflicts with the new Discovery GUI page
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2017-12-18 20:10:03 +00:00
Krasi Georgiev 80182a5d82 use poolKey as the pool map key to avoid multi dimensional maps 2017-12-18 17:23:47 +00:00
Krasi Georgiev 1ec76d1950 rearange the contexts variables and logic
split the groupsMerge function to set and get
other small nits
2017-12-18 17:23:47 +00:00
Krasi Georgiev f2df712166 updated README 2017-12-18 17:22:50 +00:00
Krasi Georgiev aca8f85699 fixed the tests 2017-12-18 17:22:50 +00:00
Krasi Georgiev fe6c544532 some renaming and comments fixes.
remove some select state that is most likely obsoleete and hoepfully doesn't braje anything :)
merge targets will sort by Discoverer name so we can have consistent tests for the maps.
2017-12-18 17:22:50 +00:00
Krasi Georgiev f5c2c5ff8f brake the start provider func so that can run unit tests against it. 2017-12-18 17:22:50 +00:00
Krasi Georgiev c5cb0d2910 simplify naming and API. 2017-12-18 17:22:50 +00:00
Krasi Georgiev 9c61f0e8a0 scrape pool doesn't rely on context as Stop() needs to be blocking to prevent Scrape loops trying to write to a closed TSDB storage. 2017-12-18 17:22:49 +00:00
Krasi Georgiev e405e2f1ea refactored discovery 2017-12-18 17:22:49 +00:00
Brian Brazil 81db4716c1
Mention SD moratorium in README (#3573) 2017-12-11 15:38:23 +00:00
Will Howard 6a80fc24cf Parse the normalized container.PortMappings presented by the Marathon 1.5.x API
Fixes #3465
2017-12-06 11:23:12 -05:00
Brian Brazil d7b3df5ae1 Fix staticcheck errors 2017-12-02 14:52:13 +00:00
Krasi Georgiev 29506e0bca one meaningless write to the config file to trigger anothe fsnotify (#3492) 2017-12-01 17:32:27 +00:00
Tom Wilkie 099c50ce93 Avoid empty pod UID in test. 2017-11-24 15:02:42 +00:00
Tom Wilkie 9811e90d65 Fix tests. 2017-11-24 12:24:13 +00:00