gnmic supports exposing gnmi updates on a prometheus server, for a prometheus client to scrape.
A Prometheus output can be defined using the below format in
gnmic config file under
outputs: output1: type: prometheus # require # address to listen on for incoming scrape requests listen: :9804 # path to query to get the metrics path: /metrics # maximum lifetime of metrics in the local cache, # # a zero value defaults to 60s, a negative duration (e.g: -1s) disables the expiration expiration: 60s # a string to be used as the metric namespace metric-prefix: "" # a boolean, if true the subscription name will be appended to the metric name after the prefix append-subscription-name: false # a boolean, enables exporting timestamps received from the gNMI target as part of the metrics export-timestamps: false # a boolean, enables setting string type values as prometheus metric labels. strings-as-labels: false # enable debug for prometheus output debug: false # list of processors to apply on the message before writing event-processors: # Enables Consul service registration service-registration: # Consul server address, default to localhost:8500 address: # Consul Data center, defaults to dc1 datacenter: # Consul username, to be used as part of HTTP basicAuth username: # Consul password, to be used as part of HTTP basicAuth password: # Consul Token, is used to provide a per-request ACL token which overrides the agent's default token token: # Prometheus service check interval, for both http and TTL Consul checks, # defaults to 5s check-interval: # Maximum number of failed checks before the service is deleted by Consul # defaults to 3 max-fail: # Consul service name name: # List of tags to be added to the service registration, # if available, the instance-name and cluster-name will be added as tags, # in the format: gnmic-instance=$instance-name and gnmic-cluster=$cluster-name tags: # bool, enables http service check on top of the TTL check enable-http-check: # string, if enable-http-check is true, this field can be used to specify the http endpoint to be used to the check # if provided, this filed with be prepended with 'http://' (if not already present), # and appended with the value in 'path' field. # if not specified, it will be derived from the fields 'listen' and 'path' http-check-address: # if set to true, the gnmic instance will try to ac quire a lock before registering the prometheus output in consul. # this allows to register a single instance of the cluster in consul. # if the instance which acquired the lock fails, one of the remaining ones will take over. use-lock: false
gnmic creates the prometheus metric name and its labels from the subscription name, the gnmic path and the value name.
The below diagram shows an example of a prometheus metric generation from a gnmi update
The metric name starts with the string configured under metric-prefix.
Then if append-subscription-name is
true, the subscription-name as specified in
gnmic configuraiton file is appended.
The resulting string is followed by the gNMI path stripped from its keys if there are any.
All non-alphanumeric characters are replaced with an underscore "
The 3 strings are then joined with an underscore "
If further customization of the metric name is required, the processors can be used to transform the metric name.
For example, a gNMI update from subscription
port-stats with path:
is exposed as a metric named:
The metrics labels are generated from the subscription metadata (e.g:
source) and the keys present in the gNMI path elements.
For the previous example the labels would be:
prometheus_output service registration via
prometheus to dynamically discover new instances of
gnmic exposing a prometheus server ready for scraping via its service discovery feature.
If the configuration section
service-registration is present under the output definition,
gnmic will register the
prometheus_output service in
The below configuration, registers a service name
# gnmic.yaml outputs: output1: type: prometheus listen: 10.1.1.1:9804 path: /metrics service-registration: address: consul-agent.local:8500 name: gnmic-prom-srv
gnmicwith minimal configuration changes by using
prometheusservice discovery feature.
Simplified scrape configuration in the prometheus client:
# prometheus.yaml scrape_configs: - job_name: 'gnmic' scrape_interval: 10s consul_sd_configs: - server: $CONSUL_ADDRESS services: - $service_name
$service_name to be discovered by
prometheus is configured under
If the service registration name field is not present, it will be populated with
gnmic instance-name (if present) and the
prometheus_output name, joined with a
gnmic registers the service in
Consul with a
ttl check enabled by default:
gnmicperiodically updates the service definition in
Consul. The goal is to allow
Consulto detect a same instance restarting with a different service name.
http check is added: *
Consul periodically scrapes the prometheus server endpoint to check its availability.
# gnmic.yaml outputs: output1: type: prometheus listen: 10.1.1.1:9804 path: /metrics service-registration: address: consul-agent.local:8500 name: gnmic-prom-srv enable-http-check: true
Note that for the
http check to work properly, a routable address ( IP or name ) should be specified under
Otherwise, a routable address should be added under