====== Graphite Plugin ======

===== Purpose =====

The Graphite plugin gives the possibility to collect metrics, alarms and metadata in Graphite plateform.
Once active, this plugin will collect all generated alerts, metrics and metadata and store them waiting for the probe to collect them
To limit memory usage the number of element temporally stored is limited

{{..:plugins:pasted:20190225-154022.png}}

----
===== Create/Edit =====

  * Press **Create** or select **Edit** in the table action menu to open the Graphite Plugin editor
  * Set Graphite Plugin parameter:
    * Active
    * Name
    * Host
    * Port
    * URL
    * Secure mode
    * Send alarms
    * Send metrics
    * Send Redpeaks stats
    * Max chunk size
{{..:settings:pasted:20210618-170209.png}}
----
===== Installing graphite =====

=== Prerequisites ===
  * Linux VM with a Cent OS/RedHat distribution
  * 200 GB disk space
  * 2 cores
  * 4 GB RAM

=== Installation ===
  * sudo docker run -d --name graphite --restart=always -p 80:80 -p 2003-2004:2003-2004 -p 2023-2024:2023-2024 -p 8125:8125/udp -p 8126:8126 graphiteapp/graphite-statsd
  * open ports 2003/2004
    * **CentOS/RHEL6 :**
    * sudo iptables -I INPUT -p tcp -m tcp --dport 2003 -j ACCEPT
    * service iptables save
    * **CentOS/RHEL7 :**
    * sudo firewall-cmd --zone=public --add-port=2003/tcp --permanent
    * Default credentials: root/root

=== Configure ===
  * docker exec -i -t graphite bash
  * vi /opt/graphite/conf/storage-schemas.conf
  * Configure the retention granularity.
    * The following will define retention as:
      * Keep 7 days with a granularity of 1 minute
      * Keep 30 days with a granularity of 5 minutes
      * Keep 90 days with a granularity of 90d
<code>
[agentil metrics availability]
pattern = ^agentil\.promonitor\.
retentions = 1m:7d,5m:30d,1h:90d

[agentil sap]
pattern = ^agentil\.sap\.
retentions = 1m:7d,5m:30d,1h:90d
</code>

  * you can define custom granularity based on metric path 
  * Then remove everything in /opt/graphite/storage/whisper
  * Then restart graphite via docker:
    * **sudo docker stop graphite**
  * If iptables problem issue when starting graphite, try to restart docker:  
    * **sudo systemctl restart docker**

----
===== Installing Grafana =====

=== Install === 

  * sudo  docker run -d -p 3000:3000 -v /var/lib/grafana:/var/lib/grafana -e "GF_SECURITY_ADMIN_PASSWORD=secret" grafana/grafana
  * open port 3000

=== Start ===

  * Check /var/lib/grafana is writable
  * sudo docker start <image ID>
  * **Does not restart automatically if docker is restarted**

=== Configure ===

  * Go to http://host:3000
  * Login: admin/secret
  * Admin->Datasource->Add data source
  * Choose graphite
  * HTTP settings:
    * URL: Graphite FQDN
    * Access: server

----
===== Troubleshooting =====

  * No metrics created in Graphite
    * Check plugin connectivity (port number must be 2003), test must show success.
    * Check disk space is not full
    * Try the metric sender script from local and distant severs.

  * Some metrics seems to be missing
    * Check "use title as source". If title contains spaces, metric will be rejected.
    * Check /var/log/carbon.log for ignore messages
    * Restart Redpeaks, can be some cache issue

  * Disk full.
    * Can be caused by huge log files : carbon.log
    * run the following commands:
<code>
sudo docker exec -i -t graphite bash
cd /var/log
rm -f carbon.log
exit
sudo docker stop graphite
sudo docker start graphite
</code>