====== Elasticsearch Plugin ====== This plugin allows you to send alarms, metrics and metadata directly to an Elasticsearch instance Each data type can be indexed in separate Elasticsearch indices ===== Configuration ===== * From the plugin menu of Redpeaks Cockpit, press the button ''+ Create'' then select ''Elasticsearch'' in the drop-down. ===== Connection parameters ===== ^ **Parameter** ^ **Description** ^ **Mandatory** ^ | **Active** | Enables or disables the Elasticsearch plugin | Yes | | **Configuration** | Choose between Standard Configuration or Cloud Configuration | Yes | | **Name** | A unique name for the plugin instance | Yes | | **URL** | The URL (with HTTPS for a secure connection) | Yes (if Standard Config selected) | | **Port** | The port used to connect to Elasticsearch (Default: 9200) | Yes (if Standard Config selected) | | **Cloud ID** | The Cloud ID used for connecting to a cloud-based Elasticsearch instance | Yes (if Cloud Config selected) | | **API Key** | The API key for authentication | Yes (if Cloud Config selected) | | **Username** | The username for Elasticsearch authentication | No | | **Password** | The password for Elasticsearch authentication | No | | **Properties** | A semicolon-separated list of additional Elasticsearch properties | No | **Example:** {{:products:cockpit:1.0:userguide:settings:pasted:20241122-125319.png}} ==== Properties ==== * Properties are key/value pairs separated by semicolon: ''setting1=value1;setting2=value2'' * Common Elasticsearch properties include settings for index refresh intervals, number of replicas, etc. ===== Data configuration ===== * Open the Data Configuration tab to define how data is sent and indexed in Elasticsearch. ^ Alarms parameters ^^^ ^ **Parameter** ^ **Description** ^ **Mandatory** ^ | **Send alarms** | If active, alarms received by the plugin will be sent to Elasticsearch | No | | **Track alarms history** | If active, enables tracking of alarm send history in the plugin statistics | No | | **Alarm Index** | The Elasticsearch index prefix where alarms will be stored (Default: ''alarms-redpeaks'') | Yes (if Send alarms checked) | ^ Metrics parameters ^^^ ^ **Parameter** ^ **Description** ^ **Mandatory** ^ | **Send metrics** | If active, metrics received by the plugin will be sent to Elasticsearch | No | | **Track metrics history** | If active, enables tracking of sent metrics in the plugin statistics | No | | **Metric Index** | The Elasticsearch index prefix where metrics will be stored (Default: ''metrics-redpeaks'') | Yes (if Send metrics checked) | ^ Metadata parameters ^^^ ^ **Parameter** ^ **Description** ^ **Mandatory** ^ | **Send metadata** | If active, metadata received by the plugin will be sent to Elasticsearch | No | | **Track metadata history** | If active, enables tracking of sent metadata in the plugin statistics | No | | **Split metadata** | If active, flattens metadata before sending it | No | | **Metadata Index** | The Elasticsearch index prefix where metadata will be stored (Default: ''metadata-redpeaks'') | Yes (if Send metadata checked) | * You can specify different index prefixes for alarms, metadata and metrics. * The data will be indexed according to the configured settings, allowing for efficient search and retrieval. ===== Performance & advanced settings ===== ^ Queue & batching ^^^ ^ **Parameter** ^ **Description** ^ **Mandatory** ^ | **Max queue size** | Maximum number of items held in the internal queue before dropping new data (Default: 1000000) | Yes | | **Batch size** | Maximum number of items sent per bulk request (Default: 25000, min: 1) | Yes | | **Async threads** | Number of parallel send threads per data type. 1 = synchronous, >1 = async (Default: 1, max: 20) | Yes | | **Socket timeout (ms)** | Socket timeout for Elasticsearch requests in milliseconds (Default: 60000, range: 1000-600000) | Yes | ^ Elasticsearch features ^^^ ^ **Parameter** ^ **Description** ^ **Mandatory** ^ | **Create templates** | Automatically create Elasticsearch index templates on startup (Default: enabled) | No | | **Use datastream** | Use Elasticsearch datastreams instead of regular indices (Default: disabled) | No | | **Use compression** | Enable gzip compression for data transmission to Elasticsearch (Default: enabled) | No | **Example:** {{:products:cockpit:1.0:userguide:settings:pasted:20250116-161145.png}} ===== Custom alarm payload ===== * When ''Enable Custom Payload'' is active, you can define a custom payload template for alarms. * This allows you to customize the data sent to Elasticsearch when an alarm is triggered. * You can see more examples on [[..:settings:httpplugin|]] | **Enable Custom Payload for Alarms** | If active, allows customization of the alarm payload | No | | **Alarm Payload** | The custom payload template for alarms | Yes (if Custom Payload is enabled) | **Example:** {{:products:cockpit:1.0:userguide:settings:pasted:20241108-174906.png}} ==== Using Custom Payload ===== * **Formatting the Payload** : Use the ''Format'' button to format your payload template. * **Testing the Payload** : Use the ''Test Payload'' button to test your payload with sample data. * **Payload Variable Helper** : Use the ''Payload Variable Helper'' button to insert variables into your payload template. ==== Available Variables ==== You can use variables from the following categories in your payload template: * **Alarm Variables**: Information about the alarm * **Organization Variables**: Information about the organization * **Group Variables**: Information about the group * **System Variables**: Information about the system * **Connector Variables**: Information about the connector * **Properties Variables**: Custom properties defined in your system **Notes:** * Ensure that all variables used in your template are correctly formatted and correspond to existing data fields. * Ensure that the Elasticsearch instance is reachable and properly configured to accept data from your application. ===== Troubleshooting ===== If you encounter issues: * Verify the hostname and port (Standard Configuration) or the Cloud ID and API key (Cloud Configuration). * Check Elasticsearch logs for any errors related to authentication or index operations. * Ensure that the indices specified in the configuration exist in Elasticsearch and have the appropriate permissions set for the configured user. * **PKIX / Certificate errors**: If you see ''PKIX path building failed'' errors, you need to import the Elasticsearch server certificate into the Java truststore of the Cockpit server: * Export the certificate from your Elasticsearch server (or get it from your CA) * Import it using: ''keytool -importcert -alias es-cert -file your-cert.pem -keystore $JAVA_HOME/lib/security/cacerts -storepass changeit'' * Restart the Cockpit application * **Plugin shows stats at 0 after config update**: Fixed in latest version. If the issue persists, recreate the plugin. * **Log spam when Elasticsearch is unreachable**: Connection error logs are throttled to avoid flooding. Check the plugin statistics panel for current error status.