Skip to content
This repository was archived by the owner on Oct 5, 2022. It is now read-only.

Initial Druid Metrics Integration #27

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Initial Druid Metrics Integration #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added Druid/Images/120x60.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Druid/Images/128x128.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Druid/Images/200x128.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
70 changes: 70 additions & 0 deletions Druid/description.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
<html>

<head>
<link rel="stylesheet" type="text/css" href="preview.css">
</head>

<body>
<!--<%def name="title()">Your Integration</%def>-->
<div class="sectiontitle" id="previewtitle"><h2>Tile preview</h2></div>
<div class="saas_tile" id="tile">
<div class="saas_avatar saas_in">
<img src="Images/200x128.png">
</div>
<div class="saas_content">
<p class="saas_name saas_in">Druid</p>
<div class="saas_controls">
<button class="meh">
<span class="js-icon text-success">+</span>
<span class="saas_msg" data-state_msg="Available" data-action_msg="Install">Available</span>
</button>
</div>
<div class="saas_description saas_in" id="tiledescription">Druid is an open-source data store designed for sub-second queries on real-time and historical data. </div>
</div>
</div>

<div class="integration_content" id="int-overview">
<h2>Overview</h2>
<p>Get metrics from Druid related to queries, ingestion, and coordination in real time</p>
<ul>
<li>Visualize performance metrics</li>
</ul>
<p>
<img class="screenshot" src="Images/snapshot.png" />
</p>
</div>

<div class="integration_content" id="int-configuration">
<h2>Configuration</h2>
<p>
<em>To capture metrics you need to...</em>
</p>
<ol>
<li>Configure Druid(
<a href="http://druid.io/docs/latest/development/extensions-contrib/statsd.html">see doc<a/>)<br>
<pre class="textfile">
<code> Example Configuration in common.runtime.properties:
# Monitoring
#
druid.emitter=composing
druid.emitter.composing.emitters=["logging", "statsd"]

druid.monitoring.monitors=["com.metamx.metrics.JvmMonitor"]
druid.emitter.logging.logLevel=info

druid.emitter.statsd.hostname=dogstatsd
druid.emitter.statsd.port=8125
druid.emitter.statsd.dimensionMapPath=dist/druid/extensions/statsd-emitter/metricDimensions.json

</code>
</pre>
</li>
<li>If you want to take advantage of tagging system from Datadog, need to change some code in statsd-emitter and rebuild it with <a href="https://github.com/DataDog/java-dogstatsd-client">java-dogstatsd-client<a/><br>
We're working on open source those changes to Druid statsd-emitter.
<pre class="textfile">
</pre>
</li>
</ol>
</div>
</body>
</html>
121 changes: 121 additions & 0 deletions Druid/metrics.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
druid.broker.jetty.numOpenConnections,count,,connection,per_unit_name,Number of open jetty connections.,0,druid,druid
druid.broker.jvm.bufferpool.capacity,gauge,,byte,,Bufferpool capacity.,0,,
druid.broker.jvm.bufferpool.count,count,,,,Bufferpool count.,0,,
druid.broker.jvm.bufferpool.used,gauge,,byte,,Bufferpool used.,1,,
druid.broker.jvm.gc.count,count,,,,Garbage collection count.,0,,
druid.broker.jvm.gc.time,timer,,second,,Garbage collection time.,0,,
druid.broker.jvm.mem.committed,gauge,,byte,,Committed memory.,1,,
druid.broker.jvm.mem.init,gauge,,byte,,Initial memory.,0,,
druid.broker.jvm.mem.max,gauge,,byte,,Max memory.,0,,
druid.broker.jvm.mem.used,gauge,,byte,,Used memory.,0,,
druid.broker.jvm.pool.committed,gauge,,byte,,Committed pool.,1,,
druid.broker.jvm.pool.init,gauge,,byte,,Initial pool.,0,,
druid.broker.jvm.pool.max,gauge,,byte,,Max pool.,0,,
druid.broker.jvm.pool.used,gauge,,byte,,Pool used.,-1,,
druid.broker.query.bytes,gauge,,byte,,number of bytes returned in query response.,0,,
druid.broker.query.cache.delta.averageBytes,count,,byte,,Average cache entry byte size since the last emission.,0,,
druid.broker.query.cache.delta.errors,count,,error,,Number of cache errors since the last emission.,-1,,
druid.broker.query.cache.delta.evictions,count,,eviction,,Number of cache evictions since the last emission.,0,,
druid.broker.query.cache.delta.hitRate,count,,hit,second,Cache hit rate since the last emission.,1,,
druid.broker.query.cache.delta.hits,count,,hit,,Number of cache hits since the last emission.,0,,
druid.broker.query.cache.delta.misses,count,,miss,,Number of cache misses since the last emission since the last emission.,-1,,
druid.broker.query.cache.delta.numEntries,count,,entry,,Number of cache entries since the last emission.,0,,
druid.broker.query.cache.delta.sizeBytes,count,,byte,entry,Size in bytes of cache entries since the last emission.,0,,
druid.broker.query.cache.delta.timeouts,count,,rate,,Number of cache timeouts since the last emission.,0,,
druid.broker.query.cache.total.averageBytes,gauge,,byte,,Average total cache byte size.,0,,
druid.broker.query.cache.total.errors,gauge,,error,,Number of total cache errors.,-1,,
druid.broker.query.cache.total.evictions,gauge,,eviction,,Number of total cache evictions.,0,,
druid.broker.query.cache.total.hitRate,gauge,,hit,second,Number of total cache hits.,0,,
druid.broker.query.cache.total.hits,gauge,,hit,,Number of total cache hits.,0,,
druid.broker.query.cache.total.misses,gauge,,miss,,Number of total cache misses.,0,,
druid.broker.query.cache.total.numEntries,gauge,,entry,,Number of total cache entries.,0,,
druid.broker.query.cache.total.sizeBytes,gauge,,byte,entry,Size in bytes of total cache entries.,0,,
druid.broker.query.cache.total.timeouts,gauge,,rate,,Number of total cache timeouts.,0,,
druid.broker.query.cpu.time,timer,,microsecond,query,Microseconds of CPU time taken to complete a query,0,,
druid.broker.query.node.bytes,count,,byte,historical node,number of bytes returned from querying individual historical/realtime nodes.,0,,
druid.broker.query.node.time,timer,,millisecond,historical node,Milliseconds taken to query individual historical/realtime nodes.,0,,
druid.broker.query.node.ttfb,timer,,millisecond,,Time to first byte. Milliseconds elapsed until broker starts receiving the response from individual historical/realtime nodes.,0,,
druid.broker.query.time,timer,,millisecond,,Milliseconds taken to complete a query.,0,,
druid.broker.segment.scan.pending,gauge,,,,Number of segments in queue waiting to be scanned.,0,,
druid.coordinator.jetty.numOpenConnections,count,,connection,per_unit_name,Number of open jetty connections.,0,,
druid.coordinator.jvm.bufferpool.capacity,gauge,,byte,,Bufferpool capacity.,0,,
druid.coordinator.jvm.bufferpool.count,count,,,,Bufferpool count.,0,,
druid.coordinator.jvm.bufferpool.used,gauge,,byte,,Bufferpool used.,1,,
druid.coordinator.jvm.gc.count,count,,,,Garbage collection count.,0,,
druid.coordinator.jvm.gc.time,timer,,second,,Garbage collection time.,0,,
druid.coordinator.jvm.mem.committed,gauge,,byte,,Committed memory.,1,,
druid.coordinator.jvm.mem.init,gauge,,byte,,Initial memory.,0,,
druid.coordinator.jvm.mem.max,gauge,,byte,,Max memory.,0,,
druid.coordinator.jvm.mem.used,gauge,,byte,,Used memory.,0,,
druid.coordinator.jvm.pool.committed,gauge,,,,Committed pool.,1,,
druid.coordinator.jvm.pool.init,gauge,,byte,,Initial pool.,0,,
druid.coordinator.jvm.pool.max,gauge,,byte,,Max pool.,0,,
druid.coordinator.jvm.pool.used,gauge,,byte,,Pool used.,-1,,
druid.coordinator.segment.assigned.count,gauge,,,,Number of segments assigned to be loaded in the cluster.,0,,
druid.coordinator.segment.count,gauge,,,,Number of available segments.,0,,
druid.coordinator.segment.dropped.count,gauge,,,,Number of segments dropped due to being overshadowed.,0,,
druid.coordinator.segment.dropQueue.count,gauge,,,,Number of segments to drop.,0,,
druid.coordinator.segment.loadQueue.count,gauge,,,,Number of segments to load.,0,,
druid.coordinator.segment.loadQueue.failed,gauge,,,,Number of segments that failed to load.,-1,,
druid.coordinator.segment.loadQueue.size,gauge,,byte,,Size in bytes of segments to load.,0,,
druid.coordinator.segment.moved.count,count,,,,Number of segments moved in the cluster.,0,,
druid.coordinator.segment.overShadowed.count,gauge,,,,Number of overShadowed segments.,0,,
druid.coordinator.segment.size,gauge,,byte,,Size in bytes of available segments.,0,,
druid.coordinator.segment.unavailable.count,gauge,,,,Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.,-1,,
druid.coordinator.segment.underReplicated.count,gauge,,,,Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.,-1,,
druid.coordinator.segment.unneeded.count,count,,,,Number of segments dropped due to being marked as unused.,0,,
druid.historical.jetty.numOpenConnections,count,,connection,per_unit_name,Number of open jetty connections.,0,,
druid.historical.jvm.bufferpool.capacity,gauge,,byte,,Bufferpool capacity.,0,,
druid.historical.jvm.bufferpool.count,count,,,,Bufferpool count.,0,,
druid.historical.jvm.bufferpool.used,gauge,,byte,,Bufferpool used.,1,,
druid.historical.jvm.gc.count,count,,,,Garbage collection count.,-1,,
druid.historical.jvm.gc.time,timer,,second,,Garbage collection time.,0,,
druid.historical.jvm.mem.committed,gauge,,byte,,Committed memory.,1,,
druid.historical.jvm.mem.init,gauge,,byte,,Initial memory.,0,,
druid.historical.jvm.mem.max,gauge,,byte,,Max memory.,0,,
druid.historical.jvm.mem.used,gauge,,byte,,Used memory.,0,,
druid.historical.jvm.pool.committed,gauge,,,,Committed pool.,1,,
druid.historical.jvm.pool.init,gauge,,byte,,Initial pool.,0,,
druid.historical.jvm.pool.max,gauge,,byte,,Max pool.,0,,
druid.historical.jvm.pool.used,gauge,,byte,,Pool used.,-1,,
druid.historical.query.bytes,gauge,,byte,,number of bytes returned in query response.,0,,
druid.historical.query.cache.delta.averageBytes,count,,byte,,Average cache entry byte size since the last emission.,0,,
druid.historical.query.cache.delta.errors,count,,error,,Number of cache errors since the last emission.,-1,,
druid.historical.query.cache.delta.evictions,count,,eviction,,Number of cache evictions since the last emission.,0,,
druid.historical.query.cache.delta.hitRate,count,,hit,second,Cache hit rate since the last emission.,0,,
druid.historical.query.cache.delta.hits,count,,hit,,Number of cache hits since the last emission.,0,,
druid.historical.query.cache.delta.misses,count,,miss,,Number of cache misses since the last emission since the last emission.,-1,,
druid.historical.query.cache.delta.numEntries,count,,entry,,Number of cache entries since the last emission.,0,,
druid.historical.query.cache.delta.sizeBytes,count,,byte,entry,Size in bytes of cache entries since the last emission.,0,,
druid.historical.query.cache.delta.timeouts,count,,rate,,Number of cache timeouts since the last emission.,0,,
druid.historical.query.cache.total.averageBytes,gauge,,byte,,Average total cache byte size.,0,,
druid.historical.query.cache.total.errors,gauge,,error,,Number of total cache errors.,-1,,
druid.historical.query.cache.total.evictions,gauge,,eviction,,Number of total cache evictions.,0,,
druid.historical.query.cache.total.hitRate,gauge,,hit,second,Number of total cache hits.,0,,
druid.historical.query.cache.total.hits,gauge,,hit,,Number of total cache hits.,0,,
druid.historical.query.cache.total.misses,gauge,,miss,,Number of total cache misses.,0,,
druid.historical.query.cache.total.numEntries,gauge,,entry,,Number of total cache entries.,0,,
druid.historical.query.cache.total.sizeBytes,gauge,,byte,entry,Size in bytes of total cache entries.,0,,
druid.historical.query.cache.total.timeouts,gauge,,rate,,Number of total cache timeouts.,0,,
druid.historical.query.cpu.time,timer,,microsecond,query,Microseconds of CPU time taken to complete a query,0,,
druid.historical.query.segment.time,timer,,millisecond,segment,Milliseconds taken to query individual segment. Includes time to page in the segment from disk.,-1,,
druid.historical.query.segmentAndCache.time,timer,,millisecond,,Milliseconds taken to query individual segment or hit the cache (if it is enabled on the historical node).,-1,,
druid.historical.query.time,timer,,millisecond,query,Milliseconds taken to complete a query.,-1,,
druid.historical.query.wait.time,timer,,millisecond,segment,Milliseconds spent waiting for a segment to be scanned.,-1,,
druid.historical.segment.scan.pending,gauge,,,,Number of segments in queue waiting to be scanned.,-1,,
druid.overlord.jetty.numOpenConnections,count,,connection,per_unit_name,Number of open jetty connections.,0,,
druid.overlord.jvm.bufferpool.capacity,gauge,,byte,,Bufferpool capacity.,0,,
druid.overlord.jvm.bufferpool.count,count,,,,Bufferpool count.,0,,
druid.overlord.jvm.bufferpool.used,gauge,,byte,,Bufferpool used.,1,,
druid.overlord.jvm.gc.count,count,,,,Garbage collection count.,0,,
druid.overlord.jvm.gc.time,timer,,second,,Garbage collection time.,0,,
druid.overlord.jvm.mem.committed,gauge,,byte,,Committed memory.,1,,
druid.overlord.jvm.mem.init,gauge,,byte,,Initial memory.,0,,
druid.overlord.jvm.mem.max,gauge,,byte,,Max memory.,0,,
druid.overlord.jvm.mem.used,gauge,,byte,,Used memory.,0,,
druid.overlord.jvm.pool.committed,gauge,,,,Committed pool.,1,,
druid.overlord.jvm.pool.init,gauge,,byte,,Initial pool.,0,,
druid.overlord.jvm.pool.max,gauge,,byte,,Max pool.,0,,
druid.overlord.jvm.pool.used,gauge,,byte,,Pool used.,-1,,
druid.overlord.segment.added.bytes,count,,byte,,Size in bytes of new segments created.,0,,
druid.overlord.task.run.time,timer,,millisecond,,Milliseconds taken to run task.,0,,
Loading