com.github.dimovelev:metrics-sampler
metrics-sampler is a java program which regularly queries metrics from a configured set of inputs, selects and renames them using regular expressions and sends them to a configured set of outputs. It supports JMX and JDBC as inputs and Graphite as output out of the box.
metrics-sampler is a java program which regularly queries metrics from a configured set of inputs, selects and renames them using regular expressions and sends them to a configured set of outputs. It supports JMX and JDBC as inputs and Graphite as output out of the box. Writing new extensions containing new inputs, outputs, samplers and selectors is pretty straight-forward.
Check out the following configuration as a quick-start:
<configuration>
<!-- Here we include all XML files in the selectors/ directory (relative to the location of the configuration file). Includes can be used in any file (so an
included file may include further files itself. Included files are typically just fragments of the overall configuration - they are parsed and then merged
into one XML representation in memory which is then evaluated. As a result you can do anything you would be able to do in one single file - for example
refer to elements defined in other files. -->
<includes>
<!-- The location attribute is actually a glob pattern so you can use ** to denote any number of directories, e.g. fragments/**/*.xml -->
<include location="selectors/*.xml" />
</includes>
<!-- Here we define pools that have their own life cycle, independent of the sampler they are used in. -->
<shared-resources>
<!-- This is the default thread pool used by all samplers (unless you explicitly specify a thread pool in the sampler) -->
<thread-pool name="samplers" size="10" />
<!-- This is a custom thread pool that some of the samplers will use -->
<thread-pool name="custom.samplers" size="2" />
<!-- This is a JDBC connection pool of 1 to 5 connections to an Oracle Database. -->
<jdbc-connection-pool name="oracle01" url="jdbc:oracle:thin:@//oracle1.metrics-sampler.org:1521/EXAMPLE" username="user" password="password" driver="oracle.jdbc.OracleDriver" min-size="1" max-size="5" />
</shared-resources>
<inputs>
<!-- This is an example of a template - its attributes and child elements will be copied to any input that references it
using its "parent" attribute. It is important to set template="true" so that it can never be used in a sampler
and is not forced to define all mandatory fields. -->
<jmx name="wls-template" template="true" username="admin" password="weblogic1" provider-packages="weblogic.management.remote" persistent-connection="true">
<!-- We can choose to ignore certain object names using a list of regular expressions. -->
<ignore-object-names>
<ignore-object-name regexp="^com\.oracle\.jrockit:type=Flight.+" />
</ignore-object-names>
<!-- A map of properties to pass to the JMX connector factory. You usually do not need this. -->
<connection-properties>
<entry key="jmx.remote.x.request.waiting.timeout" value="100" />
</connection-properties>
<!-- Using the socket-options you can configure some low level socket options for the RMI connections -
most notably the SO_TIMEOUT (in ms) and the socket connection timeout (in ms) -->
<socket-options connect-timeout="100" so-timeout="200" keep-alive="false" send-buffer-size="16384" receive-buffer-size="16384" />
<!-- You can also define variables in the inputs (or their templates) -->
<variables>
<string name="tomcat.port" value="8080" />
</variables>
</jmx>
<!-- WebLogic JMX server. These inputs inherit all properties (username, password, variables, ignores, etc.)
from the template input named "wls-template" -->
<jmx name="wls01" url="service:jmx:t3://weblogic1.metrics-sampler.org:6001/jndi/weblogic.management.mbeanservers.runtime" parent="wls-template" />
<jmx name="wls02" url="service:jmx:t3://weblogic2.metrics-sampler.org:6001/jndi/weblogic.management.mbeanservers.runtime" parent="wls-template" />
<!-- Tomcat JMX server -->
<jmx name="tomcat01" url="service:jmx:rmi:///jndi/rmi://tomcat.metrics-sampler.org:7001/jmxrmi" persistent-connection="true" />
<!-- Execute the given query(ies) over JDBC and use the first column as metric name, the second as metric value and the
third one as timestamp. You will need to have the JDBC drivers in the lib/ directory. If the query returns only two
columns the timestamp is set automatically to the current time when the query returned this row. -->
<jdbc name="oracle01" pool="oracle01">
<query>select replace(T2.host_name||'.'||T2.instance_name||'.'||replace(replace(replace(replace(metric_name,'/',''),'%','Perc'),'(',''),')',''),' ','_') as metric, value, (25200 + round((end_time - to_date('01-JAN-1970','DD-MON-YYYY')) * (86400),0))*1000 as dt from gv$sysmetric T1, gv$instance T2 where T1.intsize_csec between 1400 and 1600 and T1.inst_id = T2.INST_ID</query>
</jdbc>
<!-- Apache mod_status page containing information about the state of an apache httpd server. -->
<apache-status name="apache01" url="http://apache1.metrics-sampler.org:80/qos-viewer?auto" username="user" password="pass" />
<!-- Use metrics from the "info" command on the given redis instance -->
<redis name="redis" host="redis.metrics-sampler.org" port="6379">
<commands>
<!-- Additionally to the info statistics, sample the number of elements in the hash with key "hash1" and the number of elements in the list with key "list1" -->
<redis-hlen database="0" key="hash1" />
<redis-llen key="list1" />
</commands>
</redis>
<!-- Fetch metrics from the perfmaps of the given oracle NoSQL hosts -->
<oracle-nosql name="oracle-nosql" hosts="kv1.metrics-sampler.org:5000 kv2.metrics-sampler.org:5000 kv3.metrics-sampler.org:5000 kv4.metrics-sampler.org:5000" />
<!-- Fetch metrics from the diagnostics data of the given webmethods server. Only extract ZIP entries if they would require less then 10'000'000 bytes when uncompressed. Use "yyyy-MM-dd HH:mm:ss z" as simple
date format when parsing timestamps in the files -->
<webmethods name="webmethods1" url="http://webmethods1.example.com:1234/invoke/wm.server.admin/getDiagnosticData" username="user" password="pass" max-entry-size="10000000" date-format="yyyy-MM-dd HH:mm:ss z" />
<!-- Provide self-monitoring metrics of the application - statistics about every sampler, thread-pool and jdbc connection pool utilizations, etc. -->
<self name="self" />
<!-- Execute 'cmd /C "echo %METRIC%=28"' in the current working directory, passing it METRIC=a.b.c as environment (additionally to the current processes' environment) and
parsing the output as metrics. In this case always the metric "a.b.c" with value "28". You could also provide a working directory with attribute named directory -->
<exec name="exec1" command="cmd">
<arguments>
<argument>/C</argument>
<argument>echo %METRIC%=28</argument>
</arguments>
<environment>
<entry key="METRIC" value="a.b.c" />
</environment>
</exec>
<!-- Fetch the given URL over HTTP (you can also provide credentials, custom headers, etc). The response is parsed line by line using regular expressions. The first expression that matches the line wins.
In this example we expect the lines to have either name=value or name:value formats. If non of the regexes match, the line is ignored. There will probably be more response parsers coming in the future -->
<http name="http1" url="http://localhost/query_metrics.php?format=csv" username="user" password="pass">
<regexp-response-parser>
<!-- In this case we could have used one single regular expression for both line formats. For the sake of the example we used two to show that we can have as many as we want -->
<regexp-line-format expression="\s*(\S+)\s*=\s*(\S+)\s*" name-index="1" value-index="2" />
<regexp-line-format expression="\s*(\S+)\s*:\s*(\S+)\s*" name-index="1" value-index="2" />
</regexp-response-parser>
</http>
</inputs>
<outputs>
<!-- Write to the standard output -->
<console name="console" />
<!-- Send metrics to graphite running on port 2003. This is the default output - if no outputs are specified in the samplers, all outputs marked as default will be used -->
<graphite name="graphite" host="graphite.metrics-sampler.org" port="2003" default="true" />
</outputs>
<!-- We can also define some global variables that will be available in all samplers (unless overridden) -->
<variables>
<string name="tomcat.port" value="8080" />
</variables>
<!-- We define some regular expressions in groups so that we can reuse them later in the samplers -->
<selector-groups>
<selector-group name="wls">
<!-- "from-name" is a regular expression that is matched against e.g. the JMX Metric Name (consisting of canonical
object name # attribute name). The string can also contain references to variables in the form ${name}.
to-name is an expression (not a regular expression) that can use variables for things like captured groups
from the name's regular expression. -->
<regexp from-name="com\.bea:Name=DataSource_(.+),ServerRuntime=.+,Type=JDBCOracleDataSourceRuntime\.(ActiveConnectionsAverageCount|ActiveConnectionsCurrentCount|ActiveConnectionsHighCount|ConnectionDelayTime|ConnectionsTotalCount|CurrCapacity|CurrCapacityHighCount|FailuresToReconnectCount|HighestNumAvailable|HighestNumUnavailable|LeakedConnectionCount|NumAvailable|NumUnavailable|ReserveRequestCountWaitSecondsHighCount|WaitingForConnection.*)" to-name="${prefix}.jdbc.${name[1]}.${name[2]}" />
<regexp from-name="com\.bea:Name=JTARuntime,ServerRuntime=.*,Type=JTARuntime\.(.*TotalCount)" to-name="${prefix}.jta.${name[1]}" />
<regexp from-name="com\.bea:Name=ThreadPoolRuntime,ServerRuntime=.*,Type=ThreadPoolRuntime\.(CompletedRequestCount|ExecuteThreadIdleCount|ExecuteThreadTotalCount|HoggingThreadCount|MinThreadsConstraintsCompleted|MinThreadsConstraintsPending|PendingUserRequestCount|QueueLength|SharedCapacityForWorkManagers|StandbyThreadCount|Throughput)" to-name="${prefix}.threads.${name[1]}"/>
<regexp from-name="com\.bea:Name=.*,ServerRuntime=.*,Type=JRockitRuntime\.(JvmProcessorLoad|TotalGarbageCollectionCount|TotalGarbageCollectionTime|FreePhysicalMemory|UsedPhysicalMemory|Uptime)" to-name="${prefix}.jrockit.${name[1]}" />
</selector-group>
<selector-group name="tomcat">
<!-- Note that you can use variables in the "from-name" too (in this case tomcat.port). These must be explicitly
defined in the sampler (or in the input / global context) -->
<regexp from-name="Catalina:type=GlobalRequestProcessor,name=http-${tomcat.port}.\.(requestCount|bytesSent|bytesReceived)" to-name="${prefix}.http.${name[1]}"/>
</selector-group>
<selector-group name="mod_qos">
<regexp from-name=".*,metric=([^,]+),path=/([^.]+)\.(current|limit)" to-name="${prefix}.${name[2]}.${name[1]}.${name[3]}"/>
<regexp from-name=".*,metric=([^,]+)$" to-name="${prefix}.${name[1]}"/>
<regexp from-name=".*,metric=([^,]+)\.(current|limit)" to-name="${prefix}.${name[1]}.${name[2]}"/>
</selector-group>
</selector-groups>
<!-- These are the actual active runtime components that sample the data from the given input, use the given selectors to
determine which metrics are relevant (and rename them) and to send them to the given outputs. An input/output without a
sampler does not do anything or consume any resources. The samplers are scheduled at a constant rate (with the given
interval) to a thread pool of the size defined above. -->
<samplers>
<!-- Template defining common values for weblogic samplers. If you define any of the attributes / child elements in the
samplers that use this template, these values here will be lost (not appended to).
The reset-timeout is an optional attribute. If set to any positive value (the time unit is seconds) it will cause the
sampler to keep track of the number of selected metrics after each reconnect of its input. If this number of selected
metrics differs, a reset of the input and the selectors will be reset after a random timeout in the range
between 0.8*reset-timeout and 1.2*reset-timeout. This is useful if you are sampling e.g. a JMX input which takes long to
startup and exposes a partial number of matching JMX beans during startup.
-->
<sampler name="wls" template="true" interval="10" reset-timeout="60">
<selectors>
<use-group name="wls" />
</selectors>
<!-- these variables can override the ones defined in the global context and the ones defined in the input -->
<variables>
<string name="prefix" value="backend.${input.name}" />
</variables>
<value-transformers>
<!-- divide the GC pauseTime and time by 1000000 -->
<el-value-transformer name=".*\.jrockit\.gc.*\.(pauseTime|time)" expression="value / 1000000" />
</value-transformers>
</sampler>
<!-- Fetch data from "wls01" input, use the regular expressions in a group named "wls" to select and rename metrics and
send them to graphite every 10 seconds. If you specify a child element (e.g. selectors) its value will replace
the selectors defined in the template - lists of selectors will not be merged but rather replaced -->
<sampler input="wls01" parent="wls" />
<sampler input="wls02" parent="wls" />
<sampler input="tomcat01" interval="10">
<variables>
<string name="prefix" value="frontend.${input.name}" />
<!-- We override the global variable here -->
<string name="tomcat.port" value="9080" />
</variables>
<selectors>
<use-group name="tomcat" />
</selectors>
</sampler>
<!-- Setting quiet to true causes the sampler to log connection problems using debug level - thus preventing the problem
to be logged in the standard configuration. This is useful if the input is a source that is not always available but
you want to still get metrics when it is available while not flooding your logs with exceptions. -->
<sampler input="apache01" interval="10" quiet="true">
<variables>
<string name="prefix" value="frontend.${input.name}" />
</variables>
<selectors>
<use-group name="mod_qos" />
</selectors>
</sampler>
<!-- You can use ignored="true" to completely deactivate a sampler without removing / commenting it out. Note that it still needs to be valid.
This sampler also uses a custom thread pool named "custom.samplers" -->
<sampler input="oracle01" interval="10" ignored="true" pool="custom.samplers">
<variables>
<string name="prefix" value="database.${input.name}" />
</variables>
<selectors>
<!-- We can of course specify regular expressions directly here too. -->
<regexp from-name="(.*)" to-name="${name[1]}"/>
</selectors>
</sampler>
<sampler input="redis" interval="10">
<selectors>
<regexp from-name="(.*)" to-name="${name[1]}" />
</selectors>
</sampler>
<sampler input="oracle-nosql" interval="10">
<selectors>
<regexp from-name="(.*)" to-name="${name[1]}" />
</selectors>
</sampler>
<sampler input="webmethods1" interval="60">
<selectors>
<!-- Lets say we are just interested in the memory stats here -->
<regexp from-name="ServerStats\.Memory\.(.+)" to-name="${input.name}.memory.${name[1]}" />
</selectors>
</sampler>
<sampler input="exec1" interval="10">
<selectors>
<regexp from-name="(.*)" to-name="${name[1]}" />
</selectors>
</sampler>
<sampler input="http1" interval="10">
<selectors>
<regexp from-name="(.*)" to-name="${name[1]}" />
</selectors>
</sampler>
</samplers>
</configuration>
Variables can be defined in the global context, in the inputs and in the samplers. Additionally there are some variables that are automatically generated by the inputs like input.name. If a variable with the same name is defined in multiple contexts, its value will be taken from the definition in the most specific context - global variables will be overridden by variables defined in the inputs and in the samplers. Variables defined in an input will be overridden by variables defined in the samplers.
It should be pretty easy to extend the program with new inputs, outputs, samplers and selectors. For this you will need to create a new module/project like this (you could also check out the extensions-* modules which use the same mechanism):
sonatype-nexus-releases otherwise you will get HTTP 401 when trying to uploadexport GPG_TTY=$(tty)mvn clean deploy -Dgpg.keyname="<name>" -Dgpg.passphrase="<passphrase>" -Dmaven.test.skip=true -P publishgit checkout mastergit push --tags).FAQs
metrics-sampler is a java program which regularly queries metrics from a configured set of inputs, selects and renames them using regular expressions and sends them to a configured set of outputs. It supports JMX and JDBC as inputs and Graphite as output out of the box.
We found that com.github.dimovelev:metrics-sampler demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.