Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
A modern Cron replacement that is Docker-friendly
"Crontab" is in YAML format;
Builtin sending of Sentry and Mail outputs when cron jobs fail;
Flexible configuration: you decide how to determine if a cron job fails or not;
Designed for running in Docker, Kubernetes, or 12 factor environments:
Option to automatically retry failing cron jobs, with exponential backoff;
Optional HTTP REST API, to fetch status and start jobs on demand;
Arbitrary timezone support;
.. [1] Whereas vixie cron only logs to syslog, requiring a syslog daemon to be running in the background or else you don't get logs!
The project is in beta stage: essential features are complete, and the focus is finding and fixing bugs before the first stable release.
Install using pip +++++++++++++++++
yacron requires Python >= 3.6 (for systems with older Python, use the binary instead). It is advisable to install it in a Python virtual environment, for example:
.. code-block:: shell
python3 -m venv yacronenv
. yacronenv/bin/activate
pip install yacron
Install using pipx ++++++++++++++++++
pipx_ automates creating a virtualenv and installing a python program in the newly created virtualenv. It is as simple as:
.. code-block:: shell
pipx install yacron
.. _pipx: https://github.com/pipxproject/pipx
Install using binary ++++++++++++++++++++
Alternatively, a self-contained binary can be downloaded from github: https://github.com/gjcarneiro/yacron/releases. This binary should work on any Linux 64-bit system post glibc 2.23 (e.g. Ubuntu:16.04). Python is not required on the target system (it is embedded in the executable).
Configuration is in YAML format. To start yacron, give it a configuration file
or directory path as the -c
argument. For example::
yacron -c /tmp/my-crontab.yaml
This starts yacron (always in the foreground!), reading
/tmp/my-crontab.yaml
as configuration file. If the path is a directory,
any *.yaml
or *.yml
files inside this directory are taken as
configuration files.
Configuration basics ++++++++++++++++++++
This configuration runs a command every 5 minutes:
.. code-block:: yaml
jobs:
- name: test-01
command: echo "foobar"
shell: /bin/bash
schedule: "*/5 * * * *"
The command can be a string or a list of strings. If command is a string,
yacron runs it through a shell, which is /bin/bash
in the above example, but
is /bin/sh
by default.
If the command is a list of strings, the command is executed directly, without a shell. The ARGV of the command to execute is extracted directly from the configuration:
.. code-block:: yaml
jobs:
- name: test-01
command:
- echo
- foobar
schedule: "*/5 * * * *"
The schedule
option can be a string in a crontab format specified by https://github.com/josiahcarlson/parse-crontab (this module is used by yacron).
Additionally @reboot can be included , which will only run the job when yacron is initially
executed. Further schedule
can be an object with properties. The following configuration
runs a command every 5 minutes, but only on the specific date 2017-07-19, and
doesn't run it in any other date:
.. code-block:: yaml
jobs:
- name: test-01
command: echo "foobar"
schedule:
minute: "*/5"
dayOfMonth: 19
month: 7
year: 2017
dayOfWeek: "*"
Important: by default all time is interpreted to be in UTC, but you can
request to use local time instead. For instance, the cron job below runs
every day at 19h27 local time because of the utc: false
option:
.. code-block:: yaml
jobs: - name: test-01 command: echo "hello" schedule: "27 19 * * *" utc: false captureStdout: true
Since Yacron version 0.11, you can also request that the schedule be
interpreted in an arbitrary timezone, using the timezone
attribute:
.. code-block:: yaml
jobs: - name: test-01 command: echo "hello" schedule: "27 19 * * *" timezone: America/Los_Angeles captureStdout: true
You can ask for environment variables to be defined for command execution:
.. code-block:: yaml
jobs:
- name: test-01
command: echo "foobar"
shell: /bin/bash
schedule: "*/5 * * * *"
environment:
- key: PATH
value: /bin:/usr/bin
You can also provide an environment file to define environments for command execution:
.. code-block:: yaml
jobs:
- name: test-01
command: echo "foobar"
shell: /bin/bash
schedule: "*/5 * * * *"
env_file: .env
The env file must be a list of KEY=VALUE
pairs. Empty lines and lines starting with #
will be ignored.
Variables declared in the environment
option will override those found in the env_file
.
Specifying defaults +++++++++++++++++++
There can be a special defaults
section in the config. Any attributes
defined in this section provide default values for cron jobs to inherit.
Although cron jobs can still override the defaults, as needed:
.. code-block:: yaml
defaults:
environment:
- key: PATH
value: /bin:/usr/bin
shell: /bin/bash
utc: false
jobs:
- name: test-01
command: echo "foobar" # runs with /bin/bash as shell
schedule: "*/5 * * * *"
- name: test-02 # runs with /bin/sh as shell
command: echo "zbr"
shell: /bin/sh
schedule: "*/5 * * * *"
Note: if the configuration option is a directory and there are multiple configuration files in that directory, then the defaults
section in each configuration file provides default options only for cron jobs inside that same file; the defaults have no effect beyond any individual YAML file.
Reporting +++++++++
Yacron has builtin support for reporting jobs failure (more on that below) by email, Sentry and shell command (additional reporting methods might be added in the future):
.. code-block:: yaml
Here, the onFailure
object indicates that what to do when a job failure
is detected. In this case we ask for it to be reported both to sentry and by
sending an email.
The captureStderr: true
part instructs yacron to capture output from the the
program's standard error
, so that it can be included in the report. We could
also turn on standard output
capturing via the captureStdout: true
option.
By default, yacron captures only standard error. If a cron job's standard error
or standard output capturing is not enabled, these streams will simply write to
the same standard output and standard error as yacron itself.
Both stdout
and stderr
stream lines are by default prefixed with
[{job_name} {stream_name}]
, i.e. [test-01 stdout]
, if for any reason you
need to change this, provide the option streamPrefix
(new in version 0.16)
with your own custom string.
.. code-block:: yaml
In some cases, for instance when you're logging JSON objects you might want to completely get rid of the prefix altogether:
.. code-block:: yaml
It is possible also to report job success, as well as failure, via the
onSuccess
option.
.. code-block:: yaml
Since yacron 0.5, it is possible to customise the format of the report. For
mail
reporting, the option subject
indicates what is the subject of the
email, while body
formats the email body. For Sentry reporting, there is
only body
. In all cases, the values of those options are strings that are
processed by the jinja2_ templating engine. The following variables are
available in templating:
.. _jinja2: http://jinja.pocoo.org/
Example:
.. code-block:: yaml
The shell reporter (since yacron 0.13) executes a user given shell command in the specified shell. It passes all environment variables from the python executable and specifies some additional ones to inform about the state of the job:
A simple example configuration:
.. code-block:: yaml
Since yacron 0.15, it is possible to send emails formatted as html, by adding
the html: true
property. For example, here the standard output of a shell
command is captured and interpreted as html and placed in the email message:
.. code-block:: yaml
Metrics +++++++++
Yacron has builtin support for writing job metrics to Statsd_:
.. _Statsd: https://github.com/etsy/statsd
.. code-block:: yaml
jobs:
- name: test01
command: echo "hello"
schedule: "* * * * *"
statsd:
host: my-statsd.example.com
port: 8125
prefix: my.cron.jobs.prefix.test01
With this config Yacron will write the following metrics over UDP
to the Statsd listening on my-statsd.example.com:8125
:
.. code-block::
my.cron.jobs.prefix.test01.start:1|g # this one is sent when the job starts my.cron.jobs.prefix.test01.stop:1|g # the rest are sent when the job stops my.cron.jobs.prefix.test01.success:1|g my.cron.jobs.prefix.test01.duration:3|ms|@0.1
Handling failure ++++++++++++++++
By default, yacron considers that a job has failed
if either the process
returns a non-zero code or if it generates output to standard error
(and
standard error capturing is enabled, of course).
You can instruct yacron how to determine if a job has failed or not via the
failsWhen
option:
.. code-block:: yaml
failsWhen: producesStdout: false producesStderr: true nonzeroReturn: true always: false
producesStdout If true, any captured standard output causes yacron to consider the job as failed. This is false by default.
producesStderr If true, any captured standard error causes yacron to consider the job as failed. This is true by default.
nonzeroReturn If true, if the job process returns a code other than zero causes yacron to consider the job as failed. This is true by default.
always If true, if the job process exits that causes yacron to consider the job as failed. This is false by default.
It is possible to instruct yacron to retry failing cron jobs by adding a
retry
option inside onFailure
:
.. code-block:: yaml
The above settings tell yacron to retry the job up to 10 times, with the delay between retries defined by an exponential backoff process: initially 1 second, doubling for every retry up to a maximum of 30 seconds. A value of -1 for maximumRetries will mean yacron will keep retrying forever, this is mostly useful with a schedule of "@reboot" to restart a long running process when it has failed.
If the cron job is expected to fail sometimes, you may wish to report only in
the case the cron job ultimately fails after all retries and we give up on it.
For that situation, you can use the onPermanentFailure
option:
.. code-block:: yaml
Concurrency
+++++++++++
Sometimes it may happen that a cron job takes so long to execute that when the moment its next scheduled execution is reached a previous instance may still be running. How yacron handles this situation is controlled by the option concurrencyPolicy
, which takes one of the following values:
Allow allows concurrently running jobs (default) Forbid forbids concurrent runs, skipping next run if previous hasn’t finished yet Replace cancels currently running job and replaces it with a new one
Execution timeout +++++++++++++++++
(new in version 0.4)
If you have a cron job that may possibly hang sometimes, you can instruct yacron
to terminate the process after N seconds if it's still running by then, via the
executionTimeout
option. For example, the following cron job takes 2
seconds to complete, yacron will terminate it after 1 second:
.. code-block:: yaml
When terminating a job, it is always a good idea to give that job process some time to terminate properly. For example, it may have opened a file, and even if you tell it to shutdown, the process may need a few seconds to flush buffers and avoid losing data.
On the other hand, there are times when programs are buggy and simply get stuck,
refusing to terminate nicely no matter what. For this reason, yacron always
checks if a process exited some time after being asked to do so. If it hasn't,
it tries to forcefully kill the process. The option killTimeout
option
indicates how many seconds to wait for the process to gracefully terminate
before killing it more forcefully. In Unix systems, we first send a SIGTERM,
but if the process doesn't exit after killTimeout
seconds (30 by default)
then we send SIGKILL. For example, this cron job ignores SIGTERM, and so yacron
will send it a SIGKILL after half a second:
.. code-block:: yaml
Change to another user/group ++++++++++++++++++++++++++++
(new in version 0.11)
You can request that Yacron change to another user and/or group for a specific
cron job. The field user
indicates the user (uid or userame) under which
the subprocess must be executed. The field group
(gid or group name)
indicates the group id. If only user
is given, the group defaults to the
main group of that user. Example:
.. code-block:: yaml
Naturally, yacron must be running as root in order to have permissions to change to another user.
Remote web/HTTP interface +++++++++++++++++++++++++
(new in version 0.10)
If you wish to remotely control yacron, you can optionally enable an HTTP REST interface, with the following configuration (example):
.. code-block:: yaml
web: listen: - http://127.0.0.1:8080 - unix:///tmp/yacron.sock
Now you have the following options to control it (using HTTPie as example):
Get the version of yacron: ##########################
.. code-block:: shell
$ http get http://127.0.0.1:8080/version HTTP/1.1 200 OK Content-Length: 22 Content-Type: text/plain; charset=utf-8 Date: Sun, 03 Nov 2019 19:48:15 GMT Server: Python/3.7 aiohttp/3.6.2
0.10.0b3.dev7+g45bc4ce
Get the status of cron jobs: ############################
.. code-block:: shell
$ http get http://127.0.0.1:8080/status HTTP/1.1 200 OK Content-Length: 104 Content-Type: text/plain; charset=utf-8 Date: Sun, 03 Nov 2019 19:44:45 GMT Server: Python/3.7 aiohttp/3.6.2
test-01: scheduled (in 14 seconds) test-02: scheduled (in 74 seconds) test-03: scheduled (in 14 seconds)
You may also get status info in json format:
.. code-block:: shell
$ http get http://127.0.0.1:8080/status Accept:application/json HTTP/1.1 200 OK Content-Length: 206 Content-Type: application/json; charset=utf-8 Date: Sun, 03 Nov 2019 19:45:53 GMT Server: Python/3.7 aiohttp/3.6.2
[ { "job": "test-01", "scheduled_in": 6.16588, "status": "scheduled" }, { "job": "test-02", "scheduled_in": 6.165787, "status": "scheduled" }, { "job": "test-03", "scheduled_in": 6.165757, "status": "scheduled" } ]
Start a job right now: ######################
Sometimes it's useful to start a cron job right now, even if it's not scheduled to run yet, for example for testing:
.. code-block:: shell
$ http post http://127.0.0.1:8080/jobs/test-02/start HTTP/1.1 200 OK Content-Length: 0 Content-Type: application/octet-stream Date: Sun, 03 Nov 2019 19:50:20 GMT Server: Python/3.7 aiohttp/3.6.2
Includes ++++++++
(new in version 0.13)
You may have a use case where it's convenient to have multiple config files, and choose at runtime which one to use. In that case, it might be useful if you can put common definitions (such as defaults for reporting, shell, etc.) in a separate file, that is included by the other files.
To support this use case, it is possible to ask one config file to include
another one, via the include
directive. It takes a list of file names:
those files will be parsed as configuration and merged in with this file.
Example, your main config file could be:
.. code-block:: yaml
include: - _inc.yaml
jobs:
- name: my job
...
And your included _inc.yaml
file could contain some useful defaults:
.. code-block:: yaml
defaults: shell: /bin/bash onPermanentFailure: report: sentry: ...
Custom logging ++++++++++++++
It's possible to provide a custom logging configuration, via the logging
configuration section. For example, the following configuration displays log lines with
an embedded timestamp for each message.
.. code-block:: yaml
logging:
# In the format of:
# https://docs.python.org/3/library/logging.config.html#dictionary-schema-details
version: 1
disable_existing_loggers: false
formatters:
simple:
format: '%(asctime)s [%(processName)s/%(threadName)s] %(levelname)s (%(name)s): %(message)s'
handlers:
console:
class: logging.StreamHandler
level: DEBUG
formatter: simple
stream: ext://sys.stdout
root:
level: INFO
handlers:
- console
Obscure configuration options +++++++++++++++++++++++++++++
enabled: true|false (default true) ##################################
(new in yacron 0.18)
It is possible to disable a specific cron job by adding a enabled: false
option. Jobs
with enabled: false
will simply be skipped, as if they aren't there, apart from
validating the configuration.
.. code-block:: yaml
jobs:
- name: test-01
enabled: false # this cron job will not run until you change this to `true`
command: echo "foobar"
shell: /bin/bash
schedule: "* * * * *"
enabled
option in jobs (#73)0.13.0 (2021-06-28)
body
in the schema for sentry config@reboot
in schedule (#22)extra
and level
options.utc
option and document that times are utc by default (#17);always
flag to consider any cron job that exits to be failed
(thanks evanjardineskinner)maximumRetries
can now be -1
to never stop retrying (evanjardineskinner)schedule
can be the string @reboot
to always run that cron job on startup
(evanjardineskinner)saveLimit
can be set to zero (evanjardineskinner)executionTimeout
, to terminate jobs that get stuck;killTimeout
controls how much time to wait for graceful termination before killing it;FAQs
A modern Cron replacement that is Docker-friendly
We found that yacron demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.