Media Processing

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package cgi implements the common gateway interface (CGI) for Caddy, a modern, full-featured, easy-to-use web server. This plugin lets you generate dynamic content on your website by means of command line scripts. To collect information about the inbound HTTP request, your script examines certain environment variables such as PATH_INFO and QUERY_STRING. Then, to return a dynamically generated web page to the client, your script simply writes content to standard output. In the case of POST requests, your script reads additional inbound content from standard input. The advantage of CGI is that you do not need to fuss with server startup and persistence, long term memory management, sockets, and crash recovery. Your script is called when a request matches one of the patterns that you specify in your Caddyfile. As soon as your script completes its response, it terminates. This simplicity makes CGI a perfect complement to the straightforward operation and configuration of Caddy. The benefits of Caddy, including HTTPS by default, basic access authentication, and lots of middleware options extend easily to your CGI scripts. CGI has some disadvantages. For one, Caddy needs to start a new process for each request. This can adversely impact performance and, if resources are shared between CGI applications, may require the use of some interprocess synchronization mechanism such as a file lock. Your server’s responsiveness could in some circumstances be affected, such as when your web server is hit with very high demand, when your script’s dependencies require a long startup, or when concurrently running scripts take a long time to respond. However, in many cases, such as using a pre-compiled CGI application like fossil or a Lua script, the impact will generally be insignificant. Another restriction of CGI is that scripts will be run with the same permissions as Caddy itself. This can sometimes be less than ideal, for example when your script needs to read or write files associated with a different owner. Serving dynamic content exposes your server to more potential threats than serving static pages. There are a number of considerations of which you should be aware when using CGI applications. CGI SCRIPTS SHOULD BE LOCATED OUTSIDE OF CADDY’S DOCUMENT ROOT. Otherwise, an inadvertent misconfiguration could result in Caddy delivering the script as an ordinary static resource. At best, this could merely confuse the site visitor. At worst, it could expose sensitive internal information that should not leave the server. MISTRUST THE CONTENTS OF PATH_INFO, QUERY_STRING AND STANDARD INPUT. Most of the environment variables available to your CGI program are inherently safe because they originate with Caddy and cannot be modified by external users. This is not the case with PATH_INFO, QUERY_STRING and, in the case of POST actions, the contents of standard input. Be sure to validate and sanitize all inbound content. If you use a CGI library or framework to process your scripts, make sure you understand its limitations. An error in a CGI application is generally handled within the application itself and reported in the headers it returns. Additionally, if the Caddy errors directive is enabled, any content the application writes to its standard error stream will be written to the error log. This can be useful to diagnose problems with the execution of the CGI application. Your CGI application can be executed directly or indirectly. In the direct case, the application can be a compiled native executable or it can be a shell script that contains as its first line a shebang that identifies the interpreter to which the file’s name should be passed. Caddy must have permission to execute the application. On Posix systems this will mean making sure the application’s ownership and permission bits are set appropriately; on Windows, this may involve properly setting up the filename extension association. In the indirect case, the name of the CGI script is passed to an interpreter such as lua, perl or python. The basic cgi directive lets you associate a single pattern with a particular script. The directive can be repeated any reasonable number of times. Here is the basic syntax: For example: When a request such as https://example.com/report or https://example.com/report/weekly arrives, the cgi middleware will detect the match and invoke the script named /usr/local/cgi-bin/report. The current working directory will be the same as Caddy itself. Here, it is assumed that the script is self-contained, for example a pre-compiled CGI application or a shell script. Here is an example of a standalone script, similar to one used in the cgi plugin’s test suite: The environment variables PATH_INFO and QUERY_STRING are populated and passed to the script automatically. There are a number of other standard CGI variables included that are described below. If you need to pass any special environment variables or allow any environment variables that are part of Caddy’s process to pass to your script, you will need to use the advanced directive syntax described below. The values used for the script name and its arguments are subject to placeholder replacement. In addition to the standard Caddy placeholders such as {method} and {host}, the following placeholder substitutions are made: - {.} is replaced with Caddy’s current working directory - {match} is replaced with the portion of the request that satisfies the match directive - {root} is replaced with Caddy’s specified root directory You can include glob wildcards in your matches. Basically, an asterisk represents a sequence of zero or more non-slash characters and a question mark represents a single non-slash character. These wildcards can be used multiple times in a match expression. See the documentation for path/Match in the Go standard library for more details about glob matching. Here is an example directive: In this case, the cgi middleware will match requests such as https://example.com/report/weekly.lua and https://example.com/report/report.lua/weekly but not https://example.com/report.lua. The use of the asterisk expands to any character sequence within a directory. For example, if the request is made, the following command is executed: Note that the portion of the request that follows the match is not included. That information is conveyed to the script by means of environment variables. In this example, the Lua interpreter is invoked directly from Caddy, so the Lua script does not need the shebang that would be needed in a standalone script. This method facilitates the use of CGI on the Windows platform. In order to specify custom environment variables, pass along one or more environment variables known to Caddy, or specify more than one match pattern for a given rule, you will need to use the advanced directive syntax. That looks like this: For example, With the advanced syntax, the exec subdirective must appear exactly once. The match subdirective must appear at least once. The env, pass_env, empty_env, and except subdirectives can appear any reasonable number of times. pass_all_env, dir may appear once. The dir subdirective specifies the CGI executable’s working directory. If it is not specified, Caddy’s current working directory is used. The except subdirective uses the same pattern matching logic that is used with the match subdirective except that the request must match a rule fully; no request path prefix matching is performed. Any request that matches a match pattern is then checked with the patterns in except, if any. If any matches are made with the except pattern, the request is rejected and passed along to subsequent handlers. This is a convenient way to have static file resources served properly rather than being confused as CGI applications. The empty_env subdirective is used to pass one or more empty environment variables. Some CGI scripts may expect the server to pass certain empty variables rather than leaving them unset. This subdirective allows you to deal with those situations. The values associated with environment variable keys are all subject to placeholder substitution, just as with the script name and arguments. If your CGI application runs properly at the command line but fails to run from Caddy it is possible that certain environment variables may be missing. For example, the ruby gem loader evidently requires the HOME environment variable to be set; you can do this with the subdirective pass_env HOME. Another class of problematic applications require the COMPUTERNAME variable. The pass_all_env subdirective instructs Caddy to pass each environment variable it knows about to the CGI excutable. This addresses a common frustration that is caused when an executable requires an environment variable and fails without a descriptive error message when the variable cannot be found. These applications often run fine from the command prompt but fail when invoked with CGI. The risk with this subdirective is that a lot of server information is shared with the CGI executable. Use this subdirective only with CGI applications that you trust not to leak this information. If you protect your CGI application with the Caddy JWT middleware, your program will have access to the token’s payload claims by means of environment variables. For example, the following token claims will be available with the following environment variables All values are conveyed as strings, so some conversion may be necessary in your program. No placeholder substitutions are made on these values. If you run into unexpected results with the CGI plugin, you are able to examine the environment in which your CGI application runs. To enter inspection mode, add the subdirective inspect to your CGI configuration block. This is a development option that should not be used in production. When in inspection mode, the plugin will respond to matching requests with a page that displays variables of interest. In particular, it will show the replacement value of {match} and the environment variables to which your CGI application has access. For example, consider this example CGI block: When you request a matching URL, for example, the Caddy server will deliver a text page similar to the following. The CGI application (in this case, wapptclsh) will not be called. This information can be used to diagnose problems with how a CGI application is called. To return to operation mode, remove or comment out the inspect subdirective. In this example, the Caddyfile looks like this: Note that a request for /show gets mapped to a script named /usr/local/cgi-bin/report/gen. There is no need for any element of the script name to match any element of the match pattern. The contents of /usr/local/cgi-bin/report/gen are: The purpose of this script is to show how request information gets communicated to a CGI script. Note that POST data must be read from standard input. In this particular case, posted data gets stored in the variable POST_DATA. Your script may use a different method to read POST content. Secondly, the SCRIPT_EXEC variable is not a CGI standard. It is provided by this middleware and contains the entire command line, including all arguments, with which the CGI script was executed. When a browser requests the response looks like When a client makes a POST request, such as with the following command the response looks the same except for the following lines: The fossil distributed software management tool is a native executable that supports interaction as a CGI application. In this example, /usr/bin/fossil is the executable and /home/quixote/projects.fossil is the fossil repository. To configure Caddy to serve it, use a cgi directive something like this in your Caddyfile: In your /usr/local/cgi-bin directory, make a file named projects with the following single line: The fossil documentation calls this a command file. When fossil is invoked after a request to /projects, it examines the relevant environment variables and responds as a CGI application. If you protect /projects with basic HTTP authentication, you may wish to enable the ALLOW REMOTE_USER AUTHENTICATION option when setting up fossil. This lets fossil dispense with its own authentication, assuming it has an account for the user. The agedu utility can be used to identify unused files that are taking up space on your storage media. Like fossil, it can be used in different modes including CGI. First, use it from the command line to generate an index of a directory, for example In your Caddyfile, include a directive that references the generated index: You will want to protect the /agedu resource with some sort of access control, for example HTTP Basic Authentication. This small example demonstrates how to write a CGI program in Go. The use of a bytes.Buffer makes it easy to report the content length in the CGI header. When this program is compiled and installed as /usr/local/bin/servertime, the following directive in your Caddy file will make it available: The cgit application provides an attractive and useful web interface to git repositories. Here is how to run it with Caddy. After compiling cgit, you can place the executable somewhere out of Caddy’s document root. In this example, it is located in /usr/local/cgi-bin. A sample configuration file is included in the project’s cgitrc.5.txt file. You can use it as a starting point for your configuration. The default location for this file is /etc/cgitrc but in this example the location /home/quixote/caddy/cgitrc. Note that changing the location of this file from its default will necessitate the inclusion of the environment variable CGIT_CONFIG in the Caddyfile cgi directive. When you edit the repository stanzas in this file, be sure each repo.path item refers to the .git directory within a working checkout. Here is an example stanza: Also, you will likely want to change cgit’s cache directory from its default in /var/cache (generally accessible only to root) to a location writeable by Caddy. In this example, cgitrc contains the line You may need to create the cgit subdirectory. There are some static cgit resources (namely, cgit.css, favicon.ico, and cgit.png) that will be accessed from Caddy’s document tree. For this example, these files are placed in a directory named cgit-resource. The following lines are part of the cgitrc file: Additionally, you will likely need to tweak the various file viewer filters such source-filter and about-filter based on your system. The following Caddyfile directive will allow you to access the cgit application at /cgit: Feeling reckless? You can run PHP in CGI mode. In general, FastCGI is the preferred method to run PHP if your application has many pages or a fair amount of database activity. But for small PHP programs that are seldom used, CGI can work fine. You’ll need the php-cgi interpreter for your platform. This may involve downloading the executable or downloading and then compiling the source code. For this example, assume the interpreter is installed as /usr/local/bin/php-cgi. Additionally, because of the way PHP operates in CGI mode, you will need an intermediate script. This one works in Posix environments: This script can be reused for multiple cgi directives. In this example, it is installed as /usr/local/cgi-bin/phpwrap. The argument following -c is your initialization file for PHP. In this example, it is named /home/quixote/.config/php/php-cgi.ini. Two PHP files will be used for this example. The first, /usr/local/cgi-bin/sample/min.php, looks like this: The second, /usr/local/cgi-bin/sample/action.php, follows: The following directive in your Caddyfile will make the application available at sample/min.php: This examples demonstrates printing a CGI rule

Streamserve is a live media streaming server: it reads data from a pipe and distributes it to many HTTP clients. In general: PCM audio: MP3 audio: Show all options: Unlike general-purpose web servers, streamserve it suitable when: 1. The process supplying the data stream is expensive. For example, you want to encode media on the fly and send it to 100 clients, but you can't run 100 encoding processes at once. 2. It's acceptable for a given client (a) to start receiving data mid-stream, and (b) if it has been receiving data too slowly, to skip some data segments to catch up with everyone else. (For this to work, the server must understand enough about the data stream format to interrupt and resume at appropriate positions.) 3. Clients are expected to read data at the same speed data is received from the source. By default, streamserve listens for connections at port 80 on all network interfaces. Note: Use 'sudo setcap cap_net_bind_service=+ep [...]/bin/streamserve' if you want to listen on port 80 or another privileged port. Don't run streamserve as root. Specify an IP address and port number: Specify a port number, listen on all interfaces: Choose any unused port (can be useful for testing): Default: Input data is split into frames. When a client first connects, it starts receiving data at a frame boundary. When a client is receiving data too slowly and skips some of the stream, the skipped part starts and ends at frame boundaries. The raw filter chunks its input into fixed-size blocks. It pays no attention to the data content. The mp3 filter accepts physical MP3 frames. It skips past input data that doesn't look like MP3 frames. The -frame-bytes argument is a maximum: it must be big enough to hold the biggest frame. Data from the input FIFO is read into a fixed-size ring buffer, with a static number of frames. The size of the buffer determines how far a client can fall behind before it catches up by skipping part of the stream. The buffer size is given in frames. This is a 1048576-byte buffer: Maximum client lag is determined by the source buffer argument and the actual frame sizes: If using -filter=mp3, the above example will skip frames when a client lags behind by 1024 mp3 frames (not 1048576 bytes). You can control streamserve's behaviour when a data source closes, and when it becomes idle (no clients are connected). Keep reading data even when no clients are connected: Close the FIFO when no clients are connected: If the input FIFO reaches EOF or encounters an error, reopen it and keep reading. If the input FIFO reaches EOF or encounters an error, disconnect all clients and exit. Limit CPU usage. The default is to use as many threads as you have CPU cores. Disconnect clients after a specified number of bytes. Limit how fast the input is read. (This is meant for testing. You could also use it to serve static content from a regular file as if it were a live stream, although a regular static file server would probably be a better choice.) Speed is given in bytes per second. The default is to read data as fast as the input FIFO supplies it.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

go-xvid are Go bindings to xvidcore from Xvid 1.3.X (which uses the MPEG-4 Part 2, MPEG-4 Visual, ISO/IEC 14496-2 video codec standard). This library can encode a sequence of images to an encoded Xvid stream, decode images from an encoded Xvid stream, and convert images between different color spaces. go-xvid only handles raw Xvid streams. Nearly all video files commonly found are stored in a media container, that encapsulate, but are not, raw Xvid video streams. go-xvid cannot decode or encode container data, and the raw video streams must be encapsulated or decapsulated. go-xvid tries to not abbreviate names and identifiers so that all the names used can easily be searched on the Internet when they are not known. This means that this documentation will not redefine or explain common codec concepts like macroblocks, quantizers, rate-control, and such. Most of the complex configuration structures can be initialized to sane default values in case the user is not familiar with advanced encoding concepts. Before any other function in the package can be called, Init or InitWithFlags must be called once to initialize all internal Xvid state. There is no Close method corresponding to the Init call. As an exception, GetGlobalInfo, which returns general information about the runtime Xvid build, can be called at any time before and after Init. go-xvid defines a specific error type, Error, which is used to represent internal xvidcore errors. Images in go-xvid is stored in the Image structure, which stores both an image color space and its data as an array of planes, which are themselves arrays of data. Each plane has a specific stride. The classic RGBA color space has only one plane and data array but some color spaces can have up to three. See Image for more information. Images can be converted from one color space to another with the Convert function. go-xvid can decode a sequence of images from a raw encoded Xvid stream. Decoder is the struct used to decode from a stream. A Decoder is created with NewDecoder, which takes a DecoderInit configuration struct to initialize it. Once created, Decoder.Decode can be called in a loop to decode a single frame at a time until the entire stream has been processed. Each decoded frame contains extra statistics returned by Decoder.Decode. When the Decoder is no longer needed, it must be closed with Decoder.Close to free any internal data. go-xvid can encode a sequence of images to a raw encoded Xvid stream. Encoder is the struct used to encode from a stream. An Encoder is created with NewEncoder, which takes an EncoderInit configuration struct to initialize it, which itself should be initialized with NewEncoderInit to sane default values. Once created, Encoder.Encode can be called in a loop to encode a single image at a time until all the images have been processed. Each encoded frame contains extra statistics returned by Encoder.Encode. When the Encoder is no longer needed, it must be closed with Encoder.Close to free any internal data. Plugins are used to read and write internal frame data when encoding. Some standard plugins are defined in the library but custom ones can be created by implementing the Plugin interface. In Xvid, rate-control is achieved by using plugins (for both 1-pass rate-control and 2-pass rate-control). You will probably need to use one of these rate-control plugins when encoding (otherwise the smallest quantizer is always used).

secfileshare is an experimental tool to share files securely. What is it? The internet is rife with commercial and other insecure file sharing solutions. secfileshare is trying to address this by implementing client side crypto while remaining easy to use. It consists of two programs. A client side program that encrypts and uploads the blob and a server that stores said blob. Once a client finishes uploading the blob it'll receive a link from the server that can be used to retrieve the blob at a later time. Note: The server side currently does not do any form of authentication and downloads blobs fully into memory first. This obviously can be abused and crashed etc. Do realize that at this time this software is considered experimental. Why do we need this? Hackers, social media, governments etc are increasingly infringing on our privacy and being able to safely and securely share information with others is vital in this day and age. Closed source crypto solutions are bad since they can not be audited and verified by 3rd parties. Commercial entities have proven to be unscrupulous and can not be trusted. How does it work? The tool leverages the ideas of the outstanding NaCl (http://nacl.cr.yp.to/) crypto suite. It uses only free, not patented algorithms that were developed in academia that are seemingly untainted by government influence. All messages are encrypted using NaCl boxes that use Curve25519, XSalsa20 and Poly1305 for encryption and public-key authenticators. Additionally the mcrypt package (https://github.com/marcopeereboom/mcrypt) adds public-key signatures. secfileshare builds on top of this and adds an ephemeral shared secret that encrypts the file that will be shared. The ephemeral shared secret is then encrypted by the originator's private key and N recipients public keys. The resulting blob does not contain any identifiable information regarding the originator or the recipients. How do I start using it? Install Go! See the following link for details http://golang.org/doc/install Install secfileshare Run secfileshare without any parameters. This will create your keys. Note that secfileshare uses HTTPS URLs to distinguish if it is dealing with files or servers. The process of sharing files is simple. Encrypt and upload a file to a single recipient Encrypt and upload a file to a multiple recipients Encrypt and upload a file to a single recipient with an optional description Note that if you want to be able to decrypt the blob yourself that you must provide your public-key as well. Download and decrypt a single file Download and decrypt multiple files Download and decrypt a single file to a specific filename Do read the tool's help as well since it explains the -out parameter that is used to manipulate the out filenames. If you want to share a blob without using a backend server you can use the file to file mode. This code is proof-of-concept and the following list is what needs to happen to mature

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

package media defines common methods and operations for producing Who's On First (WOF) style GeoJSON Feature documents for media records. Common operations include: Gathering files, cloning files, processing files, rotating files and removing files.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.

Package thumbnailer provides a more efficient media thumbnailer than \ available with native Go processing libraries through ffmpeg bindings.