AllProxy: HTTP, SQL, gRPC Debugging Tool
AllProxy is similar to Fiddler and Charles, but in addition to HTTP(S), it also can captures SQL, gRPC, MongoDB, Redis, Memcached, TCP, and log messages.
Features:
- captures HTTP and/or HTTPS messages as either a forward or reverse proxy
- captures SQL, MongoDB, Redis, gRPC, and other protocol messages sent to backend services
- captures log messages from dockers logs
- modify and resend HTTP requests
- search entire request/response message for matching text
- stop/start recording
- take snapshots of captured messages
- export and import captured messages
- supports multiple dashboard browser tabs
- HTTP/2 support
Table of Contents
Quick Start
Node Version
Version 10 or higher is required. Use nvm to install the appropriate node version.
Install AllProxy
The AllProxy may be install from either NPM or GitHub.
Install From NPM
$ npm install -g allproxy
Install From GitHub Project
$ cd ~/git/allproxy
allproxy$ npm install
Start the AllProxy Server
The AllProxy server is started using either the NPM installed allproxy script or using npm start, depending on how it was installed.
Start NPM Package
$ allproxy
Listening on http: 8888
Open browser to http://localhost:8888/allproxy
Listening on https: 9999
Start GitHub Project
allproxy$ npm start
Listening on http: 8888
Open browser to http://localhost:8888/allproxy
Listening on https: 9999
Install CA Certificate
The allproxy-ca can be used to install the CA certificate so it is trusted by your browser.
The ca.pem location:
- GitHub Project: ~/git/allproxy/ca.pem # assuming the allproxy project was cloned to ~/git/
- NPM Package: ~/allproxy/ca.pem
If you need to manually install the AllProxy ca.pem. These online tutorials many help:
NPM Package
$ allproxy-ca
GitHub Project
$ cd ~/git/allproxy/bin
bin$ ./allproxy-ca
Open Dashboard in Browser
Enter http://localhost:8888/allproxy in browser.
Configure Browser Proxy
Chrome will also honor the system proxy configuration for MacOS. The allproxy-system-proxy command can be used to configure the system proxy.
NPM Package
$ cd ~/git/allproxy
allproxy$ allproxy-system-proxy enable
GitHub Project
$ allproxy-system-proxy enable
Firefox Proxy Configuration
To capture HTTP and HTTPS messages, configure your browser to proxy HTTP/HTTPS messages to the AllProxy. The default is to proxy HTTP messages to port 8888, and HTTPS messages to port 9999. This is how Firefox can be configured to proxy HTTP and HTTPS messages.
Linux Proxy Configuration
For chrome and chromium you can set the browser proxy using environment variables http_proxy and https_proxy.
$ http_proxy=http://localhost:8888 https_proxy://9999 chromium-browser
Screenshots
Dashboard
Settings
Reachable Hosts
Configuration
This section gives example on how to configure the AllProxy. Clicking the settings icon in the upper right corner opens the Setting modal.
HTTP/HTTPS Proxy
Both a forward and reverse proxy is supported for HTTP/HTTPS messages. Your browser must be configured to proxy HTTP/HTTPS messages to the forward proxy. See Configure Browser Proxy for more information on configuring your browser.
The reverse proxy can be used to transparently capture HTTP/HTTPS messages sent by backend services. The backend service is configured to send the HTTP/HTTPS messages to the AllProxy. For example, a -search- microservice could be configured to send Elasticsearch messages to the AllProxy by setting environment variables.
Example -search- microservice configuration:
ELASTIC_HOST=elasticsearch
ELASTIC_PORT=9200
Modified -search- micorservice configuration:
ELASTIC_HOST=allproxy
ELASTIC_PORT=8888
An HTTP path is added to proxy HTTP requests to the elasticsearch host. All HTTP requests matching path /_search are proxied to the elasticsearch host on port 9200.
HTTP/2 Support
You can use HTTP/2 to connect to HTTP/2 enabled servers (e.g., duckduckgo.com). To enable HTTP/2:
```sh
$ allproxy --http2
```
MySQL Proxy
The SQL proxy can transparently capture SQL messages sent by backend microservices to a MySQL server.
Example microservice config file:
MYSQL_HOST=mysql
MYSQL_PORT=3306
Modified microservice config file:
MYSQL_HOST=allproxy
MYSQL_PORT=3306
The AllProxy is configured to proxy MySQL requests to the MySQL server:
gRPC Proxy
The gRPC proxy can transparently capture gRPC HTTP/2 messages sent to backend microservices. Only unsecure connections are supported. Secure TLS support may be added in the future.
Example gRPC microservice config file:
GRPC_HOST=grpchost
GRPC_PORT=12345
Modified gRPC microservice config file:
GRPC_HOST=allproxy
GRPC_PORT=12345
The AllProxy is configured to proxy gRPC requests to a microservice:
MongoDB Proxy
The MongoDB proxy can transparently capture MongoDB messages sent by backend microservices.
Example MongoDB microservice config file:
MONGO_HOST=mongodb
MONGO_PORT=27017
Modified MongoDB microservice config file:
MONGO_HOST=allproxy
MONGO_PORT=27017
The AllProxy is configured to proxy MongoDB requests to a microservice:
Redis Proxy
The Redis proxy can transparently capture Redis messages sent by backend microservices.
Example Redis microservice config file:
REDIS_HOST=redis
REDIS_PORT=6379
Modified Redis microservice config file:
REDIS_HOST=allproxy
REDIS_PORT=6379
The AllProxy is configured to proxy Redis requests to a microservice:
TCP Proxy
The TCP proxy can transparently capture TCP request/response messages sent by backend microservices. For example, the TCP proxy can be used to capture memcached messages.
Example Memcached microservice config file:
MEMCACHED_HOST=memcached
MEMCACHED_PORT=11211
Modified Memcached microservice config file:
MEMCACHED_HOST=allproxy
MEMCACHED_PORT=11211
The AllProxy is configured to proxy Memcached requests to a microservice:
Dockers Logs
The Docker log proxy can capture log messages.
The AllProxy is configured to capture Dockers log messages:
Dashboard
The AllProxy dashboard is stated from the browser with URL http://localhost:8888/allproxy.
Pause Recording
The recording of messages can be temporarily stopped, to allow time to examine the messages without the log wrapping.
Filter Messages
Filtering allows you to find messages matching a search filter, and hide other messages. The entire message is search for a match. The filter may be case insensitive, case sensitive, a logical expression, or a regular expression.
Types of filters:
- case insensitive - If Aa is not selected, a case insensitive search is performed.
- case ensensitive - If Aa is selected, a case sensitive search is performed.
- logical expression - If && is selected, &&, ||, (), and ! operators may be used to build a logical expression.
- regular expression - If .* is selected, regular expression match in performed.
Boolean filters can use &&, ||, !, and parenthesis.
Resend HTTP Requests
To resend an HTTP or HTTPS request, click on the icon next to the request to open a modal. Optionally modify the request body, and then click the send button. If the dashboard is not paused, the resent request should appear at the bottom of the dashboard request log.
Modify HTTPS Responses
An HTTPS response can be replaced with your own hard coded response. Simply create a file in the replace-responses directory matching the request URL path.
For example:
Create test.js file in replace-responses directory. It's path must match the URL: /lib/js/test.js.
$ cd allproxy/replace-responses
replace-responses$ mkdir -p lib/js
replace-responses$ echo 'My test response' > test.js
WHenever AllProxy receives URL /lib/js/test.js, it will replace the response with the content from file replace-responses/lib/js/test.js.
Note, the AllProxy must be restarted after adding new files to the replace-responses directory.
Snapshots
Clicking on the camera icon will take a snapshot of the currently captured messages, and create a new snapshot tab. A snapshot tab may be exported to a file, and later imported again.
Multiple Browser Tabs
Multiple Dashboard instances can be opened in separate browser tabs, and all of the open Dashboards will record messages.
Each Dashboard instance keeps its own copy of the messages, so clearing or stopping recording in one Dashboard instance, does not affect another other Dashboard instances.
Certificates
Certificates are managed by the node-http-mitm-proxy package.
Generated certificates are stored in allproxy/.http-mitm-proxy/certs/. Import allproxy/ca.pem to your browser to trust all AllProxy generated certificates.
The allproxy-ca script can be used to install the CA certificate on MacOS and Linux.
For Windows, execute the allproxy-ca script to get the path of the CA certificate, and manually import it to your browser.
Implementation:
- HTTP proxy - The http package is used to proxy HTTP traffic as either a forward or reverse proxy.
- HTTPS proxy - The node-http-mitm-proxy package is used to build certificates to capture decrypted HTTPS traffic as either a forward or reverse proxy.
- TCP proxy - The net package is used to listen on a TCP port for non-HTTP messages, and proxy the protocol messages to the target host.
- Socket.IO - The node socket.io package is used to pass messages between the server and browser where they are recorded and displayed in a dashboard.
- stdout/stderr - Spawn a child process to read stdout and stderr from any docker log or log file, and display the log messages in the dashboard.
Configuration File
- When running from a GitHub package, config.json file is stored in the root directory of your GitHub project.
- When running from an NPM package (allproxy script), the config.json file is stored your home directory at $HOME/allconfig/config.json.
Command Line Parameters
Usage: allproxy [--listen [host:]port] [--listenHttps [host:]port]
Options:
--listen - listen for incoming http connections. Default is 8888.
--listenHttps - listen for incoming https connections.
Example: allproxy --listen 8888 --listenHttps 9999
Limitations
- Only HTTP/2 reverse proxy is supported. HTTP/2 forward proxy is not supported.
License
This code is licensed under the MIT License.