ProxyAll: HTTP, SQL, gRPC... Debugging Tool
ProxyAll is similar to Fiddler and Charles, but in addition to HTTP(S), it also can captures SQL, gRPC, MongoDB, Redis, Memcached, TCP, and log messages.
Features:
- captures HTTP and/or HTTPS messages as either a forward or reverse proxy
- captures SQL, MongoDB, Redis, gRPC, and other protocol messages sent to backend services
- captures log messages from dockers logs
- modify and resend HTTP requests
- search entire request/response message for matching text
- stop/start recording
- take snapshots of captured messages
- export and import captured messages
- supports multiple dashboard browser tabs
Table of Contents
Quick Start
Node Version
The ProxyAll has been tested with node versions v10.22.1, and v12.19.1. Version 10 or higher is recommended. Use nvm to install the appropriate node version.
Install ProxyAll
$ cd ~/git/proxyall
proxyall$ npm install
Build ProxyAll
proxyall$ npm run build
ProxyAll Parameters
Usage: npm start [--listen [host:]port] [--listenHttps [host:]port]
Options:
--listen - listen for incoming http connections. Default is 8888.
--listenHttps - listen for incoming https connections.
Example: npm start -- --listen 8888 --listenHttps 9999
Start the ProxyAll
proxyall$ npm start
> proxyall@1.0.0 start /home/davechri/proxyall-proxy
> if ./scripts/noDir.sh ./build; then npm run build; fi; if ./scripts/noDir.sh ./client/build; then npm run build-client; fi; NODE_ENV=production node ./build/app.js
Listening on http: 8888
Open browser to http://localhost:8888/proxyall
Listening on https: 9999
Open Dashboard in Browser
Enter http://localhost:8888/proxyall in browser.
Configure Browser Proxy
To capture HTTP and HTTPS messages, configure your browser to proxy HTTP/HTTPS messages to the ProxyAll. The default is to proxy HTTP messages to port 8888, and HTTPS messages to port 9999. This is how Firefox can be configured to proxy HTTP and HTTPS messages.
for chrome and chromium you can set the browser proxy using environment variables http_proxy and https_proxy.
$ http_proxy=http://localhost:8888 https_proxy://9999 chromium-browser
Screenshots
Dashboard
Settings
Reachable Hosts
Configuration
This section gives example on how to configure the ProxyAll. Clicking the settings icon in the upper right corner opens the Setting modal.
HTTP/HTTPS Proxy
Both a forward and reverse proxy is supported for HTTP/HTTPS messages. Your browser must be configured to proxy HTTP/HTTPS messages to the forward proxy. See Configure Browser Proxy for more information on configuring your browser.
The reverse proxy can be used to transparently capture HTTP/HTTPS messages sent by backend services. The backend service is configured to send the HTTP/HTTPS messages to the ProxyAll. For example, a -search- microservice could be configured to send Elasticsearch messages to the ProxyAll by setting environment variables.
Example -search- microservice configuration:
ELASTIC_HOST=elasticsearch
ELASTIC_PORT=9200
Modified -search- micorservice configuration:
ELASTIC_HOST=proxyall
ELASTIC_PORT=8888
An HTTP path is added to proxy HTTP requests to the elasticsearch host. All HTTP requests matching path /_search are proxied to the elasticsearch host on port 9200.
MySQL Proxy
The SQL proxy can transparently capture SQL messages sent by backend microservices to a MySQL server.
Example microservice config file:
MYSQL_HOST=mysql
MYSQL_PORT=3306
Modified microservice config file:
MYSQL_HOST=proxyall
MYSQL_PORT=3306
The ProxyAll is configured to proxy MySQL requests to the MySQL server:
gRPC Proxy
The gRPC proxy can transparently capture gRPC HTTP/2 messages sent to backend microservices. Only unsecure connections are supported. Secure TLS support may be added in the future.
Example gRPC microservice config file:
GRPC_HOST=grpchost
GRPC_PORT=12345
Modified gRPC microservice config file:
GRPC_HOST=proxyall
GRPC_PORT=12345
The ProxyAll is configured to proxy gRPC requests to a microservice:
MongoDB Proxy
The MongoDB proxy can transparently capture MongoDB messages sent by backend microservices.
Example MongoDB microservice config file:
MONGO_HOST=mongodb
MONGO_PORT=27017
Modified MongoDB microservice config file:
MONGO_HOST=proxyall
MONGO_PORT=27017
The ProxyAll is configured to proxy MongoDB requests to a microservice:
Redis Proxy
The Redis proxy can transparently capture Redis messages sent by backend microservices.
Example Redis microservice config file:
REDIS_HOST=redis
REDIS_PORT=6379
Modified Redis microservice config file:
REDIS_HOST=proxyall
REDIS_PORT=6379
The ProxyAll is configured to proxy Redis requests to a microservice:
TCP Proxy
The TCP proxy can transparently capture TCP request/response messages sent by backend microservices. For example, the TCP proxy can be used to capture memcached messages.
Example Memcached microservice config file:
MEMCACHED_HOST=memcached
MEMCACHED_PORT=11211
Modified Memcached microservice config file:
MEMCACHED_HOST=proxyall
MEMCACHED_PORT=11211
The ProxyAll is configured to proxy Memcached requests to a microservice:
Dockers Logs
The Docker log proxy can capture log messages.
The ProxyAll is configured to capture Dockers log messages:
Dashboard
The ProxyAll dashboard is stated from the browser with URL http://localhost:8888/proxyall.
Pause Recording
The recording of messages can be temporarily stopped, to allow time to examine the messages without the log wrapping.
Filter Messages
Filtering allows you to find messages matching a search filter, and hide other messages. The entire message is search for a match. The filter may be case insensitive, case sensitive, a logical expression, or a regular expression.
Types of filters:
- case insensitive - If Aa is not selected, a case insensitive search is performed.
- case ensensitive - If Aa is selected, a case sensitive search is performed.
- logical expression - If && is selected, &&, ||, (), and ! operators may be used to build a logical expression.
- regular expression - If .* is selected, regular expression match in performed.
Boolean filters can use &&, ||, !, and parenthesis.
Resend HTTP Requests
To resend an HTTP or HTTPS request, click on the icon next to the request to open a modal. Optionally modify the request body, and then click the send button. If the dashboard is not paused, the resent request should appear at the bottom of the dashboard request log.
Snapshots
Clicking on the camera icon will take a snapshot of the currently captured messages, and create a new snapshot tab. A snapshot tab may be exported to a file, and later imported again.
Multiple Browser Tabs
Multiple Dashboard instances can be opened in separate browser tabs, and all of the open Dashboards will record messages.
Each Dashboard instance keeps its own copy of the messages, so clearing or stopping recording in one Dashboard instance, does not affect another other Dashboard instances.
Certificates
Certificates are managed by the node-http-mitm-proxy package.
Generated certificates are stored in proxyall/.http-mitm-proxy/certs/. The '/proxyall/.http-mitm-proxy/certs/ca.pem' CA certificate can be imported to your browser to trust certificates generated by the ProxyAll.
Implementation:
- HTTP proxy - The http package is used to proxy HTTP traffic as either a forward or reverse proxy.
- HTTPS proxy - The node-http-mitm-proxy package is used to build certificates to capture decrypted HTTPS traffic as either a forward or reverse proxy.
- TCP proxy - The net package is used to listen on a TCP port for non-HTTP messages, and proxy the protocol messages to the target host.
- Socket.IO - The node socket.io package is used to pass messages between the server and browser where they are recorded and displayed in a dashboard.
- stdout/stderr - Spawn a child process to read stdout and stderr from any docker log or log file, and display the log messages in the dashboard.
Limitations
- Only unsecure HTTP/2 is supported. You can only connect to gRPC without TLS.
- Only HTTP/2 reverse proxy is supported. HTTP/2 forward proxy is not supported.
- The TCP proxy does not support TLS. You can only connect to memcached without TLS.
License
This code is licensed under the MIT License.