0xacab.org/leap/menshen

Menshen

Menshen is a gateway & bridge distribution service for LEAP VPN.

In chinese folk religions, menshen are guardians of doors and gates.

Overview

menshen is a resource distributor for LEAP VPN. Here, resource means information for accessing either gateways, bridges, or a combination of the two. This information includes not only endpoint discovery but also crendentials (although the credential distribution is not fully integrated at this point).

After adopting 002-vpnweb-deprecation, menshen is responsible for centrally handling the resource inventory, and assigning each item on the inventory to the users of the system. How this is assigned can change from deployment to deployment, but the general idea is that menshen takes several parameters into account, including current node capacity, geolocation, user preferences (i.e., opting-in to resource marked as experimental) and possible ACLs in place.

Architecture

menshen is a single service that can bootstrapped with or without the legacy service inventory information coming from vpnweb (in the form of eip-service.json, remotely or from an in-disk file). As specified in 002-vpnweb-deprecation, this is intended as a compatibility measure that will have to be phased out at some point.

menshen also listens on an internal port where it receives information from menshen_agent running on each gateway via gRPC calls (receiving information from bridges is TBD).

The auto-generated clients from the menshen spec are feeded to bitmask-core to get clients compatible with the latest version of the spec.

graph
  eip-service.json --> menshen
  inventory -.-> menshen
  menshen_agent_gw --lb--> menshen
  menshen_agent_br -.-lb-.-> menshen
  control_br -.-> menshen
  control_gw -.-> menshen

  subgraph "gateway"
    menshen_agent_gw
    control_gw 
    openvpn
  end

  subgraph "bridge"
    menshen_agent_br
    control_br 
    obfsvpn
  end

menshen --> bitmask-core

In the diagram above, the dashed line from "inventory" marks a yet unresolved way of specifying resources in the inventory (see #26).

OpenAPI spec generation

We generate an OpenAPI spec from comments in the source code. Every commit that changes the API must update the relevant documentation (more information how to change the API).

To update the OpenAPI spec, you need to install swag:

go install github.com/swaggo/swag/cmd/swag@latest

To generate clients from the spec, you need to install swagger in your system.

To do both steps at once:

make swag

:warning: - we still miss a good way to move/copy over the generated client to the bitmask-core codebase. This should probably be automated somehow, with manual overview.

Build

go build ./cmd/menshen

Run

The parameter --from-provider-json is a file path to a provider.json. It is always required to start menshen. Use --verbose to get debug output.

by pointing to another upstream provider

a) with self-signed provider API endpoints

Further required parameters are --ca-file, --client-cert-url and an eip source (--from-eip-file or --from-eip-url ). You can start menshen from a v3 eip-service file:

go run ./cmd/menshen/main.go --verbose \
--from-provider-json-file test/data/provider.json \
--from-eip-file test/data/eip-service.json \
--ca-file test/data/demo.crt \
--client-cert-url https://api.demo.bitmask.net:4430/3/cert

Alternatively you can start menshen with a eip-service.json from an URL:

go run ./cmd/menshen/main.go --verbose \
--from-provider-json-file test/data/provider.json \
--from-eip-url https://api.demo.bitmask.net:4430/3/config/eip-service.json \
--ca-file test/demo.crt \
--client-cert-url https://api.demo.bitmask.net:4430/3/cert

b) with ca-signed provider endpoints

Additional required parameters are --client-cert-url and an eip source (--from-eip-file or --from-eip-url ). You can start menshen from a v3 eip-service file:

go run ./cmd/menshen/main.go --verbose \
--from-provider-json-file test/data/provider.json \
--from-eip-url https://api.demo.bitmask.net/3/config/eip-service.json \
--client-cert-url https://api.demo.bitmask.net/3/cert

Alternatively you can start menshen with a eip-service.json from an URL:

go run ./cmd/menshen/main.go --verbose \
--from-provider-json-file test/data/provider.json \
--from-eip-file test/data/eip-service.json \
--client-cert-url https://api.demo.bitmask.net/3/cert

Note: Replace test/data/eip-service.json and test/data/provider.json with the relevant configuration files, as they are just samples.

by configuring for local cert generation

Additional required parameters are --ovpn-ca-crt, --ovpn-ca-key and an eip source (--from-eip-file or --from-eip-url ). You can start menshen from a v3 eip-service file:

go run ./cmd/menshen/main.go --verbose \
--from-provider-json-file test/data/provider.json \
--from-eip-file test/data/eip-service.json \
--ovpn-ca-crt test/data/ovpn_client_ca.crt \
--ovpn-ca-key test/data/ovpn_client_ca.key

To visit the public url, you can point your browser to:

http://localhost:8443/api/swagger/index.html

If you want to make the general gateway and bridges endpoints public enable it by using flags --allow-gateway-list --allow-bridge-list

go run ./cmd/menshen/main.go --verbose \
--from-provider-json-file test/data/provider.json \
--from-eip-url https://api.demo.bitmask.net:4430/3/config/eip-service.json \
--ca-file test/demo.crt \
--client-cert-url https://api.demo.bitmask.net:4430/3/cert \
--allow-gateway-list --allow-bridge-list

If you're not running menshen as part of an orchestration platform that can set up TLS certificates for you, you can leverate the auto-tls functionality (for instance, while running standalone nodes for development or quick testing):

./menshen --auto-tls --server-name example.com

For help on the optional flags, run with -h:

❯ ./menshen -h
Usage of ./menshen:
      --algo string                      Select the preferred algorithm for certificate generation currently supported ecdsa, ed25519 (default "ed25519")
      --allow-bridge-list                allow public bridge listing
      --allow-gateway-list               allow public gateway listing
      --auto-tls                         configure auto TLS using Lets Encrypt
      --ca-file string                   filename with CA certificate used for validating certificates
      --client-cert-url string           url that returns a valid OpenVPN certificate and private key in plain text
      --db-file string                   location of sqlite database file (default "./sqlite.db")
      --enable-cert-v3                   enable /3/cert endpoint for rsa cert generation
      --from-eip-file string             start from eip-service file (legacy)
      --from-eip-url string              start from eip-service url (legacy)
      --from-provider-json-file string   file path to provider.json
      --lb-addr string                   Address for load balancer to listen on (default ":9003")
      --localbridges string              comma-separated list of addresses for the control port of bridges
      --metrics-port int                 port for /metrics (prometheus) to listen on (default 9002)
      --ovpn-ca-crt string               openvpn ca crt for client cert generation
      --ovpn-ca-key string               openvpn ca key for signing client cert
      --ovpn-client-crt-expiry int       openvpn client cert expiry (default 23)
      --port int                         port to listen on (default 8443)
      --server-name string               server name (for LetsEncrypt AutoTLS)
      --verbose                          set log verbosity to DEBUG

Config

You can either pass flags, as in the example above, or a yaml file, or environment variables.

The environment variables follow the uppercase-underscore convention and are prefixed with MENSHEN_. So, to pass the equivalent of --port, you'd pass MENSHEN_PORT.

Other utilities

For convenience, there's a binary that can be used to get coordinates for the locations of the remotes in your inventory with the same geolocation library used by the API:

❯ go build ./cmd/locations
❯ ./locations /tmp/eip.json get-location
New York        40.71   -74.01
Paris           48.85   2.35
Seattle         47.61   -122.33
Amsterdam       52.37   4.90
Miami           25.77   -80.19
Montreal        45.52   -73.65

Bucket token auth

Resources (bridges and gateways) can be given a "bucket" property to denote that they should be restricted/not part of publicly available assets.

Admins can then generate auth tokens which would give end users access to those resources.

You can use the accompanying CLI to generate said tokens:

❯ go run ./cmd/tokens --buckets "bucket1,bucket2" --number-tokens 10
Tokens:
solitech_z6tV8f+1jpYMBgmE0J6jZQ==
solitech_bYnFp+YzPTp/71/edSDEiw==
solitech_pusRycrUBH9lxvkVcbIyXQ==
solitech_e9JckBd9QzYmSE4wo2jQBQ==
solitech_564x5U+rANCS1jbpvIeQvA==
solitech_zNcbJeFyMANj0kIxGruIpw==
solitech_t3/B7mdKB2tPbcnNdIejpw==
solitech_tLdCZT0QVPyPuNJFdlpvvQ==
solitech_G5ACBRpwqcivbMEkC708OA==
solitech_u6Zoe3ftUcVmLb3lm/KsSw==

These tokens can then be passed in as a x-menshen-auth-token header when making requests to menshen which will allow that request access to private resources in "bucket1" and "bucket2".