S3App simplifies the access to a S3Buckets with a provider independent web based frontend which allows the visualizing and the management of the content of S3 buckets with an S3 provider independent web application.
YOTRON is a consultancy company which is focused on DevOps, Cloudmanagement and Data Management. Visit us on www.yotron.de.
S3 (Simple Storage Solution) is a object storage services to provide a web bases storages which works with simple http requests. It is known for its scalability, data availability, security and performance. Introduced by Amazon in AWS, other provider of S3 compatible object storages are available just like possibilities for self-hosted solutions.
S3App simplifies the access to a S3Buckets with a provider independent web based frontend which allows the visualizing and the management of the content of S3 buckets with an S3 provider independent web application, ....
For further information and the manual, please see YOTRON/s3app
The project contains code, build packages, container ... . Below an overview:
| Type | Provider | URL |
|---|---|---|
| S3App Manual | yotron.de | https://www.yotron.de/s3app/ |
| Container | hub.docker.com | https://hub.docker.com/r/yotronpublic/s3app |
| Python Package (PyPi) | pypi.org | https://pypi.org/project/s3app/ |
| HELM package | artifacthub.io / helm.yotron.de | https://artifacthub.io/packages/helm/yotron-helm-charts/s3app |
| Code/Contribution | github.com | https://github.com/yotron/s3app/ |
| Problems/Feedback | github.com | https://github.com/yotron/s3app/issues |
We provide two installation methods. A native installation on any OS which supports Python3 and a installation in Kubernetes via HELM. Per default, S3App runs on a sqlite database. Sqlite is recommended only for testing and on a standalone installation.
In production, you should use a PostgreSQL database for a better data persistency and to allow a HA setup with more S3App nodes.
The S3App has no TLS-termination ("https"). We recommend to use a Reverse Proxy like NGINX or Apache Web Server in front of S3App Web App for TLS termination.
The HELM project of S3App for Kubernetes contains all needed and recommended components. You can optionally install a NGINX Reverse Proxy and a PostgreSQL database with the project, but you can also use separated applications.
S3App supports all current Python3 versions. It is tested with Python 3.9.x and 3.10.x
To check which Python version is running:
vagrant@bullseye:~$ python3 -V
Python 3.9.2
To check which Pip version is running:
vagrant@bullseye:~$ pip3 -V
pip 20.3.4 from /home/vagrant/s3app/lib/python3.9/site-packages/pip (python 3.9)
It is possible, that in your distribution Python must be called with python and pip.
If you need a Python installation from the scratch. Python3, Pip3 and python3-venv are available in the main OS distribution.
For Debian/Ubuntu:
sudo apt-get install python3 python3-pip python3-venv
For RedHat/CentOS:
sudo yum install python3 python3-pip python3-venv
For authentication packages for authentication like OpenLDAP and SSL are needed
For Debian/Ubuntu:
sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-dev
For RedHat/CentOS:
sudo yum install python-devel openldap-devel
There are a lot of variants how to start a Python web project. We use a Python virtual environment for setting up S3App on a virtual environment. Depend on your environment and knowledge in Python the process can be different.
Create a virtual environment of Python
Create a virtual environment for S3App
vagrant@bullseye:~$ python3 -m venv venv
Activate your virtual environment
This steps is to repeat every time you want to restart S3App.
vagrant@bullseye:~$ source venv/bin/activate
(venv) vagrant@bullseye:~$
Install all Python dependencies needed for the project:
(s3app) vagrant@bullseye:~$ pip install s3app --upgrade
Start S3App server
(s3app) vagrant@bullseye:~$ s3app-run --host=0.0.0.0 --port=8080
2022-10-06 15:21:07,382:INFO:flask_appbuilder.base:Registering class S3View on menu
2022-10-06 15:21:07,383:INFO:flask_appbuilder.baseviews:Registering route /s3/<string:prefixUrl> ('GET',)
2022-10-06 15:21:07,383:INFO:flask_appbuilder.baseviews:Registering route /s3/ ('GET',)
...
2022-10-10 11:18:22,384:INFO:flask_appbuilder.base:Registering class S3View on menu
2022-10-10 11:18:22,385:WARNING:flask_appbuilder.base:View already exists S3View ignoring
2022-10-10 11:18:22,385:INFO:flask_appbuilder.base:Registering class S3IndexView on menu
2022-10-10 11:18:22,385:INFO:flask_appbuilder.baseviews:Registering route / ('GET',)
2022-10-10 11:18:22,424:INFO:waitress:Serving on http://0.0.0.0:8080
Open Browser and start S3App with one of the IPs mentioned above:
Your will automatically redirected to http://x.x.x.x:8080/login
Login as a admin user
The default credentials for the admin user are:
Username: admin
Password: admin
Please see the manual to allow access to buckets.
You find the installation instructions for Kubernetes here
The manual for S3App you find here
S3APP_CONF_FILE variableYou can customize S3App with a setting defined in a dotenv file. The dotenv file can be chosen freely and you can put the file in a folder of your choice but the file must be reachable by the S3App App.
Additionally, you need a environment variable S3APP_CONF_FILE with the path to your config file.
For example, for Linux you can set the environment variable with:
export S3APP_CONF_FILE=/etc/s3app/.s3app
with .s3app as the dotenv file name.
The file contains setting to the following topics:
| name | example | description | possible values | default |
|---|---|---|---|---|
| S3APP_APP_NAME | My Fancy S3 App | The name of your S3App. It is display in the head of your application. | ||
| S3APP_APP_ICON | https://myserver.com/my_own_logo.svg | The icon of your S3App as a html link. It is display in the head of your application. | ||
| S3APP_LOG_LEVEL | info | LogLevel for the Logging. | fatal, error, warning, info, debug | info |
| S3APP_LOG_FILE | /var/log/s3app.log | When set logs are written to the mentioned file. If not set it is written to console (stdout). | not set | |
| S3APP_DB_TYPE | postgresql | Type of database to use. Can be a SQLite or a PostgreSQL Database | sqlite, postgres | sqlite |
| S3APP_PG_DB_HOST | k8s-1.mydatabse.com | Only if db-type postgres: Host name of the PostresSQL server. | ||
| S3APP_PG_DB_PORT | 5432 | Only if db-type postgres: Port number of the PostresSQL server. | 5432 | |
| S3APP_PG_DB_NAME | s3app | Only if db-type postgres: Name of the PostgreSQL Database. | ||
| S3APP_PG_DB_USER_NAME | s3app | Only if db-type postgres: Username to authenticate against the PostgreSQL Database | ||
| S3APP_PG_DB_USER_PW | s3app | Only if db-type postgres: Password to authenticate against the PostgreSQL Database | ||
| S3APP_SECRET_KEY | thisIsMyHiddenSecretKey | A key which used to sign session cookies for protection against cookie data tampering. In production please change it. |
| name | example | description | possible values | default |
|---|---|---|---|---|
| S3APP_AUTH_TYPE | database | Parameter to define the authentication method. It can be an authentication via LDAP or via the default database with name and password. | ldap, database | database |
| AUTH_USER_REGISTRATION | Parameter to define if a user can self registrate to S3App. With LDAP it must be set to True. | True, False | False | |
| AUTH_USER_REGISTRATION_ROLE | Default role a user, when registered or authenticated via LDAP firstly. | S3User, Admin | S3User | |
| AUTH_ROLES_MAPPING | Mapping a LDAP group onto a S3App role S3Manager, S3User or Admin. | |||
| S3APP_SESSION_LIFETIME | 1800 | Seconds of inactivity after which a user must re-login. | 600 |
Example of a role mapping
AUTH_ROLES_MAPPING = {
"cn=s3user,ou=groups,dc=example,dc=com": ["S3Users"],
"cn=s3appadmins,ou=groups,dc=example,dc=com": ["Admin"],
}
To allow authentication with LDAP, a LDAP group must be mapped onto one the S3App Role S3Manager, S3User or Admin . After the first authentication a new user, the user must be added to the S3Access
or S3Group. Please see the manual of S3App.
| name | example | description | possible values | default |
|---|---|---|---|---|
| AUTH_LDAP_SERVER | ldap://ldap.example.com | The URL of the LDAP server. | ||
| AUTH_LDAP_USE_TLS | False | If the LDAP server allows TLS secured communication set to True. | True, False | |
| AUTH_LDAP_FIRSTNAME_FIELD | givenName | Name of the field of the person LDAP entity with the given name. | ||
| AUTH_LDAP_LASTNAME_FIELD | sn | Name of the field of the person LDAP entity with the last name. | ||
| AUTH_LDAP_EMAIL_FIELD | Name of the field of the person LDAP entity with the email address. | |||
| AUTH_LDAP_USERNAME_FORMAT | uid=%s,ou=users,dc=example,dc=com | Distinguished name of the user to authenticate. %s will be replaced by the username of the S3App login. | ||
| AUTH_LDAP_APPEND_DOMAIN | example.com | When a username always has a domain appendix. | ||
| AUTH_LDAP_SEARCH | ou=users,dc=example,dc=com | LDAP search string if a user is part of an organizational unit. | ||
| AUTH_LDAP_UID_FIELD | uid | When using a LDAP search the filed name with the username of the organizational unit. | ||
| AUTH_LDAP_BIND_USER | ldapadmin | The bind user used for authentication against LDAP. | ||
| AUTH_LDAP_BIND_PASSWORD | myHiddenPassword | The password of the bin user sed for authentication against LDAP. | ||
| AUTH_LDAP_SEARCH_FILTER | (memberOf=cn=myTeam,ou=teams,dc=example,dc=com) | Filter the user which are allowed to access S3App generally. | ||
| AUTH_LDAP_GROUP_FIELD | memberOf | When using AUTH_ROLES_MAPPING the name of the field with the role DN. |
You find further information about how to configure LDAP against Microsoft AD or OpenLDAP here.
To start of the S3App server simply call s3app-run as shown above. You have has the following parameter:
| name | short | example | description | default |
|---|---|---|---|---|
| HOST | -s, --host | 127.0.0.1 | Listener Host IP. 0.0.0.0 for all IPs. | 0.0.0.0 |
| PORT | -P, --port | 8090 | Listener Host Port | 8080 |
| THREADS | -t --threads | 1 | Threads for parallelization | 4 |
With s3app-run -h you get the possible configuration:
(s3app) vagrant@bullseye:~$ s3app-run -h
...
usage: s3app-run [-h] [-s HOST] [-p PORT] [-t THREADS]
Start parameter for the S3App web server.
optional arguments:
-h, --help show this help message and exit
-s HOST, --host HOST Listener Host IP. Default: 0.0.0.0
-p PORT, --port PORT Listener Host Port. Default: 8080.
-t THREADS, --threads THREADS Threads for parallelization. Default: 4.
FAQs
S3App simplifies the access to a S3Buckets with a provider independent web based frontend which allows the visualizing and the management of the content of S3 buckets with an S3 provider independent web application.
We found that s3app demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.