Automate with computer-use agent (ComputerUse / CUA) for macOS, Windows, Linux, Browser, iOS & Android. It sees the screen, clicks, types, browses and writes the code too. One agent, every computer.
Filebin2 is a web application that facilitates convenient file sharing over the web. It is the software that powers https://filebin.net/. It is still in development status and will see breaking changes.
A couple of (in hindsight) bad architectural decisions in the previous version of filebin paved the road for filebin2. Filebin2 is using a PostgreSQL database to handle meta data and S3 to store files. I decided to move to a new repository because of breaking changes from the previous verson of filebin.
Getting started
Filebin2 requires read write access to an S3 bucket for file storage and a PostgreSQL database that it will use for meta data. The Filebin application itself is written in Go and builds to a single binary that is configured using command line arguments and/or environment variables.
Using Docker Compose
The quickest way to get a fully working setup is to use the included docker-compose.yml, which builds Filebin from source and starts it together with PostgreSQL and Stupid Simple S3):
Review and adjust the environment variables in docker-compose.yml to suit your needs. See the Configuration section for all available options.
Using the container image
Multi-arch container images (linux/amd64 and linux/arm64) are published to GitHub Container Registry for each release. The container images are distroless and contain only the filebin binary:
ghcr.io/espebra/filebin2:latest
Run the container with the required environment variables pointing to your PostgreSQL database and S3 bucket:
Filebin can be configured using command line arguments or environment variables. Environment variables use the FILEBIN_ prefix with uppercase letters and underscores instead of hyphens. Command line flags take precedence over environment variables.
General
Base URL
Environment Variable: FILEBIN_BASEURL
Command Line Argument: --baseurl
Default: https://filebin.net
The base URL to use, which impacts URLs that are presented to the user for files and bins. It needs to point to the hostname of the filebin instance.
Contact
Environment Variable: FILEBIN_CONTACT
Command Line Argument: --contact
Default: (required)
The e-mail address to show on the website on the contact page.
Expiration
Environment Variable: FILEBIN_EXPIRATION
Command Line Argument: --expiration
Default: 604800
Bin expiration time in seconds since the last bin update. Bins will be inaccessible after this time, and files will be removed by the lurker.
Temporary Directory
Environment Variable: FILEBIN_TMPDIR
Command Line Argument: --tmpdir
Default: /tmp
Directory for temporary files for upload and download.
Command Line Argument: --tmpdir-capacity-threshold
Default: 4.0
Workspace capacity threshold multiplier. A workspace must have at least this multiplier times the file size available to be selected (e.g., 4.0 requires 4x the file size available).
Manual Approval
Environment Variable: FILEBIN_MANUAL_APPROVAL
Command Line Argument: --manual-approval
Default: false
If enabled, the administrator needs to manually approve new bins before files and archives can be downloaded. Bin and file operations except downloading are accepted while a bin is pending approval. This is a mechanism added to limit abuse. The API request used to approve a bin is an authenticated PUT /admin/approve/{bin} to the admin listener.
Command Line Argument: --require-verification-cookie
Default: false
If enabled, a warning page will be shown before a user can download files. If the user accepts to continue past the warning page, a cookie will be set to identify that the user understands the risk of downloading files.
Which cookie value to expect to avoid showing a warning message. See --require-verification-cookie.
GeoIP City Database
Environment Variable: FILEBIN_MMDB_CITY
Command Line Argument: --mmdb-city
Default: (not set)
The path to an mmdb formatted geoip database like GeoLite2-City.mmdb. This is optional.
GeoIP ASN Database
Environment Variable: FILEBIN_MMDB_ASN
Command Line Argument: --mmdb-asn
Default: (not set)
The path to an mmdb formatted geoip database like GeoLite2-ASN.mmdb. This is optional.
Allow Robots
Environment Variable: FILEBIN_ALLOW_ROBOTS
Command Line Argument: --allow-robots
Default: false
If enabled, the X-Robots-Tag response header will allow search engines to index and show Filebin in search results. Otherwise, robots will be instructed to not show files and bins in search results.
Pre-Upload Hook
Environment Variable: FILEBIN_PRE_UPLOAD_HOOK
Command Line Argument: --pre-upload-hook
Default: (not set)
Command to execute before every file upload is accepted. The command runs after the file has been received from the client and its content type and checksums have been calculated, but before it is stored in S3 or its metadata persisted to the database. It is intended for validation, for example checking the checksums against a blocklist or a malware scanning service. It is invoked with the following named arguments: --request-id, --bin, --filename, --content-type, --size, --md5, --sha1, and --sha256. The checksums are hex encoded.
The exit code determines the outcome:
1 — Reject the upload with HTTP 403 (Forbidden). The last line of stdout from the command is returned to the client as the response message.
0 — Accept the upload.
Any other exit code — Accept the upload. The exit code and output are logged.
Only exit code 1 rejects an upload. A command that cannot be started, or that does not finish within the timeout, is logged and the upload is accepted, so that a broken or hung hook cannot block uploads. An example hook script is provided in misc/pre-upload-hook-example.
Timeout for the pre-upload hook command execution. If the command does not complete within this duration, it is killed and the timeout is logged, but the upload is accepted so that a slow or hung hook does not block uploads. The value is specified using Go duration format, examples: 5s, 30s, 1m.
Post-Upload Hook
Environment Variable: FILEBIN_POST_UPLOAD_HOOK
Command Line Argument: --post-upload-hook
Default: (not set)
Command to execute after every successful file upload. The command runs after the file has been stored in S3 and its metadata persisted to the database, and is intended for notifications and post-processing (for example, triggering an indexing job or webhook). It is invoked with the following named arguments: --request-id, --bin, --filename, --content-type, --size, --md5, --sha1, and --sha256. The checksums are hex encoded.
The hook does not affect the response to the client: any non-zero exit code or stdout/stderr is logged but the upload is always reported as successful. An example hook script is provided in misc/post-upload-hook-example.
Timeout for the post-upload hook command execution. If the command does not complete within this duration, it is killed and the failure is logged, but the upload is still reported as successful to the client. The value is specified using Go duration format, examples: 5s, 30s, 1m.
Limit the number of downloads per file. 0 means no limit. If the value is 100, then each file can be downloaded 100 times before further downloads are rejected.
Limit Storage
Environment Variable: FILEBIN_LIMIT_STORAGE
Command Line Argument: --limit-storage
Default: 0
Limit the storage capacity that filebin will use. 0 means no limit. If set to "200GB", filebin will allow file uploads until the total storage surpasses 200 GB. New file uploads will be rejected until storage consumption is below 200 GB again.
A whitespace separated list of file extensions that will be rejected. Example: "exe bat dll".
HTTP Server
Filebin serves HTTP on two listeners:
The public listener (--listen-host and --listen-port) serves the web interface, the API, uploads and downloads. None of its endpoints require authentication, so it is safe to expose to the internet.
The admin listener (--admin-listen-host and --admin-listen-port) serves the admin interface (/admin), the Prometheus metrics (/metrics) and the Go profiling endpoints (/debug/pprof/). These endpoints are not available on the public listener. The admin listener is not meant to be exposed to the internet, but to be reached from the host itself, a private network or through a tunnel such as ssh -L 8081:127.0.0.1:8081 or kubectl port-forward. The admin interface and the profiling endpoints still require the admin credentials. Links from the admin interface to bins and files point to the public listener using the base URL, so make sure it is set correctly.
Both listeners serve /filebin-status, /storage-status and the static files, so that health checks can be pointed at either of them.
Listen Host
Environment Variable: FILEBIN_LISTEN_HOST
Command Line Argument: --listen-host
Default: 127.0.0.1
Which IP address Filebin will bind to. The default value of 127.0.0.1 is a safe default that does not expose Filebin outside of the host.
Listen Port
Environment Variable: FILEBIN_LISTEN_PORT
Command Line Argument: --listen-port
Default: 8080
Which port Filebin will listen to. The default of 8080 does not require privileged access.
Admin Listen Host
Environment Variable: FILEBIN_ADMIN_LISTEN_HOST
Command Line Argument: --admin-listen-host
Default: 127.0.0.1
Which IP address the admin listener will bind to. The default value of 127.0.0.1 only allows access from the host itself.
Admin Listen Port
Environment Variable: FILEBIN_ADMIN_LISTEN_PORT
Command Line Argument: --admin-listen-port
Default: 8081
Which port the admin listener will listen to. It must be different from --listen-port. Set to 0 to disable the admin listener.
Access Log
Environment Variable: FILEBIN_ACCESS_LOG
Command Line Argument: --access-log
Default: not set
Path to a filename for the access log output, in the combined log format. Requests to both the public and the admin listener are logged. The access log is disabled if this is not set. A typical value is /var/log/filebin/access.log.
Audit Log Directory
Environment Variable: FILEBIN_AUDIT_LOG_DIR
Command Line Argument: --audit-log-dir
Default: not set
Directory for the audit log, which records uploads, downloads, archive downloads, deletions, admin actions and bin expiry with timestamps, request ids, client IP addresses, network and location details, and the checksums of the files transferred. It is written as one JSON lines file per day (UTC), named audit-YYYY-MM-DD.jsonl. The file of the current day is plain text, and the files of previous days are gzip compressed. The audit log is written outside of the database, and is intended to be kept long after the files themselves have been deleted. The audit log is disabled if this is not set. A typical value is /var/log/filebin/audit.
Audit Log Retention
Environment Variable: FILEBIN_AUDIT_RETENTION
Command Line Argument: --audit-retention
Default: 14
The number of days to keep audit log files. The lurker removes the files of days that are older than this, regardless of whether the bins and files they refer to still exist. Set to 0 to keep the files forever.
Proxy Headers
Environment Variable: FILEBIN_PROXY_HEADERS
Command Line Argument: --proxy-headers
Default: false
If enabled, the client IP will be read from the proxy headers provided in the incoming HTTP requests on the public listener. This should only be enabled if there is an HTTP proxy running in front of Filebin that uses proxy headers to tell Filebin the original client IP address. This setting does not affect the admin listener, see --admin-proxy-headers.
Admin Proxy Headers
Environment Variable: FILEBIN_ADMIN_PROXY_HEADERS
Command Line Argument: --admin-proxy-headers
Default: false
If enabled, the client IP will be read from the proxy headers provided in the incoming HTTP requests on the admin listener. This should only be enabled if the admin listener is reachable exclusively through an HTTP proxy that uses proxy headers to tell Filebin the original client IP address. Otherwise, clients can spoof their IP address in the admin login tracking and the logs.
Timeouts
Read Timeout
Environment Variable: FILEBIN_READ_TIMEOUT
Command Line Argument: --read-timeout
Default: 1h
Read timeout for the HTTP server. File uploads need to complete within this timeout before they are terminated.
Read Header Timeout
Environment Variable: FILEBIN_READ_HEADER_TIMEOUT
Command Line Argument: --read-header-timeout
Default: 2s
Read header timeout for the HTTP server.
Write Timeout
Environment Variable: FILEBIN_WRITE_TIMEOUT
Command Line Argument: --write-timeout
Default: 1h
Write timeout for the HTTP server.
Idle Timeout
Environment Variable: FILEBIN_IDLE_TIMEOUT
Command Line Argument: --idle-timeout
Default: 30s
Idle timeout for the HTTP server.
Database
Database Host
Environment Variable: FILEBIN_DATABASE_HOST
Command Line Argument: --db-host
Default: (required)
Which PostgreSQL host to connect to. This can be an IP address or a hostname.
Database Port
Environment Variable: FILEBIN_DATABASE_PORT
Command Line Argument: --db-port
Default: 5432
The port to use when connecting to the PostgreSQL database.
Database Name
Environment Variable: FILEBIN_DATABASE_NAME
Command Line Argument: --db-name
Default: (required)
The name of the PostgreSQL database to use.
Database Username
Environment Variable: FILEBIN_DATABASE_USERNAME
Command Line Argument: --db-username
Default: (required)
The username to use when authenticating to the PostgreSQL database.
Database Password
Environment Variable: FILEBIN_DATABASE_PASSWORD
Command Line Argument: --db-password
Default: (required)
The password to use when authenticating to the PostgreSQL database.
Maximum time a database connection may be reused before it is closed and replaced. The value is specified using Go duration format, examples: 5m, 10m, 1h.
Maximum time a database connection may sit idle before it is closed. The value is specified using Go duration format, examples: 1m, 5m, 10m.
S3 Storage
S3 Endpoint
Environment Variable: FILEBIN_S3_ENDPOINT
Command Line Argument: --s3-endpoint
Default: (required)
The S3 endpoint to connect to. This can be the hostname or IP address. When self hosting S3 on a non-standard port, the port can be specified using hostname:port.
S3 Bucket
Environment Variable: FILEBIN_S3_BUCKET
Command Line Argument: --s3-bucket
Default: (required)
The name of the bucket in S3 where files will be stored.
S3 Region
Environment Variable: FILEBIN_S3_REGION
Command Line Argument: --s3-region
Default: (required)
The S3 region where the bucket lives.
S3 Access Key
Environment Variable: FILEBIN_S3_ACCESS_KEY
Command Line Argument: --s3-access-key
Default: (required)
The access key to use when connecting to the S3 bucket where files will be stored.
S3 Secret Key
Environment Variable: FILEBIN_S3_SECRET_KEY
Command Line Argument: --s3-secret-key
Default: (required)
The secret key to use when connecting to the S3 bucket where files will be stored.
S3 Secure
Environment Variable: FILEBIN_S3_SECURE
Command Line Argument: --s3-secure
Default: true
Whether or not Filebin will require the connection to S3 to be TLS encrypted using https. If set to false, Filebin will attempt connecting to S3 using plain http.
S3 URL TTL
Environment Variable: FILEBIN_S3_URL_TTL
Command Line Argument: --s3-url-ttl
Default: 1m
When a Filebin user downloads a file, that is done using a presigned URL that contains a token with limited time to live. The default allows presigned URLs to be used for 1 minute before they expire. The value is specified using the time unit, examples: 30s, 5m, 2h.
S3 Timeout
Environment Variable: FILEBIN_S3_TIMEOUT
Command Line Argument: --s3-timeout
Default: 30s
Timeout for quick S3 operations (delete, head, stat).
S3 Transfer Timeout
Environment Variable: FILEBIN_S3_TRANSFER_TIMEOUT
Command Line Argument: --s3-transfer-timeout
Default: 10m
Timeout for S3 data transfers (put, get, copy).
Lurker (Background Jobs)
Lurker Interval
Environment Variable: FILEBIN_LURKER_INTERVAL
Command Line Argument: --lurker-interval
Default: 300
The lurker is a batch job that runs automatically and in the background to delete expired bins and remove old log entries from the database. This specifies the time in seconds for the lurker to sleep between each execution.
Lurker Throttle
Environment Variable: FILEBIN_LURKER_THROTTLE
Command Line Argument: --lurker-throttle
Default: 250
Milliseconds to wait between S3 deletions to throttle the deletion rate.
Log Retention
Environment Variable: FILEBIN_LOG_RETENTION
Command Line Argument: --log-retention
Default: 7
The number of days to keep transaction log entries in the PostgreSQL database before they are removed by the lurker.
Admin Authentication
Admin Username
Environment Variable: FILEBIN_ADMIN_USERNAME
Command Line Argument: --admin-username
Default: (not set)
Username to require for access to the /admin and /debug/pprof/ endpoints on the admin listener. If the username is not set, then these endpoints will not be available.
Admin Password
Environment Variable: FILEBIN_ADMIN_PASSWORD
Command Line Argument: --admin-password
Default: (not set)
Password to require for access to the /admin and /debug/pprof/ endpoints on the admin listener. If the password is not set, then these endpoints will not be available. Make sure to keep this password a secret.
Metrics
Enable Metrics
Environment Variable: FILEBIN_METRICS
Command Line Argument: --metrics
Default: false
Enables the /metrics endpoint on the admin listener. If this is not set, the endpoint will not return any metrics.
Metrics Username
Environment Variable: FILEBIN_METRICS_USERNAME
Command Line Argument: --metrics-username
Default: (not set)
The username used for authentication to the /metrics endpoint for Prometheus metrics.
Metrics Password
Environment Variable: FILEBIN_METRICS_PASSWORD
Command Line Argument: --metrics-password
Default: (not set)
The password used for authentication to the /metrics endpoint for Prometheus metrics.
Metrics Auth
Environment Variable: FILEBIN_METRICS_AUTH
Command Line Argument: --metrics-auth
Default: (not set)
Enables authentication. Currently only basic auth is supported. If set to basic, basic auth will be required. If not set, the endpoint is open to anyone who can reach the admin listener.
Metrics ID
Environment Variable: FILEBIN_METRICS_ID
Command Line Argument: --metrics-id
Default: $HOSTNAME
The string used as the identification of the filebin instance in the Prometheus metrics. By default, this is the $HOSTNAME environment variable.
Slack Integration
Slack Secret
Environment Variable: FILEBIN_SLACK_SECRET
Command Line Argument: --slack-secret
Default: (not set)
The secret that Slack will need to use when connecting to the http api. If this secret is not set, the Slack integration will be disabled.
Slack Domain
Environment Variable: FILEBIN_SLACK_DOMAIN
Command Line Argument: --slack-domain
Default: (not set)
Limits which Slack domain is allowed to access the http api. Other domains will be rejected.
Slack Channel
Environment Variable: FILEBIN_SLACK_CHANNEL
Command Line Argument: --slack-channel
Default: (not set)
If Filebin is set to require manual approval of new bins, then this approval can be given using the user interface, the http api directly or via Slack (using the http api). The http endpoint /integration/slack can be accessed using a webhook from Slack. This parameter limits which Slack channel is allowed to access the http api. Requests from other channels will be rejected.
Upgrading
Admin endpoints moved to a separate listener
The admin interface (/admin), the Prometheus metrics (/metrics) and the profiling endpoints (/debug/pprof/) are no longer served on the public listener. They are only served on the new admin listener, which listens on 127.0.0.1:8081 by default. On the public listener, these paths are now handled like any other bin or file path, and no longer give an authentication challenge.
When upgrading:
Update Prometheus scrape targets to point at the admin listener, for example http://127.0.0.1:8081/metrics.
Update API clients that approve bins with PUT /admin/approve/{bin} to use the admin listener. The Slack integration is unaffected, as it is served on the public listener.
Remove any proxy or load balancer rules that forward or restrict /admin, /metrics and /debug/pprof/ on the public listener.
If the admin listener must be reached from another host, set --admin-listen-host to an address on a private network and restrict access to it with a firewall. Avoid exposing it to the internet.
Make sure --baseurl is set correctly, as the admin interface uses it to link to bins and files on the public listener.
If you run Filebin in a container, publish port 8081 in addition to 8080 if you need to reach the admin listener from outside of the container, and set --admin-listen-host 0.0.0.0 (or FILEBIN_ADMIN_LISTEN_HOST=0.0.0.0).
Development
Local development environment
The development environment consists of one PostgreSQL instance, one Stupid Simple S3 object storage instance and an instance of filebin2 built from source. The easiest way to set up this environment is to clone this repository and do:
# With Docker
docker compose up --build
# With Podman
podman compose up --build
The easiest way to run the test suite is to run it in docker compose. Docker will exit successfully (return code 0) if the tests succeed, and exit with an error code other than 0 if the tests fail.
# Docker
docker compose -f integration-tests.yml up --abort-on-container-exit
# Podman
podman compose -f integration-tests.yml up --abort-on-container-exit
Building
The program can be built using:
# Build for Linux amd64 only
make linux
# Build for all platforms:
make build-all
The output will be the Filebin program as binaries in the artifacts/ folder called filebin2-linux-amd64 (depending on the platform). The filebin binary take the following environment variables and command line parameters.
Releasing
Releases are built and published by the release workflow, which runs when a tag matching v* is pushed. There is no version number to bump in the source; the version is taken from the tag.
Make sure master is up to date and that the CI workflow passes on the commit to release.
Pick the next version following semantic versioning, based on the changes since the previous release:
Built the binaries for Linux and macOS (amd64 and arm64).
Built .deb and .rpm packages for Linux (amd64 and arm64).
Built multi-arch container images and pushed them to ghcr.io/espebra/filebin2, tagged with the full version (1.4.0), the minor version (1.4), the major version (1), and latest.
Created a GitHub release for the tag with the binaries, the packages, a checksums.txt file, and release notes generated from the commits and pull requests since the previous release.
The tag and commit are compiled into the binaries and container images. They are logged at startup and printed by filebin2 --version.
If the workflow fails, fix the problem on master, then delete the tag and create it again on the fixed commit:
git push --delete origin v1.4.0
git tag -d v1.4.0
git tag v1.4.0 master
git push origin v1.4.0
If the GitHub release was already created before the failure, delete it on the releases page before pushing the tag again.
Integrations
Grafana and Prometheus
Filebin2 comes with a /metrics endpoint on the admin listener that is compatible with Prometheus. There is an example dashboard that visualizes this data.
Slack
This integration may be useful if manual approval is required (see --manual-approval). The integration allows members of a Slack channel to list the recently updated bins and approve specific bins directly in the Slack channel using slash commands. The slash commands available are:
Slash command
Description
/filebin approve bin_id
Approve the bin bin_id
/filebin lastupdated
List the 10 last updated bins
/filebin lastupdated n
List the n last updated bins
The documention on how to configure Slack to work with this integration does not exist currently.
