pirate

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 8, 2025 License: BSD-3-Clause Imports: 16 Imported by: 0

README

= pirate 

image:https://github.com/aalbacetef/pirate/actions/workflows/ci.yml/badge.svg[CI status] 
image:https://img.shields.io/badge/License-BSD_3--Clause-blue.svg[License] 
image:https://goreportcard.com/badge/github.com/aalbacetef/pirate[Go Report Card]

Note: this is a WIP.

:toc: 

== Introduction 

image::misc/pirate-ship.jpg[] 

**Pirate** is a webhooks task runner. It aims to satisfy the need for running tasks based on webhook events.

Example config (see link:ship.sample.yml[ship.sample.yml]).
[source,yaml]
----
server:
  # optional: defaults to 'localhost'
  host: localhost

  # required: port on which to listen to 
  port: 3939

  # optional: maximum time allowed for a request, defaults to 5m0s 
  request-timeout: '2m30s'

  logging:
    # required: logging directory.
    #   Will be created with permission 744 if it doesn't exist
    #   Filename will be pirate.YYYY-MM-DD--hh:mm:ss.log
    dir: './logs' 

    ## NOTE: special value :stdout: writes to standard output
    # dir: ':stdout:'


handlers:
  # NOTE: all fields of the handler are required unless stated otherwise
  - endpoint: /webhooks/simple
    name: simple webhook handler

    # optional: handler execution policy, one of: drop, queue, parallel. Defaults to queue.
    policy: drop 

    # authenticates the handler based on the value of the X-Authorization header 
    auth:
      # a list validator will check if the token matches one of .token
      validator: list
      token: 
        - alpha
        - beta
    # script to run, the request headers and body are available as env vars.
    run: |
      SOME_VAR="some-variable"
      echo "SOME_VAR: $SOME_VAR"
      echo "body: $PIRATE_BODY"
      echo "headers: $PIRATE_HEADERS" 
      echo "header param: $PIRATE_HEADERS_SOME_PARAM"


  - endpoint: /new-release
    name: new release
    policy: queue
    auth:
      # a command validator will pass if the run block exits with code = 0.
      validator: command
      run: |
        echo "offloading validation to another program"
        ./path/to/validator --token="$PIRATE_TOKEN" --name="$PIRATE_NAME"
    run: | 
      # one can call scripts from the run block, this which makes it easier
      # to implement complex workflows
      ./scripts/handle-new-release.sh  "$PIRATE_BODY"
----

== Installation

=== Binaries

We provide pre-built binaries for Linux on the releases page.

See link:https://github.com/aalbacetef/pirate/releases[Releases].

=== Docker 

We provide a docker image for ease of use. 

The recommended way of using it is to mount your ship config as well as any needed directories. 

Example:

[source,bash]
----
docker run --rm -it \
  -v ./logs/:/app/logs \
  -v ./ship.yml:/app/ship.yml \
  -v /var/www/html:/app/blog-html \
  -p 39390:39390 \
  aalbacetef/pirate:latest
----

**Tips**

Don't forget to set `server.host` to `0.0.0.0`. 
Some users might find it useful to set the logging to standard output, while others would prefer to mount the log directory.


== Configuration

Pirate uses a YAML configuration file (`ship.yml`) to define server settings, logging, and webhook handlers.

=== Server Configuration

The `server` section defines how Pirate listens for incoming webhook requests.

[source,yaml]
----
server:
  host: localhost       # Optional: Defaults to 'localhost'
  port: 3939            # Required: The port Pirate listens on
  request-timeout: '5m0s' # Optional: Defaults to 5 minutes
----

- *`host`* (optional) - The address Pirate binds to. Defaults to `localhost`.
- *`port`* (required) - The port number Pirate listens on.
- *`request-timeout`* (optional) - Maximum duration for processing a request. Defaults to `5m0s`.

=== Logging Configuration

The `logging` section controls where logs are stored.

[source,yaml]
----
logging:
  dir: './logs'  # Required: Log directory or `:stdout:` for console output
----

* *`dir`* (required) - Directory where logs are saved.
** If the directory does not exist, Pirate creates it with **744 permissions**.
** Log files follow the format: `pirate.YYYY-MM-DD--HH:mm:ss.log`.
** Special value `:stdout:` writes logs to standard output.

=== Webhook Handlers

The `handlers` section defines webhook endpoints, authentication, and execution scripts.

==== Example Handler

[source,yaml]
----
handlers:
  - endpoint: /webhooks/simple
    name: simple webhook handler
    policy: drop
    auth:
      validator: list
      token: 
        - alpha
        - beta
    run: |
      echo "body: $PIRATE_BODY"
      echo "headers: $PIRATE_HEADERS"
----

Each handler includes:

* *`endpoint`* (required) - The URL path for this webhook (e.g., `/webhooks/simple`).
* *`name`* (required) - A human-readable name for the handler.
* *`policy`* (optional) - Execution policy. One of `drop`, `parallel`, `queue`. Defaults to `queue`. 
** `drop`: if webhook events come in while the handler is already running, they will be dropped.
** `parallel`: handlers will run as webhooks come in.
** `queue`: handlers will be queued as they come in.
* *`auth`* (required, one of `list` or `command`) - Authentication method:
** *`validator: list`* - Checks if the `X-Authorization` header matches one of the provided tokens.
** *`validator: command`* - Runs a script and passes authentication if it exits with `0`.
* *`run`* (required) - A shell script executed when the webhook is triggered. Available environment variables:
** `$PIRATE_BODY`: The request body.
** `$PIRATE_HEADERS`: All request headers.
** `$PIRATE_HEADERS_<HEADER_NAME>`: A specific header value.

==== Authentication Methods

===== Token-based Authentication

[source,yaml]
----
auth:
  validator: list
  token: 
    - alpha
    - beta
----

Passes if `X-Authorization` header matches one of the values of the `token` list, in this case: `alpha` or `beta`.

===== Command-based Authentication

[source,yaml]
----
auth:
  validator: command
  run: |
    echo "running validation via a script"
    ./scripts/validate-user.sh "$PIRATE_TOKEN"
----

Passes if the run block exits with exit code 0. 
The `X-Authorization` header's value is exposed as an environment variable: `PIRATE_TOKEN`.
The handler name is exposed as an environment variable: `PIRATE_NAME`.

=== Running External Scripts

Pirate allows running external scripts to handle complex workflows.

[source,yaml]
----
run: |
  ./scripts/handle-new-release.sh
----

== Notes On Security

- We assume users are running **pirate** behind some reverse-proxy like NGINX so not much care has been given to reimplement features offered by it (for the MVP), like rate-limiting, but will be added in the future.

- Don't use easy tokens for auth. If you need stricter checks use the command validator for more complex auth logic. In the future this will probably be passed a lot more request metadata.

- **Pirate** creates its scripts by default under /tmp (which it cleans up after running). In the future this will be configurable.

- **Pirate** responds with 404 even if validation fails, to not leak information. It does return a 405 if any method other than POST is used, but this shouldn't leak more information than only POST is accepted.

This tool assumes you trust yourself. If you're exposing it to the internet, make sure you know what you're doing. You’re the captain here, pirate doesn’t stop you from walking the plank if you tell it to.

Documentation

Index

Constants

View Source
const (
	LogToStdOut        = ":stdout:"
	LogTimestampFormat = "2006-01-02--15-04-05"
)
View Source
const (
	// Environment variable to read the pirate config path from.
	ConfigEnvVar = "PIRATE_CONFIG_PATH"
)
View Source
const DoTimeout = 5 * time.Minute
View Source
const TokenHeaderField = "X-Authorization"

Variables

View Source
var (
	ErrAuthFailed       = errors.New("authentication failed")
	ErrUnknownValidator = errors.New("unknown validator")
)
View Source
var ErrHandlerNotFound = errors.New("no matching handler was found")

Functions

func Load

func Load(fpath string) (Config, Source, error)

Load will attempt to load the config from the following sources (in order):

  • flag value (if passed)
  • Env Var ($PIRATE_CONFIG_PATH)
  • current working directory

It will return the source used, which aids in debugging.

Types

type Auth

type Auth struct {
	Token     []string  `yaml:"token"`
	Validator Validator `yaml:"validator"`
	Run       string    `yaml:"run"`
}

Auth specifies the authentication of the incoming request. If Validator is a ListValidator, then the token of the request must match a token of the list If Validator is a CommandValidator, then the value of Run is executed and considered successful if exit code = 0.

type Config

type Config struct {
	Server struct {
		Host           string   `yaml:"host"`
		Port           int      `yaml:"port"`
		Logging        Logging  `yaml:"logging"`
		RequestTimeout Duration `yaml:"request-timeout"`
	} `yaml:"server"`
	Handlers []Handler `yaml:"handlers"`
}

Config defines the configuration for the pirate server and its handlers.

func (Config) Valid

func (cfg Config) Valid() error

Valid will fail if fields are missing. Note that it expects optional fields to be set before being called.

type Duration

type Duration struct {
	time.Duration
}

Duration is a wrapper around time.Duration that supports JSON and YAML marshaling/unmarshaling.

func (*Duration) MarshalJSON

func (d *Duration) MarshalJSON() ([]byte, error)

MarshalJSON marshals the Duration to JSON.

func (*Duration) MarshalYAML

func (d *Duration) MarshalYAML() ([]byte, error)

MarshalYAML marshals the Duration to YAML.

func (*Duration) UnmarshalJSON

func (d *Duration) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals the Duration from JSON.

func (*Duration) UnmarshalYAML

func (d *Duration) UnmarshalYAML(node *yaml.Node) error

UnmarshalYAML unmarshals the Duration from YAML.

type ExecutionPolicy

type ExecutionPolicy string
const (
	Drop     ExecutionPolicy = "drop"
	Parallel ExecutionPolicy = "parallel"
	Queue    ExecutionPolicy = "queue"
)

type FileNotFoundError

type FileNotFoundError struct {
	Path string
}

FileNotFoundError represents an error when a config file is not found.

func (FileNotFoundError) Error

func (e FileNotFoundError) Error() string

type Handler

type Handler struct {
	Auth     Auth            `yaml:"auth"`
	Endpoint string          `yaml:"endpoint"`
	Name     string          `yaml:"name"`
	Run      string          `yaml:"run"`
	Policy   ExecutionPolicy `yaml:"policy,omitempty"`
}

Handler waits for a webhook handler to come in and runs it if authenatication passes.

type Logging

type Logging struct {
	Dir string `yaml:"dir"`
}

Logging defines the directory where logs should be written.

type MustBeSetError

type MustBeSetError struct {
	// contains filtered or unexported fields
}

MustBeSetError represents an error indicating a required field is missing.

func (MustBeSetError) Error

func (e MustBeSetError) Error() string

type Scheduler

type Scheduler interface {
	Start() error
	Pause() error
	Name() string
	Add(*scheduler.Job) error
}

type Server

type Server struct {
	// contains filtered or unexported fields
}

func NewServer

func NewServer(cfg Config) (*Server, error)

@TODO: handle log to Stdout.

func (*Server) Close

func (srv *Server) Close()

func (*Server) Do

func (srv *Server) Do(handler *Handler, headers map[string]string, payload []byte)

Do runs after a request has been validated. @TODO: maybe enforce Content-Type: application/json ? @TODO: add optional shell setting to config. @TODO: add handler timeout setting.

func (*Server) FindHandler

func (srv *Server) FindHandler(endpoint string) (Handler, error)

func (*Server) HandleRequest

func (srv *Server) HandleRequest(w http.ResponseWriter, req *http.Request)

HandleRequest is the main entrypoint of the server. It will first check if the request is a valid endpoint and passes auth checks. Then it will spin off a goroutine that executes the actual task. @TODO: queue multiple executions of the same endpoint.

type Source

type Source string

Source is where the config was read from.

const (
	LoadFromFlag   Source = "load-from-flag"
	LoadFromEnv    Source = "load-from-env"
	LoadFromCurDir Source = "load-from-cur-dir"
)

type Validator

type Validator string

Validator is the method of validation being used, either a command or a token from a list.

const (
	ListValidator    Validator = "list"
	CommandValidator Validator = "command"
)

Directories

Path Synopsis
cmd
pirate command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL