Added first version of Readme (#35)

This also contains code to generate documentation of the command line parameters from our parameter definition in the source code and this is then imported into the readme file.

This PR also restructures the entire project a little in that it now is a library crate with two binaries in the src/bin directory.

Author: soenkeliebau

README.adoc

@@ -0,0 +1,99 @@
+= Stackable Agent
+
+== Purpose
+The Stackable Agent is an alternative to the Kubernetes Kubelet that executes Pods not in containers but using systemd as its backend.
+It is implemented in Rust as a https://github.com/deislabs/krustlet[Krustlet] provider.
+
+In principle this behavior is similar to that of the the https://github.com/virtual-kubelet/virtual-kubelet[virtual kubelet] project.
+More specifically the functionality of this agent closely resembles that of https://github.com/virtual-kubelet/systemk[systemk].
+The reason why both projects currently exist is simply that systemk was not around when we started Stackable and so we had to go and build our own.
+
+WARNING: We are currently evaluating whether or not we should instead use systemk and contribute to that project instead of building our own agent under https://github.com/stackabletech/agent/issues/42[issue #42]
+
+The agent registers itself as a node with a kube-apiserver and will be considered by the Kubernetes scheduler for workloads (pods).
+To avoid _normal_ Kubernetes pods being scheduled on the Stackable agent (it would not know what to do with these) the agent assigns the following taints to its node object:
+
+|===
+|Taint |Type|Value
+
+|kubernetes.io/arch
+|NoSchedule
+|stackable-linux
+
+|kubernetes.io/arch
+|NoExecute
+|stackable-linux
+|===
+
+These taints _suggest_ to the Kubernetes scheduler that only pods with matching tolerations should be scheduled on this node.
+
+== Installation
+=== Build from source
+Building from source is fairly straightforward if you have the rust toolchain installed, our CI chain tests against the latest stable version at the time the checks are run.
+If you need to install this, generally the recommended way is to use https://rustup.rs/[rustup].
+
+After rust is installed simply run
+
+    cargo build
+
+To create the binary file in _target/debug_.
+
+=== Download binary
+We do not at this time provide pre-compiled binaries, as we are still in the process of setting up the build jobs to create these.
+This readme will be updated as soon as binary downloads are available!
+
+=== OS Packages
+We do not at this time provide OS packages, that is due to change in the near future and this readme will be updated accordingly!
+
+== Configuration
+
+=== Command Line Parameters
+The agent accepts the following command line parameters:
+
+include::documentation/commandline_args.adoc[]
+
+=== Config File
+In addition to directly specifying them on the command line, the agent allows specifying a config file via the environment variable `CONFIG_FILE`. Values specified in the file will have to adhere to the format `--parameter=value`.
+
+This file can contain all command line parameters and will be parsed before the actual command line.
+For parameters that are present in the file and on the command line, the command line will take precedence, unless it is a parameter that can be specified multiple times, in which case parameters from both, file and commandline, will be merged.
+
+.Example config file
+    --package-directory=/opt/stackable/agent/work/packages
+    --config-directory=/etc/stackable/agent
+    --server-cert-file=/etc/stackable/agent/secure/cert.crt
+    --server-key-file=/etc/stackable/agent/secure/key.key
+
+=== Kubernetes Config
+The agent uses the default way of looking for a kube-apiserver, so if your system is already set up to connect to Kubernetes with kubectl you should be good to go right of the bat.
+
+The default location for the Kubernetes client config is `~/.kube/config`, if you want to change this location you can override this via the `KUBECONFIG` environment variable.
+
+    export KUBECONFIG=/etc/stackable/agent/kubeconfig
+
+
+=== Certificates
+The agent requires a keypair and signed certificate to start a webserver which can be used to handle callbacks.
+If these are not specified on the commandline, the agent will create a keypair, upload a certificate signing request to Kubernetes and wait for the certificate before continuing.
+These steps require a certificate manager to be set up and running in your Kubernetes cluster, which may or may not be the case.
+
+You can also manually create these files and specify them on the command line.
+The following example shows how to create these files using https://github.com/OpenVPN/easy-rsa[easy-rsa], but this can be done in any number of different ways as well.
+
+    ./easyrsa init-pki
+    ./easyrsa build-ca
+    ./easyrsa gen-req krustlet1
+    ./easyrsa import-req pki/reqs/krustlet1.req krustlet1-req
+    ./easyrsa sign-req serverClient krustlet1-req
+    # Convert key to pksc8 format
+    openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in pki/private/krustlet1.key -out pkcs8.key
+
+== Contributing
+The agent is developed as an open source tool and we absolutely welcome any and all contributions!
+Don't hesitate to drop us a line at [email protected] or reach out directly to any of our committers / contributors.
+
+We will run a bi-weekly dev call during which we will discuss current development, issues, our children and the general state of the world.
+Please feel free to drop in if you have the time!
+
+Dial in info coming soon!
+

documentation/commandline_args.adoc

@@ -0,0 +1,166 @@
+
+
+
+=== tag
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* true
+
+
+A "key=value" pair that should be assigned to this agent as tag. This can be specified multiple times to assign additional tags.
+
+Tags are the main way of identifying nodes to assign services to later on.
+
+
+=== server-key-file
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+Private key file (in PKCS8 format) to use for the local webserver the Krustlet starts.
+
+
+=== log-directory
+
+*Default value*: /opt/stackable/logs
+
+*Required*: false
+
+*Multiple values:* false
+
+
+This directory will serve as starting point for all log files which this service creates.
+Every service will get its own subdirectory created within this directory.
+Anything that is then specified in the log4j config or similar files will be resolved relatively to this directory.
+
+The agent will need full access to this directory and tries to create it if it does not exist.
+
+
+=== hostname
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The hostname to register the node under in Kubernetes - defaults to system hostname.
+
+
+=== server-cert-file
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The certificate file for the local webserver which the Krustlet starts.
+
+
+=== no-config
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+If this option is specified, any file referenced in AGENT_CONF environment variable will be ignored.
+
+
+=== bootstrap-file
+
+*Default value*: /etc/kubernetes/bootstrap-kubelet.conf
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The bootstrap file to use in case Kubernetes bootstraping is used to add the agent.
+
+
+=== server-bind-ip
+
+*Default value*: No default value
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The local IP to register as the node's ip with the apiserver. Will be automatically set to the first address of the first non-loopback interface if not specified.
+
+
+=== data-directory
+
+*Default value*: /var/stackable/agent/data
+
+*Required*: false
+
+*Multiple values:* false
+
+
+The directory where the stackable agent should keep its working data.
+
+
+=== config-directory
+
+*Default value*: /opt/stackable/config
+
+*Required*: false
+
+*Multiple values:* false
+
+
+This directory will serve as starting point for all log files which this service creates.
+
+Every service will get its own subdirectories created within this directory - for every service start a
+new subdirectory will be created to show a full history of configuration that was used for this service.
+
+ConfigMaps which are specified in the pod that describes this service will be created relative to these run
+directories - unless the mounts specify an absolute path, in which case it is allowed to break out of this directory.
+
+WARNING: This allows anybody who can specify pods more or less full access to the file system on the machine running the agent!
+
+The agent will need full access to this directory and tries to create it if it does not exist.
+
+
+=== server-port
+
+*Default value*: 3000
+
+*Required*: false
+
+*Multiple values:* false
+
+
+Port to listen on for callbacks.
+
+
+=== package-directory
+
+*Default value*: /opt/stackable/packages
+
+*Required*: false
+
+*Multiple values:* false
+
+
+This directory will serve as starting point for packages that are needed by pods assigned to this node.\n Packages will be downloaded into the "_download" folder at the top level of this folder as archives and remain there for potential future use.
+
+Archives will the be extracted directly into this folder in subdirectories following the naming
+scheme of "productname-productversion".
+
+The agent will need full access to this directory and tries to create it if it does not exist.

src/main.rs renamed to src/bin/agent.rs

@@ -8,11 +8,8 @@ use kubelet::Kubelet;
 use log::{info, warn};
 use stackable_config::ConfigBuilder;
 
-use crate::agentconfig::AgentConfig;
-use crate::provider::StackableProvider;
-
-mod agentconfig;
-mod provider;
+use stackable_agent::config::AgentConfig;
+use stackable_agent::provider::StackableProvider;
 
 #[tokio::main(threaded_scheduler)]
 async fn main() -> anyhow::Result<()> {

src/bin/generate_doc.rs

@@ -0,0 +1,20 @@
+/// This is a helper binary which generates the file _documentation/commandline_args.adoc_ which
+/// contains documentation of the available command line options for the agent binary.
+///
+/// It gets the content by calling [`stackable_agent::config::AgentConfig::get_documentation()`]
+///
+/// # Panics
+/// This will panic if an error occurs when trying to write the file.
+
+fn main() {
+    use stackable_agent::config::AgentConfig;
+    use std::fs;
+
+    let target_file = "documentation/commandline_args.adoc";
+    fs::write(target_file, AgentConfig::get_documentation()).unwrap_or_else(|err| {
+        panic!(
+            "Could not write documentation to [{}]: {}",
+            target_file, err
+        )
+    });
+}