Secret Management - Subcommands to interact with Secrets

[mariafoo]

After a vault is created. We should be able to get/put secrets into the filesystem.

If we were able to expose the filesystem a real FUSE filesystem on Unix systems, then we would be able to make use any Unix CLI tool to create secrets. However I think this feature can be pushed till later, since the CLI is just a prototype while we focus on releasing a GUI as well (and in the GUI we'll end up translating GUI actions into low level commands that doesn't even touch the real filesystem anyway).

This is why we end up with a few basic commands to put secrets in and get secrets out. Almost like FTP in a way.

Note that vaults are always the root path. So /a/file means we are referring to a vault address. However since many commands need to refer to the real filesystem and the polykey filesystem, then we need some way to disambiguate. We can do something similar to AWS CLI with a special protocol used.

Note that we can do all these operations within 1 keynode. But interactions between keynodes is different as that means we are using the vault sharing system.

# copying a file from one place to another (within the same keynode)
polykey secrets cp pk://a/file ./file
polykey secrets cp ./file pk://a/file
polykey secrets cp pk://a/file pk://b/file
# copy from stdin
polykey secrets cp - pk://b/file
# copy to stdout (same as cat)
polykey secrets cp pk://a/file -
polykey secrets cat pk://a/file # you can use this to pipe the file contents to other commands
polykey secrets mv pk://a/file ./file
polykey secrets mv ./file /a/file
polykey secrets rm ./file pk://a/file
polykey secrets ls pk://a/directory
polykey secrets touch pk://a/file # creates an empty file
# maybe you can launch a subshell like this (then you can run the above commands without rewriting pk s)...
polykey secrets pk://a/file

Note that I'm not sure I like the usage of pk:// to disambiguate path for polykey paths vs the normal paths. It's possible we might be able to make use of another sigil similar to ~ is used to refer to the home directory. Potential sigils for us to use: @, %, +, -. All the other symbols are commonly used for various things in the shell.

The above commands are just sub commands that cover copying, reading, moving, deleting, and listing directories. We can add more into it. We don't actually re-implement all the shell commands in Polykey. Just to cover the basics for now. For more advanced uses, it's better to then present a FUSE interface.

A very important command is the ability to edit file. Now we could do something like use $EDITOR. But the problem is that this expects a real file somewhere. Doing this means the secret has to be leaked onto disk for random access unless we present a FUSE interface. Otherwise you have to pipe the contents to the editor, but then the editor cannot pipe back out to Polykey without some other kind of wrapper. See: https://loige.co/aws-command-line-s3-content-from-stdin-or-to-stdout/ This is a difficult problem to solve, and the GUI won't have to deal with it obviously as it can have its own editor.

Another thing is that we may be able to make use of the multiaddr in libp2p so that means keynodes themselves have a p2p peer ID, but then each vault within should also have a unique id as well. I'm thinking some content-addressed ID but I'm not sure... if that may leak secrets. Otherwise what we end up is instead with some sort of unique id generation that we want to be unique across the network. So you have to ensure that each peer has a unique id (probabilistic via the public key), and then each vault also has a unique id also probabilistic via a public key as well (but if we aren't using public keys then...?). Anyway then you can have a globally unique id for vault and globally unique id for peer. So a peer could have /a vault, and any peer could have that. But one could also then refer to vaults regardless of the peer itself for "vault discovery".

[CMCDragonkai]

Rewrote the above for some secret command interaction. @ll-aashwin-ll your opinion?

[CMCDragonkai]

As for editing something like this might be possible: https://unix.stackexchange.com/a/282384/56970

It seems in general, it might be possible to pipe the output into an editor. Or pipe it to a file, where you then open the file descriptor but delete the file contents (thus you end up with a read/write descriptor). Then pipe that file back into the same command.

Note that one has to be careful with concurrent usage. We should expect similar concurrency issues.

Also note the usage of echo 'abc' >> file. It is also possible to do echo 'abc' >> >(cat > file). But this is more advanced.

[CMCDragonkai]

I really want to be able to run a polykey command to create a secret, that launches the $EDITOR or $VISUAL and upon saving, pipe the output to the polykey command without leaking any data on disk. Not entirely sure how to achieve this though.

Oh it seems vipe might be the right way to do it. There's also how git launches the editor when you ask to do rebases.

[CMCDragonkai]

Based on my reading of https://en.wikipedia.org/wiki/Uniform_Resource_Identifier I think this should mean that we can just use pk:/a/b. As well. Or even just pk:a/b. But without the root slash it could mean something else like the current context. I don't think it's specified. There are some good discussion about this here? https://github.com/ipfs/in-web-browsers/blob/master/ADDRESSING.md

[CMCDragonkai]

polykey secrets ed pk:/a/b

Launches editor for secret, and allows one to save the file.

It works!

tmpfile=$(mktemp /tmp/abc.XXXXXX)
exec {fd}<> "$tmpfile"
rm -f "$tmpfile"
vim "/proc/self/fd/$fd"
cat <&${fd}

The above works in ZSH and Bash using automatic fd allocation.

We would do this inside JS which should be alot more robust.

[CMCDragonkai]

@robert-cronin Given that you've created the demo already, could we get an update on the status of this issue?

In relation to #3 as well.

[robert-cronin]

The cli now exposes a sub-command for interacting with secrets, the pattern looks like this:

pk secrets <action>

Under this sub command, the user can manipulate secrets with the following commands:

pk secrets ls|list --vault-name <vault-name> # lists the secrets in a particular vault
pk secrets create --vault-name <vault-name> --secret-name <secret-name> --secret-path /path/to/secret # add a secret to a vault
pk secrets get --vault-name <vault-name> --secret-name <secret-name> # get an existing secret

There is still the option of doing away with command options like --vault-name and --secret-name and just doing something like a file path: <vault-name>/<secret-name> but lets just stick with the existing options for now and it can be a convenience later.

The other thing that is useful is environment variables to set default values:

export KEYNODE_PATH=/home/user/Polykey
export DEFAULT_VAULT=Vault1

[robert-cronin]

secrets are now referenced on the CLI with the format <vaultName>:<secretName>
The only thing remaining is to develop the editor for the secrets. I will make a new issue for this

polykey secrets ed pk:/a/b

Launches editor for secret, and allows one to save the file.

It works!

tmpfile=$(mktemp /tmp/abc.XXXXXX)
exec {fd}<> "$tmpfile"
rm -f "$tmpfile"
vim "/proc/self/fd/$fd"
cat <&${fd}

The above works in ZSH and Bash using automatic fd allocation.

We would do this inside JS which should be alot more robust.