Skip to content
This repository was archived by the owner on Aug 2, 2021. It is now read-only.

Commit 427316a

Browse files
jpeletiernonsense
authored andcommitted
swarm/storage/mru: Client-side MRU signatures (#784)
* swarm/storage/mru: Add embedded publickey and remove ENS dep This commit breaks swarm, swarm/api... but tests in swarm/storage/mru pass * swarm: Refactor swarm, swarm/api to mru changes, make tests pass * swarm/storage/mru: Remove self from recv, remove test ens vldtr * swarm/storage/mru: Remove redundant test, expose ResourceHash mthd * swarm/storage/mru: Make HeaderGetter mandatory + godoc fixes * swarm/storage: Remove validator prefix for metadata chunk * swarm/storage/mru: Use Address instead of PublicKey * swarm/storage/mru: Change index from name to metadata chunk addr * swarm/storage/mru: Refactor swarm/api/... to MRU index changes * swarm/storage/mru: Refactor cleanup * swarm/storage/mru: Rebase cleanup * swarm: Use constructor for GenericSigner MRU in swarm.go * swarm/storage: Change to BMTHash for MRU hashing * swarm/storage: Reduce loglevel on chunk validator logs * swarm/storage/mru: Delint * swarm: MRU Rebase cleanup * swarm/storage/mru: client-side mru signatures Rebase to PR #668 and fix all conflicts * swarm/storage/mru: refactor and documentation * swarm/resource/mru: error-checking tests for parseUpdate/newUpdateChunk * swarm/storage/mru: Added resourcemetadata tests * swarm/storage/mru: Added tests for UpdateRequest * swarm/storage/mru: more test coverage for UpdateRequest and comments * swarm/storage/mru: Avoid fake chunks in parseUpdate() * swarm/storage/mru: Documented resource.go extensively moved some functions where they make most sense * swarm/storage/mru: increase test coverage for UpdateRequest and variable name changes throughout to increase consistency * swarm/storage/mru: moved default timestamp to NewCreateRequest- * swarm/storage/mru: lookup refactor * swarm/storage/mru: added comments and renamed raw flag to rawmru * swarm/storage/mru: fix receiver typo * swarm/storage/mru: refactored update chunk new/create * swarm/storage/mru: refactored signature digest to avoid malleability * swarm/storage/mru: optimize update data serialization * swarm/storage/mru: refactor and cleanup * swarm/storage/mru: add timestamp struct and serialization * swarm/storage/mru: fix lint error and mark some old code for deletion * swarm/storage/mru: remove unnecessary variable * swarm/storage/mru: Added more comments throughout * swarm/storage/mru: Refactored metadata chunk layout + extensive error... * swarm/storage/mru: refactor cli parser Changed resource info output to JSON * swarm/storage/mru: refactor serialization for extensibility refactored error messages to NewErrorf * swarm/storage/mru: Moved Signature to resource_sign. Check Sign errors in server tests * swarm/storage/mru: Remove isSafeName() checks * swarm/storage/mru: scrubbed off all references to "block" for time * swarm/storage/mru: removed superfluous isSynced() call. * swarm/storage/mru: remove isMultihash() and ToSafeName functions * swarm/storage/mru: various fixes and comments * swarm/storage/mru: decoupled cli for independent create/update * Made resource name optional * Removed unused LookupPrevious * swarm/storage/mru: Decoupled resource create / update & refactor * swarm/storage/mru: Fixed some comments as per issues raised in PR #743 * swarm/storage/mru: Cosmetic changes as per #743 comments * swarm/storage/mru: refct request encoder/decoder > marshal/unmarshal * swarm/storage/mru: Cosmetic changes as per review in #748 * swarm/storage/mru: removed timestamp proof placeholder * swarm/storage/mru: cosmetic/doc/fixes changes as per comments in #704 * swarm/storage/mru: removed unnecessary check in Handler.update * swarm/storage/mru: Implemented Marshaler/Unmarshaler iface in Request * swarm/storage/mru: Fixed linter error * swarm/storage/mru: removed redundant address in signature digest * swarm/storage/mru: fixed bug: LookupLatestVersionInPeriod not working * swarm/storage/mru: Unfold Request creation API for create or update+create set common time source for mru package * swarm/api/http: fix HandleGetResource error variable shadowed when requesting a resource that does not exist * swarm/storage/mru: Add simple check to detect duplicate updates * swarm/storage/mru: moved Multihash() to the right place. * cmd/swarm: remove unneeded clientaccountmanager.go * swarm/storage/mru: Changed some comments as per reviews in #784 * swarm/storage/mru: Made SignedResourceUpdate.GetDigest() public * swarm/storage/mru: cosmetic changes as per comments in #784 * cmd/swarm: Inverted --multihash flag default * swarm/storage/mru: removed Verify from SignedResourceUpdate.fromChunk * swarm/storage/mru: Moved validation code out of serializer Cosmetic / comment changes * swarm/storage/mru: Added unit tests for UpdateLookup * swarm/storage/mru: Increased coverage of metadata serialization * swarm/storage/mru: Increased test coverage of updateHeader serializers * swarm/storage/mru: Add resourceUpdate serializer test
1 parent 0647c4d commit 427316a

28 files changed

+3518
-1550
lines changed

cmd/swarm/main.go

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,18 @@ var (
182182
Usage: "Number of recent chunks cached in memory (default 5000)",
183183
EnvVar: SWARM_ENV_STORE_CACHE_CAPACITY,
184184
}
185+
SwarmResourceMultihashFlag = cli.BoolFlag{
186+
Name: "multihash",
187+
Usage: "Determines how to interpret data for a resource update. If not present, data will be interpreted as raw, literal data that will be included in the resource",
188+
}
189+
SwarmResourceNameFlag = cli.StringFlag{
190+
Name: "name",
191+
Usage: "User-defined name for the new resource",
192+
}
193+
SwarmResourceDataOnCreateFlag = cli.StringFlag{
194+
Name: "data",
195+
Usage: "Initializes the resource with the given hex-encoded data. Data must be prefixed by 0x",
196+
}
185197
)
186198

187199
//declare a few constant error messages, useful for later error check comparisons in test
@@ -235,6 +247,41 @@ func init() {
235247
Flags: []cli.Flag{SwarmEncryptedFlag},
236248
Description: "uploads a file or directory to swarm using the HTTP API and prints the root hash",
237249
},
250+
{
251+
CustomHelpTemplate: helpTemplate,
252+
Name: "resource",
253+
Usage: "(Advanced) Create and update Mutable Resources",
254+
ArgsUsage: "<create|update|info>",
255+
Description: "Works with Mutable Resource Updates",
256+
Subcommands: []cli.Command{
257+
{
258+
Action: resourceCreate,
259+
CustomHelpTemplate: helpTemplate,
260+
Name: "create",
261+
Usage: "creates a new Mutable Resource",
262+
ArgsUsage: "<frequency>",
263+
Description: "creates a new Mutable Resource",
264+
Flags: []cli.Flag{SwarmResourceNameFlag, SwarmResourceDataOnCreateFlag, SwarmResourceMultihashFlag},
265+
},
266+
{
267+
Action: resourceUpdate,
268+
CustomHelpTemplate: helpTemplate,
269+
Name: "update",
270+
Usage: "updates the content of an existing Mutable Resource",
271+
ArgsUsage: "<Manifest Address or ENS domain> <0x Hex data>",
272+
Description: "updates the content of an existing Mutable Resource",
273+
Flags: []cli.Flag{SwarmResourceMultihashFlag},
274+
},
275+
{
276+
Action: resourceInfo,
277+
CustomHelpTemplate: helpTemplate,
278+
Name: "info",
279+
Usage: "obtains information about an existing Mutable Resource",
280+
ArgsUsage: "<Manifest Address or ENS domain>",
281+
Description: "obtains information about an existing Mutable Resource",
282+
},
283+
},
284+
},
238285
{
239286
Action: list,
240287
CustomHelpTemplate: helpTemplate,
@@ -563,6 +610,26 @@ func getAccount(bzzaccount string, ctx *cli.Context, stack *node.Node) *ecdsa.Pr
563610
return decryptStoreAccount(ks, bzzaccount, utils.MakePasswordList(ctx))
564611
}
565612

613+
// getPrivKey returns the private key of the specified bzzaccount
614+
// Used only by client commands, such as `resource`
615+
func getPrivKey(ctx *cli.Context) *ecdsa.PrivateKey {
616+
// booting up the swarm node just as we do in bzzd action
617+
bzzconfig, err := buildConfig(ctx)
618+
if err != nil {
619+
utils.Fatalf("unable to configure swarm: %v", err)
620+
}
621+
cfg := defaultNodeConfig
622+
if _, err := os.Stat(bzzconfig.Path); err == nil {
623+
cfg.DataDir = bzzconfig.Path
624+
}
625+
utils.SetNodeConfig(ctx, &cfg)
626+
stack, err := node.New(&cfg)
627+
if err != nil {
628+
utils.Fatalf("can't create node: %v", err)
629+
}
630+
return getAccount(bzzconfig.BzzAccount, ctx, stack)
631+
}
632+
566633
func decryptStoreAccount(ks *keystore.KeyStore, account string, passwords []string) *ecdsa.PrivateKey {
567634
var a accounts.Account
568635
var err error

cmd/swarm/mru.go

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
// Copyright 2016 The go-ethereum Authors
2+
// This file is part of go-ethereum.
3+
//
4+
// go-ethereum is free software: you can redistribute it and/or modify
5+
// it under the terms of the GNU General Public License as published by
6+
// the Free Software Foundation, either version 3 of the License, or
7+
// (at your option) any later version.
8+
//
9+
// go-ethereum is distributed in the hope that it will be useful,
10+
// but WITHOUT ANY WARRANTY; without even the implied warranty of
11+
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12+
// GNU General Public License for more details.
13+
//
14+
// You should have received a copy of the GNU General Public License
15+
// along with go-ethereum. If not, see <http://www.gnu.org/licenses/>.
16+
17+
// Command resource allows the user to create and update signed mutable resource updates
18+
package main
19+
20+
import (
21+
"fmt"
22+
"strconv"
23+
"strings"
24+
25+
"github.com/ethereum/go-ethereum/common/hexutil"
26+
27+
"github.com/ethereum/go-ethereum/cmd/utils"
28+
swarm "github.com/ethereum/go-ethereum/swarm/api/client"
29+
"github.com/ethereum/go-ethereum/swarm/storage/mru"
30+
"gopkg.in/urfave/cli.v1"
31+
)
32+
33+
func NewGenericSigner(ctx *cli.Context) mru.Signer {
34+
return mru.NewGenericSigner(getPrivKey(ctx))
35+
}
36+
37+
// swarm resource create <frequency> [--name <name>] [--data <0x Hexdata> [--multihash=false]]
38+
// swarm resource update <Manifest Address or ENS domain> <0x Hexdata> [--multihash=false]
39+
// swarm resource info <Manifest Address or ENS domain>
40+
41+
func resourceCreate(ctx *cli.Context) {
42+
args := ctx.Args()
43+
44+
var (
45+
bzzapi = strings.TrimRight(ctx.GlobalString(SwarmApiFlag.Name), "/")
46+
client = swarm.NewClient(bzzapi)
47+
multihash = ctx.Bool(SwarmResourceMultihashFlag.Name)
48+
initialData = ctx.String(SwarmResourceDataOnCreateFlag.Name)
49+
name = ctx.String(SwarmResourceNameFlag.Name)
50+
)
51+
52+
if len(args) < 1 {
53+
fmt.Println("Incorrect number of arguments")
54+
cli.ShowCommandHelpAndExit(ctx, "create", 1)
55+
return
56+
}
57+
signer := NewGenericSigner(ctx)
58+
frequency, err := strconv.ParseUint(args[0], 10, 64)
59+
if err != nil {
60+
fmt.Printf("Frequency formatting error: %s\n", err.Error())
61+
cli.ShowCommandHelpAndExit(ctx, "create", 1)
62+
return
63+
}
64+
65+
metadata := mru.ResourceMetadata{
66+
Name: name,
67+
Frequency: frequency,
68+
Owner: signer.Address(),
69+
}
70+
71+
var newResourceRequest *mru.Request
72+
if initialData != "" {
73+
initialDataBytes, err := hexutil.Decode(initialData)
74+
if err != nil {
75+
fmt.Printf("Error parsing data: %s\n", err.Error())
76+
cli.ShowCommandHelpAndExit(ctx, "create", 1)
77+
return
78+
}
79+
newResourceRequest, err = mru.NewCreateUpdateRequest(&metadata)
80+
if err != nil {
81+
utils.Fatalf("Error creating new resource request: %s", err)
82+
}
83+
newResourceRequest.SetData(initialDataBytes, multihash)
84+
if err = newResourceRequest.Sign(signer); err != nil {
85+
utils.Fatalf("Error signing resource update: %s", err.Error())
86+
}
87+
} else {
88+
newResourceRequest, err = mru.NewCreateRequest(&metadata)
89+
if err != nil {
90+
utils.Fatalf("Error creating new resource request: %s", err)
91+
}
92+
}
93+
94+
manifestAddress, err := client.CreateResource(newResourceRequest)
95+
if err != nil {
96+
utils.Fatalf("Error creating resource: %s", err.Error())
97+
return
98+
}
99+
fmt.Println(manifestAddress) // output manifest address to the user in a single line (useful for other commands to pick up)
100+
101+
}
102+
103+
func resourceUpdate(ctx *cli.Context) {
104+
args := ctx.Args()
105+
106+
var (
107+
bzzapi = strings.TrimRight(ctx.GlobalString(SwarmApiFlag.Name), "/")
108+
client = swarm.NewClient(bzzapi)
109+
multihash = ctx.Bool(SwarmResourceMultihashFlag.Name)
110+
)
111+
112+
if len(args) < 2 {
113+
fmt.Println("Incorrect number of arguments")
114+
cli.ShowCommandHelpAndExit(ctx, "update", 1)
115+
return
116+
}
117+
signer := NewGenericSigner(ctx)
118+
manifestAddressOrDomain := args[0]
119+
data, err := hexutil.Decode(args[1])
120+
if err != nil {
121+
utils.Fatalf("Error parsing data: %s", err.Error())
122+
return
123+
}
124+
125+
// Retrieve resource status and metadata out of the manifest
126+
updateRequest, err := client.GetResourceMetadata(manifestAddressOrDomain)
127+
if err != nil {
128+
utils.Fatalf("Error retrieving resource status: %s", err.Error())
129+
}
130+
131+
// set the new data
132+
updateRequest.SetData(data, multihash)
133+
134+
// sign update
135+
if err = updateRequest.Sign(signer); err != nil {
136+
utils.Fatalf("Error signing resource update: %s", err.Error())
137+
}
138+
139+
// post update
140+
err = client.UpdateResource(updateRequest)
141+
if err != nil {
142+
utils.Fatalf("Error updating resource: %s", err.Error())
143+
return
144+
}
145+
}
146+
147+
func resourceInfo(ctx *cli.Context) {
148+
var (
149+
bzzapi = strings.TrimRight(ctx.GlobalString(SwarmApiFlag.Name), "/")
150+
client = swarm.NewClient(bzzapi)
151+
)
152+
args := ctx.Args()
153+
if len(args) < 1 {
154+
fmt.Println("Incorrect number of arguments.")
155+
cli.ShowCommandHelpAndExit(ctx, "info", 1)
156+
return
157+
}
158+
manifestAddressOrDomain := args[0]
159+
metadata, err := client.GetResourceMetadata(manifestAddressOrDomain)
160+
if err != nil {
161+
utils.Fatalf("Error retrieving resource metadata: %s", err.Error())
162+
return
163+
}
164+
encodedMetadata, err := metadata.MarshalJSON()
165+
if err != nil {
166+
utils.Fatalf("Error encoding metadata to JSON for display:%s", err)
167+
}
168+
fmt.Println(string(encodedMetadata))
169+
}

swarm/api/api.go

Lines changed: 21 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -351,11 +351,12 @@ func (a *API) Get(ctx context.Context, manifestAddr storage.Address, path string
351351
// we need to do some extra work if this is a mutable resource manifest
352352
if entry.ContentType == ResourceContentType {
353353

354-
// get the resource root chunk key
355-
log.Trace("resource type", "key", manifestAddr, "hash", entry.Hash)
354+
// get the resource rootAddr
355+
log.Trace("resource type", "menifestAddr", manifestAddr, "hash", entry.Hash)
356356
ctx, cancel := context.WithCancel(context.Background())
357357
defer cancel()
358-
rsrc, err := a.resource.Load(ctx, storage.Address(common.FromHex(entry.Hash)))
358+
rootAddr := storage.Address(common.FromHex(entry.Hash))
359+
rsrc, err := a.resource.Load(ctx, rootAddr)
359360
if err != nil {
360361
apiGetNotFound.Inc(1)
361362
status = http.StatusNotFound
@@ -364,7 +365,8 @@ func (a *API) Get(ctx context.Context, manifestAddr storage.Address, path string
364365
}
365366

366367
// use this key to retrieve the latest update
367-
rsrc, err = a.resource.LookupLatest(ctx, rsrc.NameHash(), true, &mru.LookupParams{})
368+
params := mru.LookupLatest(rootAddr)
369+
rsrc, err = a.resource.Lookup(ctx, params)
368370
if err != nil {
369371
apiGetNotFound.Inc(1)
370372
status = http.StatusNotFound
@@ -374,10 +376,10 @@ func (a *API) Get(ctx context.Context, manifestAddr storage.Address, path string
374376

375377
// if it's multihash, we will transparently serve the content this multihash points to
376378
// \TODO this resolve is rather expensive all in all, review to see if it can be achieved cheaper
377-
if rsrc.Multihash {
379+
if rsrc.Multihash() {
378380

379381
// get the data of the update
380-
_, rsrcData, err := a.resource.GetContent(rsrc.NameHash().Hex())
382+
_, rsrcData, err := a.resource.GetContent(rootAddr)
381383
if err != nil {
382384
apiGetNotFound.Inc(1)
383385
status = http.StatusNotFound
@@ -888,78 +890,46 @@ func (a *API) BuildDirectoryTree(ctx context.Context, mhash string, nameresolver
888890
return addr, manifestEntryMap, nil
889891
}
890892

891-
// ResourceLookup Looks up mutable resource updates at specific periods and versions
892-
func (a *API) ResourceLookup(ctx context.Context, addr storage.Address, period uint32, version uint32, maxLookup *mru.LookupParams) (string, []byte, error) {
893+
// ResourceLookup finds mutable resource updates at specific periods and versions
894+
func (a *API) ResourceLookup(ctx context.Context, params *mru.LookupParams) (string, []byte, error) {
893895
var err error
894-
rsrc, err := a.resource.Load(ctx, addr)
896+
rsrc, err := a.resource.Load(ctx, params.RootAddr())
895897
if err != nil {
896898
return "", nil, err
897899
}
898-
if version != 0 {
899-
if period == 0 {
900-
return "", nil, mru.NewError(mru.ErrInvalidValue, "Period can't be 0")
901-
}
902-
_, err = a.resource.LookupVersion(ctx, rsrc.NameHash(), period, version, true, maxLookup)
903-
} else if period != 0 {
904-
_, err = a.resource.LookupHistorical(ctx, rsrc.NameHash(), period, true, maxLookup)
905-
} else {
906-
_, err = a.resource.LookupLatest(ctx, rsrc.NameHash(), true, maxLookup)
907-
}
900+
_, err = a.resource.Lookup(ctx, params)
908901
if err != nil {
909902
return "", nil, err
910903
}
911904
var data []byte
912-
_, data, err = a.resource.GetContent(rsrc.NameHash().Hex())
905+
_, data, err = a.resource.GetContent(params.RootAddr())
913906
if err != nil {
914907
return "", nil, err
915908
}
916909
return rsrc.Name(), data, nil
917910
}
918911

919-
// ResourceCreate creates Resource and returns its key
920-
func (a *API) ResourceCreate(ctx context.Context, name string, frequency uint64) (storage.Address, error) {
921-
key, _, err := a.resource.New(ctx, name, frequency)
922-
if err != nil {
923-
return nil, err
924-
}
925-
return key, nil
912+
// Create Mutable resource
913+
func (a *API) ResourceCreate(ctx context.Context, request *mru.Request) error {
914+
return a.resource.New(ctx, request)
926915
}
927916

928-
// ResourceUpdateMultihash updates a Mutable Resource and marks the update's content to be of multihash type, which will be recognized upon retrieval.
929-
// It will fail if the data is not a valid multihash.
930-
func (a *API) ResourceUpdateMultihash(ctx context.Context, name string, data []byte) (storage.Address, uint32, uint32, error) {
931-
return a.resourceUpdate(ctx, name, data, true)
917+
// ResourceNewRequest creates a Request object to update a specific mutable resource
918+
func (a *API) ResourceNewRequest(ctx context.Context, rootAddr storage.Address) (*mru.Request, error) {
919+
return a.resource.NewUpdateRequest(ctx, rootAddr)
932920
}
933921

934922
// ResourceUpdate updates a Mutable Resource with arbitrary data.
935923
// Upon retrieval the update will be retrieved verbatim as bytes.
936-
func (a *API) ResourceUpdate(ctx context.Context, name string, data []byte) (storage.Address, uint32, uint32, error) {
937-
return a.resourceUpdate(ctx, name, data, false)
938-
}
939-
940-
func (a *API) resourceUpdate(ctx context.Context, name string, data []byte, multihash bool) (storage.Address, uint32, uint32, error) {
941-
var addr storage.Address
942-
var err error
943-
if multihash {
944-
addr, err = a.resource.UpdateMultihash(ctx, name, data)
945-
} else {
946-
addr, err = a.resource.Update(ctx, name, data)
947-
}
948-
period, _ := a.resource.GetLastPeriod(name)
949-
version, _ := a.resource.GetVersion(name)
950-
return addr, period, version, err
924+
func (a *API) ResourceUpdate(ctx context.Context, request *mru.SignedResourceUpdate) (storage.Address, error) {
925+
return a.resource.Update(ctx, request)
951926
}
952927

953928
// ResourceHashSize returned the size of the digest produced by the Mutable Resource hashing function
954929
func (a *API) ResourceHashSize() int {
955930
return a.resource.HashSize
956931
}
957932

958-
// ResourceIsValidated checks if the Mutable Resource has an active content validator.
959-
func (a *API) ResourceIsValidated() bool {
960-
return a.resource.IsValidated()
961-
}
962-
963933
// ResolveResourceManifest retrieves the Mutable Resource manifest for the given address, and returns the address of the metadata chunk.
964934
func (a *API) ResolveResourceManifest(ctx context.Context, addr storage.Address) (storage.Address, error) {
965935
trie, err := loadManifest(ctx, a.fileStore, addr, nil)

0 commit comments

Comments
 (0)