Multi-line item value function calls

[theoephraim]

Currently config item values (other than multi-line strings) must be on a single line. For complex nested function calls, this obviously makes things hard to read. We'd like a way of splitting up long complex function calls across multiple lines to make things clearer.

Some context on why it works this way currently:
Aside from making the parser simpler, the idea here is to maintain some backwards compatibility with most .env/bash parsers. Of course many things wouldn't work if not using varlock (or another env-spec compatible parser), but at least parsing would not fail. There is a discussion to be had around whether that goal is worthwhile at all - so please chime in on that front...

Env-based switch example
Currently we encourage users to split env-specific config into multiple files and use @envFlag, but there are cases where a user may want to use a single file, more like DMNO. We do currently provide a remap helper function, but it has limitations, and one can imagine a better helper function for this purpose.

problems with remap

• entire function call must be on one line
• we use key=val syntax to try to make things clearer but...
• key and val are seemingly backwards from what is expected, because we wanted to support matching against a regex()
• which means remapped values must also be valid key names

---

Consider a situation where we want to switch a value based on APP_ENV, with production, preview, and development have different values. We'll introduce a new function switchBy which should take in an input (usually just another item), then look through a list of possible cases, returning a new value if a match is found. The values themselves could be long function calls, especially if fetching data from a remote source, so right away that points to breaking things up on multiple lines. The matching cases could also be simple static values, or regular expressions. We could also provide a special default case, or rely on wrapping the entire thing with fallback().

Option 1 - disregard backwards compat

Simplest seeming option (to end users) is to stop caring about backwards compatibility for legacy env parsers. Of course there are many syntax options (arrays, key value objects, etc), but here is a simple one where we rely on item order and providing pairs of key/value.

SWITCHED_ITEM=switchBy($APP,
  production, exec('op read "op://api-prod/xyz/api-key"'),   # can I put a comment here??
  staging, exec('op read "op://api-preview/xyz/api-key"'),
  development, 'static-non-sensitive-value'
)

✅ it's fairly intuitive
❌ totally breaks legacy parsers
❌ adds quite a bit of complexity to the parser and language syntax
❌ there is now the possibility of comments within the middle of a function call?

Option 2 - repeating keys, coupled functions + merging definitions

Instead we could repeat the key, and do some merging of the definitions.

SWITCHED_ITEM=switchBy($APP)
SWITCHED_ITEM=switchByCase(production, exec('op read "op://api-prod/xyz/api-key"'))
SWITCHED_ITEM=switchByCase(preview, exec('op read "op://api-preview/xyz/api-key"'))
SWITCHED_ITEM=switchByCase(development, 'static-non-sensitive-value')

✅ fairly clear
❌ bit verbose
❌ new coupled function calls, need to merge together definitions which could get a bit weird

Option 3 - use special key

A slight variation of above, where we dont repeat the key, but instead use a special key

SWITCHED_ITEM=switchBy($APP)
_=switchByCase(production, exec('op read "op://api-prod/xyz/api-key"'))
_=switchByCase(preview, exec('op read "op://api-preview/xyz/api-key"'))
_=switchByCase(development, 'static-non-sensitive-value')

✅ bit less verbose
✅ more clear that something special is happening instead of defining a thing twice
❌ need to find a key that is unlikely to never conflict with any real env var

Option 4 - use comments/decorators

There are variations where we could use decorators/comments to mark specific lines as matching certain cases. The main downside here is that at least currently, the decorator value are not the same thing as the item values, so doing things like regexes would be a totally new and custom system. Maybe not the end of the world though?

SWITCHED_ITEM=switchBy($APP)
SWITCHED_ITEM=exec('op read "op://api-prod/xyz/api-key"') # @switchCase=production
SWITCHED_ITEM=exec('op read "op://api-preview/xyz/api-key"') # @switchCase=preview
SWITCHED_ITEM='static-non-sensitive-value' # @switchCase=development

❌ custom implementation for regex matching, etc
❌ I dont love the matching criteria being after the value

---

Very open to ideas and opinions here!

[Outc4sted]

Option 1 looks the most normal to me but I don't know how much you value supporting legacy parsers. The other options that show repeating keys or functions would get a little verbose for me with my setup since I'd be defining env vars for 7 different environments (local, tests, docker, dev, impl, preprod, and prod). Supporting a default case also runs into the same issue as Option 3 where you'd need to match against something special that would not conflict. Perhaps it would be a special syntax on top of option 1 where instead of commas you'd use a : and then the final one without a key which would avoid that

SWITCHED_ITEM=switchBy($APP,
  "production": exec('op read "op://api-prod/xyz/api-key"'),
  "staging": exec('op read "op://api-preview/xyz/api-key"'),
  "static-non-sensitive-value"  # no key so considered the default
)

Another nice-to-have would be to define multiple cases to a single value somehow, either by something like falling through or matching in array of keys

SWITCHED_ITEM=switchBy($APP,
  "preprod":  # no value so falls through to the next
  "production": exec('op read "op://api-prod/xyz/api-key"'),
  "staging": exec('op read "op://api-preview/xyz/api-key"'),
  "static-non-sensitive-value"
)

SWITCHED_ITEM=switchBy($APP,
  ["preprod", "production"]: exec('op read "op://api-prod/xyz/api-key"'),  # 2 cases mapped to a single value
  "staging": exec('op read "op://api-preview/xyz/api-key"'),
  "static-non-sensitive-value"
)

[theoephraim]

I dont think I'd want to introduce a totally new syntax of floating key: value within function args, although I have been wanting to eventually add explicit arrays and objects, in which case fn({ key: val }) could make sense. But that said, in this case, using keys could still suffer from the same limitation of being valid key names which are limited. But if we supported explicit arrays/objects and newlines it could open things up to:

switchBy($ENV, [
  { match: [val1, val2], replace: 'new-val' },
  ...
])

But I've been trying to avoid making it all super javascript-y.

I do think the idea of multiple matching keys is a good one, and is compatible with other ideas here, either by using an explicit array (if we add support for it) or using a function call. For example switchBy($APP, or(match1, match2), newVal) or switchBy($APP, array(match1, match2), newVal).

[Outc4sted]

Ah it hadn't clicked for me that the DSL transform was expecting some js-compatible structure. In that case maybe it could just follow the classic switch/case syntax such as:

SWITCHED_ITEM=switchBy($APP,
  case(["preprod", "prod"], exec('op read "op://api-prod/xyz/api-key"')),  # multiple matches requires array
  case("staging", exec('op read "op://api-preview/xyz/api-key"')),  # single match doesn't require array
  case(["development"], "some_val"),  # using an array syntax for single value still allowed
  "static-non-sensitive-value",  # default/fallback value. if it would be more intuitive maybe allow for a default() function to match the typical switch statement
)