-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Use Labeled blocks in TailRec, instead of label-defs.
#5115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello, and thank you for opening this PR! 🎉
All contributors have signed the CLA, thank you! ❤️
Commit Messages
We want to keep history, but for that to actually be useful we have
some rules on how to format our commit messages (relevant xkcd).
Please stick to these guidelines for commit messages:
- Separate subject from body with a blank line
- When fixing an issue, start your commit message with
Fix #<ISSUE-NBR>:- Limit the subject line to 72 characters
- Capitalize the subject line
- Do not end the subject line with a period
- Use the imperative mood in the subject line ("Add" instead of "Added")
- Wrap the body at 80 characters
- Use the body to explain what and why vs. how
adapted from https://chris.beams.io/posts/git-commit
Have an awesome day! ☀️
|
Can we avoid the def fact(n: Int, acc: Int): Int = {
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
var result$tailLocal1: Int = _
tailLabel1[Unit]: {
while (true) {
if (n$tailLocal1 == 0) {
result$tailLocal1 = acc$tailLocal1
return[tailLabel1] ()
} else {
val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp
}
}
}
result$tailLocal1
}EDIT: Introduced a new local variable for the result |
|
No, in general we cannot do that, because the right-hand-side to the inner return can refer to values that are not in scope at the top-level of the method. Besides, if you have several exit points, it doesn't really work either. It would be possible to do something like this: def fact(n: Int, acc: Int): Int = {
var tailResult: Int = 0
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
tailReturn1[Unit]: {
while (true) {
tailLabel1[Unit]: {
tailResult = {
if (n$tailLocal1 == 0) {
acc
} else {
val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp
return[tailLabel1] ()
}
}
return[tailReturn1] ()
}
}
}
tailResult
}but that is actually less efficient than leaving the dead code, even if less "ugly". I also thought that instead of generating |
A more reliable alternative would be to allow |
b920577 to
a64ba26
Compare
c2aa05f to
4adc949
Compare
Labeled blocks in TailRec, instead of label-defs.Labeled blocks in TailRec, instead of label-defs.
|
Rebased. Now ready for full review. |
|
I think there is alternate translation that we may want to consider: def fact(n: Int, acc: Int): Int = {
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
tailReturn1[Int]: {
while (true) {
tailLabel1[Unit]: {
return tailReturn1[Int] {
if (n$tailLocal1 == 0) {
acc
} else {
val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp
return[tailLabel1] ()
}
}
}
}
}@sjrd Can you clarify why we need temporaries for arguments of tail recursive call? I.e. why val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmpinstead of: |
That does not work. You'll still need a dead code
In your alternative translation without temporaries, the computation of the new value for |
|
I added a commit with my own alternative proposal to use |
|
test performance please |
|
Looks like the performance bot doesn't like me :p |
|
test performance please |
|
performance test scheduled: 1 job(s) in queue, 0 running. |
|
@liufengyun can you add @sjrd to the whitelist for running benchmarks ? |
|
Performance test finished successfully: Visit http://dotty-bench.epfl.ch/5115/ to see the changes. Benchmarks is based on merging with master (fd93b46) |
|
Now the bot is for your service @sjrd . |
|
@liufengyun Thank you :) Since you're there, it seems the performance test launched above had an issue: the page it points to is a 404 for me :s (http://dotty-bench.epfl.ch/5115/) |
|
It is an issue with Github pages. Sometimes you can use the URL
http://dotty-bench.epfl.ch/5115/index.html , sometimes you need to wait a
little while before it's available.
Le Wed, Oct 3, 2018 à 2:25 PM, Sébastien Doeraene <[email protected]>
a écrit :
… @liufengyun <https://github.com/liufengyun> Thank you :)
Since you're there, it seems the performance test launched above had an
issue: the page it points to is a 404 for me :s (
http://dotty-bench.epfl.ch/5115/)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5115 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAuDyQecAanMXxY1FubJLpuynatKoX_Vks5uhKyygaJpZM4WshD9>
.
|
52d8c77 to
abaf97e
Compare
|
Rebased. |
|
What's the status here. Can this be merged? |
|
It needs a review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also add the missing case for While in the tree traversal:
case tree @ While(cond, body) =>
tpd.cpy.While(tree)(noTailTransform(cond), noTailTransform(body))And a test that exercises it
| val varsForRewrittenParamSyms = transformer.varsForRewrittenParamSyms | ||
|
|
||
| val initialValDefs = { | ||
| val initialParamValDefs = for ((param, local) <- rewrittenParamSyms.zip(varsForRewrittenParamSyms)) yield { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(rewrittenParamSyms, varsForRewrittenParamSyms).zipped to avoid intermediate collection?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not even know about this thing 🤣
| val rewrittenParamSyms = transformer.rewrittenParamSyms | ||
| val varsForRewrittenParamSyms = transformer.varsForRewrittenParamSyms | ||
|
|
||
| val initialValDefs = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
initialVarDefs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. Depends on what the coding style in Dotty is. There are var definitions, but they are still tpd.ValDefs because there is no such thing as a VarDef. That's why I called them initialValDefs. Is it customary to refer to tpd.ValDefs that represents vars as VarDefs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quickly going through the codebase, I cannot infer a coding style. No strong feeling about this one
|
|
||
| val rhsFullyTransformed = varForRewrittenThis match { | ||
| case Some(localThisSym) => | ||
| val classSym = tree.symbol.owner.asClass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is equivalent to owner defined on line 99
| var continueLabel: Option[Symbol] = None | ||
| var varForRewrittenThis: Option[Symbol] = None | ||
| var rewrittenParamSyms: List[Symbol] = Nil | ||
| var varsForRewrittenParamSyms: List[Symbol] = Nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you could simplify the implementation here. If I understand correctly rewrittenParamSyms is just paramSyms that you reorder (for reasons I don't understand). Would it be possible to get rid of rewrittenParamSyms? Something like:
private[this] var myVarsForRewrittenParamSyms: List[Symbol] = _
def varsForRewrittenParamSyms(implicit ctx: Context) = {
if (myVarsForRewrittenParamSyms == null) {
myVarsForRewrittenParamSyms = paramSyms.map(param =>
ctx.newSymbol(method, TailLocalName.fresh(param.name.toTermName), Flags.Synthetic | Flags.Mutable, param.info))
}
myVarsForRewrittenParamSyms
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, rewrittenParamSyms is a subset of paramSyms. It only contains parameters whose value is changed in at least one tail-recursive call. This is quite important to avoid declaring and assigning useless variables.
| var continueLabel: Option[Symbol] = None | ||
| var varForRewrittenThis: Option[Symbol] = None | ||
| var rewrittenParamSyms: List[Symbol] = Nil | ||
| var varsForRewrittenParamSyms: List[Symbol] = Nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These four variables would benefit from comments explaining what they are
| } | ||
|
|
||
| val tpt = TypeTree(method.info.resultType) | ||
| Block(assignments, Typed(Return(Literal(Constant(())).withPos(tree.pos), ref(getContinueLabel())), tpt)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seq(assignments, ...)
| * while (true) { | ||
| * tailResult[ResultType]: { | ||
| * return { | ||
| * // original rhs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understand correctly, it is not original rhs here. It is original rhs with tail recursive call rewritten as you describe below
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is true, but if you follow the "story" in the comment, it later says
Self-recursive calls in tail-position are then replaced by
So at this point in the story, it is still the original rhs.
I can also rewrite the story to make everything happen "at the same time", if you prefer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's very helpful to have a self contained example that describes the whole transformation
| } | ||
|
|
||
| val tpt = TypeTree(method.info.resultType) | ||
| Block(assignments, Typed(Return(Literal(Constant(())).withPos(tree.pos), ref(getContinueLabel())), tpt)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it is worth adding a method to tpd:
def Return(expr: Tree, from: Symbol)(implicit ctx: Context): Return =
Return(expr, Ident(from.termRef))I see multiple repetitions of Return(..., ref(...)) and Return(..., Ident(... .termRef)) (I believe both are equivalent, but the latter is more efficient)
| } | ||
|
|
||
| val tpt = TypeTree(method.info.resultType) | ||
| Block(assignments, Typed(Return(Literal(Constant(())).withPos(tree.pos), ref(getContinueLabel())), tpt)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the Return tree wrapped in a TypedTree node?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To perfectly preserve the type of the node (otherwise it's Nothing), which in turn is very important for the TypeAssigner not to infer different types as lubs of branches/cases when one of the branches has been replaced by a return. Otherwise Ycheck failures arise.
|
|
||
| var rewrote: Boolean = false | ||
|
|
||
| var continueLabel: Option[Symbol] = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is weird to use an Option here since the value is not really optional. Here is the convention we tend to follow for idempotent lazily computed values:
private[this] myContinueLabel: Symbol = _
def continueLabel(implicit ctx: Context) = {
if (myContinueLabel == null) {
myContinueLabel = ...
}
myContinueLabel
}| * } | ||
| * } | ||
| * </pre> | ||
| * <p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While you're at it, you could remove the html tags in this comment, it should use markdown instead.
|
|
||
| def assignType(tree: untpd.WhileDo)(implicit ctx: Context): WhileDo = | ||
| tree.withType(defn.UnitType) | ||
| tree.withType(if (tree.cond eq EmptyTree) defn.NothingType else defn.UnitType) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does this occur ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because we want a way to represent an infinite while loop, whose type is therefore Nothing. It's the whole point of the second commit in this PR.
abaf97e to
437a828
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated. AFAICT I should have addressed the points raised in the review.
| case _ :: _ => | ||
| val (tempValDefs, assigns) = (for ((lhs, rhs) <- assignThisAndParamPairs) yield { | ||
| val temp = c.newSymbol(method, TailTempName.fresh(lhs.name.toTermName), Flags.Synthetic, lhs.info) | ||
| (ValDef(temp, rhs), Assign(ref(lhs), ref(temp)).withPos(tree.pos)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't work, because it does not use the correct owner for the new symbol.
8f408bb to
92458fc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment is not addressed:
We should also add the missing case for
Whilein the tree traversal:
case tree @ While(cond, body) =>
tpd.cpy.While(tree)(noTailTransform(cond), noTailTransform(body))And tests that exercise it
Although this could be addressed as separate PR. Otherwise LGTM!
Let's discuss in the meeting if we want to keep the last commit
| val initialVarDefs = { | ||
| val initialParamVarDefs = (for ((param, local) <- (rewrittenParamSyms, varsForRewrittenParamSyms).zipped) yield { | ||
| ValDef(local.asTerm, ref(param)) | ||
| }).toList |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you don't need toList if you do:
val initialParamVarDefs = (rewrittenParamSyms, varsForRewrittenParamSyms).zipped.map(
(local, param) => ValDef(local.asTerm, ref(param)))Somehow, for .. yield is not equivalent to map here
It's easier to first explain on an example. Consider the following
tail-recursive method:
def fact(n: Int, acc: Int): Int =
if (n == 0) acc
else fact(n - 1, n * acc)
It is now translated as follows by the `tailrec` transform:
def fact(n: Int, acc: Int): Int = {
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
while (true) {
tailLabel1[Unit]: {
return {
if (n$tailLocal1 == 0) {
acc
} else {
val n$tailLocal1$tmp1: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp1: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp1
acc$tailLocal1 = acc$tailLocal1$tmp1
(return[tailLabel1] ()): Int
}
}
}
}
throw null // unreachable code
}
First, we allocate local `var`s for every parameter, as well as `this`
if necessary.
When we find a tail-recursive call, we evaluate the arguments into
temporaries, then assign them to the `var`s. It is necessary to use
temporaries in order not to use the new contents of a param local when
computing the new value of another param local.
We avoid reassigning param locals if their rhs (i.e., the actual
argument to the recursive call) is itself, which does happen quite
often in practice. In particular, we thus avoid reassigning the local
var for `this` if the prefix is empty. We could further optimize this
by avoiding the reassignment if the prefix is non-empty but equivalent
to `this`.
If only one parameter ends up changing value in any particular
tail-recursive call, we can avoid the temporaries and directly assign
it. This is also a fairly common situation, especially after discarding
useless assignments to the local for `this`.
After all that, we `return` from a labeled block, which is right inside
an infinite `while` loop. The net result is to loop back to the
beginning, implementing the jump. The `return` node is explicitly
ascribed with the previous result type, so that lubs upstream are not
affected (not doing so can cause Ycheck errors).
For control flows that do *not* end up in a tail-recursive call, the
result value is given to an explicit `return` out of the enclosing
method, which prevents the looping.
There is one pretty ugly artifact: after the `while` loop, we must
insert a `throw null` for the body to still typecheck as an `Int` (the
result type of the `def`). This could be avoided if we dared type a
`WhileDo(Literal(Constant(true)), body)` as having type `Nothing`
rather than `Unit`. This is probably dangerous, though, as we have no
guarantee that further transformations will leave the `true` alone,
especially in the presence of compiler plugins. If the `true` gets
wrapped in any way, the type of the `WhileDo` will be altered, and
chaos will ensue.
In the future, we could enhance the codegen to avoid emitting that dead
code. This should not be too difficult:
* emitting a `WhileDo` whose argument is `true` would set the generated
`BType` to `Nothing`.
* then, when emitting a `Block`, we would drop any statements and expr
following a statement whose generated `BType` was `Nothing`.
This commit does not go to such lengths, however.
This change removes the last source of label-defs in the compiler.
After this commit, we will be able to entirely remove label-defs.
92458fc to
0c24a34
Compare
|
Updated to avoid the I have also tried the variant where we cast to I have therefore kept the variant that uses |
Such `WhileDo` node have type `Nothing` instead of `Unit`. This is used in the loops generated by `TailRec` in order not to need some spurious dead code.
0c24a34 to
3084b1a
Compare
| override def typedWhileDo(tree: untpd.WhileDo)(implicit ctx: Context): Tree = { | ||
| assert((tree.cond ne EmptyTree) || ctx.phase.refChecked, i"invalid empty condition in while at $tree") | ||
| super.typedWhileDo(tree) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And here is the addition of an explicit Ycheck test that WhileDo(EmptyTree, _) is not used too early, as requested by @smarter.
| } | ||
|
|
||
| override def typedWhileDo(tree: untpd.WhileDo)(implicit ctx: Context): Tree = { | ||
| assert((tree.cond ne EmptyTree) || ctx.phase.refChecked, i"invalid empty condition in while at $tree") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why ctx.phase.refChecked? Why not an equivalent method for TailRec?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because it existed, and the only requirement is that an infinite loop must not be pickled. It doesn't have anything to do with tailrec per se.
Probably fixed by scala#5115
This PR depends on #5113 and #5112.
It's easier to first explain on an example. Consider the following tail-recursive method:
It is now translated as follows by the
tailrectransform:First, we allocate local
vars for every parameter, as well asthisif necessary.When we find a tail-recursive call, we evaluate the arguments into temporaries, then assign them to the
vars. It is necessary to use temporaries in order not to use the new contents of a param local when computing the new value of another param local.We avoid reassigning param locals if their rhs (i.e., the actual argument to the recursive call) is itself, which does happen quite often in practice. In particular, we thus avoid reassigning the local var for
thisif the prefix is empty. We could further optimize this by avoiding the reassignment if the prefix is non-empty but equivalent tothis.If only one parameter ends up changing value in any particular tail-recursive call, we can avoid the temporaries and directly assign it. This is also a fairly common situation, especially after discarding useless assignments to the local for
this.After all that, we
returnfrom a labeled block, which is right inside an infinitewhileloop. The net result is to loop back to the beginning, implementing the jump.For control flows that do not end up in a tail-recursive call, the result value is given to an explicit
returnout of the enclosing method, which prevents the looping.There is one pretty ugly artifact: after the
whileloop, we must insert athrow nullfor the body to still typecheck as anInt(the result type of thedef). This could be avoided if we dared type aWhileDo(Literal(Constant(true)), body)as having typeNothingrather thanUnit. This is probably dangerous, though, as we have no guarantee that further transformations will leave thetruealone, especially in the presence of compiler plugins. If thetruegets wrapped in any way, the type of theWhileDowill be altered, and chaos will ensue.In the future, we could enhance the codegen to avoid emitting that dead code. This should not be too difficult:
WhileDowhose argument istruewould set the generatedBTypetoNothing.Block, we would drop any statements and expr following a statement whose generatedBTypewasNothing.This commit does not go to such lengths, however.
This change removes the last source of label-defs in the compiler. After this commit, we will be able to entirely remove label-defs.