-
An earlier attempt at visualizing JS execution, by hand-constructing a basic interpreter for the whole language. This is tedious of course, but does lead to very fine-grained control, with all the accompanying visualization perks.
-
A React hooks execution visualizer that sparked renewed interested in execution visualization efforts, because of the strictly harder context.
- Proof-of-concept visualizer, needs work
- AST explorer with the transpiler
- CodeSandbox with the React interop idea
The idea is to transpile ("lift") a piece of normal JavaScript code into one giant iterator, that performs two tasks simultaneously, while stepping through:
- execution the original code as it was;
- and yielding back execution information for visualization purposes.
For example, we'd take this code:
function app() {
const js = {
invented: 1995,
usage: "everywhere",
paradigms: [
"multi",
"event-driven",
"imperative",
"functional",
"object-oriented"
]
};
}and translate it into something like this:
function* app() {
yield _stm("VariableDeclaration", ...);
const js = yield _expr("ObjectExpression", ..., {
invented: yield _expr("NumericLiteral", ..., 1995),
usage: yield _expr("StringLiteral", ..., "everywhere"),
paradigms: yield _expr("ArrayLiteral", ..., [
yield _expr("StringLiteral", ..., "multi"),
yield _expr("StringLiteral", ..., "event-driven"),
yield _expr("StringLiteral", ..., "imperative"),
yield _expr("StringLiteral", ..., "functional"),
yield _expr("StringLiteral", ..., "object-oriented")
])
});
}At the places of the ellipses, we'd put information about the origin source location of the AST node, so that we would be able to highlight the relevant code while stepping through the program.
Important detail: we also need to transpile all user-defined functions into generator functions, and then yield over control to them when calling them. For example:
function* userFn() {
// ...
}
yield* userFn(...)This also means that generator functions are not supported in the user's code 🤷♀️ (until I spend some time cleverly crafting their transpilation somehow).
It was a bit tricky to get function calling to work as expected. JavaScript has an odd language feature where the semantics of a call expression depends on whether the callee is syntactically either a member expression or not. I call this syntactic call-site context binding. For example, o.f(41) sets this inside of the function to be o, whereas (yield _expr(..., o.f))(41), even though the callee still refers to the same function, sets this inside of the function to undefined (or whatever). To overcome this problem, you have to perform a more subtle transpilation. The end-result is as follows.
We transpile JS like this:
o.f(a, b);into stuff like this:
yield _expr("o.f(a, b)",
yield* (
(yield _expr("o.f", (yield _expr("o", TMP = o)).f))
.call(
TMP,
yield _expr("a", a),
yield _expr("b", b)
)
)
)Note the temporary storage of the object expression, because it might be a complex expression which should not be evaluated twice. Also, if the member expression is computed (with brackets [ ]), the computed property should also be transpiled correctly.
We're "lifting" the code to a different execution semantics. This also means that interop is broken unless cleverly avoided/fixed. There's two "directions" in which the interopability needs fixing: gettings results back from non-user-defined functions, and passing callbacks to other code. (Data is still data, so there's no worry there.)
The first problem is reasonably easily solved (disregarding some edge-cases that would require more work). We can just conditionally lift the result of function calls:
yield* _lift(r = (/* some call expression */))If the function call result is an iterator as a result of our own making, we can leave it be, and if it's just data, we simply lift it to be a trivial iterator:
(function*() {
return r;
})();The second problem is considerably harder, because we will be passing our lifted callback functions into all kinds of unknowing APIs:
(/* some array expression */).map(function* (number) {
yield _stm("ReturnExpression");
return yield _expr(...);
})I don't think there's any way to cleverly side-step this problem, we just have to make sure these APIs happen to work with iterators as well. For the sake of our visualizer though, that usually doesn't mean much more than the common standard library functions like Array::map, etc. And this is totally achievable with a bit of old-school (gotta love JavaScript!) monkey-patching:
const _Array_map = Array.prototype.map;
Array.prototype.map = function(callback) {
return (callback instanceof function*() {}.constructor
? function*(callback) {
console.log("!! using lifted version of Array::map");
const mapped = [];
for (let i = 0; i < this.length; i++) {
mapped[i] = yield* callback(this[i], i, this);
}
return mapped;
}
: _Array_map
).apply(this, arguments);
};
// etc.But then here's the kicker: I don't only want to use this visualizer for plain JavaScript execution, but also (separately) for the visualization of the React hooks render process. (To show the hooks' "hidden" underlying state, and how the whole function gets called on every render, etc.) This is another (quircky) case of an API that needs to be "lifted" to be able to work with iterators/generators. Because how do we deal with a situation like this?
function* MyTranspiledComponent({ someProp }) {
yield _stm("VariableDeclaration");
const [count, set_count] = ...;
// etc.
}
<MyTranspiledComponent /> // ???(Mathematically speaking, you might say we just run the transpiler on the whole React codebase. Although I admit it didn't even try, I can't imagine this would work, haha.)
I haven't solved this problem entirely yet, but my current idea is to make it work with a special-purpose transpilation step for component definition functions which basically just implements own versions of the most important hooks. (The real problem is not so much the delayed render, but the conditional/delayed hook calls. Hence the need to reimplement them.) This will look something like this (TENTATIVE SKETCH ALERT):
function MyTranspiledComponent(props) {
const [renders, set_renders] = React.useState([makeNewRenderState()]);
const [scheduled, set_scheduled] = React.useState(false);
const startRerender = () => {
set_renders([makeNewRenderState(), ...renders]);
set_scheduled(false);
};
function makeNewRenderState() {
return {
iterator: MyTranspiledComponent(props),
props,
i: 0,
hookno: -1,
done: false
};
}
const stateHooks = React.useRef({}).current;
function useState(initialValue) {
const key = ++renders[0].hookno;
console.log(
`useState#${key}`,
stateHooks[key] ? stateHooks[key][0] : "<new>"
);
if (!stateHooks[key]) {
stateHooks[key] = [
initialValue,
newValue => {
console.log(`setting state of useState#${key} to`, newValue);
stateHooks[key][0] = newValue;
set_scheduled(true);
}
];
}
return stateHooks[key];
}
// And similar for useEffect (and useMemo/useCallback/useRef/...)
function* MyTranspiledComponent({ someProp }) {
yield _stm("VariableDeclaration");
const [count, set_count] = ...;
// etc.
}
const step = () => {
const { done, value } = renders[0].iterator.next(renders[0].value);
set_renders([
{
...renders[0],
i: renders[0].i + 1,
done,
value
},
...renders.slice(1)
]);
};
const needsRerender = scheduled || _differentProps(renders[0].props, props);
const lastCompleteRender = renders.find(r => r.done);
// Either the minimal:
return lastCompleteRender ? lastCompleteRender.value : null;
// Or some stepper control UI:
return (
<div>
<div>
{lastCompleteRender ? (
lastCompleteRender.value
) : (
<span>Has not completed initial rendering yet</span>
)}
</div>
<p>
At {renders[0].i}{" "}
{renders[0].done ? (
<strong>DONE</strong>
) : (
<button onClick={step}>Step</button>
)}{" "}
{needsRerender &&
(renders[0].done ? (
<span>
<em>Needs a rerender</em>{" "}
<button onClick={startRerender}>Rerender now</button>
</span>
) : (
<em>Will need a rerender</em>
))}
</p>
</div>
);
}