Proposal: Modifying PaperWM to utilize the benefits of TypeScript and LSP

[lost-melody]

Context

I've been delving into the codes of PaperWM for some time now, trying to implement certain tweaks. However I'm having some difficulties in comprehending and modifying the codes. I've spent much time in navigating back and forth, trying to locate the declaration/initialization of a variable (e.g. its type and available fields) or to figure out the usage of a function or a class (e.g. arguments that an api accepts).

According to GJS Guid/TypeScript and LSP, it is now possible to develop gnome extensions with LSP support, so that editors would provide auto-completion, hover-documentation and diagnostics. I believe most of us are used to coding with code completions and diagnostic warnings so it should help.

Efforts & Problems

As I need the support of LSP for coding, I spared some time trying to let it work. I'm not an expert in TypeScript so what I can do is simply to follow the steps of the guide and to say "wow it works", then navigating to every diagnostic warning and trying to fix it. I renamed every *.js file to *.ts and moved them into the src/ directory, and then tweaked a couple of files to eliminate their warnings. Mostly it works as expected:

Code completion (screenshot)

API documentation (screenshot)

Still, there are pieces that I can't fix:

Undeclared imports (screenshot)

According to their source codes, I found that the file utils.js exists in GNOME Shell/js/misc while does not exist in Gjsify/gnome-shell/src/misc. So there are libraries that do exist in GNOME Shell and can be imported and used, but are not declared in @girs modules.

Next Steps

I believe it's possible to modify PaperWM to use TypeScript but there's still so much work to do. And I think there should be one who's proficient in it to lead this work. At least these tasks should be done in my eyes:

• Follow GJS's guide, create tsconfig.json, makefile, etc.
• Update scripts install.sh, uninstall.sh, etc.
• Move files *.js to *.ts and fix those LSP warnings, solving those undefined imports.
• Supplement type declarations of object fields which are currently confusing.

I've put forward this proposal with the hope that it will be realized. However, upon further consideration, I recognized my limitations in overcoming all these challenges above. Looking forward to your ideas.

[Lythenas]

In general I wouldn't be opposed to this. I also thought about it in the past but I decided that there were to many problems. But if gnome now officially supports this and has a guide for it, it might be worth it.

One issue that we will have if we move to typescript is that anyone that the extension will break for anyone that still has the extension installed through the git repo and pulls. But I think that is a minor issue now, since most people probably have it installed through EGO.

Regarding the missing util.js it is probably just a matter of adding the file to the Gjsify/gnome-shell repo. It says it is still experimental and there are missing types.

Do you have the code pushed somewhere? Then I might take another look at how much is still missing.

[lost-melody]

Regarding the missing util.js it is probably just a matter of adding the file to the Gjsify/gnome-shell repo.

Yes, I agree. But what I'm concerning is the case that these files are not designed to be exposed and are not planned to be added eventually. If so, then I think we'll need to find some other ways.

Do you have the code pushed somewhere?

I've dealt with only few files currently but I've pushed my changes here if it helps: lost-melody/PaperWM:gjs-ts. Besides the base configurations for TypeScript, I've only submitted changes in file extension.ts, since it's too much work for me to finish all those files. If you are going to test it, pull my branch and run npm install, and then it should work hopefully.

[Lythenas]

I poked a little around in the code. Here are my findings so far, in no particular order:

• Converting properly to typescript will be quite a bit of effort and it should be done (or at least thoroughly reviewed) by someone who understands the code well
• I think it would make the code a lot more clear in many areas because often I have the problem that I have absolutely no idea what the content (i.e. type) of the variable is at run time. Simplest example would be: You read a config value and pass it along somewhere, what type is it.
• prefs.js might make sense to just rewrite in code instead of loading the .ui files (same way they do it in the TS guide for the extensions). With a separate .ui file you have all the builder.get_object calls and need to cast them to a type.
• For a gradual transition we can
  • keep most files as *.js and turn off noImplicitAny
  • then gradually (i.e. across multiple PRs, over time) convert the files to type script and add types
  • eventually turn off noImplicitAny
• We need to fix some missing type definitions for files that we use (e.g. resource:///org/gnome/shell/misc/util.js). This can be contributes quite easily to https://github.com/gjsify/gnome-shell, I believe. But is a bit of effort.
• I noticed some methods had different names (e.g. Variant.deepUnpack vs Variant.deep_unpack). Not sure if that is a bug in our code or an error in the type declarations.
• I'm getting some strange errors about duplicate types in node_modules/@girs/atk-1.0/atk-1.0-ambient.d.ts but it still compiles. This should probably be fixed upstream before we continue here
• At the moment there is no version for Gnome 46 yet. So when updating to future versions of gnome we might be blocked waiting on the types to be updated (if there are changes). Although I think you could work around most of this by just casting to any until the types are updated.

My code is at: https://github.com/Lythenas/PaperWM/tree/gjs-ts

Before I spend any more time on this. What do you think about this @jtaala

[jtaala]

Hey @Lythenas,

I've been watching this one. A few points/thoughts from me (in no particular order):

• I'm not familiar with, nor have I used, TypeScript;
• I think I'd be able to pick it up fine, but tbh, I've got limited time as it is to work on PaperWM, and am not motivated to put much/any effort in to converting to parts to TypeScript (probably because I don't know / use TS);
• LSP would be (very) nice, but it's not a showstopper for me, and wouldn't justify for me the very large amount of work required to convert PaperWM modules to TS;
• it still looks very experimental on the Gnome side, and I've been stung (become jaded!) by quite a few large rewrites & refactors of all the PaperWM modules (e.g. Gnome 45 being the last one - about 11,000+ line changes). This comes with a inordinate amount of testing etc. I'm not interested in putting myself through that just to get LSP and other TS things (although I admit I'm not familiar with the advantages etc.);
• I'm happy if someone wants to take this on, but sounds like it would be better as a fork - that way, it could progress by those with the right skillsets / motivations. If it progresses and proves useful (and if/when I get into TS) we could look at bringing all those changes in. Being upfront though, I don't forsee me having the motivation / time to learn TS from a PaperWM pov.

Jay.

[Lythenas]

Hi @jtaala thanks for your comments. That's about what I expected :D

I use typescript sometimes in my job, mainly for react. And even for react (that has a big community and good support), the types are sometimes annoying... If you are less strict about types (do more type casting, use any often, etc.) this is less of a problem.

it still looks very experimental on the Gnome side

I think you are right about that. I'm fine if we delay this until e.g. gnome 47 or later and then check again. I'm also still a bit confused by some things e.g. some names are in camelCase instead of snake_case (global.display.focusWindow and global.display.focus_window both work in looking glass), Meta.Window.clone is missing etc. I think they are just wrong on the type definitions. I think that may be because a lot of types are generated from GObject introspection and not from the actual available fields in JS.

But besides that, I think there is a way for us to use the lsp support without actually using typescript. Just putting the doc comments with the types in the js file, gets picked up by the LSP. And for everything that is imported you also get the types.

IMO that is already helpful, even if we don't actually have any more typechecking during build.

I will try to look into how to set this up in our current project structure. More doc comments and some config file probably won't hurt anything. This way we can try starting to use the LSP and maybe help improve the types along the way.
With this, in the worst case you would get no or wrong types on hover and incorrect autocompletion.

[smichel17]

I'm here because I had an idea for a behavior change, went to add it, and immediately missed types very much 😅

@Thesola10 How do you feel about typescript?

I'm a big fan of static types. The main benefit compared to doc comments is that when you make (certain types of) mistakes, the type checker will warn you about them, and the code would fail to "compile"* to javascript. I find this significantly reduces the number of cycles of manual testing I need to do. Importantly, it also means the types can't get out of date like doc types do. E.g.

PaperWM/tiling.js

Lines 5201 to 5208 in 2b565b2

	/**
	* "Slurps" a window into the currently active column, vertically
	* stacking it.
	* @param {Meta.Window} metaWindow
	* @param {Boolean} below
	* @returns
	*/
	export function slurp(metaWindow, insertAt = SlurpInsertPosition.BOTTOM) {

I'd be happy to help with a transition to Typescript. Most of my professional work has been in typescript (4-5 years depending on how you want to count) and I am very comfortable with all but its most advanced features (E.g. {in/co/contra}variance still sometimes trips me up — usually the error involving "X could be instantiated with a different subtype of Y").

The main risk is around how incomplete the gnome-shell types may be. If we constantly need to suppress errors (e.g. accessing a function which the types don't know exists) then it is significantly less useful and can be a time sink. On the other hand, any improvements we can contribute back upstream help everyone.

*Quotes because it mostly just strips the types and outputs the rest verbatim. If we switched to TS and wanted to switch back, it would be a relatively easy process. In general I would not hesitate to maintain javascript code generated by the typescript compiler.

[Thesola10]

I've seen projects use gnome-shell types and I think we can already get reasonably far with what is already defined. My main worry however is that it introduces a build system, which in JS isn't always all that reliable, but we should be pretty frugal in dependencies for now.

I don't know how much of our current code has doc types, but a good start would be to transition mostly- or fully-documented modules to Typescript and feed them to the type checker in its own branch, and eventually move to migrate everything.

This means we need to get ahead of open PRs because switching to Typescript will make them all outdated and their respective authors will have to update them to the new project structure.

We should be careful and avoid the temptation of a full rewrite. I've seen too many OSS projects get bogged down and fail doing just that.