Why an IDE?

Some time ago I wrote a reddit comment explaining the benefits of IDEs. Folks refer to it from time to time, so I decided to edit it into an article form. Enjoy!

I think I have a rather balanced perspective on IDEs. I used to be a heavy Emacs user (old config, current config). I worked at JetBrains on IntelliJ Rust for several years. I used evil mode and vim for a bit, and tried tmux and kakoune. Nowadays, I primarily use VS Code to develop rust-analyzer: LSP-based editor-independent IDE backend for Rust.

I will be focusing on IntelliJ family of IDEs, as I believe these are the most advanced IDEs today.

The main distinguishing feature of IntelliJ is semantic understanding of code. The core of IntelliJ is a compiler which parses, type checks and otherwise understands your code. PostIntelliJ is the canonical post about this. That article also refutes the claim that Smalltalk IDE is the best weve ever had.

Note that semantic understanding is mostly unrelated to the traditional interpretation of IDE as Integrated Development Environment. I personally dont feel that the Integrated bit is all that important. I commit&push from the command line using Julia scripts, rebase in magit, and do code reviews in a browser. If anything, theres an ample room for improvement for the integration bits. For me, I in IDE stands for intelligent, smart.

Keep in mind this terminology difference. I feel it is a common source of misunderstanding.Unix and command line can do anything an IDE can do is correct about integrated bits, but is wrong about semantical bits.

Traditional editors like Vim or Emacs understand programming languages very approximately, mostly via regular expressions. For me, this feels very wrong. Its common knowledge that HTML shall not be parsed with regex. Yet this is exactly what happens every time one does vim index.html with syntax highlighting on. I sincerely think that almost every syntax highlighter out there is wrong and we, as an industry, should do better. I also understand that this is a tall order, but I do my best to change the status quo here :-)

These are mostly theoretical concerns though. The question is, does semantic understanding help in practice? I am pretty sure that it is non-essential, especially for smaller code bases. My first non-trivial Rust program was written in Emacs, and it was fine. Most of rust-analyzer was written using pretty spartan IDE support. There are a lot of insanely-productive folks who are like sometimes I type vim, sometimes I type vi, they are sufficiently similar. Regex-based syntax highlighting and regex based fuzzy symbol search (ctags) get you a really long way.

However, I do believe that features unlocked by deep understanding of the language help. The funniest example here is extend/shrink selection. This features allows you to extend current selection to the next encompassing syntactic construct. Its the simplest feature a PostIntelliJ IDE can have, it only needs the parser. But it is sooo helpful when writing code, it just completely blows vims text objects out of the water, especially when combined with multiple cursors. In a sense, this is structural editing which works for text.

If you add further knowledge of the language into a mix, youll get the assists system: micro-refactoring which available in a particular context. For example, if the cursor is on a comma in a list of function arguments, you can alt + enter > swap arguments, and the order of arguments will be changed in the declaration and on various call-sites as well. (See this post to learn how assists are implemented).

These small dwim things add up to a really nice editing experience, where you mostly express the intention, and the IDE deals with boring syntactical aspects of code editing:

For larger projects, complex refactors are a huge time-saver. Doing project-wide renames and signature changes automatically and without thinking reduces the cost of keeping the code clean.

Another transformative experience is navigation. In IntelliJ, you generally dont open a file. Instead you think directly in terms of functions, types and modules, and navigate to those using file structure, goto symbol, to do definition/implementation/type, etc:

https://www.jetbrains.com/help/idea/navigating-through-the-source-code.html

When I used Emacs, I really admired its buffer management facilities, because they made opening a file I want a breeze. When I later switched to IntelliJ, I stopped thinking in terms of a set of opened files altogether. I disabled editor tabs and started using editor splits less often you dont need bookmarks if you can just find things.

For me, theres one aspect of traditional editors which is typically not matched in IDEs out of the box basic cursor motion. Using arrow keys for that is slow and flow-breaking, because one needs to move the hand from the home row. Even Emacs horrific C-p, C-n are a big improvement, and vims hjkl go even further. One fix here is to configure each tool to use your favorite shortcuts, but this is a whack-a-mole game. What I do is remapping CapsLock to act as an extra modifier, such that ijkl are arrow keys. (There are also keyboards with hardware support for this). This works in all applications the same way. Easy motion / ace jump functionality for jumping to any visible character is also handy, and usually is available via a plugin.

Recent advancements with LSP protocol promise to give one the best of both worlds, where semantic-aware backend and light-weight editor frontend are different processes, which can be mixed and matched. This is nice in theory, but not as nice in practice as IntelliJ yet, mostly because IntelliJ is way more polished.

To give a simple example, in IntelliJ for go to symbol by fuzzy name functionality, I can filter the search scope by:

VS Code and LSP simply do not have capabilities for such filters yet, they have to be bolted on using hacks. Support for LSP in other editors is even more hit-and-miss.

LSP did achieve a significant breakthrough it made people care about implementing IDE backends. Experience shows that re-engineering an existing compiler to power an IDE is often impossible, or isomorphic to a rewrite. How a compiler talks to an editor is the smaller problem. The hard one is building a compiler that can do IDE stuff in the first place. Check out this post for some of the technical details. Starting with this use-case in mind saves a lot of effort down the road.

This I think is a big deal. I hypothesize that the reason why IDEs do not completely dominate tooling landscape is the lack of good IDE backends.

If we look at the set of languages fairly popular recently, a significant fraction of them is dynamically typed: PHP, JavaScript, Python, Ruby. The helpfulness of an IDE for dynamically typed languages is severely limited: while approximations and heuristics can get you a long way, you still need humans in the loop to verify IDEs guesses.

Theres C++, but its templates are effectively dynamically typed, with exactly the same issues (and a very complex base language to boot). Curiously, C looks like a language for which implementing a near-perfect IDE is pretty feasible. I dont know why it didnt happen before CLion.

This leaves C# and Java. Indeed, these languages are dominated by IDEs. Theres a saying that you cant write Java without an IDE. I think it gets the causation direction backwards: Java is one of the few languages for which it is possible to implement a great IDE without great pain. Supporting evidence here is Go. According to survey results, text editors are stably declining in popularity in favor of IDEs.

I think this is because Go actually has good IDEs. This is possible because the language is sufficiently statically typed for an IDE to be a marked improvement. Additionally, the language is very simple, so the amount of work you need to put in to make a decent IDE is much lower than for other languages. If you have something like JavaScriptWell, you first need to build an alternative language for which you can actually implement an IDE (TypeScript) and only then you can build the IDE itself (VS Code).