In November 2018, SuperCollider 3.10 was released, and all was well. Although not an exciting release from a user standpoint, it involved an important and painful overhaul of the built-in help browser in the SuperCollider IDE. The help browser was based on QtWebKit, which Qt suddenly dropped in Qt 5.7 in favor of QtWebEngine. Migrating SuperCollider to QtWebEngine was quite a project, and we knew there'd be some bugs and regressions due to mistakes of our own and due to Qt breaking things.
The bugs rolled in. Around a dozen issues concerning the help browser, many of them duplicates of each other due to certain users not searching the issue tracker before writing tickets. The situation got so chaotic that I assembled a Help browser issues board to get this all straight in my head and help condense redundant issues. Most have been diligently addressed in the 3.10.3 patch release.
Audit Removable Storage
When I made the help browser issues board, one bug stuck out. A total of seven different people filed tickets for the same issue — a bug affecting windows, reporting that while sclang boots, the IDE can't run code and the help browser is stuck at "Sending Request...".
When this issue first came up, a rumor went around that there was a race condition between SCIDE and sclang, but nobody in the SC crew felt like taking a serious look at it. After gathering together all the duplicate reports, the importance of the issue became pretty clear and I nominated it as a critical issue for 3.10.4. James Surgenor and I began looking into it in September 2019.
GitHub user @rjsuni tracked down a specific security setting in Windows. Disabling it makes the bug go away. This security setting is much more likely to be enabled if you are on a machine restricted by a work or school IT department, such as a machine on the Microsoft Security Baseline. This explains why many Windows users aren't seeing this issue, but it's 100% reproducible for a small minority of users. When I enabled this security restriction on my machine, I was able to reproduce the bug.
The security setting in question is Audit Removable Storage, which "allows you to audit user attempts to access file system objects on a removable storage device." This is insane. I was able to reproduce the bug, but my setup doesn't involve any removable storage devices, and I assume that's also true of our bug reporters.
"Audit Removable Storage" doesn't seem to be the full answer, since other users reported that disabling this option doesn't fix the bug. Still, all these users seem to be on school or work machines. There are likely other security settings in Windows that get you stuck at "Sending Request..."
The Help Browser Deep Dive
It was speculated that there was a race condition somewhere in the help browser, so let's dig into the code and try to see if there's a deadlock.
- The IDE launches sclang as a subprocess known as Main::scProcess, an instance of ScProcess which inherits from Qt's QProcess class.
- scProcess sends the signal started() after the sclang subprocess has started running.
- The HelpBrowser widget listens for a started() signal in the slot onInterpreterStart,
- which calls HelpBrowser::goHome(),
- which calls HelpBrowser::sendRequest("HelpBrowser.goHome"),
- which displays the "Sending Request..." message,
- and then calls Main::scProcess()->evaluateCode("HelpBrowser.goHome"),
- which writes HelpBrowser.goHome to sclang's standard input, followed by the special byte 0x1b indicating end of input.
The infamous "Sending Request..." message is displayed by the HelpBrowser's mLoadProgressIndicator. Its API is simple: mLoadProgressIndicator->start() displays a message, and mLoadProgressIndicator->stop() clears any displayed message.
When does mLoadProgressIndicator->stop() get called? The four places where this method can get called are:
- In HelpBrowser::onScResponse, if the HelpBrowser receives a response while communicating with sclang.
- When the mWebView is finished loading.
- When the scProcess shuts down.
- When the class library is recompiled.
That's all sensible, but if none of these things ever happen, then nothing ever tells the mLoadProgressIndicator to stop displaying "Sending Request...". "Sending Request..." is a bit of bad UX, since it really means that the help browser has sent a request, but hasn't heard back from sclang.
So if IPC between the IDE and sclang has completely broken down, then the "Sending Request..." message is never cleared. Despite being the most obvious visual feedback in the case of an unresponsive IDE, it turns out that this isn't a help browser issue at all. "Sending Request..." looks like a deadlock in the help browser, but it's actually a symptom of IPC breakdown between the IDE and sclang.