Socket npm Wrapper Feedback Update

Whenever you release something you wonder how people want to use it, how they will use it, and what doesn't work out. To that end, we released a feedback phase for socket npm to understand this better. During that feedback phase we didn't fully integrate against a few things so that the product could focus on feature feedback, particularly on what might not be working. Luckily, the feedback phase of our initial release of socket npm has been a great success!

We wanted to give and update on the ups and downs of socket npm that we have seen so far and what we are looking at to address them! These will mostly be technical but will try to explain some directions we are seeing to drive the product towards as well.

Aliasing ups and downs#

We documented and support aliasing npm to socket-npm to some great usability gains, but we saw some cracks in this approach. Some of this was mitigated by shipping the socket-npm executable to shortcut calling socket npm for tools expecting a single executable to work but other issues did show up.

• Aliasing requires local user configuration, there was some concern about this, and instead some minor feedback about wanting to just install globally did occur. Doing so would likely want us to vendor npm inside of socket-npm and prevent calling the normal npm .
• Some tools actually are using shell tooling to parse things and don't understand a tool named socket or socket-npm shadowing npm using an alias. This would be solved by having a means to install globally and clobber the normal npm as well.

What is causing the error?#

Whenever you integrate against another tool like npm there is a chance for the tools to conflict. There was some potential for confusion on where a message is coming from so we are investigating things like message prefixing to differentiate and avoid confusion. This is important for people to discover the source and potential fix for any issues they find.

Is my wrapper up to date?#

We didn't put the update notification banner on the wrapper around npm because it could be a bit confusing as to what is going on, however, it became apparent with this release that keeping things up to date for both npm and Socket's CLI are both desirable so we have now included it when running the wrapper.

OSS always finds a way (to use install scripts)#

We tried to fully cover all the ways npm can install things and prevent any suspicious behavior, but our expectation of what was suspicious actually came up as an incorrect assumption. In particular, people like to use npx and npm inside of scripts in a few ways that we would intentionally prevent. This leads to a couple of things.

One was that often when using npx it actually ends up doing a global install unless you are careful. Avoid using npx -p to avoid this accident usually, but it is in the wild and we saw a few issues come back to us about it. We are allowing this global install to be performed but likely will need to refine the messages shown to explain to users what is going on when swapping from local to global installations and perhaps add some flags. As always, protect from accidental global installs when possible using OS provided protections like directory permissions if desired.

This next one will take a bit of explaining. When using scripts to install other modules, npm normally will spawn the scripts without access to the terminal's TTY interface:

This behavior can be changed using npm --foreground-scripts but it has all sorts of other affects like sequentializing and slowing down installations. Additionally it can create a lot of noise by not hiding things like diagnostic messages in install scripts:

socket npm would to what we thought was the correct user experience intentionally error out in this case because it would not have a way to prompt for if something was ok to perform. However, we noticed a number of these surprisingly needed to get prompted during install scripts. This early error when unable to prompt differs from tools like npx which only prompt when they can; otherwise, they continue operating normally. We still wanted to error before any risk was accepted so we had to figure out a solution.

We didn't need everything to run in the foreground to get STDIO actually, just the prompting. If you look at how other tools do things, we took a fairly standard approach (like what VSCode does) of dropping an IPC server down that can be called back to using an environment variable and socket. This IPC server will block concurrent prompting as a sequential queue to capture STDIO by manually piping it through the root process until it is done but leaving scripts operating normally instead of being brought to the foreground:

Doing this we encountered a few gross things to work around, particularly around having conflicting CLI spinners fighting to show up on the same line, but we worked around them by pausing the root process spinner while a different process captured STDIO. We are excited about the IPC server and other potential uses it may provide us with, but will need another round of feedback around it.

The basics of how the IPC server works is the nested subprocess creates a pipe to the root installation process using a filesystem based pipe and waits until the root process is not in the middle of requesting a prompt of some kind. The root process could be prompting from a root process reason or because a different subprocess is also needing to perform a prompt. When there is no pending prompt workflow, the subprocess "captures" STDIO by a means similar in purpose to what setPointerCapture does, diverting all STDIO to the subprocess and pausing / preventing events for the normal root process handlers. Once the subprocess is done, it closes the connection and the root process releases STDIO for any new prompts to capture. It is vitally important to not share STDIO among subprocesses as it would cause flickering for things like CLI spinners or garbled text from async concurrent writes to the terminal.

Feature Requests#

We got feedback and questions about a few features to support.

• Other package managers

We have an incredible amount of requests not just for alternative JS package managers like pnpm and yarn but also for other ecosystems like Python and Ruby. We are actively looking at supporting these and talking to some maintainers of these projects for their feelings on doing so. These are actively being pursued.

• Telemetry

People are interesting in how often a specific risk is accepted or not accepted when using socket-npm. The current implementation does not gather any telemetry data but we are looking at an opt-in way to enable telemetry. This likely needs a bit of care, but we need to watch out around PII concerns and how we want to manage this data. This work hasn't been started and needs more research.

• Organization Settings

During feedback we have not enabled organization settings synchronization so all issues should be reproducible during that feedback. However, people are hesitant to use this without integration around organization settings. We plan on finishing up our feedback phase soon and then will be integrating against this, unblocking those users.

• Install script configurations

Install scripts are always contentious regarding usability versus maximal security. We actually faced criticism in both ways of wanting more usability with less interruption but also wanting more interruption instead of being quiet. These come down to user configuration missing around how people want their tools to function. Noise to signal configuration is a great configuration option that wouldn't vary the behavior of tooling in a way that could complicate things terribly for debugging.

• Secondary integrations

Some tools like npq and lavamoat can still function with our wrapper but the experience suffers a little, just like how we faced problems leading to our IPC server. We are looking at how to better build a generic plugin system that can be reused across tooling as we discover more nuance to the experience. Doing so will likely be very generic and allow custom tools to be written against our integrations but is not ready for further discussion at this time.

We will be winding down the feedback phase soon as we finish up what customers are specifically requesting and then be moving forward with some of the newer features. This is an exciting time and we will be wanting to continually improve but feel we are at a good state. Keep watching for more updates as this progressing and as always we are open to more feedback!