Skip to content
This repository was archived by the owner on Apr 16, 2020. It is now read-only.

What would decentralized publishing on IPFS for npm actually look like? #58

Description

@andrew

There was some talk of decentralized publishing for npm-on-ipfs on the IPFS GUI call yesterday as a potentially interesting area to explore.

After exploring some decentralized package manager ideas I thought I'd try to apply some of them to an npm implementation.

Assuming ipfs client support can be added to npm or yarn and that IPNS isn't slow (or DNSLink is used), how does one go about publishing a package directly to IPFS and then having someone else consume it?

One key aim here is to not require hosted infrastructure, can we just use standard IPFS tooling and keep the logic in the clients?

Also of note, none of these solve the discovery problem of "where do I find the decentrally published packages", which likely would be an opt-in search engine where publishing tools announce the cid of new published packages to be indexed.

Level 1

The simplest option would be to add the a tarball directly to ipfs:

ipfs add big-number-1.1.0.tgz

Then any user could specify it as a dependency in package.json:

{
  "name": "my-module",
  "dependencies": {
    "big-number": "ipfs://Qmfoo",
  }
}

Downsides to this:

  • hardcoding specific version
  • no update mechanism

Level 2

To add an update mechanism to level one, you could use ipns:

ipfs name publish $(ipfs add big-number-1.1.0.tgz)

Then any user could specify it as a dependency in package.json:

{
  "name": "my-module",
  "dependencies": {
    "big-number": "ipns://Qmfoo",
  }
}

Then when publishing a new version, just update ipns to point to the new one:

ipfs name publish $(ipfs add big-number-1.1.1.tgz)

Downsides to this:

  • end user can't control which version they are getting
  • may cause integrity errors with previous versions stored in lockfiles

Level 3

To allow end users to control which version they are installing, publish a modified index file (eg bignumber) along side the tarball.

Rather than adding a new field cid, like npm-on-ipfs does, instead make the tarball field the cid of that versions tarball:

"dist": {
  "shasum": "e6ab0a743da5f3ea018e5c17597d121f7868c159",
  "tarball": "ipfs://bafybeihmelfwcg664jeznipvrx2qzc6acrme5ztea2r4rljdqzrj72bx6u"
}

As in level 2, using an IPNS name for the index will allow for an update mechanism:

ipfs name publish $(ipfs add bignumber)

Then any user could specify it as a dependency in package.json:

{
  "name": "my-module",
  "dependencies": {
    "big-number": "ipns://Qmfoo^1.1.0",
  }
}

Missing parts:

  • A tool for generating an index file (packument) locally for one or more versions of package

Downsides:

  • Referencing an ipns hash in package.json isn't very human readable and provides no information about who the publisher of it is
  • By default dependencies will still be installed from npmjs.org over http, unless someone goes through and manually republishes all dependencies with their dependencies declared as ipns hashes as well

Level 4

Publishing a small registry directly to IPFS.

npm-on-ipfs has already shown how you can setup an npm registry on ipfs, by publishing multiple versions indexes (packuments) and adding them to a folder (or mfs), an IPFS address can then be used as a registry rather than a the location for one particular package.

small registries could be collections of useful modules for individual personal use, but the most likely use case is per-project registries that contain all the dependencies for that application.

Command to point your local npm client at an ipfs registry

npm config set registry ipns://QmT8wUy9CV3FHbjQeDG49ZL7at24u5fynBrkjum4MznNA7

The setting can also be added to a .npmrc file which can be commited in a repository so that all users of that project use the same registry:

registry=ipns://QmT8wUy9CV3FHbjQeDG49ZL7at24u5fynBrkjum4MznNA7

Then packages can be referenced in your package json using regular names:

{
  "name": "my-module",
  "dependencies": {
    "big-number": "^1.1.0",
  }
}

This has the added benefit that all sub-dependencies will also be pulled from IPFS automatically, but will require the publisher of that registry to ensure they have added all required versions of every sub-dependency to that IPFS folder before using it.

Missing parts:

  • A tool for generating an index file (packument) locally for one or more versions of package
  • A tool that uploads a number of generated packuments and tarballs to an IPFS folder (would probably want to also pin the root on a pinning service to ensure future availability)
  • A tool that monitors upstream sources of third party packages added to a registry and provides a way to bring in new releases and republish to IPFS

Downsides:

  • requires all end users of repos with a ipfs .npmrc to use IPFS for installing modules, as they won't be able to get the same packages from npmjs.org
  • only the one owner of the IPNS can publish new versions of the registry, which is restrictive for teams of multiple people
  • all other features that involve communicating with a http registry would be broken, only fetching packuments would work
  • having the registry config being a hash isn't very human friendly, would likely want to use DNSlink to make it friendlier

Note:

  • I'm not sure if @achingbrain's PR to npm would work with pointing whole registries at ipfs addresses, needs further investigation

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions