Pre-RFC: User namespaces on crates.io

Created: 04/23/17 10:39 PM Resolved: 04/24/17 11:07 PM

So a little over 24h from the time the ticket was opened to resolution.

Last thing I'll say on this thread: the whole point is that OSSRH works by a JIRA ticket system. crates.io does not have one, nor does it have staff to provide this kind of support. It needs fully automated, immutable solutions.

Package systems like Maven Central which require this sort of human support staff aren't good examples for proposed changes to crates.io

(ticket link for those reading along: Loading...)

The ticket was created on 04/23/17 10:39 PM, with response by a human at sonatype by 04/23/17 10:39 PM asking for user action (first release + a comment). This happened at 04/25/17 04:06 PM. This is "about 2 days". The last action may be in the hands of the requester, but it's still part of the process. You could argue that the reviewer part is the end of the process at the side of sonatype, but in this case, you'd need to express this here, instead of just claiming that @bascule is factually wrong.

In general, looking at "namespace" tickets in the repository does show that sonatype is rather fast (good!), but quite a lot of them have had manual intervention - especially if people don't own a domain or need access to a namespace on behalf of their employer.

1 Like

Moderator note: Thanks for the extra details, everyone, but let's not dive any further into Sonatype's exact process unless absolutely necessary. Please steer this back to specific lessons/suggestions for the Rust project.

6 Likes

(post deleted by author)

It does not work on Windows, as I found out a few months ago, pretty painfully. (for something totally unrelated)

1 Like

I've read this thread, and I do not think they have been shown incorrect at all. I wonder what you are referring to here.

Moreover, "names must be immutable" is a value proposition, a design goal, it's not per se a statement that can be true or false. I doubt you'll convince anyone by telling them that their design goal is "incorrect". The benefits of this design goal have been stated repeatedly (most importantly: no "left-pad"-style issues), and I haven't seen good arguments against it in this thread. You seem to value proper immutable names less than other concerns, which is fair, but what is not fair is expecting everyone to agree with that.

Anything DNS-based will either end up in a situation where you own a DNS name but not the corresponding cargo namespace (as a previous owner of that hostname already used the cargo namespace), or else ownership of a package changes suddenly and without an action by the previous owner when ownership of the DNS name changes. Ergo, using DNS is a no-go for an immutable registry like crates.io.

3 Likes

I read and quoted that comment. It did not contain any arguments for how to achieve immutable names with DNS, or for why using DNS has sufficiently many benefits to make it worth dropping immutable names.

2 Likes

One problem with a DNS-based solution: domain names cost money. Would we really be comfortable telling someone if they want a namespace then they need to pay up?

Link? I don't recall seeing that anywhere in this thread, and imo it's a major issue.

There’s plenty of public suffixes, like github.io, neocities.org, and js.org, for anyone that wants a domain for free.

I'm not seeing a good rationale for using domain names. If a crate namespace owner has to prove that they own a particular web domain, what problem does that solve? How is it better than just using arbitrary identifiers?

Also, since many domain owners shield their ownership from public view by using a privacy service (e.g., https://www.dreamhost.com/domains/private-registration/), how is ownership to be proved in such cases?

Moderator note: If you don't want to discuss it any more, then don't. Announcing that you're done with a thread like that is already needlessly dramatic. Continuing to participate afterward only makes it look worse.

Telling everyone "this has been addressed" like a broken record is the kind of behaviour that gets you in trouble. You are not acting your best here.

12 Likes

I agree With @burntsushi's comment above regarding high level design - seems like most "Pre-FRC" style discussions waste a lot of energy on implementation details and cause a lot of redundant and fruitless frustration. As the saying goes "Can't see the forest for the trees".

So let's start from first principles and define our goals. Here's my take:

  1. As a Rust developer I'd like to be able to easily find and "install" crates.
  2. As a Rust developer I'd like to see a clear retention policy and local caching measures so that I can avoid the "left-pad" fiasco.
  3. As a Rust developer I'd like to make available crates I own to be searchable and usable by other developers, while retaining ownership and control over the code and associated policies (licensing, retention policy, etc.)
  4. As a rust developer I should be able to change my hosting provider, be able to rename my project or transfer ownership to someone else without impacting other users unreasonably.

Non goals:

  1. Crate names do not have to be globally unique - global properties are expensive in complexity terms. It is sufficient to require that the local set of crates on my machine is unique (local reasoning is a better design). For example, this allows to have mirrors and backups and competing crates (think: "facebook/log" vs. "google/log"). Two crates named "foo" would conflict and therefore Cargo should fetch only a single instance of "foo" after resolving my mirror list for the crate "foo". In case of name conflict, I, the user, must resolve the conflict locally by e.g. renaming one of the crates.
  2. Immutable repository/global singular repository/ etc - These are all unnecessary and they complicate other aspects of the design. Keeping crates indefinitely is an overreach and raises legal questions. It is sufficient to have a clear retention policy that is well publicised and explained to users. Currently, cargo downloads sources from crates.io and compiles locally and the sources are stored on github. This is both redundant if my repo is already on github, and insufficient if I prefer to use a different provider or if github T&Cs conflict with my circumstances. Also, A federated-like system where I retain control over my own code is better suited to confer ownership. it is sufficient if crates.io can search and redirect users to my own repository from which I vendor my code. Ideally, there should be some cloud-like hosting solutions to alleviate some of the hassles involved.

As alluded to above, we need some aspects of a federated system. For starters, It confers trust explicitly in my local list of trusted repositories. crates.io would not own the namespace in any way, it simply provides a plugable way for me to add my repository with my retention and other policies available to be searched. The user must make an explicit decision to trust my repository and accept my policies before downloading artefacts. If I'm an OSS project owner I can allow crates.io to be an official mirror and cache my artefacts or if I work for a company such as google I may very well prefer to host my artefacts on company servers with company retention policies. Instead of registering the company name/user name/project name/dns addres on creates.io I rather have an API to plug-in my repo to the official index and make it explicit to users that the artifacts are from my repo and adhere to my retention policies.

5 Likes

To add to the above, I'd envision the following scenario:

Let's say, I have a project with multiple crates that, for instance, is managed under a github organisation. I should be able to fork a template crate.io style repo project and host it at my_project.github.io. I would than need to connect my "my_project.github.io" repo to crates.io. I can then explicitly set my retention policy so that crate.io will retain a cache of artefacts.

My crates will then be available to crate.io website users with a web-page that explicitly explains the above retention policy and a link to a "port-of-origin" (my_project.github.io) as well as instructions how to add the repo. The repo itself should also be searchable by name.

Users would need to explicitly add "my_project.github.io" to their trusted repos list. users can then depend on "my_project/foo" which is mapped locally to my_project.github.io.

There need to be a certificate or something that both crates.io itself and the end user both use to identify my crates, which would allow cargo to resolve to other mirrors (including crates.io itself) in case my repo isn't reachable.

I really like the reframing of the discussion as problems and use-cases to solve.

Namespaces are a particular solution, but they aren't a problem to solve.

This hits the XY problem pattern of miscommunication. We've had a lot of posts written from the perspective of "I want this solution, here's how you implement it", but without explicit definition of why, and what problem does it solve.

Maybe the same problems that are solved with namespaces could be solved in different ways? Probably not one single thing, but a combination of them.

  • Cargo already supports package aliases, names of libraries independent of crate names, and alternative registries. Those are existing tools to work with. If they're not sufficient, maybe they'd work if they were extended/improved?

  • Crates are tied to crates.io website. The website could do a lot more to display ownership, groups of crates, TOML snippets with package aliases.

  • There's crates.io API with ownership information. Crates can verifiably associate themselves with GitHub repos via repository link. There are many ways in which tooling could surface and leverage that information.

Please brainstorm ideas around these things, instead of assuming a particular solution from the beginning and then trying to jam it in.

6 Likes

I mean, no it isn't an extraordinary claim. Immutably is not actually difficult -- just do not provide any way to mutate something (except if the original crate owner explicitly transfers ownership). That's what crates.io does. End of evidence. Contrast this with DNS where changing the owner of a domain without the consent of the old owner is an explicit feature called "domain expiry". (Not sure what platonic ideals have to do with this though, that sounds like a strawman.)

But anyway, since your idea of "evidence" seems to be "it has been addressed [somewhere on the internet but I won't tell you where so you have no way to verify]" (many of your comments do not have links to back up your claim, like this and this and this), which is not evidence at all, and also since you seem unable to accept that different viewpoints than yours can legitimately exist, there is no point in continuing this.

11 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.