About the new plans

[Winterhuman]

Just read https://n0.computer/blog/a-new-direction-for-iroh after investigating what the new Beetle repo is for, and wanted to express my interest in what you have planned.

I wanted to open this discussion about any overarching ideas you're thinking about that you're willing to share at this stage (and maybe fulfill that request at the end of the blog post)

[DougAnderson444]

I'm also curious about the level of changes. Which modules will change / versus / stay the same?

[Winterhuman]

I will make one prediction I'm confident with: Bitswap is being replaced. n0 have been very involved in the Move the Bytes working group, and they're talking about breaking interoperability with Kubo which doesn't have many non-libp2p features to go against, so my guess is that's the main target

[Winterhuman]

But, it'd be interesting to know if "breaking the IPFS spec" also involves breaking some of the Libp2p spec too, changes on that front are less clear to me

[DougAnderson444]

Isn't Bitswap pretty basic though? You either have the bits or you want the bits?

I am thinking it's the DHT. The DHT has been slow since the beginning, everyone complains about it (along with IPNS), perhaps they're thinking of a way to speed it up. I understand that Rust is much faster than Go, so perhaps they plan to leverage this fact.

[Winterhuman]

Bitswap is pretty rigid, and the subject of the working group, so it seems like it'd be thrown out first in favour of smarter fetching protocols (n0 also had to do a line-by-line conversion of go-bitswap to get it working since the spec wasn't enough, so that's not ideal either).

They did mention "Build bridges to other IPFS implementations." though, so I think you might be right about the DHT changing (plus they have videos on their YouTube channel looking at alternative DHTs)

[b5]

yep. Bitswap is already gone. As mentioned, we're keeping beetle because one day we want to build a bridge from Iroh to bitswap.

[SionoiS]

IPFS is the coolest thing until you use it then your enthusiasm goes down a notch.

Here's my wish list.

IPNS - ANY apps will need some way to mutate data at some point. Some want a bulletproof system others something fast. Maybe build 2 systems? Also, I want use my favorite cryptography, make the system flexible!

Pubsub - I want to sub to a 1000 topics because I'm a crazy person.

Data transfer - I want to live stream in AV1 encoded 4K and saturate my connection. I also want to sync big decentralized DBs.

DHT - Meridian for peer records and build a small-world DHT but based on "content distance" for everything else.

[bmann]

For IPNS, you may be interested in NNS https://talk.fission.codes/t/nns-the-name-name-system/3684 that @blaine is working on.

And yes, we're doing some early stage stuff around decentralized database.

We work closely with n0 at @fission-codes and generally are aligned on wanting a system that "just works" and scales.

[b5]

I've met with Blaine on NNS, excited to keep going on research!

I see no reason to implement IPNS in this new paradigm, because it solves for the wrong 2 parts of zookoo's triangle.

[Winterhuman]

To throw my thoughts in:

• CircuitV2 is vital: Having tons of nodes which are willing to send holepunch packets by default has hugely helped DHT health and making connections "just work", so having Iroh mimic this (and hopefully still interopt with Libp2p nodes to take advantage of the existing network) is pretty important.
• +1 on IPNS needing changes: It's been pretty underwhelming in usefulness, having a name system where you have to be online regularly for it to stay around has hurt it's adoption, so a more persistent & resilient method for mutability would be great.
• Signing for all: Peer records and Provider records are still unsigned, having these records signed by default (while still accepting unsigned records for now) would be a big security gain.
• IPLD: Yeah, it's no ones favourite to put bluntly, I have no idea what to suggest here.

[Zorlin]

IPLD: Yeah, it's no ones favourite to put bluntly, I have no idea what to suggest here.

IPLD is hugely important and good, IMHO, but has suffered from a lack of clear tutorials that teach you how the hell to see through its magic and get working with it. There's been some efforts to fix that (I've been vaguely involved, in a tiny way) but it's happening in fits and starts.

[Winterhuman]

Having DAG schemas which are commonly understood is crucial, but I don't think it's fully a docs issue (even if https://ipld.io/docs scares me). IPLD feels over-engineered, however, I've never been able to figure out why it's designed like how it is, so I can't contribute on how it should be done differently; it doesn't set out any particular goal beyond "make sure every DAG can be parsed".

[RangerMauve]

One thing to note too is that now that I'm no longer working on IPLD and with the recent layoffs, the entire project has been put into a standby mode with just basic maintenance. :P

There was a chance we could have had a working group running and some devs/writers to fix the onboarding, but I don't see that happening any time soon. At least not through PL initiatives directly.

[Winterhuman]

The IPFS team seem to be interested in a DHT called R5N (was mentioned on a Notion page), thought I'd mention it here since I know @b5 was looking into alternatives to Kademlia:

https://grothoff.org/christian/teaching/2011/2194/dht.pdf
https://lsd.gnunet.org/lsd0004/

[b5]

I had not seen this paper, thanks for the recommendation! The bloom filters for preventing routing loops is at least doing a decent job of highlighting the danger of routing loops. I'm not a massive fan of the approach, but we'll at least need to account for the problem 😄

[b5]

Thanks so much for kicking off this discussion @Winterhuman! We're working on a set of design documents, and will share it next week regardless of it's completion state. In the meantime, a few of our biases to help orient this discussion.

1. Our approach this go round is to make the absolute smallest thing we can that fits the majority of use cases.
2. BLAKE3 is our secure hashing function, we BLAKE3 Verified streaming for incremental verification. In Iroh there will be a single CID for every "file" in the system ("files" are just blobs of bytes). Iroh "chunks" are just blake intermediate hash values stored as outboard metadata.
3. We want to deal in bytes & sets. We'll build at least a primitive prefix-oriented selector syntax over sets. This means no DAGs inside of iroh, until we can find a use case sets can't solve.
4. No IPLD or UnixFS. decisions around pathing & data structuring are both pushed up into application space, where developers should have far more freedom do decide how things like pathing & data serialization work. You should totally be able to build all of IPLD on top of Iroh, but we won't bake it into the protocol.
5. We're moving away from libp2p. Here's our rationalization: https://number-zero.notion.site/Why-is-Iroh-moving-away-from-libp2p-57838618157f43c280b1fec381546604 I can copy-paste that link into a discussion if that helps.
6. New DHT. As mentioned, we think the DHT needs a re-think that collages existing literature into something purpose-built for content routing.
7. New Data transfer protocol. A primitive version of this should land in an iroh release this week.

Ok hopefully that's enough for now. More soon!

[b5]

oh did that sound super hand-wavy & not at all well defined 😇? Yeah, we haven't even started into work on that yet, so it's not even remotely figured out.

The baseline assumption though: if we're working in sets, there are many scenarios where you'll want to be able to run a query over that set & get a graph structure back, for example: building a file system out of sets using prefix queries.

I don't expect us to have a robust answer for this before July, but we'll need to get a sense of the cost (particularly at read time) of such a choice. Too high a cost for too many use cases may justify just baking in graphs.

[Winterhuman]

Just gonna bundle a few more questions:

• How are "sets" defined? Closest thing I can think of is the "2 week old paper" video
• Are HTTP gateways still part of the plan? Answered at iroh 2.0 - What about WebAssembly? #787 (comment)
• How is Iroh going to handle NATs? Answered at iroh 2.0 - What about WebAssembly? #787 (comment)

[Nuhvi]

@b5
Not sure if this is the place for discussing this, but I have a question about the Range-Based Set Reconciliation video: why do all this live recalculation when a Binary Merkle tree or a Radix trie can do the same thing but only compute the tree once on write?

So Want and Have can include the branch node for the target range.

What am I missing?

[dignifiedquire]

So Want and Have can include the branch node for the target range.

Usually this doesn't quite work out in practice for large trees/sets, because you don't know what side has which pieces when you are doing reconciliation, and so the amount of data you have to transfer (especially if you use hashes) becomes quite large.

[flokli]

Thanks so much for kicking off this discussion @Winterhuman! We're working on a set of design documents, and will share it next week regardless of it's completion state. In the meantime, a few of our biases to help orient this discussion.

1. Our approach this go round is to make the absolute smallest thing we can that fits the majority of use cases.
2. BLAKE3 is our secure hashing function, we BLAKE3 Verified streaming for incremental verification. In Iroh there will be a single CID for every "file" in the system ("files" are just blobs of bytes). Iroh "chunks" are just blake intermediate hash values stored as outboard metadata.

Using the blake3 hash of raw blobs is exactly what we've also settled on for tvix-store. Considering this, it should be relatively easy to plug in substitution over "the ipfs network".

I've also been experimenting with baos and various CDC to nicely deduplicate data transferred, as similar Nix store paths (two versions of the same package) contain a lot of mostly-similar blobs.

I'd love to exchange ideas about this, so we can ideally just use the same protocol.

3. We want to deal in bytes & sets. We'll build at least a primitive prefix-oriented selector syntax over sets. This means no DAGs inside of iroh, until we can find a use case sets can't solve.
4. No IPLD or UnixFS. decisions around pathing & data structuring are both pushed up into application space, where developers should have far more freedom do decide how things like pathing & data serialization work. You should totally be able to build all of IPLD on top of Iroh, but we won't bake it into the protocol.

We built a whole content-addressed "filesystem" (similar to git tree objects, but without the legacy cruft), pointers are referring to the blake3 hash of the canonical serialization of child "tree objects".

There is some potential to avoid N+1 query problems by having the other side traverse and push the whole closure (which would require some understanding of the data), but I /think/ this is just another performance optimization which could be done via a more specialized interface, and doesn't need to change the fact this is just blake3 hashes of blobs.

[andrewheadricke]

I've been wrestling with IPFS for many years, the project has such potential and the devs do their best to put a lot of flexibility and future proofing into their implementations. But i've always disliked the P2P stack, it doesn't seem scalable in its current form. I use CIDs / IPLD and a lot of IPFS code heavily but none of it on the main libp2p swarm.

I read a few comments on IPLD being over engineered or unnecessarily complicated, in my opinion they have tried to make it as flexible as possible whilst still providing a standard way for pathing / DAG traversal which I think is important to becoming the new web standard. Lots of competing DAG formats and pathing strategies could hinder adoption. I do like IPLD, even if their libraries are super complicated for doing easy tasks.

Anyway, keen to see what you come up with.