Skip to main content
Internet Archive's 25th Anniversary Logo

View Post [edit]

Poster: Nethacker Date: Apr 28, 2006 8:19pm
Forum: forums Subject: Distributing data over p2p networks

I don't know if anybody at the archive already thought about it, but why don't you offer Bittorrent or eDonkey links for the files? eDonkey would be suitable for all types of files, Bittorrent could be used for bigger files like video or audio data, because of the torrent file you have to download first. The whole thing would take off a huge part of the load hitting your servers. Content-distribution networks like CoralCDN (http://coraldn.org) could be used for the web archive.

Reply [edit]

Poster: Diana Hamilton Date: Apr 28, 2006 10:39pm
Forum: forums Subject: Re: Distributing data over p2p networks

Bittorrent come up before, and there was even a BT experiment here on one collection a few years ago. One of the big problems is that individual items don't sustain the all-at-once, multiuser interest that makes BT work so well. So IA would be dishing out copies anyway (but I think with higher overhead?).

As for other routes, I've seen that a lot of items here do have have Magnet Links on them allowing you to get the item via P2P services.
http://www.archive.org/about/faq.php?faq_id=192

This post was modified by Diana Hamilton on 2006-04-29 05:39:07

Reply [edit]

Poster: Nethacker Date: Apr 29, 2006 8:20pm
Forum: forums Subject: Re: Distributing data over p2p networks

So why don't all of items have Magnet links? Can adding Magnet links to all items be done automatically? If yes, why hasn't it be done yet?

Reply [edit]

Poster: Diana Hamilton Date: Apr 30, 2006 1:38am
Forum: forums Subject: Re: Distributing data over p2p networks

I don't know, good question- perhaps it's in the smaller-scale experiment stage, and/or was only put on some high-traffic items or collections?

Reply [edit]

Poster: Nethacker Date: Jun 6, 2006 8:08am
Forum: forums Subject: Re: Distributing data over p2p networks

An even better solution could be the use of metalinks. (www.metalinker.org) It uses an XML file that contains information about the file, publisher, hashes, and so on. It supports multiple download resources, currently HTTP, FTP, Magnetlinks, ed2k links und Bittorrent are supported. With a client supporting Metalink, it is possible to load a file from all the different resources in parallel.