Skip to main content
Internet Archive's 25th Anniversary Logo

View Post [edit]

Poster: Brak Date: Jan 30, 2005 2:31am
Forum: web Subject: Re: Full-text search of the Wayback archive

Now that's a fascinating idea. It's not the reason it was taken down. We hope to have that functionality available again.

Doing a distributed index sounds like a worthy p2p project for all sorts of reasons. The problem with projects of this scale might be the sheer size of the data. If the index were, let's say, 25 Terabytes, then if each user donated 20GB of hard drive space, it would take, in theory, a minimum of 1250 users, and more realistically 5000 users (taking into account redundancy and load sharing). That's a taller order than most p2p apps. Anything is possible, however.

For now, stay tuned...