Cloudflare and the Wayback Machine, joining forces for a more reliable Web

[mostakimvip04]

Cloudflare now populating and using the Internet Archive’s Wayback Machine in its content distribution network application
Cloudflare and the Internet Archive are now working together to help make the web more reliable. Websites that enable Cloudflare’s Always Online service will now have their content automatically archived, and if by chance the original host is not available to Cloudflare, then the Internet Archive will step in to make sure the pages get through to users.

Cloudflare has become core infrastructure for the Web, and we are glad we can be helpful in making a more reliable web for everyone.

“The Internet Archive’s Wayback Machine has an impressive recent mobile phone number data infrastructure that can archive the web at scale,” said Matthew Prince, co-founder and CEO of Cloudflare. “By working together, we can take another step toward making the Internet more resilient by stopping server issues for our customers and in turn from interrupting businesses and users online.”

For more than 20 years the Internet Archive’s Wayback Machine has been archiving much of the public Web, and making those archives available to journalists, researchers, activists, academics and the general public, in total to hundreds of thousands of people a day. To date more than 468 billion Web pages are available via the Wayback Machine and we are adding more than 1 billion new archived URLs/day.

We archive URLs that are identified via a variety of different methods, such as “crawling” from lists of millions of sites, as submitted by users via the Wayback Machine’s “Save Page Now” feature, added to Wikipedia articles, referenced in Tweets, and based on a number of other “signals” and sources, such multiple feeds of “news” stories.

An additional source of URLs we will preserve now originates from customers of Cloudflare’s Always Online service. As new URLs are added to sites that use that service they are submitted for archiving to the Wayback Machine. In some cases this will be the first time a URL will be seen by our system and result in a “First Archive” event.

In all cases those archived URLs will be available to anyone who uses the Wayback Machine.

By joining forces on this project we can do a better job of backing up more of the public Web, and in so doing help make the Web more useful and reliable.