Twitter now indexes every tweet ever

Gigaom

Twitter nows lets users search for every public tweet since the service was launched in 2006, the company announced via blog post on Tuesday. This capability was a long time coming, and was no doubt made more difficult as Twitter’s use began to grow in the past couple years.

The post goes into a lot of detail about how Twitter built its new historical search index, but the main challenges are obvious to anyone who has followed the evolution of Twitter’s infrastructure (or Facebook’s, Google’s or any other large web service’s infrastructure) over the years — speed, scale and cost. According to the post, the full search index now includes “roughly half a trillion documents” and “is more than 100 times larger than our real-time index and grows by several billion Tweets a week.”

One aspects of Twitter's new historical index, which shards tweet records based on when they were produce. Source: Twitter One aspects of Twitter’s new historical index, which shards tweet records based on when they were produce…

Ver la entrada original 241 palabras más

Anuncios

Responder

Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Cerrar sesión / Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Cerrar sesión / Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Cerrar sesión / Cambiar )

Google+ photo

Estás comentando usando tu cuenta de Google+. Cerrar sesión / Cambiar )

Conectando a %s