Real-time search that goes all the way back

Eric Rumsey of Seeing the Picture posts that Twitter search is only extending back five days now, while it used to extend back at least ten. I wasn’t aware of that specific drop in half, but I had noticed in the last few days that after paging back in Twitter search results, I often hit a dead end, generally on page four or five or so with: “We’re givin’ ya all she’s got, Captain! Please wait a moment and try again.” I figured it was just like the fail wail, but now I wonder if this is related to the lessened archive-reach Eric has noticed.

Either way, I imagine this is volume-related, and only temporary. But whether it’s five days, or ten, or a month, the archiving of Tweets and the real-time Web is of great interest to us. Real-time engines are generally judged by how much, and how well, they capture what’s happening right now. But it’s also of real value to see what was happening “right now” three months ago. Especially when it was a helpful conversation that’s still relevant today.

With Replyz, which has been described as a specialized real-time search engine (focused on Q&A and helpful conversations), our archive of Q&A conversations at Twitter goes all the way back to launch. In fact, the vast majority of the Tweets and conversations found at Replyz can no longer be found through Twitter search. And we’re probably not alone in reaching further back into Twitter’s past than Twitter search itself does. In our case, we’re aiming for a super-deep and searchable archive of helpful Q&As (from Twitter and beyond) for search posterity.

  • Eric Rumsey

    Thanks for noticing my article. I've been seeing the reduction in Twitter Search from 10 days to 4-5 consistently for ca 3 months, so it's looking like it's more than temporary.

    Replyz looks interesting — I'll try it sometime.

    Eric Rumsey

  • Replyz

    Thanks Eric, you're right, that's a long time, so looks like it's staying put at 4-5. Hope to hear your thoughts Replyz and our approach to Twitter archiving sometime too. Thanks again.