Having fast turn around on data of this size is immeasurably important and a significant competitive advantage. I have worked on large search-engine systems up until very recently and I can say without a doubt that turnaround time is more important than overall dataset size.

Our not-so-awesome mitigations were to have strictly defined preprocessed pipes to plug into. This meant having to either rigidly define queries or suck it up and wait hours, if not days for results.