Exploring the Second Order Sparsity in Large Scale Optimization