Over the last few years, holography has been emerging as an alternative to stereoscopic imaging since it provides users with the most realistic and comfortable three-dimensional (3D) experience. However, high quality holograms enabling a free-viewpoint visualization contain tremendous amount of data. Therefore, a user willing to access to a remote hologram repository would face high downloading time, even with high speed networks. To reduce transmission time, a joint viewpoint-quality scalable compression scheme is proposed. At the encoder side, the hologram is first decomposed into a sparse set of diffracted light rays using Matching Pursuit over a Gabor atoms dictionary. Then, the atoms corresponding to a given user's viewpoint are selected to form a sub-hologram. Finally, the pruned atoms are sorted and encoded according to their importance for the reconstructed view. The proposed approach allows a progressive decoding of the sub-hologram from the first received atom. Streaming simulations for a moving user reveal that our approach outperforms conventional scalable codecs such as scalable H.265 and enables a practical streaming with a better quality of experience.