Digital holography is an emerging technology for 3D visualization which is expected to dethrone conventional stereoscopic devices in the future. Aside from their specific signal properties, high quality holograms with broad viewing angles contain massive amount of data. For a reasonable transmission time, efficient scalable compression schemes are needed to bridge the gap between the overwhelming volume of data and the limited bandwidth of the communication channels. The viewpoint scalability is a powerful property since it allows to encode and transmit only the information corresponding to the observer’s view. However, this approach imposes an online encoding at the server which may increase the latency of the transmission chain. To overcome this hurdle, we propose a scalable compression framework based on Gabor-wavelets decomposition, where the whole hologram is encoded offline. First, the observer plane is divided into spatial blocks. Then, the Gabor atoms are assigned to these blocks by exploiting the duality between Gabor wavelets and light rays. The atoms of each block are then classified into different layers according to their importance for the reconstruction and encoded in packets. At the decoder side, the atoms’ packets are progressively decoded based on the viewer’s position. Then, the corresponding sub-hologram is generated using a GPU implementation. Results show that our approach enables a practical progressive streaming of digital holograms with a low latency.