Past, Present and Future

Today I want to sum up all the things which have been done in nginx-rtmp project so far.

It's been more than 4 months since the project started and about 3 months since nginx-rtmp is installed in our project production (leading Russian video service). At first we had a couple of minors problems. Now it's brilliantly replacing the old solution we had. The number of production servers has been reduced by a factor of 6. The new solution is stable, flexible and highly-configurable.

I keep receiving feedback from people all over the world who use the module in their projects. I want to thank all these people for their impact of any kind. Please keep sending patches, ideas and bug reports. I'll do my best to response.

The most challenging features in nginx-rtmp:

• RTMP encrypted handshake. OpenSSL+SHA256 with sophisticated double-encrypting. This lead to H264 support in flash clients.


• HLS. One of the most challenging tasks. Testing on iPhone was inconvenient. Apple tools were not much verbose about the stream problems. At some point I gave up using libavcodec in favor of plain reformatting. Now I'm absolutely sure encoding should be done outside nginx binary. Libavformat is also not perfect. One of the ideas for future is getting rid of libavformat and writing mpeg-ts manually.


• Relays: pull/push. These features were extremely important for load-balancing and pulling other people's streams. As for push feature I was surprised to see people using the module as RTMP push hub.


• HTTP callbacks. These make it possible to move project-specific business logic to PHP/Python/etc which receive all client info like ip, sfwUrl, pageUrl and can trigger special actions as well as reject or permit RTMP operations.


• Video on demand (FLV). I didn't plan this feature from the start since nginx has flv vod support implemented as HTTP module. However I have received a lot of requests for it. Now I have plans for MP4 streaming.

FFmpeg/AVConv role. These great projects make things easier. Following Unix philosophy I haven't implemented the things which have already been done in ffmpeg/avconf like stream encoding/resizing/filtering etc. Instead I have added exec feature to make use of them as external tools. There are more reasons for moving stream encoding out of nginx. Encoding can be CPU-greedy which is bad for single-threaded servers. It can also have high memory consumption and uncontrollable pool usage which is no good for a long-living server.

There are two features which are still not merged into master:

• Stream synchronization (peer-sync branch)


• Multi-worker live streaming (auto-push branch)

Stream synchronization includes advanced frame dropping code. Most RTMP frames have relative timestamps. Dropped frames lead to audio-video synchronization problems. The peer-sync branch fixes this problem sending empty absolute frame with the right timestamp. Directive 'sync 500ms' sets max stream divergence to 500 milliseconds.

Multi-worker support is a highly anticipated feature. Pulled live streaming and video-on-demand don't need this. It's only needed when stream is published locally. Stream pushing to all workers is implemented in auto-push branch. Unix sockets are used for that. The directives related are 'rtmp_socket_dir /tmp' setting unix socket directory and 'rtmp_auto_push on' turning on the feature. This feature can make nginx-rtmp output giant traffic on multi-core machines when proper network cards are available. You won't probably need this if your network card's bandwidth is under 2 Gbps. One core is usually enough for this case.

The 2 branches mentioned seem stable. But I still need more feedback to merge them into master. Please try them if you need these features and feel free to report any issues.

I'm leaving for 2 week vacation. After that I'll get back to work with many great ideas and plans for future.

Stay tuned!