Tuesday, March 5, 2013

Native HLS in nginx-rtmp

Until now HLS implementation in nginx-rtmp made use of ffmpeg libs to create HLS fragments. That lead to such major problems as complicated compilation process, ffmpeg/avconv confusion, libavformat API limitations, uncontrollable memory allocations in ffmpeg libs etc. Besides ffmpeg may not be available on some architectures (embedded etc). That was all too serious to keep using those libs.

Today I'm introducing native HLS support. It seems to be significantly faster that before, and does not require any third-party libs. The new code in is native-hls branch. There's no separate HLS module anymore. HLS is included in main rtmp module as one of the features.

In addition to native mpeg-ts generation new HLS engine is more consistent with stream restarts.

27 comments:

  1. Hello Arut,
    I'm using nginx-rtmp with hls support via ffmpeg libs, it's working fine and stable, however i see two limitations on hls feature:
    1. hls_path directory is stored for all streams.When an application with many streams and one stream have lot of fragment files then number of file per the directory limited by operating system.
    2. when stream restart and hls playlist is empty then first fragment will be overwrite.
    Hope you will overcome this limitation.Thanks

    ReplyDelete
  2. 1. My idea is to create a directory per stream. I will implement that later. Do you have another idea?
    2. Use "hls_continuous on" (default in native-hls)

    ReplyDelete
  3. 2. if playlist is empty then what fragments are overwritten? old or new fragments?

    ReplyDelete
  4. Hi Arut,

    thanks that you spend your time with the development of this great module, i´m impressed.

    I´m new with HLS streaming and got a working version with ffmpeg libs and your module and nginx. Now i saw that you have written an native mpegTS plexer that makes a lot of sense to me.

    I compiled the old version without native HLS with this line:
    ./configure --add-module=/Users/gary/Desktop/ngnix/nginx-rtmp-module --add-module=/Users/gary/Desktop/ngnix/nginx-rtmp-module/hls --with-cc-opt=-I/opt/ffmpeg/include --with-ld-opt=-L/opt/ffmpeg/lib --without-http_rewrite_module

    Can you please tell how to compile the new version, do i have simple to add the .../nginx-rtmp-module without any other flags?

    And in the conf settings how can i enable HLS, like the old one?

    I will download now the new version and try things out. Would be great when you can please document the new HLS implementation.

    Thanks a lot..
    Gary

    ReplyDelete
  5. your native HLS implementation works out of box on OSX...great..will play with it now and tomorrow check the ts with our anaylzers and come back with the result.

    thx
    gary

    ReplyDelete
  6. Thanks for reply,
    1. Workaround, I also have to modified your code to create each stream has a dir.
    2. hls_continuous enabled, but playlist is empty then there is no way to know the last fragment, there is no way to restore the previous fragment, so odlest fragment (first fragment, i think id is 1) is overwritten.
    I replaced start_id of fragment each time when stream start or restart by epoch time(same "date +%s" command), for example: mystream_1362532600.ts

    ReplyDelete
  7. In native-hls branch all HLS code in already included in rtmp modulde, you only need to add rtmp module. This is the command for your case:

    ./configure –add-module=/Users/gary/Desktop/ngnix/nginx-rtmp-module –without-http_rewrite_module

    Configuration is exactly the same as before (hls, hls_path, hls_fragment etc).

    ReplyDelete
  8. 1. great 2. playlist should exist if there was any previous hls streaming. If you delete it manually this is exactly what you will have.

    ReplyDelete
  9. Great, I really need more feedback

    ReplyDelete
  10. Ok! I have another question, hls_continuous on vs #EXT-X-DISCONTINUITY tag?

    ReplyDelete
  11. The module never sets that tag.

    hls_continuous is the right thing to do, in native-hls it's on by default.

    ReplyDelete
  12. I recomplied rtmp module with hls-native on my system and it works greate.FFmpeg lib's error '[error] : hls: av: muxrate VBR' disappeared in debug log.
    Many thanks.

    ReplyDelete
  13. hi arut,

    sorry for delay but i wanted to make sure that i analyze the ts segments correctly.

    It seems AAC audio looks great in ts, i found some issues in the h.264 AVC. Specially the PTS at 90 khz periods. Here is a screenshot of a analyze, encoded with FME/LE and transmuxed thru your new native mpegts muxer.

    Screenshot is here:
    https://www.amazon.de/gp/drive/share?ie=UTF8&s=pMtn-MlMR4Ah9aEQgh3j24#

    I really would like to join the project, but i´m a java expert and not c or c++, but hey..why not learn it.

    I know much about HLS. I would like to share some ideas.

    When there is a native ts parser where we can grab the h.264 NAL units, i think its not even hard to add AES encryption. Cause you use C++, i think to include openSSL is the right choice.

    There are 2 encryption method in HLS, one is to encrypt the whole TS which destroys the complete ts container structure. Another way is to add sample-aes. Sample aes only packs NAL units in a rotation of each XX packet.

    Another great idea is to extract the AAC stream to write AAC segments in 64kbit. This is required by the apple store requirements.

    As a programmer, i know that ideas makes no code... I only wanted to share some of my minds.

    Which IDE do you use for nginx module and c++ development? Do you use eclipse under lnx?

    I want to install an developer IDE and hopefully get an debugger for testing, thats why i ask...

    come back soon.

    Gary

    ReplyDelete
  14. I cannot open your link, it says "This link is unavailable" in German. Please give more details about h264 problems. I tested on iPhone & Android and it worked perfect for me, tstools also said it's ok. Make sure you have the latest *native-hls*, I've committed a lot yesterday (March 7).

    OpenSSL is already linked to nginx for RTMP handshake. I can use it here as well, but what's the reason for that?

    Writing audio-only stream is a good idea. But choosing a stream with 64k audio among all other incoming streams is tricky.

    It's not C++, it's plain C. I don't use any IDE, only vim/ctags, bash and gdb. I do some things remotely (via ssh).

    It''s really great if you help the project in *any* way.

    PS: I'm leaving for a week. I'l; probably be able to reply but not write the code. So we can discuss h264/mpegts problems until then.

    ReplyDelete
  15. hi arut,

    thanks for your fast replys. sorry for the link, it seems to work from here. Is there maybe a better way for communicating? Can you please send me your mail for some disucssions? I have some great ideas that i would like to share.

    ReplyDelete
  16. Gary has nice point there with <64kbit audio only stream.
    For example if you have incoming stream with 500kbit video+56kbit aac audio, you could have that split into 2 streams:
    1. with whole stream (video+audio, 556kbit/sec)
    2. audio only stream (56kbit, directly stripped from incoming stream)

    And have module to create multistream manifest for those.. I know it is pretty easy to create ffmpeg exec command but that feature could be very handy for IOS deliveries.

    This applies only to HLS streaming, since Apple is requiring HLS streams with one stream less than 64kbit/s for apps if 3G network is used in app. This applies to apps that are playing bigger streams than 50megabytes or longer than 5 minutes.

    ReplyDelete
  17. Hello Arut,

    great job on this module. I'm hoping to be able to use it on a project i'm working on.

    I'm having a tough time getting hls to work on an iphone4. It plays for several seconds then stops for several seconds. It continues this behavior throughout playback making it unwatchable.

    The HLS stream works fine on android via vplayer. Am I missing something in my encoding process or nginx.conf setup? My videos are encoded in h264 / aac an I'm using ffmpeg to convert the video files prior to streaming to rtmp.

    here is an example of the encoded format i'm using obtained via ffmpeg -i
    Duration: 00:05:11.33, start: 0.000000, bitrate: 1234 kb/s
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 720x480, 1100 kb/s, 29.97 tbr, 1k tbn, 59.94 tbc
    Stream #0:1: Audio: aac, 44100 Hz, stereo, fltp, 160 kb/s

    My nginx.conf hls application looks like this:

    application hls {
    live on;

    interleave on;
    wait_video on;
    sync 300ms;

    hls on;
    hls_path /tmp/streams/live1;
    hls_fragment 5s;

    }

    I experimented with all types of configuration options adjusting the hls_fragment, sync and toggling interleave & wait_video with no luck. any insight will be appreciated.


    thank you

    ReplyDelete
  18. My apologies, it looks like something was going on on my on my iphone5. It works great on my iphone4.

    ReplyDelete
  19. my e-mail arutyunyan.roman@gmail.com

    your link is ok today

    ReplyDelete
  20. Could you explain in more detail what these PTS errors mean? Does your tool give more info when you click on error entry? I use tsreport. It's happy with the stream.

    ReplyDelete
  21. Hi, Roman

    Just tested native-hls and there seems to be a problem
    If published stream is h264/mp3 no audio can be heard and VLC says that codec is MPEG AAC
    However if published stream is h264/aac there's no problem.
    Is mp3 supported as an audio for hls or transcoding with ffmpeg should be done first ?

    thanks for the great software.

    ReplyDelete
  22. mp3 is not supported in native hls, only aac.

    ReplyDelete
  23. hi rarut, really great extension, thanks for all your effort. is the HLS already in master branch?

    ReplyDelete
  24. ok, thanks, once again, thumb up (y)

    ReplyDelete
  25. Thanks for sharing the useful information. If you guys want to finish HLS streaming download task, you should make use of the downloading tools. You can check some of them in the link. Hope it helps.

    ReplyDelete