GStreamer
For a long time I've been using only ffmpeg as streaming/encoding/decoding tool. But there's another well-known project GStreamer which can do a lot of things, even more than ffmpeg. Recently I've encountered a problem which GStreamer solves but ffmpeg does not.Here's an example of GStreamer call capturing video and audio from webcam and publishing RTMP stream to server. The whole long argument is called GStreamer pipe. It consists of elements separated with "!". Elements receive input and produce output. For details please apply to GStreamer web site.
gst-launch-1.0 v4l2src ! "video/x-raw,width=640,height=480,framerate=15/1" !
h264enc bitrate=1000 ! video/x-h264,profile=high ! h264parse ! queue !
flvmux name=mux alsasrc device=hw:1 ! audioresample ! audio/x-raw,rate=48000 !
queue ! voaacenc bitrate=32000 ! queue ! mux. mux. !
rtmpsink location=\"rtmp://example.com/myapp/mystream live=1\"
GStreamer rtmpsink plugin writes long timestamps in type-3 packets in a different way than ffmpeg rtmp engine. So the following directive is needed in nginx.conf when publishing with GStreamer.
publish_time_fix off;
Encoding & decoding on Raspberry Pi
Most people already known what Raspberry Pi is. It is a small and cheap credit-card sized ARM computer. One can easily install Linux on it. It recognizes USB devices well (webcams, 3G/4G modems etc) so streaming from Raspberry Pi is an interesting thing to try.Raspberry Pi processor is too weak to encode h264. Successfully it has hardware h264 encoder & decoder. The only way to use it is through OpenMAX interface. Ffmpeg has no support for OpenMAX so we can only use GStreamer which has OpenMAX support in gst-omx project. Here's an example GStreamer pipeline streaming RTMP from Raspberry Pi to server.
gst-launch-1.0 v4l2src ! "video/x-raw,width=640,height=480,framerate=15/1" !
omxh264enc target-bitrate=1000000 control-rate=variable !
video/x-h264,profile=high ! h264parse ! queue !
flvmux name=mux alsasrc device=hw:1 ! audioresample ! audio/x-raw,rate=48000 !
queue ! voaacenc bitrate=32000 ! queue ! mux. mux. !
rtmpsink location=\"rtmp://example.com/myapp/mystream live=1\"
Note omxh264enc element which is hardware h264 encoder. Software h264 encoder is called h264enc.
Please make sure you use the recent version of gst-omx (>= July 1). I've submitted a patch which fixes setting stream bitrate for Raspberry Pi OpenMAX. Without it you can only stream at a low bitrate about 100kpbs.
You need GStreamer version 1.0 for gst-omx to compile and work on Raspberry Pi.
That is fantastic. I will try gstreamer now. Do I just pipe raspivid into that gst-launch command? It will run from the command line right?
ReplyDeleteSomething like this ...
Deleteraspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - | gst-launch-1.0 fdsrc fd=0 ! ... rest-of-parameters-by-Roman ...
I'm not sure about piping from raspivid. Please try and report. I don't have Raspberry Pi Camera module so I had no chance to try raspivid.
ReplyDeleteI'm always getting error message during configuration of gst-omx. any workaround?
ReplyDeletegstreamer 1.0 is installed.
#gst-omx $ gst-launch-1.0 --version
gst-launch-1.0 version 1.0.7
GStreamer 1.0.7
error message after ./autogen.sh
....
checking for GIO... yes
checking for GST... no
configure: No package 'gstreamer-1.0' found
configure: error: no gstreamer-1.0 >= 1.0.0 (GStreamer) found
I installed gstreamer-1.0 (everything but gst-omx) from here
Deletehttp://theiopage.blogspot.ru/2013/04/enabling-hardware-h264-encoding-with.html
I could compile everything without errors now, but if i set target-bitrate=1000000 I get:
Delete"ERROR: from element /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0: Could not initialise supporting library.
Additional debug info:
gstvideoencoder.c(1418): gst_video_encoder_change_state (): /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
Failed to open encoder"
If I don't use bitrate parameter it works fine with some default bitrate.
After following the instructions for gstreamer installation I use the steps below:
git clone git://anongit.freedesktop.org/gstreamer/gst-omx
cd gst-omx
./autogen.sh
./configure --with-omx-target=rpi
ln /opt/vc/include/IL/OMX_Broadcom.h omx/OMX_Broadcom.h -s
sudo make
sudo make install
Is there anything I missed?
Looks like I found problem but still not the solution. gst-launch does not recognize the manually compiled gst-omx files. how can I make sure all compiled files from gst-omx move to the right directories to be accepted by gst-launch?
DeleteYou should set both target-bitrate and control-rate.
ReplyDeletedoes not work. installed new raspbian image and all gstreamer related stuff again. still same error.
ReplyDeletewithout "target-bitrate=1000000 control-rate=variable " it works fine.
this my commandline
gst-launch-1.0 rtmpsrc location=rtmp://myserver.com/folder/path ! flvdemux ! h264parse! omxh264dec ! videoconvert ! omxh264enc target-bitrate=1000000 control-rate=variable ! video/x-h264,profile=high ! mpegtsmux ! udpsink host=192.168.3.187 port=1234
same error as before:
add --gst-debug for more info
ReplyDeleteHi Roman,
ReplyDeletethis is the output.
0:00:00.357480931 2408 0x1690580 ERROR omx gstomx.c:2773:plugin_init: Invalid type name 'GstOMXTheoraDec' for element 'omxtheoradec'
0:00:00.359019853 2408 0x1690580 ERROR omx gstomx.c:2773:plugin_init: Invalid type name 'GstOMXVP8Dec' for element 'omxvp8dec'
Setting pipeline to PAUSED ...
0:00:00.464287532 2408 0x1690580 ERROR omxvideoenc gstomxvideoenc.c:307:gst_omx_video_enc_open: Failed to set bitrate parameters: Version mismatch (0x8000100f)
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0: Could not initialise supporting library.
Additional debug info:
gstvideoencoder.c(1418): gst_video_encoder_change_state (): /GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0:
Failed to open encoder
Setting pipeline to NULL ...
Freeing pipeline ...
i've downloaded gst-omx with git clone git://anongit.freedesktop.org/gstreamer/gst-omx.
ReplyDeleteis your patch included in this version?
i m trying to stream video from my raspeberry to vlc player. I have the following pipeline.
ReplyDeletegst-launch-1.0 v4l2src ! omxh264enc ! video/x-h264,profile=high ! h264parse ! queue ! mpegtsmux ! tcpserversink host=134.202.84.72 port=1234
Vlc palayer is buffering the video but is unable to play it. Vlc player continuosly displays the following message
packetizer_h264 warning: waiting for SPS/PPS
I have tried to save the video using
gst-launch-1.0 v4l2src ! omxh264enc ! multipartmux ! filesink location=webcam.ts
and then copied it to my pc and it played perfectly fine but when it comes to streaming the vlc player is unable to play it please help
Hi Roman, when i try to run the test.h264 video distributed with wheezy using gst-launch-1.0 playbin uri=file://...test.h264, it errors out with
ReplyDeleteomxh264dec0: internal data stream error.
Additional debug info:
gstomxvideodec.c (2183): gst_omx_video_dec_loop(): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstOMXH264De - omxh264dec: omh264dec - omxh264dec0:
stream stopped, reason not-negotiated
ERROR: pipeline doesnt want to preroll.
Any help is greatly appreciated.
I have tried bot to compile gst from source and download the prebuilt plugin but no dice :(
Hello. I trying to use this, and video encoder works ok, but i still getting error in audo, something like:
ReplyDeleteSetting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2812): gst_base_src_loop (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 658804940 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
libv4l2: warning v4l2 mmap buffers still mapped on close()
Freeing pipeline ...
What can it be? Thanks
I fugured it out. For new gstreamer you need to add aacparse after codec, because flvmux consume ONLY raw AAC
ReplyDeleteHello guys. What is the command to stream rtmp only the raspi cam? Without audio. I tried this command :
ReplyDeleteraspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - | gst-launch-1.0 fdsrc fd=0 ! omxh264enc target-bitrate=1000000 control-rate=variable ! video/x-h264,profile=high ! h264parse ! queue ! flvmux name=mux alsasrc device=hw:1 ! audioresample ! audio/x-raw,rate=48000 ! queue ! voaacenc bitrate=32000 ! queue ! mux. mux. ! rtmpsink location=\"rtmp://example.com/myapp/mystream live=1\"
but i get this:
WARNING: erroneous pipeline: could not link queue0 to audioresample0
It is supposed to look like this raspivid -n -t 1000000 -vf -b 2000000 -fps 25 -o - | gst-launch-1.0 fdsrc fd=0 ! omxh264enc target-bitrate=1000000 control-rate=variable ! video/x-h264,profile=high ! h264parse ! rtmpsink location=\"rtmp://localhost/rtmp/live live=1\" - The error erroneus pipleine meanst that the rtmp sink is getting an unsupported stream... I am not sure what the problem is.
DeleteI got gstreamer to dump the strem ti nginx-rtmp. I can see in top that both nginx and gstreamer are consuming cpu so its doing something but I cannot get video on my iPhone or webpage.. almost there!?!
Deleteraspivid -t 999999 -h 720 -w 1080 -fps 25 -hf -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! flvmux ! rtmpsink location=\"rtmp://localhost/rtmp/live live=1\"
This comment has been removed by the author.
DeleteAny ideas on this.. I only get "Pipeline is prerolling..." with this.
DeleteThis comment has been removed by the author.
ReplyDeleteHi Roman,
ReplyDeleteI compiled the nginx module and is up and running, but i'm not able to reproduce the video through a web client using jwplayer. Do you have any idea or an nginx config file to take as reference?
This is my nginx configuration file, I've tried to static_exec the gstreamer pipeline and pushing from other process but neither work.
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
server {
listen 8080;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root html;
index index.html index.htm;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
#server {
# listen 443;
# server_name localhost;
# ssl on;
# ssl_certificate cert.pem;
# ssl_certificate_key cert.key;
# ssl_session_timeout 5m;
# ssl_protocols SSLv2 SSLv3 TLSv1;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# root html;
# index index.html index.htm;
# }
#}
}
rtmp{
server{
listen 1987;
chunk_size 4096;
application rpi{
live on;
record off;
allow publish 127.0.0.1;
deny publish all;
#exec_static gst-launch-1.0 -v -e v4l2src ! "video/x-raw,width=640,height=480,framerate=15/1" ! omxh264enc target-bitrate=1000000 control-rate=variable ! video/x-h264,profile=high ! h264parse ! queue ! flvmux name=mux alsasrc ! audioresample ! audio/x-raw,rate=48000,channels=1 ! queue ! voaacenc bitrate=32000 ! queue ! mux. mux. ! rtmpsink location="rtmp://localhost:1987/rpi/flv:mystream";
}
}
}