RTMP video packets and streaming thereof

[Radu-Adrian Popescu]

Hi guys,

First off congrats for the good work!

I'm working on an application that streams video data into FMS. I've
been through the Red5 docs and other available information and gotten
to the stage
where I've created a stream and called publish on it in live mode.
I've captured the first five video packets that a Flash movie sends
to the FMS from a webcam, and if I load these packets up into a
buffer and send them
to the FMS at this point, it works as expected (another test Flash
movie displays the stream).

The problem is I don't really seem to understand how to build these
RTMP video packets, basically, everything after the 0x09 video data
marker is completely
gibberish, up until what seems to be video data (from the visual hex
patterns).
There's the packet start byte, 3 bytes, yet more three bytes that
indicate the video data size, 0x09, some other bytes that I don't
understand and appear to
be more or less the same in each following full packets, then video
data.

I've tried to build video packets with bogus video data (0xFF for
instance) and copied in the mystery bytes from the 1st video packet
Flash sent, and after
sending a few of these packets the server closes the connection and
the FMS administration console shows an error message saying that the
client is not allowed
to broadcast. Note that if instead I load the etherealed video
packets from raw files and send those instead, everything works as
expected.

Can you guys help? Thanks a lot!

Cheers and kind regards,

[Rob Terrell]

This sounds like an issue we talked about previously on this list --
we noticed that FMS won't let clients connect and send streams that
use the screen video codec. I'm guessing that any connecting sending
a non-spark codec is rejected.

Post by Radu-Adrian Popescu
I've tried to build video packets with bogus video data (0xFF for
instance) and copied in the mystery bytes from the 1st video packet
Flash sent, and after
sending a few of these packets the server closes the connection and
the FMS administration console shows an error message saying that the
client is not allowed
to broadcast. Note that if instead I load the etherealed video
packets from raw files and send those instead, everything works as
expected.

[Tom Krcha]

Does anybody know how to create Java client, which is sending proper data
video to Red5.



The point is to create javaclient for capturing the screen and then show
this data in flash.



Thx.



Tom



_____

From: Red5-***@osflash.org [mailto:Red5-***@osflash.org] On Behalf
Of Rob Terrell
Sent: Thursday, June 01, 2006 1:12 PM
To: ***@osflash.org
Subject: Re: [Red5] RTMP video packets and streaming thereof



This sounds like an issue we talked about previously on this list -- we
noticed that FMS won't let clients connect and send streams that use the
screen video codec. I'm guessing that any connecting sending a non-spark
codec is rejected.





On Jun 1, 2006, at 7:02 AM, Radu-Adrian Popescu wrote:





I've tried to build video packets with bogus video data (0xFF for

instance) and copied in the mystery bytes from the 1st video packet

Flash sent, and after

sending a few of these packets the server closes the connection and

the FMS administration console shows an error message saying that the

client is not allowed

to broadcast. Note that if instead I load the etherealed video

packets from raw files and send those instead, everything works as

expected.

[Radu-Adrian Popescu]

Hi Rob,

Post by Tom Krcha
This sounds like an issue we talked about previously on this list -- we
noticed that FMS won't let clients connect and send streams that use the
screen video codec. I'm guessing that any connecting sending a non-spark
codec is rejected.

Thanks for the info, I'll keep that in mind and try using a proper codec.

At any rate, does anyone know:
1) what bytes 2,3,4 represent (starting with 1) and why do they change with each RTMP
video header
2) why the RTMP packet's declared size (bytes 5,6,7) is smaller than the number of bytes
found in the packet after the video marker (0x09)
3) the meaning of any of the bytes after the video marker

Thanks a lot in advance!

[Joachim Bauch]

Hi,

Radu-Adrian Popescu wrote:
[...]

Post by Radu-Adrian Popescu
Thanks for the info, I'll keep that in mind and try using a proper codec.
1) what bytes 2,3,4 represent (starting with 1) and why do they change
with each RTMP video header
2) why the RTMP packet's declared size (bytes 5,6,7) is smaller than the
number of bytes found in the packet after the video marker (0x09)
3) the meaning of any of the bytes after the video marker

The RTMP protocol is described on http://osflash.org/rtmp_os

Joachim

[Radu-Adrian Popescu]

Post by Joachim Bauch
The RTMP protocol is described on http://osflash.org/rtmp_os
Joachim

Thanks, I know, I've been through it several times over and able to create proper RTMP
packets, except for video, which are not really discussed there.

The problem I have is understanding what is inside the RTMP video packets, i.e. after the
0x09 video marker.
I've discovered that it's FLV more or less, i.e. without the FLV header, skips directly
into the body.

Red5 authors, can anyone explain why the RTMP-declared length of the packet is larger than
the size of the RTMP body?
I've ethereal traces that declare a RTMP size that's smaller than the actual body sent by
somewhere 5-6 bytes. I can't make sense of this. For INVOKE packets the numbers always add
up, from both the packets I produce and the packets captured. For my video packets,
created with FLV content from files, they add up as well, but not for the flash-originated
captured ones.
Any ideas ?

Thanks for your help,

[Luke Hubbard]

Hi,

Couple of things.. first dont read the MM spec.
If you do you cannot create server or client. So you are limited to tools.
http://www.osflash.org/flv has info on flv format.

The size difference is likely to be the chunking bytes.
This is described in the documentation joachim pointed you to.
Only where it says "The AMF", mick was talking about RTMP.
RTMP packets are split into 128 byte chunks (by default).
In between each chunk there is another header.
Normally this is a 1 byte continue header, but it can be longer.

Hope that helps.
- Luke

Post by Radu-Adrian Popescu

Post by Joachim Bauch
The RTMP protocol is described on http://osflash.org/rtmp_os
Joachim

Thanks, I know, I've been through it several times over and able to create proper RTMP
packets, except for video, which are not really discussed there.
The problem I have is understanding what is inside the RTMP video packets, i.e. after the
0x09 video marker.
I've discovered that it's FLV more or less, i.e. without the FLV header, skips directly
into the body.
Red5 authors, can anyone explain why the RTMP-declared length of the packet is larger than
the size of the RTMP body?
I've ethereal traces that declare a RTMP size that's smaller than the actual body sent by
somewhere 5-6 bytes. I can't make sense of this. For INVOKE packets the numbers always add
up, from both the packets I produce and the packets captured. For my video packets,
created with FLV content from files, they add up as well, but not for the flash-originated
captured ones.
Any ideas ?
Thanks for your help,
--
Radu-Adrian Popescu
CSA, DBA, Developer
Aldrapay MD
Aldratech Ltd.
[phone]
_______________________________________________
Red5 mailing list
http://osflash.org/mailman/listinfo/red5_osflash.org

[Radu-Adrian Popescu]

Post by Luke Hubbard
Hi,

Hi Luke,

Post by Luke Hubbard
Couple of things.. first dont read the MM spec.
If you do you cannot create server or client. So you are limited to tools.
http://www.osflash.org/flv has info on flv format.

Thanks, I've read the license agreement and decided not to read it. You can create clients
as far as I understood, though.

Post by Luke Hubbard
The size difference is likely to be the chunking bytes.
This is described in the documentation joachim pointed you to.
Only where it says "The AMF", mick was talking about RTMP.
RTMP packets are split into 128 byte chunks (by default).
In between each chunk there is another header.
Normally this is a 1 byte continue header, but it can be longer.

Right, thing is I can't seem to find any relation between bytes located at 128 offset
increments. Are they supposed to be the same byte?
What does Red5 do when dealing with video data? Does it parse it back into FLV, i.e.
without the chunking bytes? If so could you point out relevant code in Red5?

Post by Luke Hubbard
Hope that helps.
- Luke

Thank you very much,

[Steven Gong]

Post by Radu-Adrian Popescu

Post by Luke Hubbard
Hi,

Hi Luke,

Post by Luke Hubbard
Couple of things.. first dont read the MM spec.
If you do you cannot create server or client. So you are limited to

tools.

Post by Luke Hubbard
http://www.osflash.org/flv has info on flv format.

Thanks, I've read the license agreement and decided not to read it. You can create clients
as far as I understood, though.

Post by Luke Hubbard
The size difference is likely to be the chunking bytes.
This is described in the documentation joachim pointed you to.
Only where it says "The AMF", mick was talking about RTMP.
RTMP packets are split into 128 byte chunks (by default).
In between each chunk there is another header.
Normally this is a 1 byte continue header, but it can be longer.

Right, thing is I can't seem to find any relation between bytes located at 128 offset
increments. Are they supposed to be the same byte?

Well, AFAIK there's another type of RTMP packet to control the actual chunk
size whose default value is 128 bytes.

What does Red5 do when dealing with video data? Does it parse it back into

Post by Radu-Adrian Popescu
FLV, i.e.
without the chunking bytes? If so could you point out relevant code in Red5?

Red5 does not parse any of the video data. During streaming, it simply
extracts the content from FLV (stripping the header) and sends to client.

Post by Radu-Adrian Popescu
Hope that helps.

Post by Luke Hubbard
- Luke

Thank you very much,
--
Radu-Adrian Popescu
CSA, DBA, Developer
Aldrapay MD
Aldratech Ltd.
[phone]
_______________________________________________
Red5 mailing list
http://osflash.org/mailman/listinfo/red5_osflash.org

[Radu-Adrian Popescu]

I've just figured this out!

Looking at smaller, full RTMP video packets, they only have a header of 8 bytes, ending in
09. After each 128 bytes an C6 is added, and the numbers add up.
For a 670 bytes size packet, the RTMP header-declared length is 0x291 = 657.
657 can hold 5 128-byte packets, so there's 5 C6 bytes in between chunks.
The header is 8 bytes, rendering 657+8+5=670.

I've three different traces of the 1st video packet that is larger and contains a full frame.
In two of them the chunking is performed at 128 bytes, as for the smaller, delta frames.
All fine and dandy.
In the other 1st video packet, the first C6 byte appears at offset 158 in the trace file,
which is 18 bytes "later"; after that it continues at each 128. Weird!

Cheers,

[Radu-Adrian Popescu]

Ok, extremely good news!

I've just managed to send through FMS a video packet containing the 1st full frame from a
FLV file (going to use libflv to provide frames, for now I'm just hackishly reading it
from the file), and it got played in the flash player.

I've done "proper" chunking (see below for an exceptional case of chunking found through
etherealing) i.e. insert C6 each 128 bytes counting from 0x09. FMS is happy, Flash player
is happy.

Someone noted earlier on that FMS may not like all codecs. I've manually changed the codec
code from 0x12 (sorenson) to 0x14(screen) in the FLV file and sent it through. Flash
obviously didn't play it, but FMS accepted the stuff, so I don't think it does any codec
filtering as suggested, it's just that when I was sending junk to it it couldn't make
sense of the packets and therefore dropped them and said I'm not allowed to broadcast.
Hope that clears a few worries.


;) cheers

Post by Radu-Adrian Popescu
I've just figured this out!
Looking at smaller, full RTMP video packets, they only have a header of
8 bytes, ending in 09. After each 128 bytes an C6 is added, and the
numbers add up.
For a 670 bytes size packet, the RTMP header-declared length is 0x291 = 657.
657 can hold 5 128-byte packets, so there's 5 C6 bytes in between chunks.
The header is 8 bytes, rendering 657+8+5=670.
I've three different traces of the 1st video packet that is larger and
contains a full frame.
In two of them the chunking is performed at 128 bytes, as for the
smaller, delta frames. All fine and dandy.
In the other 1st video packet, the first C6 byte appears at offset 158
in the trace file, which is 18 bytes "later"; after that it continues at
each 128. Weird!
Cheers,
------------------------------------------------------------------------
_______________________________________________
Red5 mailing list
http://osflash.org/mailman/listinfo/red5_osflash.org

[Jakub Kahovec]

Hello,
I've set up Red5 server, it works perfectly when being accessed within
the local network.
If I want to access it from the Internet though(the server with Red5 is
behind a firewall which I can't change settings on and where only ports
80,443 and 22 are opened ), I can only connect to it
from Flash Player 9 beta 3, if want to connect to it with Flash 7 or 8
it doesn't work.
The app I'm trying to connect to is here
rtmp://videoconference.londongt.org/fitcDemo
Any ideas ?
Thank you

Jakub

[Joachim Bauch]

Hi Jakub,

Post by Jakub Kahovec
I've set up Red5 server, it works perfectly when being accessed within
the local network.
If I want to access it from the Internet though(the server with Red5 is
behind a firewall which I can't change settings on and where only ports
80,443 and 22 are opened ), I can only connect to it
from Flash Player 9 beta 3, if want to connect to it with Flash 7 or 8
it doesn't work.
The app I'm trying to connect to is here
rtmp://videoconference.londongt.org/fitcDemo

If you want to use RTMPT, you will have to explicitly specify the protocol
in the connection string, e.g. "rtmpt://videoconference.londongt.org/fitcDemo".
Please make sure to change the RTMPT port from the default 8088 to 80 if
you need to access Red5 on that port.

Joachim

[Saurav Mohapatra]

Hi,

Take a look at the Flash SWF specs from macromedia.. the video data format
is documented there in FLV File section.

regards
Saurav


----- Original Message -----
From: "Radu-Adrian Popescu" <***@aldratech.com>
To: <***@osflash.org>
Sent: Thursday, June 01, 2006 4:32 PM
Subject: [Red5] RTMP video packets and streaming thereof

_______________________________________________
Red5 mailing list
http://osflash.org/mailman/listinfo/red5_osflash.org

[Radu-Adrian Popescu]

Post by Saurav Mohapatra
Hi,
Take a look at the Flash SWF specs from macromedia.. the video data format
is documented there in FLV File section.
regards
Saurav

The license agreement says, to my understanding at least, that if you do read the spec you
're prevented from building servers that stream data - you can only do clients.
Thanks

[Saurav Mohapatra]

the specs EULA prevents you from making a player.. not a publisher...

that is the whole purpose of the specs... macromedia to my knowledge put the
specs out to allow people to create pieces that would publish data but not
subscribe to the data...

regards
Saurav
----- Original Message -----
From: "Radu-Adrian Popescu" <***@aldratech.com>
To: <***@osflash.org>
Sent: Thursday, June 01, 2006 10:55 PM
Subject: Re: [Red5] RTMP video packets and streaming thereof

_______________________________________________
Red5 mailing list
http://osflash.org/mailman/listinfo/red5_osflash.org

[Radu-Adrian Popescu]

Hi guys,

First off congrats for the good work!

I'm working on an application that streams video data into FMS. I've
been through the Red5 docs and other available information and gotten
to the stage
where I've created a stream and called publish on it in live mode.
I've captured the first five video packets that a Flash movie sends
to the FMS from a webcam, and if I load these packets up into a
buffer and send them
to the FMS at this point, it works as expected (another test Flash
movie displays the stream).

The problem is I don't really seem to understand how to build these
RTMP video packets, basically, everything after the 0x09 video data
marker is completely
gibberish, up until what seems to be video data (from the visual hex
patterns).
There's the packet start byte, 3 bytes, yet more three bytes that
indicate the video data size, 0x09, some other bytes that I don't
understand and appear to
be more or less the same in each following full packets, then video
data.

I've tried to build video packets with bogus video data (0xFF for
instance) and copied in the mystery bytes from the 1st video packet
Flash sent, and after
sending a few of these packets the server closes the connection and
the FMS administration console shows an error message saying that the
client is not allowed
to broadcast. Note that if instead I load the etherealed video
packets from raw files and send those instead, everything works as
expected.

Can you guys help? Thanks a lot!

Cheers and kind regards,
--
Radu-Adrian Popescu
CSA,DBA,Programmer
[phone]