Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Stream h.264 with PHP script?


Hi!

Has anybody figured out whether ...Hi!

Has anybody figured out whether it would be possible to use the 'simple php streaming script' (as shown in http://www.jeroenwijering.com/extras/streaming.html) to stream mp4-files with h.264 content to the player?
I tried it without changing the php-code and it works when you play the file from beginning - when you try to skip in the file the playback breaks (I suppose it's because the stream.php sends the FLV file format header, which does not correspond the mp4 file format...).
Does anyone know how a mp4 file format header has to look like and has managed to send it to the player when skipping in a file?
Thx in advance!

100 Community Answers

JeroenW

JW Player Support Agent  
0 rated :

First, you need the metadata of the MP4 file, and second you need an array with keyframepositions in that metadata. I’m not sure how to access the metadata, and how to insert an array with keyframe-bytepositions.

It might indeed work if you add the bytepositions and set the header to video/mp4 instead of video/flv. Please keep us posted, as I don’t know if it’s possible at all but am very interested!

JW Player

User  
0 rated :

Hi!

Just tried to simply send the bitstream from the byteposition and setting the content-type to 'video/mp4' -> no chance... :|
I also attempted to send some bytes from the beginnig of the mp4 file before the rest of the bitstream (actually I tried many combinations from some bytes to some hundred kilobytes) to "imitate" a mp4 file header: also no luck...
It seems Adobe wants to have a fully regular mp4 file with a 100% correct header and file structure -- and I have no clue where to find information about that nor how to create such a structure on the fly in a php file.
So I assume: no simple php-streamscript for mp4 files atm :s

JW Player

User  
0 rated :

I'm desperately trying to do this too.

Here's a list of the resources I've uncovered so far (The QuickTime container is basically the bluprint for mp4):

Quicktime File Format Documentation
[url=http://developer.apple.com/documentation/QuickTime/QTFF/]http://developer.apple.com/documentation/QuickTime/QTFF/[/url]

Atomic Parsley mp4 Info
[url=http://atomicparsley.sourceforge.net/mpeg-4files.html]http://atomicparsley.sourceforge.net/mpeg-4files.html[/url]

[url=http://wiki.multimedia.cx/index.php?title=QuickTime_container]http://wiki.multimedia.cx/index.php?title=QuickTime_container[/url]

Renaun Erickson has the source for an AIR app on his site ([url=http://renaun.com/blog/2007/08/22/234/]http://renaun.com/blog/2007/08/22/234/[/url]) that can move the 'moov' atom from the foot of an mp4 container to the top. If the atoms inside the file can be moved around in this way and each explicitly states the byte offsets for video content, it should be possible to manually stitch together a file with PHP-written headers and the original video file from the right offset to start http delivery from the point in the video the user has seeked.

I've found that the getid3 library for PHP allows us to introspect the atom data in an mp4 movie and can easily get at the track information and other QT meta-data but I so far can't locate the 'seekpositions' and other information (as shown in the onMetaData callback object in FP9 Update 3).

Anyone have any ideas as to the mappings between the atom heirarchy and the onMetadata properties ?

Maybe there's no close correlation between key-frame byte offsets and time-codes as there is in FLV but I think it's worth a try.

JW Player

User  
1 rated :

I've just been trying this out a bit.

I started out with a nicely encoded mpeg4 file that plays correctly in non-streaming mode. I then used the qt-faststart (http://svn.mplayerhq.hu/ffmpeg/trunk/tools/qt-faststart.c) tool to move the metadata to the front of the file.

I then adjusted the stream.php script to output a nice mpeg header and video/mp4 content type, and used it as a streamstript.

So far so good: the movie plays nicely, and starts playing right away, before the file is fully buffered (which it didn't do when the metadata was at the end of the file). It also shows the correct total length of the movie, even during buffering.

Now, if I try to skip forawrd to a part of the movie that hasn't been buffered yet, nothing happens. I checked my server logs, and it seems the player doesn't even send a new request to the streamscript with an updated position.

Jeroen, I guess this mean that the problem is indeed that the player doesn't get any metadata? Strangely enough, this article (by one of the people working on flash) http://www.kaourantin.net/2007/08/what-just-happened-to-video-on-web_20.html seems to suggest that the indices should be available in a similar way as for normal flv files: "You'll get this information through the onMetaData callback in an array with the name 'seekpoints'.".

Any idea what is going on?

JW Player

User  
0 rated :

Hi!

That could be the reason why the player stops playback when you try to skip in the file - maybe Jeroen can give us some feedback whether the player is able to send the correct 'file' and 'pos' arguments to the streamscript when using it with mp4 content?
Other question: has any of you guys figured out how a valid mp4 file header has to look like when sending a section of a mp4 file? I assume just "copying" the original header will not work as movie length and structure will change? I had a quick look at the URLs Stephen posted but I'm not sure how to build a 'new' valid mp4 file with simple methods inside a script.

JW Player

User  
0 rated :

Hi!


Looking at the mp4 file format and the ffmpeg source code, I would think a simple ftyp tag, followed by mdat and the movie data should do the trick:

So first an ftyp id tag to identify this as an mp4 file:

0000 0014 6674 7970 6973 6f6d 0000 0200 ....ftypisom.
6d70 3431 ....

0x14 (==20 bytes) is the length of the tag, 0x66747970 (=="ftyp") is the name of the tag, and the rest seems to be specific for the mp4 format (to distinguish it from .mov etc).

Then start the movie data mdat tag:

xxxx xxxx 6d64 6174

where xxxx xxxx should be replaced by the size (as a bigendian 64 bit number) of the remainder of the data plus these 8 bytes, and 0x6d646174 ("mdat") is the tag name to signal the start of the movie data.

After that, just stream the remainder of the movie data.

I'll try to make a new stream.php script with this info later.

regards,
Bas.

JW Player

User  
0 rated :

Excellent work Bas.

I've been looking in detail at what data getId3 extracts from the Backcountry Bombshells mp4 file that is used in the FP9 demo and a couple of atoms stick out as interesting.

I couldn't find the 'seekpoints' data in the atom hierarchy but the 'ctts' and 'sdtp' atoms (not recognised by getId3) seem to carry binary data (AMF format?) so perhaps there's a bunch of metadata that the flash h.264 encoder puts in there as a nested data structure.

Can anyone shed any light on the purpose of these two atoms ? Without the keyframe/seekpoint information and the corresponding byte offsets seeking within the video won't be possible.

I would post the output from getId3 here but it's really verbose.

JW Player

User  
0 rated :

Scratch that. Not AMF data - just more atom information not decoded by getId3. There was a bunch of code in the getId3 QuickTime parsing library commented out. I removed comments and can now see the following information:

moov/trak/mdia/minf/stbl/stss
- time to sample (keyframe) table

moov/trak/mdia/minf/stbl/stco
- chunk offset table

moov/trak/mdia/minf/stbl/stsz
- sample size table

moov/trak/mdia/minf/stbl/stsc
- samples-to-chunks table

Video/Audio data is stored in chunks of samples. Each chunk has an offset (in terms of bytes) and can contain one or more samples. A sample is a section of the video/audio content.

In order to make a PHP streaming script for mp4 movies, I think we probably need to use the sample/chunk information to split the file at the nearest sample and then change the data in the atoms listed above so that the file is structurally correct.

Bas - did your simpler method work? It would be great if it did.

I'm still not entirely sure about how to use this information, however, I think it's a step forward/sideways so I posted here. The QuickTime file format is complex to say the least!

JeroenW

JW Player Support Agent  
0 rated :

Great work here guys. Especially the tool for getting the metadata to the front of the MP4 file: very nice!

If any of you has an mp4 file that should have an array with keyframepositions in it, please put it’s link here or send it to me. Then I can parse the meta info to see if there’s indeed an array with keyframe-bytepositions in there.

JW Player

User  
0 rated :

Hi Jeroen!

Here's a file that should have the correct metadata included: http://www.zoetekouw.net/Zooi/trailer2.mp4

JW Player

User  
0 rated :

The relevant Metadata in mp4 files can be exctracted with the onMetaData events.
They look like :
seekpoints[i]["time"]; seekpoints[i]["offset"]
i being the number of seek points (kind of keyframe).

Here are the metadatas from the trailer2.mp4 posted by Bas :
(sorry for the length, but it might be usefull)

0 - 0;77262
1 - 0.5;144183
2 - 1;222965
3 - 1.5;293303
4 - 2;362199
5 - 2.5;431178
6 - 3;502433
7 - 3.5;574002
8 - 4;646215
9 - 4.5;719302
10 - 5;792508
11 - 5.5;806661
12 - 6;820803
13 - 6.5;835024
14 - 7;1323968
15 - 7.5;1841109
16 - 8;2235287
17 - 8.5;2642789
18 - 9;2855343
19 - 9.5;3235906
20 - 10;3594152
21 - 10.5;3902539
22 - 11;4163846
23 - 11.5;4454491
24 - 12;4708752
25 - 12.5;4978089
26 - 13;5020229
27 - 13.5;5206538
28 - 14;5459199
29 - 14.5;5684215
30 - 15;5891268
31 - 15.5;6079089
32 - 16;6214723
33 - 16.5;6330117
34 - 17;6440116
35 - 17.5;6554910
36 - 18;6716489
37 - 18.5;6873648
38 - 19;7028696
39 - 19.5;7146194
40 - 20;7174362
41 - 20.5;7357374
42 - 21;7511404
43 - 21.5;7694258
44 - 22;7854208
45 - 22.5;7967120
46 - 23;8134946
47 - 23.5;8200341
48 - 24;8245815
49 - 24.5;8333260
50 - 25;8441030
51 - 25.5;8560222
52 - 26;8688468
53 - 26.5;8811256
54 - 27;8936022
55 - 27.5;9062144
56 - 28;9191461
57 - 28.5;9311322
58 - 29;9405987
59 - 29.5;9457143
60 - 30;9475560
61 - 30.5;9583153
62 - 31;9725918
63 - 31.5;9851502
64 - 32;9973659
65 - 32.5;10101358
66 - 33;10269765
67 - 33.5;10406498
68 - 34;10567250
69 - 34.5;10646753
70 - 35;10668518
71 - 35.5;10848876
72 - 36;11116528
73 - 36.5;11313414
74 - 37;11480176
75 - 37.5;11665579
76 - 38;11956247
77 - 38.5;12169224
78 - 39;12366103
79 - 39.5;12659792
80 - 40;12861479
81 - 40.5;13025300
82 - 41;13078062
83 - 41.5;13150198
84 - 42;13345669
85 - 42.5;13516808
86 - 43;13742630
87 - 43.5;13877831
88 - 44;13995151
89 - 44.5;14097797
90 - 45;14223967
91 - 45.5;14368456
92 - 46;14444050
93 - 46.5;14481415
94 - 47;14563200
95 - 47.5;14671074
96 - 48;14724482
97 - 48.5;14756360
98 - 49;14873400
99 - 49.5;14998349
100 - 50;15118407
101 - 50.5;15234657
102 - 51;15347405
103 - 51.5;15456215
104 - 52;15576764
105 - 52.5;15725249
106 - 53;15873175
107 - 53.5;16030674
108 - 54;16088908
109 - 54.5;16269091
110 - 55;16577724
111 - 55.5;16771247
112 - 56;16877810
113 - 56.5;17045929
114 - 57;17219524
115 - 57.5;17299753
116 - 58;17382716
117 - 58.5;17557747
118 - 59;17737405
119 - 59.5;17920654
120 - 60;18093245
121 - 60.5;18272161
122 - 61;18476065
123 - 61.5;18689138
124 - 62;18883946
125 - 62.5;19000102
126 - 63;19105932
127 - 63.5;19256660
128 - 64;19407053
129 - 64.5;19548817
130 - 65;19710883
131 - 65.5;19860489
132 - 66;19896917
133 - 66.5;19953081
134 - 67;20041943
135 - 67.5;20164845
136 - 68;20295873
137 - 68.5;20379700
138 - 69;20491400
139 - 69.5;20590419
140 - 70;20675134
141 - 70.5;20759719
142 - 71;20837353
143 - 71.5;21025566
144 - 72;21353140
145 - 72.5;21651683
146 - 73;21977661
147 - 73.5;22071500
148 - 74;22183314
149 - 74.5;22329156
150 - 75;22468165
151 - 75.5;22609873
152 - 76;22708169
153 - 76.5;22828115
154 - 77;22943777
155 - 77.5;23077548
156 - 78;23138758
157 - 78.5;23155083
158 - 79;23170762
159 - 79.5;23186401
160 - 80;23321989
161 - 80.5;23471715
162 - 81;23656023
163 - 81.5;23778791
164 - 82;23895626
165 - 82.5;23970622
166 - 83;24053704
167 - 83.5;24135385
168 - 84;24214940
169 - 84.5;24303273
170 - 85;24389567
171 - 85.5;24477673
172 - 86;24570594
173 - 86.5;24665449
174 - 87;24758892
175 - 87.5;24854831
176 - 88;24952777
177 - 88.5;25064289
178 - 89;25186210
179 - 89.5;25306352
180 - 90;25413509
181 - 90.5;25519485
182 - 91;25633215
183 - 91.5;25742790
184 - 92;25851064
185 - 92.5;25937908
186 - 93;26022267
187 - 93.5;26107854
188 - 94;26197508
189 - 94.5;26288247
190 - 95;26376399
191 - 95.5;26468010
192 - 96;26569919
193 - 96.5;26680827
194 - 97;26818543
195 - 97.5;26962317
196 - 98;27077318
197 - 98.5;27235176
198 - 99;27425598
199 - 99.5;27607673
200 - 100;27782424
201 - 100.5;27924966
202 - 101;28036636
203 - 101.5;28204655
204 - 102;28443398
205 - 102.5;28593810
206 - 103;28651603
207 - 103.5;28747218
208 - 104;28833646
209 - 104.5;28917874
210 - 105;29027374
211 - 105.5;29077299
212 - 106;29241203
213 - 106.292;29361949
214 - 106.792;29599178
215 - 107.292;29725462
216 - 107.792;29951036
217 - 108.292;30167416
218 - 108.792;30356527
219 - 109.292;30553962
220 - 109.792;30785757
221 - 110.292;31056607
222 - 110.792;31247416
223 - 111.292;31437524
224 - 111.792;31663804
225 - 112.292;31922798
226 - 112.792;32110489
227 - 113.292;32295577
228 - 113.792;32455903
229 - 114.292;32609277
230 - 114.792;32756312
231 - 115.292;32878877
232 - 115.792;32954705
233 - 116.292;33056837
234 - 116.792;33213247
235 - 117.292;33379771
236 - 117.792;33576733
237 - 118.292;33737230
238 - 118.792;33861613
239 - 119.292;33976371
240 - 119.792;34110602
241 - 120.292;34253321
242 - 120.792;34421093
243 - 121.292;34497090
244 - 121.792;34615251
245 - 122.292;34767028
246 - 122.792;34890557
247 - 123.292;35090899
248 - 123.792;35275276
249 - 124.292;35422404
250 - 124.792;35566149
251 - 125.292;35691201
252 - 125.792;35891038
253 - 126.292;36045748
254 - 126.792;36308984
255 - 127.292;36489578
256 - 127.792;36682195
257 - 128.292;36947754
258 - 128.792;37159358
259 - 129.292;37391925
260 - 129.792;37601102
261 - 130.292;37800348
262 - 130.792;38073956
263 - 131.292;38237599
264 - 131.792;38440120
265 - 132.292;38704127
266 - 132.792;38971982
267 - 133.292;39201238
268 - 133.792;39348318
269 - 134.292;39534633
270 - 134.792;39717948
271 - 135.292;39915320
272 - 135.792;40097571
273 - 136.292;40244053
274 - 136.708;40394041
275 - 137.208;40543204
276 - 137.708;40614292
277 - 138.208;40749222
278 - 138.708;40808313
279 - 139.208;40998347
280 - 139.708;41166016
281 - 140.208;41331166
282 - 140.708;41479392
283 - 141.208;41600159
284 - 141.708;41708704
285 - 142.208;41801565
286 - 142.708;41903505
287 - 143.208;42008853
288 - 143.708;42107728
289 - 144.208;42231362
290 - 144.708;42389122
291 - 145.208;42564583
292 - 145.708;42720721
293 - 146.208;42863789
294 - 146.708;43003758
295 - 147.208;43192594
296 - 147.708;43289671
297 - 148.208;43364036
298 - 148.708;43443949
299 - 149.208;43524929
300 - 149.708;43628333
301 - 150.208;43670264

JW Player

User  
0 rated :

Hi Guys,

Thanks for the info and the test file. I'm looking into seeking for h264 files as well. So far I've come to the conclusion that it's not as simple as mod_flv_streaming (the lighttpd plugin). You can't just simply seek in a h264 file and pass that data to the flash player. It doesn't like that it all.

So I've been thinking along the lines that a new lighttpd plugin (mod_264_streaming?) needs to be a smarter. I.e. it reconstructs the necessary h264 headers (so-called 'moov') and then send the movie data ('mdat') starting from the seekpoint.

The thing is: this is quite a bit of work. Everytime somebody seeks in a file the plugin has to parse the original h264 headers, fix up all the offsets and then send the result back.

My guess is that this may work, but at the same time something tells me that there must be a simpler/easier way to achieve seeking.

If you guys have any ideas or links to other articles it's much appreciated.

Regards,

Arjen

JW Player

User  
0 rated :

Hi Arjen!

Do you think the metadata really needs to be sent when seeking? This is also not necessary when using normal flv files. I would asume that Flash just caches the data or so...

Greetings,
Bas.

JW Player

User  
0 rated :

From what I understand, as the H.264 file needs to be consistent to be playable, the file structure must be reconstructed "on the fly" when seeking.
IOW, as stated Arjen, the H.264 header (i.e. the atoms) must be set.

Thus, FLV files are much more easier to handle : no need to send back a sophisticated header.

JW Player

User  
0 rated :

Hi Bas, hi Chris,

Indeed, that's the main difference between flv/mpeg1/mpeg2 and h264. The first are all simple streams where you could start playing pretty much anywhere in the stream. H264 is using tables and indexes (stored in the moov atom) with absolute file offsets. So when sending only part of the stream, you'd need to patch these offsets. A bit similar to what qt-faststart and qtstreamize do.

I'm just curious what Adobe's ideas is on how this would eventually work. They could 'fix' it in the player I guess, but I haven't read anything about that.

Reconstructing the headers is a pain, but I'll keep you posted on any progress.

JW Player

User  
0 rated :

Hi all,

First of all, thanks for sharing your thoughts about h264 in flash http streaming. It seems that for consistent file playback with seeking one would need to reconstruct the entire tables and indexes every time the user seeks.

However, for live stream broadcasting (which is my own target), seeking is not required, which means that the incoming client would only have to get the stream from the "last keyframe seen" on the live server.

But does it mean that the index can only have one entry, the beginning ? In other terms, could one do h264 http progressive streaming without having to index the whole file ?

The first task i'm trying to clarify is the binary string that allows to identify an Intra frame in the bitstream, i'll keep you posted if i get to something.

Cheers

JW Player

User  
0 rated :

Hi Flo,

You mean 'live' as in not pre-recorded (E.g. a live webcam)? I haven't heard of a solution yet. You are likely to run into the same issues as with progressive downloading of h264 streams.

Or do you mean something more similar to: http://www.apple.com/quicktime/broadcaster/ ?

Being able to broadcast 'live' in h264 would solve the same problems, so if you find any more info let us know.

Regards,

Arjen

JW Player

User  
0 rated :

Hi guys,

Just to keep you informed about some progress I've made.

I've written a basic plugin for Lighttpd (named mod_h264_streaming). Similar to the mod_flv_streaming plugin you pass it a start position in the URL to tell it where to start playing.

There are still some loose ends, but the basic seeking functionality is working. I'll post a link to a demo soon.

Regards,

Arjen

JeroenW

JW Player Support Agent  
0 rated :

Great info! How did you resolve matters with regards to the correct MP4 headers Flash seems to want?

JW Player

User  
0 rated :

Hi JeroenW,

When you request an MP4 file the plugin fixes the necessary headers and chomps off the skipped data. (Specifically: it patches all the atoms in 'stbl', i.e.: stts, stss, stsc, stsz and stco).

I haven't tested it on many different streams (yet), but it works fine on streams generated with ffmpeg/xvid264 and Bas's trailer2 file.

For the flash player things are likely a bit different as well. The biggest difference is that seeking in a .MP4 file will re-trigger onMetaData events.

Arjen

JeroenW

JW Player Support Agent  
0 rated :

OK. The re-triggering of onMetaData isn’t a big problem. I’ll just have a look at this.

JW Player

User  
0 rated :

Some great progress here. Arjen - please would you upload your source somewhere, I'd love to have a look to try and get my head around it for a PHP implementation. thanks.

JW Player

User  
0 rated :

Hi,

I've put up a little page with a demo. It's using the previously mentioned trailer2.mp4 (thanks Bas!).

Note that this is *work in progress* and that we will be putting up the SVN repository with the lighttpd plugin shortly.

Jeroen: It would be really nice if you could incorporate this in your video player. We're happy to help.

Here's the wiki page: http://h264.code-shop.com/ and the demo: http://h264.code-shop.com:8080/longtail_player.html

Regards,

Arjen

JW Player

User  
0 rated :

The demo looks great!

But what's the username/password for the SVN?

JW Player

User  
0 rated :

Hi Arjen!

The demo looks excellent. What kind of player are you currently using on the page?

Bas.

JW Player

User  
0 rated :

Hi,

challefredde: SVN now has anonymous access, so no username/password should be needed. The files of interest are src/mod_h264_streaming.c and src/moov.c. The latter containing the low-level atom shuffling.

Bas: The player is self fabricated, but nothing special. I mainly use it for experimenting with the Flash beta releases (H264, Full-screen, etc...).

Thanks,

Arjen

JW Player

User  
0 rated :

I get this error when I try to compile it with make on an Intel Mac running Tiger.

..........
moov.c: In function 'atom_skip':
moov.c:85: warning: unused parameter 'buffer'
moov.c: In function 'trak_build_index':
moov.c:741: error: 'off_t' undeclared (first use in this function)
moov.c:741: error: (Each undeclared identifier is reported only once
moov.c:741: error: for each function it appears in.)
moov.c:741: error: parse error before 'pos'
moov.c:745: error: 'pos' undeclared (first use in this function)
moov.c: In function 'trak_write_index':
moov.c:886: warning: comparison between signed and unsigned
make[2]: *** [moov.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any idea what this means?

JW Player

User  
0 rated :

Hi challefredde,

Please change 'off_t' on line 741 to 'unsigned int' and you;re good to go.

The full line after the fix will read:

unsigned int pos = trak->chunks_[j].pos_;

Thanks for reporting and let me know if you run into any more troubles.

Arjen

JW Player

User  
0 rated :

It works great on all my mp4 files (with h264).

I put together a Mac installation package of lighttpd with the plugin:

[url=http://rapidshare.com/files/64649520/lighttpd-1.4.18.zip.html]http://rapidshare.com/files/64649520/lighttpd-1.4.18.zip.html[/url]

Great work Arjen!

JW Player

User  
0 rated :

Hi challefredde,

Nice to hear that it's working for you!

Would you mind if we put the mac installation package on the Wiki page next to the other downloads? It would be nice to keep them together.

Even better would be to include your installation/packaging script in SVN. What are your thoughts on that? It would make a nice contribution. :)

On a related note. Do you encode your mp4 files yourself? If so, I'm still looking for the best tool (ffmpeg?) and command line options to use.

Best regards,

Arjen

JW Player

User  
0 rated :

Hi again Arjen!

No, I don't mind if you put it on the wiki/svn.

I packaged it after I compiled it, with Apple's Packagemaker, together with my own additions: StartupItem (start at boot), start/stop shellscripts etc.

Yes I encode my mp4 files myself. I'm using different tools but mostly this combination:
On Mac OS X 10.4 Tiger:
- Perian plugin installed (watch almost all videos in Quicktime)
- Mpeg Streamclip from Squared 5
- x264 Quicktime codec, http://www.macupdate.com/info.php/id/20273/x264-quicktime-codec
Encodes better h264 videos than Quicktime's built-in.
- QTFastStart

Video: 900 kbit/s
Audio: AAC, stereo 96 kbit/s
Res: 640x360 (widescreen)
The video plays fine on iPod's.

What operatingsystem do you use?

/challefredde

JW Player

User  
0 rated :

JeroenW: It would be great if your player could handle this.

JW Player

User  
0 rated :

Hi challefredde,

Thanks, I've added your package (and your encoding tips) to the wiki.

We encode the videos on a *nix (Ubuntu) machine and use ffmpeg for this. Development for the lighttpd plugin is done both on *nix and pc, so that the code has seen at least a few different compilers. :)

Just thinking out loud here. Would there be interest in, say, a similar apache module or is lighttpd the leader in the pack when it comes to video streaming?

Take care,

JW Player

User  
0 rated :

@Arjen,

There would be interest in an Apache module.

Also, would it be feasible to do the same thing in a PHP script? I haven't looked at the code, so I don't even know what language it is written in or what you are doing within the code, but I'd be interested in translating it to PHP if you think that is possible. :)

JW Player

User  
0 rated :

I use it like an apache module by using apache as a proxy for one domain.

Apache runs on port 80.
lighttpd on port 81.
only port 80 visible externaly

When someone try to access media.domain.com (on port 80) apache forwards it to the lighttpd server.


I've made some changes in the Mac package.

Intel: [url=http://rapidshare.com/files/64941787/lighttpd-intel.zip.html]http://rapidshare.com/files/64941787/lighttpd-intel.zip.html[/url]
PowerPC: [url=http://rapidshare.com/files/64941989/lighttpd-powerpc.zip.html]http://rapidshare.com/files/64941989/lighttpd-powerpc.zip.html[/url]

/challefredde

JW Player

User  
0 rated :

Hi,

@Will: Thanks for your reply. I'll add +1 to the Apache module. :)

PHP may be possible, but I think it will be quite the challenge. A lot of different tables have to be 'parsed' and patched and it involves quite a bit of bit twiddling. Have a look at the file src/moov.c and see what you think?

Are there any advantages of using PHP over Lighttpd or Apache?

@challefredde: Thanks, I've updated the links on the wiki.

Arjen

JW Player

User  
0 rated :

@Arjen,

We were all hoping that we could use a simple PHP stream script to do fake streaming like we do now with FLVs. :d

I could certainly do it with Apache, or ffmpeg, or even the Red5 server that I have running already, but for many users, their only option on a colo is to run a PHP script.

JeroenW

JW Player Support Agent  
0 rated :

@challefredde: working on it!

@Arjen: I’ve posted a message on your site asking if you could get me going on the changes needed to setup h264 streaming with my player. Could you please send me an email? Then I can add support for this!

JW Player

User  
0 rated :

New files for Mac again! :-)

- Mac OS X 10.5 Leopard compatible

Intel: http://rapidshare.com/files/65648028/lighttpd-intel.zip.html

PowerPC: http://rapidshare.com/files/65649640/lighttpd-powerpc.zip.html

Arjen:
Tip about H264 encoding on Mac's: For a beginner in the encoding world, [url=http://www.techspansion.com/visualhub/]VisualHub[/url] i much easier.

/challefredde

JW Player

User  
0 rated :

+1 for a php implementation.
Most of us can't install any Lighttpd module (nor Apache), while php hosting services are quite common.

JW Player

User  
0 rated :

this all is not working cause the file container mp4 format is not a streaming format like flv.

you MUST! rebuild all atooms for every request seeking position.

that might work for small files, but what happen if you have an videofile >30 min, or 1 hour?

thats about 1 till 3megabytes of atooms :)

maybe you add an oldstyle flash 3 loading gfx, just joking.

you must send all atooms for jumping thru the files!

i think you will only do real seeking with rtmp(e) and flashmedia server 3.

give me feedback what you think about this.

best
tom

JW Player

User  
0 rated :

Is FMS3 not rebuilding the atooms ?

JW Player

User  
0 rated :

Jimmy (28.10.2007):

Is FMS3 not rebuilding the atooms ?


no, they dont need to do that. they pack the h.264 raw packets to rtmp(e) and maybe to rtmp packets. cause they use an "simple" encryption with rtmp(e) they have a good position to fight for illegal use of their rtmp(e) protocol. break or rebuild en/decryption is another thing then to understand what kind of bytes the flash player sends to a server.



JW Player

User  
0 rated :

Hi Tom,

Agreed. You need to rebuild the atoms on every seek, and that's exactly what the lighttpd plugin does. :)

You are right, in that for very large files the 'moov' atom can get quite big. There are a few things that can help with making smaller 'moov' atoms.

First there is the way how you encode files. There are encoding options which may help to reduce the size of certain atoms (ctts, stss).

A second option could be to include ZLIB compression in the plugin. You can compress the whole moov atom. I haven't done any tests, but the atoms taking up most of the size definitely have a lot of redundant data.

To summarize, it is not as easy as FLV pseudo-streaming, but (for me anyways) it's a step in the right direction.

Of course, ideas are more than welcome.

Take care,

JW Player

User  
0 rated :

Hi Arjen,

i look at your c-code and see much work you had. for this reason i want to help what i can.

there is one chance we have.

if we can figure out the vide codec id for the h.264 codec we can remux the h.264 raw packets into flv packets in realtime.

look:
we have 3 codecs in flash 8,9:

h2.63 byte codecID - 3
vp6 byte codecID - 4
bitmap screen codeID - 2

we also have tags for audio and video in the flv file structur.

0x08 = audio
0x09 = video

of course we have serveral data tags, that not so important for this project.

now we need to convert in realtime the h2.64 into 09 video tags for flv.

yes, there is more to do with timecode etc. but this is simple to rewrite.

when we got this, we dont need to use the mp4 filecontainer format.

read the article on kaourantin.net
Will it be possible to place H.264 streams into the traditional FLV file structure? It will, but we strongly encourage everyone to embrace the new standard file format. There are functional limits with the FLV structure when streaming H.264 which we could not overcome without a redesign of the file format. This is one reason we are moving away from the traditional FLV file structure. Specifically dealing with sequence headers and enders is tricky with FLV streams.

Will it be possible to place AAC streams into an FLV file structure? Yes, though the same limitations as for H.264 apply.

-
we will use the flv format for "pseudo" streaming.

take care,
tom

JW Player

User  
0 rated :

Hi Tom,

Sounds like a nice trick! All you need then is a preprocess tool that converts an MP4 stream into an FLV container. You could even use the existing mod_flv_streaming and PHP scripts to handle the pseudo streaming.

Do you know of anyone who has tried to put an MP4 stream into an FLV container? Any links?

Best regards,
Arjen

JW Player

User  
0 rated :

hi arjen,

not yet...we need to wait if we got the new specification from adobe or an flv with h2.64.

that means about 6 - 12 month after the fms3 will released (dont know it excatly)

i tryed to write an own muxer and add +1,+1,etc. to the known codecIDs. when i understand it right there are only 15 possible ids in the id tag for codecs, but nothing works here.

we dont know yet if the current beta flash player supports flv with h.264 packets.

we must wait until we got an flv demo file in our hands.

please let me know if you see a chance here.


best
tom

JW Player

User  
0 rated :

Hi Tom,

Somehow I doubt that Adobe will release the specification of an FLV file with H264 as they don't recommend it to begin with.

I think that for now our best bet is to stick to the actual MP4 specification for pseudo-streaming. There are still a few ways we can improve the lighttpd plugin.

What is the biggest problem for you currently? Is that the fact that you want to pseudo-stream *really* large files? If you have a scenario, we could look into that and see how we can improve things.

Take care,
Arjen

JW Player

User  
0 rated :

hi, Arjen,
Thanks for your great work and valuable practices.

Have you tried the fragmented movie format mentioned in ISO/IEC 14496-14? That is of the structure like 'ftyp/moov/moof->traf/mfra' style. It is said in A.8 that this kind of mp4 files can be friendly for the HTTP fast-start senario. It's similar to the flv format. I wonder if the beta version flash player can handle these atoms.

For testing purpose, a specific mp4 muxer (other than ffmpeg) is needed to generate the mp4 file.
P.S. It's possible to do live streaming via http, however, the current mod_h264_streaming approach can not achieve that. Let's see how it goes via fragmented mp4 files.

Regards,
trueice at gmail

JW Player

User  
0 rated :

Hi trueice,

Thanks for the pointer. If I read it correctly, a fragmented movie would help in getting the size down of the initial moov atom. Which in turn would make the movie start faster.

Do you know of an MP4 muxer that supports this feature? The first thing to try would be to see if the FlashPlayer supports these atoms.

Regards,

JW Player

User  
0 rated :

Hi,

Isn't this what gpac supports using mp4box -frag option? Tried it, flash 9.0r64 (linux) doesn't play these files..

Best,

JW Player

User  
0 rated :

Hi, Arjen,

I've only found that mplayer has fragmented demuxer support
( see http://lists-archives.org/mplayer-dev-eng/20154-movie-fragment-support-for-mov-mp4-demuxer.html). That will be helpful for us to guess the behavior of the ISO/IEC 14496-14 compliant mp4 players.

Also, mp4v2 has limited 'moof/traf' atoms support.

Maybe we could start at ffmpeg/libavcodec/movenc.c to encapsulate AVPackets into movie fragments(moof), each moof has two track fragments (a/v). Each video chunk only contain one video sample like the flv 'tag' does. Each moof is then constructed by a group of samples which has a random accessable (intra/key) sample as the first one. The 'mfra' atoms are then inserted into the stream to address these keyframes for the player. See 8.37-8.39 and C.7 for more details:)

Regards,
trueice # gmail



JW Player

User  
0 rated :

hi, EricM,

Thanks, I'll look at gpac's code for details:)

Regards,
trueice # gmail

JW Player

User  
0 rated :

Hi, EricM, all,

I've also tried with the GPAC produced fragmented movie file, and the flash player just reports:
'NetStream.Play.NoSupportedTrackFound'
However, that file also can not be played by Quicktime.

In hex mode, I found that the file contains a mdat atom prior to the moof/traf/trun atoms. That's weird:P

No enough time, just a small report:)


Regards
trueice

JW Player

User  
0 rated :

Hi trueice, EricM and all,

Any success with the fragmented movie files?

Could you perhaps post the commandlines of the different tools you've used to create these files? That may save me some time. :)

Thanks,

Arjen

JW Player

User  
0 rated :

hi, Arjen,

I just used 'MP4Box -frag 100 filename.mp4' to create the fragmented output file. I'll look into this issue within these two days.

The new flash player only supports a subset of the ISO/IEC 14496-12 standard. I've posted a comment about the moof support on kaourantin.net, but still no reply by now.

Regards,
trueice

JW Player

User  
0 rated :

Awesome work guys! (b) Any new developments? :) My company wants to "stream" 20-30min clips, but 1-2M meta data for every seek seems a bit insane. Any clues on slimming this down? We're building files with FFMPEG as suggested:

http://h264.code-shop.com/trac/wiki/Encoding

JeroenW

JW Player Support Agent  
0 rated :

@Chris: 3.13 will be there next week I think…

H.264 streaming is already working – www.jeroenwijering.com/extras/streaming.html

JeroenW

JW Player Support Agent  
0 rated :

3.13 is out – with Lighttpd H264 streaming! [url=http://www.jeroenwijering.com/extras/streaming.html]Here’s a demo[/url].

JW Player

User  
0 rated :

I am on a server using Lighttpd. Do I need to enable another mod like I did for FLVs to get the h264s to stream? mod_h264_streaming? Thanks!

[url=http://www.simplethoughtproductions.com/2007/12/16/h264_flash_streaming/]Testing Here[/url]

Edit: Looks like I do need to get the new mod installed. Found it on the website that had a demo.

JeroenW

JW Player Support Agent  
0 rated :

Check out [url=http://h264.code-shop.com/trac]this page on Lighttpd H264 streaming[/url]. It has a link to the H.264 module included.

JW Player

User  
0 rated :

I got the h264 module installed but its still not working, will have to double check its setup and readme to see if I missed something.Looking at your example it looks like you still use lighttpd as the stream script.

[url=http://www.simplethoughtproductions.com/wp-content/uploads/Other/josh/hd_test/video.html]My Test[/url]

JeroenW

JW Player Support Agent  
0 rated :

Yes indeed, you’ll have to set the flashvar “streamscript=lighttpd”. But you also need an array with keyframe-bytepositions in your H264 files. Some links on how to do this is also on the code-shop website.

JW Player

User  
0 rated :

OK, The video I am using is the one from your sample. I downloaded and uploaded it without editing it. I wouldn't think this would change the file. I wonder what is going on then?

JeroenW

JW Player Support Agent  
0 rated :

No indeed, if you just download the sample (the movietrailer – not the 5sec clip on my site!), it should work…

JW Player

User  
0 rated :

Yeah I downloaded the trailer 2 file, and uploaded it to my server. Then had my admin install the mod_h264... I will have to double check with him and make sure it was enabled and such, it seems to load up the file, ready to stream, but if you seek some where, it goes to that point, but starts playing from the begining but with the time stamp you seeked to...

JW Player

User  
0 rated :

Hey, I help admin Josh's server and we just got H.264 streaming working. Our server is using Ubuntu 7.10 (Gutsy) and lighttpd 1.4.18 from the repositories. I tried to use the Feisty repo for the binary mod_h264_streaming module, however it won't install unless you install lighty 1.4.13 or force it through dpkg, so I just grabbed the source, installed, and setup as follows:

Download and build the module:
$ svn co http://h264.code-shop.com/svn/h264/trunk/lighttpd-1.4.18 lighttpd-1.4.18
$ cd lighttpd-1.4.18
$ ./configure
$ make

Copy the module and set up lighty to use it:
$ sudo cp src/.libs/mod_h264_streaming.so /usr/lib/lighttpd/
$ cd /etc/lighttpd/conf-available
$ sudo touch 10-h264.conf
$ sudo nano 10-h264.conf

Put something like this in the file:
## Load the H.264 Streaming module
server.modules += ( "mod_h264_streaming" )
## Tell the module what to stream
h264-streaming.extensions = ( ".mp4" )

Then, enable the config and reload lighty:
$ sudo ln 10-h264.conf ../conf-enabled/
$ sudo /etc/init.d/lighttpd force-reload

After all that you should have H.264 streaming for Flash working.

JW Player

User  
0 rated :

Hi all

I was told somewhere that live streaming with h264 in upcoming Flash products won't use the regular container format... I hope live streaming will still be possible using poor-man's (no offense intended, or this would be self-offense :p) means.

Here's the deal: how can you live stream h264 if
* the moov atom is only written at the end of encoding
* you don't know the total length nor keyframes addresses before flushing the metadata

My guess is the encoders keep such metadata in memory before finishing the encoding/muxing. Maybe this progressively-enriched data could be exported before the end of the process... I'll try looking into x264 to see if this is not just a proof of dumb intuition.

If there is a way to get this data before ending the encoding process, then it should be possible to reconstruct headers (based on this very lighty mod) every once in a while.

Anyone tried the same ? Any suggestions ?

Cheers, and, again, well done !

Flo

JW Player

User  
0 rated :

Hi!

Feedback on H.264 Pseudo Streaming with Lighttpd: Installed the 3.13 player on our test server, and tested it with a 200MB/30min H.264 MP4 file. Everything worked well, but when limiting the Lighttpd bandwidth to 150K/s, skipping required nearly 10 seconds buffering. I expect this to be because of the rather large amount of meta data that needs to be downloaded every time I skip in the file :-/ Compressing this data was suggester above, but any clues on this? Does 3.13 support this? :)

JeroenW

JW Player Support Agent  
0 rated :

@Chris: a quick calculation, you’re using about 110 kbps? Please make sure that the ‘bufferlength’ variable isn’t set to 10sec or so, but to a very small amount (1?).

It might be that there’s a lot of metadata, but I doubt (hope) it’s not a couple 100s of kilobytes…

On a sidenote, with our Bits on the Run system, we are working on modifying the mod_flv_streaming and mod_h264_streaming a bit, so it supports a quick burst (non-limited bandwidth) for the first couple of seconds. That’d decrease any waiting times when skipping through Lightppd-streamed movies.

JW Player

User  
0 rated :

Hi JeroenW!

Modifying the mod_h264_streaming sounds intereseting, where can I read more about it (or download a beta version)?

JW Player

User  
0 rated :

Hi Jeroen,

Any more info on the 'quick burst' you mentioned? If you could explain a bit more, or have any pointers, I'd be happy to help.

Best regards,

Arjen

JW Player

User  
0 rated :

Hi all,

Finally, I found a valid fragmented 3gpp2 video sample:
http://samples.mplayerhq.hu/mobileVideo_3gp/MAV_0001.3G2

(It seems being recorded with a SANYO W21SA mobile phone.)

You can use the cmdline:
$ ./AtomicParsley MAV_0001.3G2 -T
to check its atom tree.

This file can be played-back by mplayer and Quicktime, while the flash player still reports
'NetStream.Play.NoSupportedTrackFound'

Conclusion: the flash player does not support fragmented movie for now.

JW Player

User  
0 rated :

Hi!

I finally set up my lighttpd server last night with mod_h264_streaming - and all I can say is, it's working like a charm; it also assorts well with mod_secdownload and bandwith limitation.
It's successfully streaming a more than 30min. h264 file with over 500MB data and skipping takes just two or three seconds (that's mainly because of the bandwith limitation, I suppose).
So thanks a lot guys for your great work!!

btw: One of my machines had flash player 9.0 r60 (the early beta with h264 support), and it couldn't skip in content, with upgrade to r115 it's working fine now.

JW Player

User  
0 rated :

where is the mod_h264_streaming.so file?

after I

./configure
make

I can't fine .so any where to copy over. Can someone help?

JW Player

User  
0 rated :

Hi trueice,

Thanks for the feedback on the fragmented movie files. Too bad the flash player doesn't support it (yet). It would have been neat.

Arjen

JW Player

User  
0 rated :

For very large files, moov.c does not shave off any bytes -- it sends the entire file.

JW Player

User  
0 rated :

Hi Jason,

Do you mean that seeking doesn't work and that it starts playback from the beginning?

There are a few cases which aren't covered yet (like 64 bit atoms). If you have a link to an example file, I can try and fix it.

Regards,

JW Player

User  
0 rated :

i couldn't make it run with mod_secdownload, any tips would be appreciated

JW Player

User  
0 rated :

ok i fixed it :D

if it happens to you just make sure you've enabled mod_h264 after mod_secdownload.

ex:

server.modules += ( "mod_secdownload" )
server.modules += ( "mod_h264_streaming" )

:D

JW Player

User  
0 rated :

I've got the same problem. Seeking isn't working very well. I use ffmpeg with libx264 for h.264 conversions and AAC sound codec.

here's the URL: http://cahe.mendrek.net/pkkb/upload.php <= upload and playlist selector.

Please help :-)

JW Player

User  
0 rated :

the new url: http://at.tasystems.pl

JW Player

User  
0 rated :

Hi Cahe,

Could you be a little more specific about what is not working? I've checked your new url, and the videos (like: http://at.tasystems.pl/watch.php?5) play and seek fine.

JW Player

User  
0 rated :

Hi,

Well... it is now :-). I user qtfaststart as suggested on h264 streaming page... and it worked ;-). Thanks.

JW Player

User  
0 rated :

But not quite...

Look at this: http://at.tasystems.pl/watch.php?17

It's streaming alright, but if you click somewhere to start from the middle you have 50-50 chance it's going to stop and get back to initial state (covered with the jpeg, stopped). How do you avoid that? flash r115/firefox/windows

JW Player

User  
0 rated :

If just tried on the same system (r115/ff/win) and been seeking like mad. It never restarted. :)

Any change if you could track down if it always fails on a certain seekpoint? (Example URL: http://movie.mendrek.net/329fe24fff767d73df8bbc2c6a106466a83b1620.mp4?start=33.133). That would certainly help.

If you have FireBug installed (Firefox plugin) you can see the exact requests that the player makes in the 'Net' tab.

Thanks,

JW Player

User  
0 rated :

Arjen

Can you please share your lighttpd.conf? I couldn't manage it to work on a linux box properly, server crashes with 100 concurrent connections :(

JW Player

User  
0 rated :

no lighty stuff here? its bad

JW Player

User  
0 rated :

Hello All,
Thanks to read,
Is RED5 capable to stream file format other than flv and mp3 , if possible than how....
Suggest what u have idea about it.

JW Player

User  
0 rated :

Hi all,

Dirty solution I use for "pseudo-stream" HD is to create 10 mp4 files of my original mp4 file which have a different start delay :

1.mp4 (0:00 -> 2:00) (33Mo)
2.mp4 (0:20 -> 2:00) (25 Mo)
3.mp4 (0:40 -> 2:00) (15 Mo)
4.mp4 (1:00 -> 2:00) (10 Mo)
........

I know it's very very dirty and it takes a lot of disk space on the server but it works well. Waiting for a cleaner solution ;-)

JW Player

User  
0 rated :

Hi Tom,

That's what this thread is all about. You can use pseudo streaming using Jeroen's latest player and a lighttpd plugin called mod_h264_streaming (http://h264.code-shop.com/trac). This would be the clean solution. :)

JW Player

User  
0 rated :

So I've tested this implementation of it:
http://h264.code-shop.com:8080/longtail_player.html

Although it's functional it doesnt seem that responsive. Is this because of the time needed to buffer (downloading the extra metadata) or the work being done on the server side to recaluclulate this?

I've been doing some testing myself and i cant see why you just can't calculate the nearest keyframe serverside, cut the video here and add a h.264 header to it, repackage the mp4 file and send this (I dont know how the audio will act though). The flash player will think it's a new video and play from the start, but if you create your own progressbar wrapper and keep track of what time the video is at, you should be able to send the time back to the server if you seek again.

Now, I've never programmed in flash so this might be what you guys have been talking about above, so sorry if i've missunderstood anything

JW Player

User  
0 rated :

Hi Dave,

Everytime you seek to a different part of the file, the MP4 header is patched (quick) and transferred to the client (slowish, depending on the size of the header).

What you describe is pretty much how it works. The video is cut and a new h264 header is created. The flash player thinks it's a new video and plays from the start; Seeking into the video with keeping track of the progressbar is what Jeroen has added in his latest player.

The responsiveness can still be improved. For example by compressing the patched h264 header.

Ideas and thoughts are more than welcome.

Regards,

JW Player

User  
0 rated :

Ok, sorry i'm still a little bit of a noob on this and just thinking how it will work (At least i can say great minds think a like)

Anyway, is the header static for each video? I've looked a raw H.264 file with mpeg4ip (h254_parse) and as far as i can tell it is would be for the whole stream (I've even tried cutting the file and putting the same header back on and both vlc and wmp10 can play it). Again audio and muxing in mp4 might be different.

So can you store the header of the video on the client and patch as you go in flash? Does the flash player allow you to connect to a local datastream where you can throw in these bytes as it plays? Kind of like a custom datasource in Java media framework.

I've started to read through the iso spec on both mpeg4 - avc and mp4 and from what i can tell the header isnt really that big. What size and what is exactly needed? I do want to really help so i'll continue to look myself but extra help would be appreciated.

JW Player

User  
0 rated :

Hi Dave,

Flash doesn't really provide any low-level access to the video played (As far as I know). It just expects a properly formatted MP4 file. When you tell it to play a video it expects a 'complete' MP4 video, and always reads the headers first and then the audio/video data.

So the lighttpd plugin does exactly that. It generates a completely new MP4 file, but does so by only needing to read the header and patch it up. The upside of this is that you don't need to store anything locally on your server, the file is patched dynamically.

So when a new start time is requested by the flash player. The plugin patches the header. It's not rocket science, it just means updating the times/frames/offsets in the different header parts (also called atoms in MP4). After the header there is this big chunk of actual audio/video data. According to the start-time we calculate how many bytes we can skip from this (mdat) chunk.

The plugin then sends the header first (from memory) and the megabytes of audio/video data is transferred using lighttpd's memory mapped feature.

I'm sure there are a lot of different configurations out there which may perform differently, but for the videos (about 30 mins long) I've encoded the seeking is rather quick, especially when you have the feedback (=timeline) of the buffering in the flash player. It really takes less than a second to start playback at a new position.

Of course, if you have any videos where it constantly takes, say, 5 seconds, I'd be happy to investigate and see how we can improve things.

Kind regards,

JW Player

User  
0 rated :

One second is fine. The above link i mentioned was taking over 10 seconds. My net connection is ok but i should have tried it at work where it's 24mbit

Maybe the site is just a little slow. Do you have any other examples i can test?

Sound like all the hard work is done already - well done to everyone involved

JW Player

User  
0 rated :

Hello
I've tested the solution, that uses lighttpd server and it works great but we're planning to introduce the on-demand internet TV in our company and the technology we're using forbids us to install new web serwers (like lighttpd). Is there a way to make this patch similar to that written in PHP for flv videos (to provide the seeking functionality)?

Regards

JW Player

User  
0 rated :

Hi Pszemus,

Currently only a Lighttpd version is available, I'm not aware of any PHP scripts providing the seeking functionality.

JW Player

User  
0 rated :

Has an apache2 module for this been released or only for lighttpd ?

JW Player

User  
0 rated :

Hi Gordon,

I havent had many requests yet for an apache2 module. If you run Apache, you can always setup Lighttpd to serve just the video files, and thats what most people seem to do. On the other hand, it shouldnt be too difficult to wrap the plugin into an Apache module.

This question has received the maximum number of answers.