| *** tpb has joined #timvideos | 00:00 | |
| *** CarlFK has joined #timvideos | 00:24 | |
| *** ChanServ sets mode: +v CarlFK | 00:24 | |
| *** thaytan has quit IRC | 01:22 | |
| *** thaytan has joined #timvideos | 01:23 | |
| *** ChanServ sets mode: +v thaytan | 01:23 | |
| *** thaytan has quit IRC | 01:35 | |
| *** thaytan has joined #timvideos | 01:36 | |
| *** ChanServ sets mode: +v thaytan | 01:36 | |
| *** CarlFK has quit IRC | 02:25 | |
| *** CarlFK has joined #timvideos | 02:45 | |
| *** ChanServ sets mode: +v CarlFK | 02:45 | |
| *** hyades_ has joined #timvideos | 03:25 | |
| *** Niharika__ has joined #timvideos | 04:29 | |
| *** Niharika__ has quit IRC | 04:34 | |
| *** CarlFK has quit IRC | 04:43 | |
| *** Niharika__ has joined #timvideos | 04:59 | |
| *** Niharika__ has quit IRC | 05:03 | |
| mithro | MaZderMind: there are a couple of ways to get strong synchronization with inter* elements | 05:16 |
|---|---|---|
| mithro | MaZderMind: I'm interested to know how you are measuring audio sync? | 05:20 |
| *** hyades_ has quit IRC | 06:18 | |
| MaZderMind | mithro: I'm playing back a testclip into the system (http://c3voc.mazdermind.de/avsync-30min.mp4) and record it on the other end | 06:29 |
| MaZderMind | I then import both into kdenlive and play back | 06:30 |
| MaZderMind | I align them by video frame | 06:30 |
| MaZderMind | one can now hear that a) the audio does not lign up at the start and b) the clicks (one from the source and one from the recording) are getting closer and closer to each other until they lign up and finally drift away again | 06:31 |
| MaZderMind | while the video-frames are exactly identical | 06:31 |
| mithro | MaZderMind: and the audio is behind or ahead of the video? | 06:34 |
| MaZderMind | the starts ahead, catches uo and then drifts to beeing behind | 06:35 |
| MaZderMind | i hat a conversation on #gstreamer with heftig, he said beside other things: | 06:36 |
| MaZderMind | 10:19 < heftig> for swapping sinks I just use the multifdsink | 06:36 |
| MaZderMind | 10:19 < heftig> for swapping sources i have a system which can swap between a videotestsrc+audiotestsrc and a bin containing fdsrc+matroskademux | 06:36 |
| MaZderMind | unfortunately he can't offer source code. I currently try to replicate that | 06:36 |
| MaZderMind | it would result in one big pipeline | 06:37 |
| mithro | one big pipeline is really dangerous in my opinion as the more complicated a pipeline is, the easier it is for something to go wrong in it | 06:37 |
| mithro | However, in your case with a very fix configuration and very controlled environment it might be reliable enough | 06:38 |
| MaZderMind | yes, my oppinion too, but driftig audio/video is a blocker | 06:38 |
| *** Guest32262 is now known as aps | 06:38 | |
| mithro | MaZderMind: are you going to have time on the weekend to work on this? | 06:39 |
| MaZderMind | mithro: yes, I'm planning to work on this | 06:39 |
| MaZderMind | I'll too have to do some InDesign juggeling for my wedding in 3 months but nothing more :) | 06:40 |
| mithro | MaZderMind: How about we schedule a 4-6 hour block on the weekend were we both sit down and do some pair coding and analysis? I'm pretty sure we can fix your drifting problem with inter stuff but I'm likely to get distracted if I don't actually schedule a time to work on it | 06:41 |
| mithro | btw hardware hacker - I started the following for a "choosing the right FPGA board" talk I'm doing at work https://docs.google.com/document/d/14MVW0ncpYKrkJtC4wGY7wMX-CbNFKmcFpsHzXl1khug/edit | 06:45 |
| tpb | Title: FPGA Development Boards - Google Docs (at docs.google.com) | 06:45 |
| MaZderMind | mithro: those timezones… how about Sunday, 6pm to 8pm Sydney Time? | 06:48 |
| mithro | MaZderMind: yeah I was thinking reserving like 6pm -> midnight ? | 06:48 |
| MaZderMind | mithro: I missed that I have some appointments over the weekend I wasn't aware of… | 06:49 |
| *** Niharika__ has joined #timvideos | 06:50 | |
| mithro | I can do Saturday or Sunday | 06:50 |
| MaZderMind | mithro: yes, I'm planning around… but (unfortunately?) the wedding preparations take a lot of time | 06:52 |
| mithro | MaZderMind: yeah, no worries | 06:52 |
| mithro | MaZderMind: this is also why I like scheduling things because it uncovers things like this :) | 06:52 |
| MaZderMind | Saturday I planned with my witness (is that the correct word?) to look for Suit and I guess we'll be in the city the whole day and after that I'll be really done | 06:53 |
| MaZderMind | Sunday I just found hat I have to leanr dancing and we have a dancing class booked :-> | 06:54 |
| MaZderMind | But what interests me most: how were you plannung to "fix" the inter*-elements? | 06:55 |
| *** [d__d] has quit IRC | 06:56 | |
| MaZderMind | I could imagine that it may be possible to rewrite the interaudiosrc to pass on timestamps but as easy as this sounds - it may break a whole lot of things like syncing correctly to the target playlist's clock and such | 06:56 |
| mithro | MaZderMind: before I figure out how to "fix" the inter*-elements I need to first understand exactly what is going on that is causing the drift | 07:00 |
| MaZderMind | ok, sorry for my ungraceful wording – I assumed you knew about that problem | 07:04 |
| *** [d__d] has joined #timvideos | 07:09 | |
| mithro | MaZderMind: I want to trace the buffers going through the inter elements and understand when they are getting dropped / duplicated and how they are getting restamped | 07:10 |
| MaZderMind | GST_BUFFER_PTS (buffer) = intervideosrc->timestamp_offset + | 07:11 |
| MaZderMind | gst_util_uint64_scale (GST_SECOND * intervideosrc->n_frames, | 07:11 |
| MaZderMind | GST_VIDEO_INFO_FPS_D (&intervideosrc->info), | 07:11 |
| MaZderMind | GST_VIDEO_INFO_FPS_N (&intervideosrc->info)); | 07:11 |
| MaZderMind | there is similar code in interaudiosrc | 07:11 |
| MaZderMind | the PTS is here calculated based on the fps * the numer of frames passed through + the startup-offset -- as far as I can read that | 07:13 |
| MaZderMind | for audio that is | 07:13 |
| MaZderMind | GST_BUFFER_TIMESTAMP (buffer) = interaudiosrc->timestamp_offset + | 07:13 |
| MaZderMind | gst_util_uint64_scale (interaudiosrc->n_samples, GST_SECOND, | 07:13 |
| MaZderMind | interaudiosrc->info.rate); | 07:13 |
| MaZderMind | so number of samples times sample length + offset | 07:14 |
| MaZderMind | maybe involving the original buffers' timestamp + offset could change that, but when there is no frame to read from the input the element need to create a buffer with a timestamp exactly ligning up with the last one received from upstream | 07:16 |
| MaZderMind | I think I have to write me a better testbench for hacking on these elements | 07:16 |
| MaZderMind | I'll have to go afk now but will report back soon | 07:17 |
| mithro | MaZderMind: okay, so if we record the buffer timestamp when it goes into the inter component and then record the buffer timestamp out of the inter we can understand any change in the timestamps | 07:17 |
| MaZderMind | yes, I already had a sample for that. I used an identity element and its handoff-signal to get notified about each passing buffer and report its pts | 07:18 |
| mithro | MaZderMind: so you have a log of the timestamps on both sides somewhere I can look at? | 07:19 |
| MaZderMind | not at hand | 07:19 |
| mithro | MaZderMind: if you can get me one, I'd love to look at it | 07:19 |
| MaZderMind | but to to real testing I'll try to construct sth. with appsink/appsrc which would allow me to pass in buffers with arbitrary timestamps and see how they get transformed | 07:20 |
| MaZderMind | did you see the screenshots in my mailinglist-post? | 07:20 |
| MaZderMind | Here is a comparison of a 15 Minute Test-Clip, on the top is the | 07:20 |
| MaZderMind | input, on the bottom the output: | 07:20 |
| MaZderMind | Minute 0: http://pasteboard.co/VeEzzF0.png | 07:20 |
| tpb | Title: Pasteboard Uploaded Image (at pasteboard.co) | 07:20 |
| MaZderMind | Minute 20: http://pasteboard.co/VeHxCU0.png | 07:20 |
| tpb | Title: Pasteboard Uploaded Image (at pasteboard.co) | 07:20 |
| MaZderMind | the top clip is the source, the bottom is the recording, they are kigned up by video frame | 07:21 |
| mithro | MaZderMind: yeah - it doesn't explain why it is happening though | 07:23 |
| *** Niharika__ is now known as Niharika_ | 07:26 | |
| MaZderMind | in the code-quotes above buffer may be an input buffer from the source. Isl PTS (video) and TIMESTAMP (audio) is completely overwritten with values calculated from the nominal fps/sample-length. The *actual* timestamp of the source buffer is completely discarded | 07:26 |
| MaZderMind | that is why it happens | 07:26 |
| MaZderMind | I'll see if I can change that later :) | 07:27 |
| mithro | MaZderMind: which means the input values have an actual timestamp which doesn't match those values, right? | 07:27 |
| MaZderMind | yes | 07:28 |
| MaZderMind | the real timestamps of real files are never exactly as calculated, because of clock drift in cameras, temperature differences of hardware clocks, … | 07:28 |
| MaZderMind | this is why the original timestamps need to be taken into account when calculating the new timestamps in the inter*src | 07:29 |
| mithro | MaZderMind: if the input values are running fast or slow then there will be an eventual discontinuity where the inter element drops or duplicates a buffer? | 07:29 |
| MaZderMind | if they exceed the timeout, yes | 07:30 |
| MaZderMind | the timeouts defaults to 1s | 07:30 |
| MaZderMind | I have to go afk now, sorry | 07:31 |
| mithro | No worries | 07:31 |
| mithro | So, I can't recall of the top of my head how much data is inside a single buffer | 07:31 |
| mithro | I think we can look at those timeouts | 07:32 |
| *** Niharika_ has quit IRC | 08:41 | |
| *** Niharika_ has joined #timvideos | 08:49 | |
| *** Niharika_ has quit IRC | 09:16 | |
| *** Niharika_ has joined #timvideos | 09:20 | |
| *** Niharika_ has quit IRC | 09:24 | |
| *** Niharika_ has joined #timvideos | 11:20 | |
| *** Niharika_ has quit IRC | 11:25 | |
| *** Niharika_ has joined #timvideos | 11:36 | |
| *** tariq786 has quit IRC | 11:43 | |
| *** tariq786 has joined #timvideos | 11:46 | |
| *** tija has joined #timvideos | 12:22 | |
| *** f15h has joined #timvideos | 12:53 | |
| *** CarlFK has joined #timvideos | 12:55 | |
| *** ChanServ sets mode: +v CarlFK | 12:55 | |
| MaZderMind | the timeouts do apply now, too | 12:59 |
| MaZderMind | they wouln't change | 12:59 |
| *** f15h has quit IRC | 13:13 | |
| *** Niharika_ has quit IRC | 15:07 | |
| *** CarlFK has quit IRC | 15:10 | |
| *** CarlFK has joined #timvideos | 15:34 | |
| *** ChanServ sets mode: +v CarlFK | 15:34 | |
| *** Niharika_ has joined #timvideos | 15:39 | |
| *** f15h has joined #timvideos | 15:49 | |
| *** f15h has quit IRC | 15:52 | |
| *** Niharika_ has quit IRC | 16:21 | |
| *** tija has quit IRC | 16:54 | |
| *** CarlFK has quit IRC | 16:59 | |
| *** hyades_ has joined #timvideos | 17:47 | |
| *** [d__d] has quit IRC | 18:33 | |
| *** [d__d] has joined #timvideos | 18:33 | |
| MaZderMind | i have a working prototype of a pts synced interaudiosrc | 18:35 |
| MaZderMind | but it seems the duration needs to be transported as well | 18:37 |
| *** hyades is now known as Guest58949 | 18:38 | |
| *** hyades_ is now known as hyades | 18:38 | |
| *** hyades_ has joined #timvideos | 18:38 | |
| *** hyades_ is now known as hyades_zombie | 18:55 | |
| MaZderMind | first test looks good so far, although i have not implemented all possibilities yet | 19:19 |
| *** _florent_ has joined #timvideos | 19:53 | |
| MaZderMind | https://github.com/MaZderMind/gst-plugins-bad/compare/inter-avsync | 20:28 |
| tpb | Title: Comparing master...inter-avsync · MaZderMind/gst-plugins-bad · GitHub (at github.com) | 20:28 |
| *** CarlFK has joined #timvideos | 20:35 | |
| *** ChanServ sets mode: +v CarlFK | 20:35 | |
| *** _florent_ has quit IRC | 21:13 | |
| MaZderMind | http://lists.freedesktop.org/archives/gstreamer-devel/2015-June/053039.html | 21:36 |
| tpb | Title: debugging a/v-sync issues, possibly inter*-element related [patch included] (at lists.freedesktop.org) | 21:36 |
| MaZderMind | i asked on the ml for feedback, we'll see what they say | 21:36 |
| CarlFK | MaZderMind: what triggers the problem? (I am wondering if it can be worked around like restart everything between talks ) | 21:39 |
| MaZderMind | the problem was that the audio-buffer-timestamps as generated by the source are not used in the target pipeline. they are recalculated based on the number of samples seen. but in real physics world, the samples are not always exactly same length, they drift over time. | 21:41 |
| MaZderMind | the patch ove uses the timestamps known in the source to build the timestamps in the target | 21:41 |
| MaZderMind | restarting is not an option for a running live-stream, btw. | 21:41 |
| CarlFK | yes it is ;) | 21:43 |
| mithro | MaZderMind: I believe your patch will solve the issue when playing back already recorded files, but I think it breaks the way the inter parts are non blocking. | 22:42 |
| mithro | But I'm currently on the way to the dentist and haven't actually looked at your code yet. | 22:43 |
| mithro | BTW playing back files and recording live video is actually slightly different scenarios when talking about clocks | 22:45 |
| mithro | MaZderMind: although in your case where you are using mkv and ffmpeg for capture things are a little confusing. | 22:50 |
| mithro | thaytan: can you tell me how long (in ms) a gstreamer buffer which contains audio is? | 22:55 |
| *** hyades has quit IRC | 23:28 | |
Generated by irclog2html.py 2.13.1 by Marius Gedminas - find it at mg.pov.lt!
