*** tpb has joined #timvideos | 00:00 | |
*** Bertl_oO is now known as Bertl_zZ | 00:00 | |
*** CarlFK has quit IRC | 00:04 | |
*** sb0 has joined #timvideos | 01:01 | |
*** paradisaeidae_ has joined #timvideos | 03:45 | |
mithro | tumbleweed: oh, where in the world are you? | 04:12 |
---|---|---|
mithro | tumbleweed: I assumed you were in the US, but guess you could still be in South Africa? | 04:12 |
tumbleweed | yep, back in US | 04:37 |
tumbleweed | mithro: so, want to have a shot at this? | 04:42 |
mithro | tumbleweed: In about 10-15 minutes? | 04:42 |
tumbleweed | works for me | 04:42 |
mithro | okay | 04:43 |
mithro | tumbleweed: okay? | 05:05 |
mithro | tumbleweed: can you tell me what you are seeing and what you have tried? | 05:05 |
tumbleweed | mithro: one sec, digging it out | 05:08 |
mithro | tumbleweed: ahh, I was hoping you would have started while I was finishing up here | 05:11 |
tumbleweed | naah, I was finishing up the dishes | 05:15 |
tumbleweed | OK, sorry about that | 05:17 |
tumbleweed | got it all up now | 05:17 |
tumbleweed | [380481.179618] usb 2-1: new high-speed USB device number 52 using xhci_hcd | 05:17 |
tumbleweed | [380481.307764] usb 2-1: New USB device found, idVendor=2a19, idProduct=5440 | 05:17 |
tumbleweed | [380481.307767] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 | 05:17 |
tumbleweed | [380486.758005] usb 2-1: USB disconnect, device number 52 | 05:17 |
tumbleweed | so, it comes up then disconnects | 05:17 |
mithro | tumbleweed: and never reconnects? | 05:17 |
tumbleweed | correct | 05:17 |
tumbleweed | jumpers are currently in the factory position, I think | 05:17 |
mithro | tumbleweed: and have you loaded anything onto it? | 05:18 |
tumbleweed | debugging I did in CPT was: | 05:18 |
tumbleweed | probe all the power supply outputs | 05:18 |
tumbleweed | all looked good | 05:18 |
tumbleweed | jumpered it into failsafe mode | 05:18 |
tumbleweed | was able to re-flash the ROM, but that didn't make any difference | 05:19 |
mithro | tumbleweed: what did you flash it with? | 05:21 |
mithro | tumbleweed: and how did you flash it? | 05:21 |
tumbleweed | from failsafe mode, I could load the ixo-jtag firmware | 05:21 |
tumbleweed | and use openocd | 05:21 |
tumbleweed | with c4646dd6ad5ac3be4edb05f08d5f72aa2fcba8d5 from the prebuilt repo | 05:22 |
tumbleweed | that's an old build, but I had it tagged as known-good | 05:22 |
mithro | tumbleweed: Probably best to start with what should be the shipping firmware? | 05:22 |
tumbleweed | I have no idea what that is | 05:22 |
mithro | tumbleweed: https://github.com/mithro/HDMI2USB-mode-switch/blob/opsis-prod/spiflash/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin | 05:23 |
tpb | Title: HDMI2USB-mode-switch/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin at opsis-prod · mithro/HDMI2USB-mode-switch · GitHub (at github.com) | 05:23 |
tumbleweed | OK, let me try to remember how to get this into failsafe (and how to drive openocd) | 05:24 |
mithro | tumbleweed: if you put it into failsafe mode, you should be able to just use hdmi2usb-mode-switch to flash it | 05:25 |
tumbleweed | ok | 05:26 |
mithro | hdmi2usb-mode-switch --flash-gateware <.bin file> | 05:26 |
tumbleweed | so, I have to disconnect P18, and move K3 to 1-2 | 05:29 |
mithro | tumbleweed: In theory you only need to move K3 | 05:30 |
tumbleweed | OK, let me try that | 05:30 |
tumbleweed | ah, yes, that's better | 05:31 |
tumbleweed | now it's actually flashing | 05:31 |
tumbleweed | it's just hanging after "Info : sector 22 took 247 ms" | 05:32 |
tumbleweed | ah, no, done | 05:32 |
tumbleweed | I think? now it's waiting on "Info : Found flash device 'micron n25q128' (ID 0x0018ba20)" | 05:32 |
tumbleweed | and done | 05:33 |
mithro | tumbleweed: Can you paste the complete output? | 05:33 |
mithro | tumbleweed: It will verify the flash after writing to it | 05:33 |
tumbleweed | https://paste.debian.net/784425/ | 05:33 |
tpb | Title: debian Pastezone (at paste.debian.net) | 05:34 |
mithro | tumbleweed: yeah - that looks all good | 05:34 |
mithro | tumbleweed: put the jumper back on and powercycle the device? | 05:34 |
tumbleweed | done | 05:34 |
tumbleweed | and that seems to be working now | 05:35 |
mithro | tumbleweed: working now? | 05:35 |
tumbleweed | bah, maybe known-good is not so good | 05:35 |
mithro | tumbleweed: well - you have multiple Opsis boards right? | 05:35 |
tumbleweed | yep | 05:36 |
mithro | tumbleweed: Can you try the "known good" on a different board? | 05:36 |
tumbleweed | tehy aren' there | 05:36 |
tumbleweed | they're in the UK | 05:36 |
tumbleweed | when I was debugging this, we were in the middle of setup day, so there wasn't time to try other boards | 05:36 |
tumbleweed | but I'm pretty sure I'd used that firmware version on other ones before | 05:36 |
mithro | tumbleweed: So, you don't have multiple Opsis boards then :P | 05:36 |
tumbleweed | not in any useful sense, no :) | 05:37 |
mithro | tumbleweed: Can you try loading the firmware your firmware now using the mode switch tool? | 05:37 |
mithro | bah | 05:37 |
mithro | tumbleweed: Can you try loading your "known good" firmware using the mode switch tool now? | 05:37 |
tumbleweed | sure | 05:38 |
tumbleweed | remind me. What do I do with a 0x181818181813d47 DNA? | 05:38 |
tumbleweed | ah, switch mode a few times | 05:39 |
mithro | tumbleweed: Yes | 05:39 |
tumbleweed | yeah, doesn't come up | 05:42 |
tumbleweed | no, it did | 05:42 |
tumbleweed | nafc | 05:42 |
mithro | nafc? | 05:42 |
tumbleweed | why it didn't work before | 05:42 |
tumbleweed | I wasn't using modeswitch to flash, I was doing it a little more manually | 05:42 |
mithro | tumbleweed: Where you maybe flashing the .bit file rather than the .bin file? | 05:42 |
tumbleweed | it's possible | 05:42 |
tumbleweed | I don't have my bash history from that long ago | 05:43 |
mithro | tumbleweed: I mean you could try that now and see if it behaves that way | 05:43 |
mithro | tumbleweed: except I think I made mode-switch refuse to do that now :P | 05:43 |
tumbleweed | anyway, it works, that's really what matters | 05:44 |
tumbleweed | what's the state of current firmware? | 05:44 |
* tumbleweed wonders what he should flash it to | 05:45 | |
mithro | tumbleweed: Working in theory but no idea in practice | 05:45 |
mithro | tumbleweed: Please always use the latest version unless you run into an issue | 05:45 |
mithro | xfxf: ping? | 05:45 |
tumbleweed | well, that's no good when you're trying to debug | 05:45 |
tumbleweed | but sure | 05:45 |
mithro | tumbleweed: I believe the current firmware is broken on the Atlys board | 05:46 |
mithro | tumbleweed: And it is unclear to me what CarlFK has been using | 05:46 |
tumbleweed | :) | 05:47 |
mithro | tumbleweed: Carl seems to do his own thing and not listen to my suggestions | 05:48 |
tumbleweed | sounds about right | 05:49 |
mithro | tumbleweed: So - any chance you could package up mode-switch for me? :P | 05:49 |
tumbleweed | any chance you can get openocd upstream to cut a release? | 05:51 |
mithro | tumbleweed: not a chance | 05:51 |
mithro | tumbleweed: However, we could ship the required TCL scripts with mode-switch rather than openocd? | 05:51 |
tumbleweed | sure | 05:51 |
tumbleweed | or ship the patches in debian's openocd package | 05:52 |
mithro | tumbleweed: I'm pretty sure that we don't need any changes to the openocd C code | 05:52 |
tumbleweed | hrm, I flashed the latest master, and am hitting the same problem | 05:55 |
mithro | tumbleweed: Okay, I have a production Opsis sitting here on my desk too, let's see if I can replicate | 05:56 |
mithro | tumbleweed: I'd really like to be able to give something like "prebuilt:<revision>" to mode-switch and have it download the firmware from the github repo | 05:57 |
tumbleweed | that'd nice, yes | 05:58 |
mithro | tumbleweed: and the stable/testing/unstable aliases too | 05:58 |
mithro | tumbleweed: wouldn't be much python to make that happen :P | 05:58 |
tumbleweed | yeah, should be simple enough | 05:58 |
mithro | tumbleweed: It would actually be pretty simple now we have SPI access in the FPGA gateware to support flashing *without* needing OpenOCD | 05:59 |
tumbleweed | that would be nice | 06:00 |
tumbleweed | but you need to be able to fall back to openocd in situations like this | 06:00 |
mithro | tumbleweed: yes | 06:01 |
tumbleweed | so, maybe not a big gain | 06:01 |
mithro | tumbleweed: Well, we need the ability to access the SPI for storing configuration stuff | 06:01 |
tumbleweed | what's nice about openocd was that when I was using a JTAG dongle, all I had to do was slightly tweak our existing flashing command | 06:01 |
tumbleweed | not figure it out from scratch | 06:02 |
mithro | tumbleweed: JTAG dongle? | 06:02 |
tumbleweed | when we were debugging our dead board | 06:02 |
tumbleweed | I borrowed an FTDI based dongle | 06:02 |
mithro | ahh | 06:05 |
mithro | tumbleweed: what exactly did you flash? | 06:08 |
tumbleweed | HDMI2USB-firmware-prebuilt/opsis/firmware/unstable/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin | 06:09 |
mithro | Flashing now.... | 06:11 |
tumbleweed | which I think resolves to HDMI2USB-firmware-prebuilt/archive/master/v0.0.2-18-g65d53e1/opsis/hdmi2usb/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin | 06:11 |
mithro | tumbleweed: I guess I should get the tool to resolve the realpath for it... | 06:14 |
mithro | tumbleweed: That firmware seems to work here... | 06:25 |
mithro | tumbleweed: HDMI2USB-firmware-prebuilt/archive/master/v0.0.2-18-g65d53e1/opsis/hdmi2usb/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin | 06:26 |
tumbleweed | not here | 06:31 |
tumbleweed | I tried again, same problem | 06:32 |
mithro | tumbleweed: Do you have a logic analyzer? | 06:32 |
tumbleweed | nope | 06:32 |
tumbleweed | I can get one :P | 06:36 |
mithro | tumbleweed: The problem is most likely the communication between the FPGA and the FX2 on the I2C lines | 06:36 |
mithro | tumbleweed: You don't have a TOFE-LowSpeedIO board either, right? | 06:37 |
tumbleweed | nope | 06:37 |
mithro | tumbleweed: A logic analyzer will let you see what is happening on the I2C lines | 06:37 |
mithro | tumbleweed: If you have a TOFE board, you could connect the extra serial port and see what the firmware thinks is happening | 06:37 |
mithro | tumbleweed: It would be good to figure out what is going on - in theory this board on my desk should be identical to your board | 06:40 |
* tumbleweed orders some hardware | 06:45 | |
mithro | tumbleweed: Did I give you one of the little FX2 dev boards at LCA? | 06:46 |
tumbleweed | nope | 06:46 |
mithro | tumbleweed: Oh well, I should have | 06:48 |
mithro | tumbleweed: That can be used as a logic analyzer | 06:48 |
* tumbleweed may have successfully avoided that | 06:48 | |
mithro | tumbleweed: they only cost ~3-5 USD | 06:49 |
tumbleweed | that's pretty darn cheap | 06:49 |
mithro | tumbleweed: a bus pirate or saele logic devices are other good options | 06:50 |
mithro | tumbleweed: but cost more | 06:50 |
tumbleweed | I went for the http://dangerousprototypes.com/docs/Open_Bench_Logic_Sniffer | 06:51 |
tpb | Title: Open Bench Logic Sniffer - DP (at dangerousprototypes.com) | 06:51 |
mithro | Never seen that device before | 06:51 |
tumbleweed | same people as the bus pirate | 06:52 |
mithro | Have heard of SUMP | 06:52 |
*** paradisaeidae_ has quit IRC | 06:53 | |
mithro | tumbleweed: be interested to see how you find it | 07:06 |
tumbleweed | yeah, we'll see :) | 07:12 |
mithro | tumbleweed: so - packaging mode-switch? :P | 07:12 |
mithro | tumbleweed: I was thinking it probably makes sense for the udev rules to end up in their own package? | 07:13 |
tumbleweed | which rules? | 07:13 |
tumbleweed | oh, permissions | 07:14 |
mithro | tumbleweed: https://github.com/timvideos/HDMI2USB-mode-switch/tree/master/udev | 07:14 |
tpb | Title: HDMI2USB-mode-switch/udev at master · timvideos/HDMI2USB-mode-switch · GitHub (at github.com) | 07:14 |
mithro | tumbleweed: they do more then permissions | 07:14 |
mithro | tumbleweed: see the README in that directory | 07:14 |
tumbleweed | those permissions are obviously terrible :P | 07:15 |
mithro | tumbleweed: yeah - I guessed you might object to that | 07:16 |
tumbleweed | I guess we need to find a suitable group | 07:16 |
mithro | tumbleweed: which why the udev rules are split into multiple files | 07:16 |
mithro | tumbleweed: you can drop in a replacement permissions rule | 07:17 |
* tumbleweed is fading fast, and must get to bed | 07:17 | |
tumbleweed | why do you think they should be in a separate package? because you don't *need* modeswitch? | 07:18 |
mithro | tumbleweed: yeah | 07:18 |
mithro | tumbleweed: there is no direct dependency between the udev rules and the tool | 07:18 |
tumbleweed | they are both support packages for the opsis, that's about it | 07:19 |
mithro | tumbleweed: but they are *highly recommended* to be used together | 07:19 |
tumbleweed | and the rules are pretty useful for the tool (possibly required?) | 07:19 |
tumbleweed | permissions at least | 07:19 |
mithro | tumbleweed: If you don't have the udev rules, you need the setuid unbind-helper tool | 07:19 |
mithro | tumbleweed: or have to run the tool as root | 07:20 |
mithro | tumbleweed: It might also make sense to ship the firmware blogs in a separate package | 07:26 |
mithro | s/blogs/blobs/ | 07:27 |
mithro | tumbleweed: incase people are worried about free/non-free type problems | 07:27 |
mithro | I'm going to find dinner | 07:32 |
mithro | I assume tumbleweed has fallen asleep :P | 07:32 |
mithro | bblr | 07:32 |
xfxf | mithro: pong. was out today grabbing LA AV gear for pycon | 07:37 |
mithro | xfxf: I'll probably be around for another hour or two if you want me to help you debug Opsis issues like I did for tumbleweed | 08:07 |
xfxf | oh, doh - i'm about to do my usual night routine with the family, i was going to do that later tonight | 08:11 |
xfxf | i've got all of the LA Opsis's + mine now, so was going to lay out a test bench after dinner | 08:11 |
mithro | xfxf: well, I might be around late this evening | 08:15 |
xfxf | cool, good to know | 08:16 |
mithro | xfxf: I give it a 50/50 chance :P | 08:16 |
xfxf | my immediate intention is to update the LCA imaging machine that CarlFK/tumbleweed setup to install latest Ubuntu + voctomix + etc | 08:17 |
xfxf | and then from there i'll figure out the best flashing tools / firmware to push out | 08:17 |
xfxf | i'll simultaneously run an opsis with the latest firmware again - what debugging steps did you want me to take outside of running a hdmi2usb console with debug enabled and redirecting stdout to a log file? | 08:18 |
xfxf | the opsis locked up on me a few weeks ago just running video overnight, didn't debug any further | 08:18 |
xfxf | not capture, just video0 in > video0 out to a hdmi monitor | 08:18 |
mithro | xfxf: That isn't particularly useful thing to say | 08:19 |
xfxf | tumbleweed: if you're around, is there any new information/lessons you've learnt re the opsis's/firmware/voctomix i should know since LCA? | 08:20 |
xfxf | mithro: this is why i'm asking specifically what you'd like me to do in a test to dertermine whether a firmware is good or not. i don't know what information you're after | 08:20 |
mithro | xfxf: Start with figuring out if the firmware can be used the way you need to | 08:21 |
xfxf | i'm lost as to how that exercise is any more explicit than what i've already told you - the device locked up on me when capturing overnight (no video coming through, could not talk to its console), needing a reset | 08:23 |
xfxf | i'm assuming if i manage to reproduce that, you want more than me going 'it happened again' | 08:23 |
mithro | xfxf: Do you need the device to run overnight? No you need it to run for a couple of hours. | 08:25 |
mithro | xfxf: Once it has locked up, there is not much you can do | 08:25 |
mithro | xfxf: assuming it has locked up | 08:25 |
mithro | xfxf: even if it hasn't, there isn't much you can do that I can think of | 08:26 |
mithro | xfxf: Being able to reproducibly cause it to lock up by doing X is useful | 08:26 |
mithro | xfxf: I mean it would be nice if you ran it overnight and everything continued to work perfectly, but that isn't a hard requirement for you usage right now | 08:28 |
xfxf | i've ran the Atlys's for days on end, and i'm pretty sure the Opsis's with an older firmware for many days | 08:35 |
xfxf | this is less about having to run them for >12 hours, more that to me it indicates a regression in the firmware somewhere, and that's bad, as it has the potential to rear its head randomly | 08:35 |
mithro | xfxf: How many times have you run it overnight and had it lock up? | 08:35 |
xfxf | only twice so far. now you're around i'll setup a test bench and try and repeat | 08:36 |
xfxf | but both times consistently (as in, both times i tried running the Opsis for more than a few hours) - i didn't try using the device after that. | 08:36 |
mithro | xfxf: There are way to many variables to make the claim you just did | 08:36 |
xfxf | i recognise the problem is likely very complex but my abstracted goal is reliable capture | 08:37 |
xfxf | this doesn't give me any confidence the current firmware can do that, so i'd like to help solve it | 08:37 |
mithro | xfxf: I'm not confident that you have found a regression | 08:37 |
xfxf | possibly, i just do not remember seeing this before | 08:38 |
xfxf | but i may be remembering experiences with the Atlys, not the Opsis | 08:38 |
xfxf | and i'd say yes I need to run the device for ~8-9 hours. i don't want it locking up mid-way through a presentation | 08:38 |
mithro | xfxf: Never trust your memory | 08:38 |
mithro | xfxf: reboot the device every break | 08:39 |
xfxf | i'm not... look at my original question. 'how should I test this' | 08:39 |
xfxf | having a hard requirement to reboot our capture device every few hours does not fill me with confidence | 08:39 |
mithro | xfxf: You don't have the time to be more picky | 08:40 |
xfxf | that to me is a workaround if everything else fails, not what we should accept as normal | 08:40 |
xfxf | i don't really think needing a device to work for a full day is picky tbh, but i have no power to fix this myself | 08:40 |
mithro | xfxf: We'll do our best to fix these problems but I at the moment we can't even say what conditions you have working capture for 2 hours | 08:41 |
xfxf | sure, which again, which is why i'm asking for what specific information you'd like from me | 08:41 |
xfxf | i don't want to spend a few days trying to reproduce the problem and then i don't have the info you need | 08:41 |
mithro | xfxf: I would like you to try and run a pretend conference and find a reproducible way of creating any problems you are running into | 08:42 |
xfxf | that is already what i'm doing, but the point i am attempting to make is i assume you want more information than 'the board stopped working' | 08:43 |
xfxf | but given we're going around in circles i'll do that and then ask, i guess... | 08:43 |
mithro | xfxf: I want "Do XYZ and you get behaviour ABC but it should be doing DEF which prevents me from achieving goal of capturing a conference" | 08:44 |
xfxf | sure, i can do that, i'm just pre-empting you wanting debug information or something | 08:44 |
xfxf | but i shall do that | 08:44 |
xfxf | (what you're suggesting) | 08:44 |
mithro | xfxf: If you can reproducibly make something happen, then we can debug | 08:45 |
mithro | xfxf: Once you can reliably reproduce a problem, the next step would be to change one thing and see if it keeps happening | 08:46 |
xfxf | sure | 08:46 |
mithro | xfxf: IE same setup, different firmware version | 08:46 |
mithro | or different inputs / outputs | 08:46 |
mithro | etc | 08:46 |
mithro | xfxf: If you can make a problem happen 5 times in a row, that is a pretty good start | 08:46 |
mithro | xfxf: We can look into work arounds if we can't fix it in time | 08:47 |
xfxf | sure, will do, my main motivation pre-empting further steps is context switching sucks and these tests will span days as they take many hours to reproduce | 08:49 |
xfxf | i'll try and counteract that by simultaneously testing multiple opsis's though | 08:50 |
xfxf | in terms of trying earlier firmware, are there any specific branches or commits i should be jumping back to, where you know major changes have occurred? | 08:50 |
xfxf | carl has a list of firmware's he's tried and what works for him but it's Atlys specific IIRC | 08:50 |
mithro | xfxf: Use the latest version unless you have a reason not too | 08:51 |
xfxf | okay, i'll try and persist with the latest firmware in master then | 08:51 |
xfxf | and ping you if i have issues with specifics | 08:51 |
mithro | xfxf: you know how to do a bisect right? | 08:52 |
mithro | xfxf: If you find a reliably reproducible problem in the latest firmware, you can try doing a bisect to figure out if it has ever worked | 08:53 |
mithro | xfxf: and then find when the regression was introduced | 08:54 |
xfxf | as in, a git bisect? | 08:55 |
mithro | xfxf: Bisect just a way to find where a regression was introduced between a working and failing revision | 08:55 |
mithro | xfxf: It's just a binary search | 08:55 |
mithro | xfxf: git bisect is just an automated way of doing it with a git history | 08:56 |
xfxf | sure, this is what i'm getting at, i'm sure i can be more useful beyond 'X doesn't work' | 08:56 |
mithro | xfxf: X used to work and now doesn't and regressed at revision XXX is *super useful* | 08:57 |
mithro | does matter if X is something big or small | 08:58 |
mithro | xfxf: "X has never worked in any previous firmware" is also useful | 08:59 |
*** Bertl_zZ is now known as Bertl | 10:03 | |
*** Bertl is now known as Bertl_oO | 12:51 | |
mithro | xfxf: Time for me to bail for the night | 13:24 |
xfxf | mithro: i'm still laying out my testing equipment/desk so that's cool, speak tomorrow? | 13:24 |
xfxf | sleep well | 13:24 |
mithro | xfxf: sure | 13:28 |
*** rohitksingh has joined #timvideos | 13:49 | |
*** nueces has quit IRC | 14:47 | |
tumbleweed | xfxf: I'm the wrong person to ask, there. I was only called in to debug a few things | 14:58 |
tumbleweed | it seems vocto worked out pretty well (single PC with 2 blackmagic cards) | 14:58 |
tumbleweed | and the biggest problem with the opsis boards was around mode switching because of VGA convertors | 14:59 |
*** rohitksingh has quit IRC | 16:24 | |
*** rohitksingh has joined #timvideos | 16:26 | |
*** rohitksingh has quit IRC | 16:41 | |
*** CarlFK has joined #timvideos | 19:35 | |
*** ChanServ sets mode: +v CarlFK | 19:35 |
Generated by irclog2html.py 2.13.1 by Marius Gedminas - find it at mg.pov.lt!