Sunday, 2016-07-24

*** tpb has joined #timvideos00:00
*** Bertl_oO is now known as Bertl_zZ00:00
*** CarlFK has quit IRC00:04
*** sb0 has joined #timvideos01:01
*** paradisaeidae_ has joined #timvideos03:45
mithrotumbleweed: oh, where in the world are you?04:12
mithrotumbleweed: I assumed you were in the US, but guess you could still be in South Africa?04:12
tumbleweedyep, back in US04:37
tumbleweedmithro: so, want to have a shot at this?04:42
mithrotumbleweed: In about 10-15 minutes?04:42
tumbleweedworks for me04:42
mithrookay04:43
mithrotumbleweed: okay?05:05
mithrotumbleweed: can you tell me what you are seeing and what you have tried?05:05
tumbleweedmithro: one sec, digging it out05:08
mithrotumbleweed: ahh, I was hoping you would have started while I was finishing up here05:11
tumbleweednaah, I was finishing up the dishes05:15
tumbleweedOK, sorry about that05:17
tumbleweedgot it all up now05:17
tumbleweed[380481.179618] usb 2-1: new high-speed USB device number 52 using xhci_hcd05:17
tumbleweed[380481.307764] usb 2-1: New USB device found, idVendor=2a19, idProduct=544005:17
tumbleweed[380481.307767] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=005:17
tumbleweed[380486.758005] usb 2-1: USB disconnect, device number 5205:17
tumbleweedso, it comes up then disconnects05:17
mithrotumbleweed: and never reconnects?05:17
tumbleweedcorrect05:17
tumbleweedjumpers are currently in the factory position, I think05:17
mithrotumbleweed: and have you loaded anything onto it?05:18
tumbleweeddebugging I did in CPT was:05:18
tumbleweedprobe all the power supply outputs05:18
tumbleweedall looked good05:18
tumbleweedjumpered it into failsafe mode05:18
tumbleweedwas able to re-flash the ROM, but that didn't make any difference05:19
mithrotumbleweed: what did you flash it with?05:21
mithrotumbleweed: and how did you flash it?05:21
tumbleweedfrom failsafe mode, I could load the ixo-jtag firmware05:21
tumbleweedand use openocd05:21
tumbleweedwith c4646dd6ad5ac3be4edb05f08d5f72aa2fcba8d5 from the prebuilt repo05:22
tumbleweedthat's an old build, but I had it tagged as known-good05:22
mithrotumbleweed: Probably best to start with what should be the shipping firmware?05:22
tumbleweedI have no idea what that is05:22
mithrotumbleweed: https://github.com/mithro/HDMI2USB-mode-switch/blob/opsis-prod/spiflash/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin05:23
tpbTitle: HDMI2USB-mode-switch/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin at opsis-prod · mithro/HDMI2USB-mode-switch · GitHub (at github.com)05:23
tumbleweedOK, let me try to remember how to get this into failsafe (and how to drive openocd)05:24
mithrotumbleweed: if you put it into failsafe mode, you should be able to just use hdmi2usb-mode-switch to flash it05:25
tumbleweedok05:26
mithrohdmi2usb-mode-switch --flash-gateware <.bin file>05:26
tumbleweedso, I have to disconnect P18, and move K3 to 1-205:29
mithrotumbleweed: In theory you only need to move K305:30
tumbleweedOK, let me try that05:30
tumbleweedah, yes, that's better05:31
tumbleweednow it's actually flashing05:31
tumbleweedit's just hanging after "Info : sector 22 took 247 ms"05:32
tumbleweedah, no, done05:32
tumbleweedI think? now it's waiting on "Info : Found flash device 'micron n25q128' (ID 0x0018ba20)"05:32
tumbleweedand done05:33
mithrotumbleweed: Can you paste the complete output?05:33
mithrotumbleweed: It will verify the flash after writing to it05:33
tumbleweedhttps://paste.debian.net/784425/05:33
tpbTitle: debian Pastezone (at paste.debian.net)05:34
mithrotumbleweed: yeah - that looks all good05:34
mithrotumbleweed: put the jumper back on and powercycle the device?05:34
tumbleweeddone05:34
tumbleweedand that seems to be working now05:35
mithrotumbleweed: working now?05:35
tumbleweedbah, maybe known-good is not so good05:35
mithrotumbleweed: well - you have multiple Opsis boards right?05:35
tumbleweedyep05:36
mithrotumbleweed: Can you try the "known good" on a different board?05:36
tumbleweedtehy aren' there05:36
tumbleweedthey're in the UK05:36
tumbleweedwhen I was debugging this, we were in the middle of setup day, so there wasn't time to try other boards05:36
tumbleweedbut I'm pretty sure I'd used that firmware version on other ones before05:36
mithrotumbleweed: So, you don't have multiple Opsis boards then :P05:36
tumbleweednot in any useful sense, no :)05:37
mithrotumbleweed: Can you try loading the firmware your firmware now using the mode switch tool?05:37
mithrobah05:37
mithrotumbleweed: Can you try loading your "known good" firmware using the mode switch tool now?05:37
tumbleweedsure05:38
tumbleweedremind me. What do I do with a 0x181818181813d47 DNA?05:38
tumbleweedah, switch mode a few times05:39
mithrotumbleweed: Yes05:39
tumbleweedyeah, doesn't come up05:42
tumbleweedno, it did05:42
tumbleweednafc05:42
mithronafc?05:42
tumbleweedwhy it didn't work before05:42
tumbleweedI wasn't using modeswitch to flash, I was doing it a little more manually05:42
mithrotumbleweed: Where you maybe flashing the .bit file rather than the .bin file?05:42
tumbleweedit's possible05:42
tumbleweedI don't have my bash history from that long ago05:43
mithrotumbleweed: I mean you could try that now and see if it behaves that way05:43
mithrotumbleweed: except I think I made mode-switch refuse to do that now :P05:43
tumbleweedanyway, it works, that's really what matters05:44
tumbleweedwhat's the state of current firmware?05:44
* tumbleweed wonders what he should flash it to05:45
mithrotumbleweed: Working in theory but no idea in practice05:45
mithrotumbleweed: Please always use the latest version unless you run into an issue05:45
mithroxfxf: ping?05:45
tumbleweedwell, that's no good when you're trying to debug05:45
tumbleweedbut sure05:45
mithrotumbleweed: I believe the current firmware is broken on the Atlys board05:46
mithrotumbleweed: And it is unclear to me what CarlFK has been using05:46
tumbleweed:)05:47
mithrotumbleweed: Carl seems to do his own thing and not listen to my suggestions05:48
tumbleweedsounds about right05:49
mithrotumbleweed: So - any chance you could package up mode-switch for me? :P05:49
tumbleweedany chance you can get openocd upstream to cut a release?05:51
mithrotumbleweed: not a chance05:51
mithrotumbleweed: However, we could ship the required TCL scripts with mode-switch rather than openocd?05:51
tumbleweedsure05:51
tumbleweedor ship the patches in debian's openocd package05:52
mithrotumbleweed: I'm pretty sure that we don't need any changes to the openocd C code05:52
tumbleweedhrm, I flashed the latest master, and am hitting the same problem05:55
mithrotumbleweed: Okay, I have a production Opsis sitting here on my desk too, let's see if I can replicate05:56
mithrotumbleweed: I'd really like to be able to give something like "prebuilt:<revision>" to mode-switch and have it download the firmware from the github repo05:57
tumbleweedthat'd nice, yes05:58
mithrotumbleweed: and the stable/testing/unstable aliases too05:58
mithrotumbleweed: wouldn't be much python to make that happen :P05:58
tumbleweedyeah, should be simple enough05:58
mithrotumbleweed: It would actually be pretty simple now we have SPI access in the FPGA gateware to support flashing *without* needing OpenOCD05:59
tumbleweedthat would be nice06:00
tumbleweedbut you need to be able to fall back to openocd in situations like this06:00
mithrotumbleweed: yes06:01
tumbleweedso, maybe not a big gain06:01
mithrotumbleweed: Well, we need the ability to access the SPI for storing configuration stuff06:01
tumbleweedwhat's nice about openocd was that when I was using a JTAG dongle, all I had to do was slightly tweak our existing flashing command06:01
tumbleweednot figure it out from scratch06:02
mithrotumbleweed: JTAG dongle?06:02
tumbleweedwhen we were debugging our dead board06:02
tumbleweedI borrowed an FTDI based dongle06:02
mithroahh06:05
mithrotumbleweed: what exactly did you flash?06:08
tumbleweedHDMI2USB-firmware-prebuilt/opsis/firmware/unstable/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin06:09
mithroFlashing now....06:11
tumbleweedwhich I think resolves to HDMI2USB-firmware-prebuilt/archive/master/v0.0.2-18-g65d53e1/opsis/hdmi2usb/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin06:11
mithrotumbleweed: I guess I should get the tool to resolve the realpath for it...06:14
mithrotumbleweed: That firmware seems to work here...06:25
mithrotumbleweed: HDMI2USB-firmware-prebuilt/archive/master/v0.0.2-18-g65d53e1/opsis/hdmi2usb/opsis_hdmi2usb-hdmi2usbsoc-opsis.bin06:26
tumbleweednot here06:31
tumbleweedI tried again, same problem06:32
mithrotumbleweed: Do you have a logic analyzer?06:32
tumbleweednope06:32
tumbleweedI can get one :P06:36
mithrotumbleweed: The problem is most likely the communication between the FPGA and the FX2 on the I2C lines06:36
mithrotumbleweed: You don't have a TOFE-LowSpeedIO board either, right?06:37
tumbleweednope06:37
mithrotumbleweed: A logic analyzer will let you see what is happening on the I2C lines06:37
mithrotumbleweed: If you have a TOFE board, you could connect the extra serial port and see what the firmware thinks is happening06:37
mithrotumbleweed: It would be good to figure out what is going on - in theory this board on my desk should be identical to your board06:40
* tumbleweed orders some hardware06:45
mithrotumbleweed: Did I give you one of the little FX2 dev boards at LCA?06:46
tumbleweednope06:46
mithrotumbleweed: Oh well, I should have06:48
mithrotumbleweed: That can be used as a logic analyzer06:48
* tumbleweed may have successfully avoided that06:48
mithrotumbleweed: they only cost ~3-5 USD06:49
tumbleweedthat's pretty darn cheap06:49
mithrotumbleweed: a bus pirate or saele logic devices are other good options06:50
mithrotumbleweed: but cost more06:50
tumbleweedI went for the http://dangerousprototypes.com/docs/Open_Bench_Logic_Sniffer06:51
tpbTitle: Open Bench Logic Sniffer - DP (at dangerousprototypes.com)06:51
mithroNever seen that device before06:51
tumbleweedsame people as the bus pirate06:52
mithroHave heard of SUMP06:52
*** paradisaeidae_ has quit IRC06:53
mithrotumbleweed: be interested to see how you find it07:06
tumbleweedyeah, we'll see :)07:12
mithrotumbleweed: so - packaging mode-switch? :P07:12
mithrotumbleweed: I was thinking it probably makes sense for the udev rules to end up in their own package?07:13
tumbleweedwhich rules?07:13
tumbleweedoh, permissions07:14
mithrotumbleweed: https://github.com/timvideos/HDMI2USB-mode-switch/tree/master/udev07:14
tpbTitle: HDMI2USB-mode-switch/udev at master · timvideos/HDMI2USB-mode-switch · GitHub (at github.com)07:14
mithrotumbleweed: they do more then permissions07:14
mithrotumbleweed: see the README in that directory07:14
tumbleweedthose permissions are obviously terrible :P07:15
mithrotumbleweed: yeah - I guessed you might object to that07:16
tumbleweedI guess we need to find a suitable group07:16
mithrotumbleweed: which why the udev rules are split into multiple files07:16
mithrotumbleweed: you can drop in a replacement permissions rule07:17
* tumbleweed is fading fast, and must get to bed07:17
tumbleweedwhy do you think they should be in a separate package? because you don't *need* modeswitch?07:18
mithrotumbleweed: yeah07:18
mithrotumbleweed: there is no direct dependency between the udev rules and the tool07:18
tumbleweedthey are both support packages for the opsis, that's about it07:19
mithrotumbleweed: but they are *highly recommended* to be used together07:19
tumbleweedand the rules are pretty useful for the tool (possibly required?)07:19
tumbleweedpermissions at least07:19
mithrotumbleweed: If you don't have the udev rules, you need the setuid unbind-helper tool07:19
mithrotumbleweed: or have to run the tool as root07:20
mithrotumbleweed: It might also make sense to ship the firmware blogs in a separate package07:26
mithros/blogs/blobs/07:27
mithrotumbleweed: incase people are worried about free/non-free type problems07:27
mithroI'm going to find dinner07:32
mithroI assume tumbleweed has fallen asleep :P07:32
mithrobblr07:32
xfxfmithro: pong. was out today grabbing LA AV gear for pycon07:37
mithroxfxf: I'll probably be around for another hour or two if you want me to help you debug Opsis issues like I did for tumbleweed08:07
xfxfoh, doh - i'm about to do my usual night routine with the family, i was going to do that later tonight08:11
xfxfi've got all of the LA Opsis's + mine now, so was going to lay out a test bench after dinner08:11
mithroxfxf: well, I might be around late this evening08:15
xfxfcool, good to know08:16
mithroxfxf: I give it a 50/50 chance :P08:16
xfxfmy immediate intention is to update the LCA imaging machine that CarlFK/tumbleweed setup to install latest Ubuntu + voctomix + etc08:17
xfxfand then from there i'll figure out the best flashing tools / firmware to push out08:17
xfxfi'll simultaneously run an opsis with the latest firmware again - what debugging steps did you want me to take outside of running a hdmi2usb console with debug enabled and redirecting stdout to a log file?08:18
xfxfthe opsis locked up on me a few weeks ago just running video overnight, didn't debug any further08:18
xfxfnot capture, just video0 in > video0 out to a hdmi monitor08:18
mithroxfxf: That isn't particularly useful thing to say08:19
xfxftumbleweed: if you're around, is there any new information/lessons you've learnt re the opsis's/firmware/voctomix i should know since LCA?08:20
xfxfmithro: this is why i'm asking specifically what you'd like me to do in a test to dertermine whether a firmware is good or not.  i don't know what information you're after08:20
mithroxfxf: Start with figuring out if the firmware can be used the way you need to08:21
xfxfi'm lost as to how that exercise is any more explicit than what i've already told you - the device locked up on me when capturing overnight (no video coming through, could not talk to its console), needing a reset08:23
xfxfi'm assuming if i manage to reproduce that, you want more than me going 'it happened again'08:23
mithroxfxf: Do you need the device to run overnight? No you need it to run for a couple of hours.08:25
mithroxfxf: Once it has locked up, there is not much you can do08:25
mithroxfxf: assuming it has locked up08:25
mithroxfxf: even if it hasn't, there isn't much you can do that I can think of08:26
mithroxfxf: Being able to reproducibly cause it to lock up by doing X is useful08:26
mithroxfxf: I mean it would be nice if you ran it overnight and everything continued to work perfectly, but that isn't a hard requirement for you usage right now08:28
xfxfi've ran the Atlys's for days on end, and i'm pretty sure the Opsis's with an older firmware for many days08:35
xfxfthis is less about having to run them for >12 hours, more that to me it indicates a regression in the firmware somewhere, and that's bad, as it has the potential to rear its head randomly08:35
mithroxfxf: How many times have you run it overnight and had it lock up?08:35
xfxfonly twice so far. now you're around i'll setup a test bench and try and repeat08:36
xfxfbut both times consistently (as in, both times i tried running the Opsis for more than a few hours) - i didn't try using the device after that.08:36
mithroxfxf: There are way to many variables to make the claim you just did08:36
xfxfi recognise the problem is likely very complex but my abstracted goal is reliable capture08:37
xfxfthis doesn't give me any confidence the current firmware can do that, so i'd like to help solve it08:37
mithroxfxf: I'm not confident that you have found a regression08:37
xfxfpossibly, i just do not remember seeing this before08:38
xfxfbut i may be remembering experiences with the Atlys, not the Opsis08:38
xfxfand i'd say yes I need to run the device for ~8-9 hours.  i don't want it locking up mid-way through a presentation08:38
mithroxfxf: Never trust your memory08:38
mithroxfxf: reboot the device every break08:39
xfxfi'm not... look at my original question.  'how should I test this'08:39
xfxfhaving a hard requirement to reboot our capture device every few hours does not fill me with confidence08:39
mithroxfxf: You don't have the time to be more picky08:40
xfxfthat to me is a workaround if everything else fails, not what we should accept as normal08:40
xfxfi don't really think needing a device to work for a full day is picky tbh, but i have no power to fix this myself08:40
mithroxfxf: We'll do our best to fix these problems but I at the moment we can't even say what conditions you have working capture for 2 hours08:41
xfxfsure, which again, which is why i'm asking for what specific information you'd like from me08:41
xfxfi don't want to spend a few days trying to reproduce the problem and then i don't have the info you need08:41
mithroxfxf: I would like you to try and run a pretend conference and find a reproducible way of creating any problems you are running into08:42
xfxfthat is already what i'm doing, but the point i am attempting to make is i assume you want more information than 'the board stopped working'08:43
xfxfbut given we're going around in circles i'll do that and then ask, i guess...08:43
mithroxfxf: I want "Do XYZ and you get behaviour ABC but it should be doing DEF which prevents me from achieving goal of capturing a conference"08:44
xfxfsure, i can do that, i'm just pre-empting you wanting debug information or something08:44
xfxfbut i shall do that08:44
xfxf(what you're suggesting)08:44
mithroxfxf: If you can reproducibly make something happen, then we can debug08:45
mithroxfxf: Once you can reliably reproduce a problem, the next step would be to change one thing and see if it keeps happening08:46
xfxfsure08:46
mithroxfxf: IE same setup, different firmware version08:46
mithroor different inputs / outputs08:46
mithroetc08:46
mithroxfxf: If you can make a problem happen 5 times in a row, that is a pretty good start08:46
mithroxfxf: We can look into work arounds if we can't fix it in time08:47
xfxfsure, will do, my main motivation pre-empting further steps is context switching sucks and these tests will span days as they take many hours to reproduce08:49
xfxfi'll try and counteract that by simultaneously testing multiple opsis's though08:50
xfxfin terms of trying earlier firmware, are there any specific branches or commits i should be jumping back to, where you know major changes have occurred?08:50
xfxfcarl has a list of firmware's he's tried and what works for him but it's Atlys specific IIRC08:50
mithroxfxf: Use the latest version unless you have a reason not too08:51
xfxfokay, i'll try and persist with the latest firmware in master then08:51
xfxfand ping you if i have issues with specifics08:51
mithroxfxf: you know how to do a bisect right?08:52
mithroxfxf: If you find a reliably reproducible problem in the latest firmware, you can try doing a bisect to figure out if it has ever worked08:53
mithroxfxf: and then find when the regression was introduced08:54
xfxfas in, a git bisect?08:55
mithroxfxf: Bisect just a way to find where a regression was introduced between a working and failing revision08:55
mithroxfxf: It's just a binary search08:55
mithroxfxf: git bisect is just an automated way of doing it with a git history08:56
xfxfsure, this is what i'm getting at, i'm sure i can be more useful beyond 'X doesn't work'08:56
mithroxfxf: X used to work and now doesn't and regressed at revision XXX is *super useful*08:57
mithrodoes matter if X is something big or small08:58
mithroxfxf: "X has never worked in any previous firmware" is also useful08:59
*** Bertl_zZ is now known as Bertl10:03
*** Bertl is now known as Bertl_oO12:51
mithroxfxf: Time for me to bail for the night13:24
xfxfmithro: i'm still laying out my testing equipment/desk so that's cool, speak tomorrow?13:24
xfxfsleep well13:24
mithroxfxf: sure13:28
*** rohitksingh has joined #timvideos13:49
*** nueces has quit IRC14:47
tumbleweedxfxf: I'm the wrong person to ask, there. I was only called in to debug a few things14:58
tumbleweedit seems vocto worked out pretty well (single PC with 2 blackmagic cards)14:58
tumbleweedand the biggest problem with the opsis boards was around mode switching because of VGA convertors14:59
*** rohitksingh has quit IRC16:24
*** rohitksingh has joined #timvideos16:26
*** rohitksingh has quit IRC16:41
*** CarlFK has joined #timvideos19:35
*** ChanServ sets mode: +v CarlFK19:35

Generated by irclog2html.py 2.13.1 by Marius Gedminas - find it at mg.pov.lt!