Friday, 2019-01-11

*** tpb has joined #symbiflow00:00
*** Vincenttl has quit IRC00:56
*** citypw has joined #symbiflow03:29
*** efg has quit IRC04:00
*** citypw has quit IRC04:17
*** citypw has joined #symbiflow04:22
*** _whitelogger has quit IRC06:18
*** _whitelogger has joined #symbiflow06:21
*** photon70 has joined #symbiflow08:13
*** photon70 has quit IRC08:38
*** acomodi has joined #symbiflow08:47
*** tmichalak has joined #symbiflow09:48
digshadownats`: good night!10:14
digshadowoh ha that was last night10:14
sxpertfrom last night, it seems like vivado is written by monkeys10:17
*** citypw has quit IRC10:22
nats`sxpert, you had doubt about that ? :D11:07
nats`I mean until vivado 2016 I could crash it by clicking too fast on popup11:07
nats`it happened to me at least one million time11:07
nats`anyway I think hopefully because we can use tcl interface we can solve almost all fuzzing problem using my patch11:08
nats`and we can have a sort of parallelism11:08
sxpertnats`: all I have ever used so far is ise because vivado doesn't support my https://www.scarabhardware.com/minispartan6/11:08
tpbTitle: miniSpartan6+ | Scarab Hardware (at www.scarabhardware.com)11:08
nats`uhhmm tbh vivado is way above ise, if you want argument we could discuss that in private11:08
sxpertso I'd like to be able to use said minispartan6 with something else at some point ;)11:09
nats`I don't know if there is a known effort on serie 6 maybe ask digshadow11:10
digshadowsxpert: theres definitely interest in 6 series, but nobody working on it11:10
digshadowis that something you might be interested to add support for?11:11
sxpertdigshadow: unfortunately, I'm rather unqualified11:11
digshadowsxpert: if you are interested but aren't sure how to start, we can help with that11:11
nats`the idea is not stupid because serie 6 despite being really expensive compared to serie 7 keeps some advantage like static consumption11:11
sxpert(and lack time)11:11
digshadowthats a bigger issue :)11:11
sxpertnats`: the 6 series is in a large number of blackmagic devices11:12
digshadowsxpert: have you used them? how was your experience?11:13
sxpertwhich are pretty cheap, and could gain from opensource firmware11:13
nats`blackmagic the video manufacturer ? (if the discussion continue we may have to switch to private or an other channel to not pollute chan)11:13
sxpertnats`: yeah11:13
digshadowI think he means the 1bitsquared product11:13
digshadowah11:13
digshadownats`: but yeah thanks for keeping OT11:13
nats`is it a good idea to have a linked chan where we could put related but not totally ontopic discussion in case it interest more people in the chan ?11:14
nats`something like symbichat11:14
nats`(I'm not fan of having tens of channel but may be useful here)11:14
digshadowwell theres ##openfpga11:14
nats`ah sure :)11:14
digshadowwhich tends to be looser11:14
nats`get it :)11:15
sxpertnoted11:15
nats`I'll add it to my chan list again, if you want to keep going on the discussion sxpert ping me there :)11:16
nats`digshadow, Bug #50111:28
nats`are you talking about the output of get_pips ?11:28
nats`if yes I tried to do that because 072 relies on it but a simple puts seems not enough each item of pips is stored with a string and it seems that's not good11:28
digshadownats`: I think someone is working on that11:29
digshadowalready11:29
digshadowmarked as assigned11:29
nats`oky so I'll wait to integrate it in 072 because i didn't manage to do that well11:29
digshadowunless kgugala wants to hand off to you?11:29
digshadowah? its used there as well?11:29
nats`in 072 yes11:30
nats`and I call it a lot of time now11:30
nats`in each job11:30
nats`but beware get_pips with no -ob_object return something like 2e6 results on the 50T11:30
digshadowgotcha. So making it common would speed that up quite a bit11:31
nats`yep11:31
nats`that's why I tried11:31
nats`but failed :D11:31
digshadownats`: just talked to him11:32
digshadowsounds like you should be good to take that11:32
digshadowhes mostly working on 05211:32
nats`ohhh if he wants to do that no problem !11:32
nats`I don't want to take it from him11:32
digshadowno, I think you should go for it11:32
nats`oky11:32
digshadowhes focusing on the core logic of 052, not list pips11:32
nats`so the idea is to find a way to store that11:33
nats`it could be binary or text ?11:33
digshadowit should be the same file format as it is now right? just only generated once11:33
nats`are we talking about the same things, the output of get_pips command ?11:34
digshadowhttps://github.com/SymbiFlow/prjxray/blob/master/fuzzers/piplist.tcl11:37
tpbTitle: prjxray/piplist.tcl at master · SymbiFlow/prjxray · GitHub (at github.com)11:37
digshadowspecifically running that file11:37
nats`maybe we have a bigger rework to do in fact11:39
nats`foreach pip [lsort [get_pips -filter {IS_DIRECTIONAL} -of_objects [get_tiles $tile]]] {11:39
nats`        set src [get_wires -uphill -of_objects $pip]11:39
nats`        set dst [get_wires -downhill -of_objects $pip]11:39
nats`        if {[llength [get_nodes -uphill -of_objects [get_nodes -of_objects $dst]]] != 1} {11:39
nats`            puts $fp "$tile_type.[regsub {.*/} $dst ""].[regsub {.*/} $src ""]"11:39
nats`        }11:39
nats`    }11:39
nats`that part is really close to what the 072 does11:39
nats`the main difference is that fuzzer starts at a speificied object11:39
nats`where in 072 we simply list all wires everywhere if I get it correctly11:39
nats`maybe in fact a lot of 05x could be called after the 072 ?11:40
nats`(the real question is optimisation worth the effort ?)11:40
digshadownats`: no. 07x series takes too long11:42
digshadowand its not related to bitstream stuff11:42
digshadowI don't want any of the fuzzers blocked on 07x that don't have to11:42
nats`did you try with the new version ?11:42
nats`on my computer it takes 30 minutes now and it could go faster with more core and finding a better share between RAM/CPU11:43
nats`what do you considere too long ?11:43
nats`real question11:43
digshadownats`: from fresh checkout on a not very powerful computer, I can start running 05x series after only maybe 10 minutes?11:45
digshadowthe longest is tilegrid11:46
nats`sure ok get it I think on a middle range computer you'll have avec something like one hour on 07x11:46
digshadowit really just seems like an unnecessary dependency11:46
nats`yep11:46
nats`you're giht11:46
nats`right11:46
nats`I'll finish the 07x problem and look into sharing piplist for 05x fuzzer if kgugala is making something else11:48
digshadowsounds good!11:48
*** Vincenttl has joined #symbiflow11:50
*** acomodi has quit IRC12:05
*** tmichalak has quit IRC12:09
*** celadon has quit IRC13:24
*** Rahix has joined #symbiflow13:33
*** celadon has joined #symbiflow13:41
*** Vincenttl has quit IRC14:39
*** citypw has joined #symbiflow14:39
nats`litghost : should I handle the possible Exception of subprocess.check_call ?18:04
nats`in that case how should I do that ?18:04
nats`I mean what would be the behavior of the python script18:04
nats`I could force exit but what will happens to other thread in that case ?18:06
nats`try:18:17
nats`        subprocess.check_call("${XRAY_VIVADO} -mode batch -source $FUZDIR/job.tcl -tclargs " + str(blockID) + " " + str(start) + " " + str(stop), shell=True)18:17
nats`    except subprocess.CalledProcessError:18:17
nats`        print("Problem happened in jobs !")18:17
nats`        #dirty way to kill all the process18:17
nats`        os.kill(os.getpid(), signal.SIGINT)18:17
nats`need to be tested18:17
nats`it reacts well with a Ctrl+C input18:21
nats`so let's say it'll work in case the vivado process fails to start18:21
nats`do I have to do something to signal that I modified the PR ?18:33
litghostDo not handle the exception18:48
litghostIf cleanup needs to happen, make sure to put it in a context handler18:48
litghostI would just us multiprocessing.Queue rather than managing it by hand18:51
litghostRevise that, use multiprocessing.Pool and imap_unordered18:54
litghostIf you get an exception, then you can use multiprocessing.Pool.terminate to stop work18:54
litghostmap_async also works if you need to preserve ordering and what to handle errors without exceptions18:55
litghosthttps://docs.python.org/2/library/multiprocessing.html#multiprocessing.pool.AsyncResult.successful18:55
tpbTitle: 16.6. multiprocessing — Process-based “threading” interface Python 2.7.15 documentation (at docs.python.org)18:55
nats`uhhmm oky i'll take a look18:59
nats`it's not a little overkill for a "run once" script ?19:00
nats`I'm looking it I'm not sure it's usefull to go into such complexity19:07
mithroIf you want to chat about alternative firmware for Blackmagic products I recommend joining #timvideos channel19:08
nats`sure but I was more discussing making an opensource board :)19:10
litghostIf you want to avoid complexity, then avoid introducing threading to being with.  If you want parallel execution, multiprocessing.Pool is the safe and recommended way to do it19:13
*** citypw has quit IRC19:13
nats`litghost what's your opinion ? I think I should modify the way I call thread and calculate the index19:13
nats`oky I'll rewrite it19:13
nats`but for what I saw the current version is working well at catching signal19:13
litghostThe code will actually get simplier using Pool, because it will handle the nbParBlock logic for you19:14
nats`I have to read about that I really don't know those module it's literally 10 years I didn't write real python code :)19:16
litghostIf should just be #1 create pool, #2 start worker, #3 add work, #4 get results (or handle error) , #5 close pool19:17
litghostOpps, #2 start workers isn't required, creating the pool starts the workers19:21
somloafter building the latest yosys, prjtrellis, and nextpnr, I successfully managed to compile one of the examples included with prjtrellis (soc_versa5g); But after connecting the LFE5UM5G-45F-VERSA-EVN board via usb, and running "make prog", I get an error from openocd (https://pastebin.com/udAHnXFQ). The board stops blinking and spinning its segment-LED thingie, so *something* happens, but not quite what *should* happen. Any clue as to what I'm19:43
somloscrewing up (I'm a n00b at openocd and jtag) much much appreciated!19:43
tpbTitle: [soc_versa5g]$ make prog openocd -f /usr/share/prjtrellis/misc/openocd/ecp5-ver - Pastebin.com (at pastebin.com)19:43
daveshahsomlo: you need to set J50 to bypass the ispclock19:44
daveshahshort (1, 2) and (3, 5)19:44
* somlo digs around for magnifying glass...19:44
daveshahfor some reason openocd can't cope with the ispclock in the JTAG chain, even if you give it the IR length and other details19:45
somlo3,5 --that would be the two upper-left-horizontal ones and the far-right vertical two (in a ":::" pattern) -- right ?19:47
somloI had (1,2) (3,4) and (5,6) from the factory19:48
daveshahyes19:52
somloI get the same behavior19:56
daveshahsame error too?19:59
daveshahwould you be able to post a photo of your jumpers?19:59
somlodaveshah: same error, same numbers (except for at the bottom, where I get different READ values each time20:00
somlodaveshah: I'll upload a picture in a few minutes -- and thanks a ton for your help !!!20:01
daveshahone more thing to try would be to put the CFG0, CFG1 and CFG2 jumpers all up (off)20:02
nats`litghost, I still have to make a loop to create pool of 4 thread no ?20:06
somlothere's four dip switches (4,3,2,1) with (3,2,1) labeled cfg0, 1, and 2, respectively20:07
somloand they're down,down,up,down20:07
daveshahsomlo: yeah, try all four up20:07
daveshahthat forces it to JTAG mode, which shouldn't be necessary but might be for some odd reason20:07
nats`ahh no if I understand correctly you tell the pool to create 4 worker and feed the data and it'll do whatever needed to process all the data while using only 4 parallel thread20:09
nats`?20:09
somlonow I don't get the blinky default behavior on power-up, and still the same error20:10
daveshahsomlo: just to double check, this is how J50 should be https://usercontent.irccloud-cdn.com/file/8CeHwSxq/j50.jpg20:12
daveshahsomlo: actually I think your versa board might be from the bad batch20:14
litghostnats: No, create the pool of 4 once.  Then add all of the work at once (either via map/map_async/imap/imap_unordered).  Once all the work is added, it will be batched to each worker.20:14
daveshahwhat is the text in the top right corner, and what is the text on the FPGA?20:14
litghostnats: ya20:14
somlodaveshah: https://imgur.com/a/m3aLqDi20:19
tpbTitle: Imgur: The magic of the Internet (at imgur.com)20:19
daveshahsomlo: yes, you have a bad versa20:19
daveshahLattice accidently built a batch of Versa 5Gs with non 5G FPGAs20:20
daveshahThis is the second I've seen now20:20
daveshahsomlo: replace the nextpnr command in the Makefile with `nextpnr-ecp5 --json attosoc.json --lpf versa.lpf --basecfg ../../misc/basecfgs/empty_lfe5um-45f.config --textcfg [email protected] --um-45k --freq 50 --speed 8` and try again20:21
somloso basically s/um5g/um/g :)20:23
sorearisn't the non-5G architecturally the same whenever both meet timing?20:24
sorearI guess, they have different IDCODEs in OTP and that confuses openocd?20:25
somlohehe, if anyone anywhere ever makes a bad batch of anything, I will probably end up with it :D20:27
daveshahsorear: yes, the idcode is in the bitstream too20:31
daveshahTiming is complicated here, because the 5G version runs at higher Vcc which accounts for the faster fabric20:32
somlowonder if I can get them to send me a replacement...20:32
daveshahI guess this board runs the non 5G version at the higher Vcc, so timing is anyone's guess, although probably closer to the 5G20:32
daveshahsomlo: yes, I'm sure you can20:32
daveshahHowever, there's a risk of getting another from the same batch if you go through the distributor20:33
soreargeneral semiconductor process question: how correlated are process variations on nearby transistors?  on a "slow" chip, are all of the transistors slow, or just a few of them?20:35
sorearI guess there must be some correlated variation since I've seen in-circuit measurement devices that rely on being "near" the operational circuit, but a spatial power spectrum would be very interesting here20:38
somlodaveshah: thanks again, make prog now doesn't error out anymore, and I get 8 flashing LEDs next to the usb connector, four red, and for yellow20:38
daveshahConnect to the second uart (ttyUSB1 with no other devices) at 9600 and you should see hello world20:39
somlowhoo hoo, it works! thanks again for the handholding!20:40
daveshah\o/20:41
nats`litghost, I wrote that but I'm sure it's good enough to handle a crash in one of the thread20:42
nats`https://pastebin.com/LM7wyAn020:42
tpbTitle: [Python] 072 split job python pool - Pastebin.com (at pastebin.com)20:42
litghostYa, that looks good20:45
nats`can you explain to me how it'll be handled ?20:52
nats`because in case of crash of one it should stop all other20:52
litghostthe pool context manager should call terminate20:54
nats`oky20:57
nats`I need to fix the fact that map can't take multiple argument20:57
somlowhile I'm on the topic of ECP5 dev boards, is there anything out there with e.g. a LFE5UM5G-85F and at least 128MB of DDR RAM (i.e. something comparable to the nexys4-ddr or Arty)?21:09
daveshahNot at the moment21:10
daveshahThe only board with significant DDR is the Versa21:10
somloIf only they made an 85 kCell version...21:11
sorearany guesses why they didn't?21:13
litghostjust send an list of arguments21:13
nats`you'll certainly laugh but I don't know how to construct it right :p21:14
nats`blockId = range(0, nbBlocks)21:14
nats`    startI = range(0, pipscount, blocksize)21:14
nats`    stopI = range(blocksize, pipscount + 1, blocksize)21:14
nats`    argList = [ blockId, startI, stopI ]21:14
nats`    with Pool(processes=nbParBlock) as pool:21:14
nats`        pool.map(start_vivado, argList)21:14
nats`this is not good21:14
daveshahsorear: no idea, maybe supply related in the early days21:14
daveshahThey are pin compatible so they could make one easily enough (from memory there's no shortage of capacity on the power supplies)21:14
litghostargList = zip(blockId, startI, stopI )21:14
nats`ahh sure zip.....21:15
nats`I always forget about that one21:15
nats`thx21:15
nats`litghost, testing the new code I hope it'll be good that time :)21:27
nats`will let it run to verify sha121:27
nats`litghost, should I close my PR and reopen one ?21:33
nats`because a lot of commit are useless now21:33
nats`I must admit I don't know how to that in git...21:34
litghostgit rebase -i master21:35
litghostbut before doing that, I would make a backup branch or write down the commit hash21:35
litghostif you are new at git21:36
nats`uhhh can I avoid that ?21:36
litghostThe rebase?21:37
nats`yep21:38
nats`I'm pretty sure I'll break something21:38
litghostFeel free to provide the PR as you feel you can.  rebase is the fastest way, but if you have another way to make a cleaner PR, feel free21:39
nats`if I understand correctly if I rebase on a commit it smash all the history and everything back to that commit ?21:40
litghostYou choose.  default is no change21:41
litghostsquash or s will cause the commit to be merged with it predecessor21:42
nats`litghost, should I close the PR anyway and reopen one rebased21:46
nats`?21:46
nats`because the current one will keep all discussion etc21:46
litghostafter rebasing, git push <remote name> <branch name> --force21:47
litghostand that will update the PR but not force a new PR to be created21:47
litghostassuming you have the same branch name21:48
litghostPR only cares about the branch name21:48
litghostthat's it21:48
litghostIn general do not use --force with push unless you are specifying intending to rewrite history (like a rebase for cleanliness purposes)21:48
nats`I'll make a clone of my local repo to test21:49
litghostsure21:50
litghostThat's overkill, but it will work21:50
nats`for safety in fact21:51
nats`because i'm not sure to understand how rebase works21:51
nats`basically I want to commit my current state to few commit before and smash all those in the pile21:52
litghostya, that's what rebase does21:53
nats`but if I understand it only works on branch21:54
nats`I made all my modification directly in master21:54
litghostthat's fine actually22:04
litghostjust make a branch at the place you are at (git checkout -b <new branch name>)22:04
litghostthen move master back to the upstream location (git checkout master then git reset --hard upstream/master or origin/master)22:05
litghostthen rebase, and you are good to go22:05
nats`I replace origin/master by the commit number I want to base on ?22:07
nats`or the one after the last commit I want to keep ?22:08
litghostgit rebase -i <oldest commit to leave alone>22:08
nats`oky i'll try that the new version is working22:12
nats`last question should I commit my last modification before rebase ?22:13
litghostyes22:14
litghostrebase manipulates commits22:14
litghoststaged and unstaged files are not allowed22:14
nats`ah oky22:14
nats`I'm rebasing should I select drop ?22:21
nats`uhhmm I have a conflict22:24
litghostsquash22:24
litghostnot drop22:24
litghostgit rebase --abort is you friend22:24
nats`yep I just made that22:26
nats`oky i'll try with squash22:26
nats`oky it opens me the comment window with all the previous comment I guess I delete everything22:30
nats`and just make a new one22:30
nats`and I push force now22:32
nats`duhhhhh it's more stressful than playing Resident Evil :)22:37
litghostnats: So that rebased version still generates the same hash?22:41
litghostIf so I'll merge it22:41
*** Foo__ has joined #symbiflow22:42
nats`let me rerun once22:44
nats`only 30 minutes left22:44
nats`but yes22:44
litghostokay, sounds good22:44
nats`last time I tried it was ok22:44
nats`but I prefer to retest :D22:44
litghostgood by me22:44
litghostGood PR, thanks for the investigation!22:44
nats`thanks :)22:45
nats`I'll try to focus on 074 now22:45
nats`and also thanks for he help refreshing my python and git knowledge ;)22:45
nats`s/he/the/22:45
*** powerbit has joined #symbiflow22:49
nats`litghost, maybe you can close https://github.com/SymbiFlow/prjxray/issues/46522:50
tpbTitle: High RAM usage in fuzzer 072-ordered-wires · Issue #465 · SymbiFlow/prjxray · GitHub (at github.com)22:50
mithronats`: How much RAM does it use now?22:50
nats`and that one https://github.com/SymbiFlow/prjxray/issues/17122:50
tpbTitle: All output products should go into "build" dir · Issue #171 · SymbiFlow/prjxray · GitHub (at github.com)22:50
nats`depends on how many thread you run and blocksize but with defaut parameter I eat lesst than 8GB and run in 30 minutes22:50
nats`wait before closing the build dires I need to test that one for 07422:52
nats`mithro I just checked it tops at 4x1.6GB22:53
litghostmithro: Not only does the new code use less memory, it runs ~3x faster22:53
nats`so we could maybe make longer blocksize22:54
nats`making even more fast build22:54
litghostnats: I would focus on fixing 074 before tuning it further22:54
nats`sure22:54
litghostthe current memory requirements are excessive and are the higher priority22:54
nats`with test we could infere an optimum and then automagically parametrize it22:54
nats`yep will take a look22:54
nats`can you run a full 07x Fuzzer build on a server ?22:55
nats`to see if all the PR is coherent ?22:55
nats`in the mean time I'll see what I can do for 07422:55
nats`litghost, 074 will be longer to fix the tcl is way more complicated, I'll try to have that done on sunday (I'm not available saturday)23:06
nats`(basically this evening and sunday I can work on it)23:06
litghostNo rush, this issue has been long standing.23:06
litghostThanks for the work23:06
nats`np :)23:10
*** temp_ has joined #symbiflow23:11
*** temp_ has left #symbiflow23:15
nats`litghost, reverified sha1sum is ok with the pushed version :)23:27
nats`so now it's sure23:27
nats`I mean 101% sure23:27
nats`my first conclusion is the same apparently it's get_nodes again which is leaking ram23:50
nats`at almost the same rate23:50
nats`you know how long it takes to run the 074 initially ?23:51

Generated by irclog2html.py 2.13.1 by Marius Gedminas - find it at mg.pov.lt!