| *** tpb has joined #symbiflow | 00:00 | |
| mithro | digshadow: So my database build failed on imuxlout | 01:25 |
|---|---|---|
| *** nonlinear has joined #symbiflow | 01:51 | |
| *** _whitelogger has quit IRC | 02:21 | |
| *** _whitelogger has joined #symbiflow | 02:24 | |
| *** citypw has joined #symbiflow | 04:05 | |
| *** bitmorse_ has joined #symbiflow | 09:01 | |
| *** karol2 has joined #symbiflow | 09:33 | |
| *** karol2 is now known as kgugala | 09:33 | |
| nats` | oky mithro got it :) | 09:54 |
| *** citypw has quit IRC | 10:24 | |
| *** bitmorse_ has quit IRC | 10:32 | |
| *** bitmorse_ has joined #symbiflow | 10:34 | |
| *** bitmorse_ has quit IRC | 11:23 | |
| *** bitmorse_ has joined #symbiflow | 11:27 | |
| *** bitmorse_ has quit IRC | 11:34 | |
| *** bitmorse_ has joined #symbiflow | 11:34 | |
| *** bitmorse_ has quit IRC | 12:19 | |
| *** bitmorse__ has joined #symbiflow | 12:19 | |
| *** bitmorse__ has quit IRC | 12:48 | |
| *** bitmorse__ has joined #symbiflow | 13:06 | |
| *** bitmorse__ has quit IRC | 13:18 | |
| *** citypw has joined #symbiflow | 13:32 | |
| *** bitmorse__ has joined #symbiflow | 13:36 | |
| *** bitmorse__ has quit IRC | 13:48 | |
| *** bitmorse__ has joined #symbiflow | 14:05 | |
| *** bitmorse__ has quit IRC | 14:19 | |
| mithro | digshadow: I'm blocked on building a new database because of the imuxlout issue | 17:41 |
| digshadow | mithro: its not a new issue, but someone is actively looking at it right onw | 17:44 |
| digshadow | now | 17:44 |
| mithro | Do I just keep rerunning the fuzzer? | 17:45 |
| kgugala | I'm looking at that | 17:45 |
| kgugala | I think I'm getting closer, but still there are some issues that needs to be resolved | 17:46 |
| nats` | https://pastebin.com/hSM0bSkd | 22:07 |
| tpb | Title: [TCL] 072 Memory Leak Split job attempt - Pastebin.com (at pastebin.com) | 22:07 |
| nats` | I'm trying something like that for the problem of the 072 fuzzer | 22:07 |
| nats` | for what I saw the problem comes from get_nodes function | 22:08 |
| nats` | my hope is that the temporary interpreter will end cleaned with everything inside | 22:08 |
| nats` | if not we may need to use sort of a bash script on top to call the tcl with good index value | 22:08 |
| nats` | and sorry for the crappy tcl but it was a long time I didn't write a line of that... language | 22:09 |
| nats` | we will soon have an answer but it may be possible that the main vivado interpreter doesn't clean everything like it should | 22:12 |
| litghost | nats: Does that lower the memory usage as expected? | 22:22 |
| nats` | litghost, uhhmm still have to wait | 22:27 |
| nats` | it doesn't fall after each interpreter delete but apparently GC couyld take care of that later when memory needed | 22:27 |
| nats` | but I think it'll not | 22:27 |
| nats` | let's see | 22:27 |
| nats` | in the worst case i'll split it at shell script level | 22:28 |
| litghost | How are you measuring peak memory usage? I've been using "/usr/bin/time -v <program>" | 22:28 |
| nats` | I'm only looking at top | 22:28 |
| nats` | is it enough ? | 22:28 |
| nats` | by the way the memory module of tcl is not available in the vivado shell | 22:28 |
| nats` | I guess they didn't compile with the good flag | 22:28 |
| litghost | :( | 22:28 |
| nats` | do you think my code is clean enough in tcl ? | 22:29 |
| litghost | using /usr/bin/time provides a nice summary of memory usage and CPU time | 22:29 |
| litghost | tcl looks fine | 22:29 |
| litghost | We should write a comment about why the extra complexity exists, but hold off until it's proven to work | 22:30 |
| nats` | sure | 22:30 |
| nats` | I made a lot of test | 22:30 |
| nats` | and it apperas that calling get_nodes no matter the way you do it eat memory until your close the vivado tcl shell | 22:30 |
| litghost | Likely vivado is opening a bunch of internal data structures, and doesn't close them | 22:31 |
| nats` | https://pastebin.com/0seQdiUJ | 22:31 |
| tpb | Title: [TCL] get_nodes leak ? - Pastebin.com (at pastebin.com) | 22:31 |
| nats` | certainly | 22:31 |
| litghost | For systems with plenty of memory, that is likely a good choice | 22:31 |
| nats` | int hat simple loop it still eat all the ram even with explicit unset | 22:31 |
| nats` | what is a lot ? :D | 22:31 |
| litghost | I have 128 GB, runs fine on a 50k part | 22:31 |
| litghost | However there are 100k and 200k and higher parts | 22:32 |
| litghost | At some point even my system will fall over | 22:32 |
| nats` | sure | 22:32 |
| nats` | what is worrying me is that if it doesn't work with slave interpreter there are some huge problem in their tcl interpreter | 22:32 |
| nats` | because a slave interpreter is deleted with all his context | 22:32 |
| nats` | at least it should | 22:33 |
| litghost | The get_nodes is their interface, there could actually be a non-tcl leak present in that interface | 22:33 |
| nats` | but if I'm not wrong GC implementation of tcl is free to manufacturer | 22:33 |
| nats` | uhhmmm good point ! | 22:33 |
| litghost | We are also using a fairly old Vivado version, so its possible this bug was already fixed in a newer version | 22:33 |
| nats` | something usual with C wrapper | 22:33 |
| nats` | I also found something interesting about old vivado | 22:34 |
| nats` | https://forums.xilinx.com/t5/Vivado-TCL-Community/Memory-Leak-in-Vivado-TCL/td-p/525475 | 22:34 |
| tpb | Title: Memory Leak in Vivado TCL - Community Forums3rd Party Header & Footer (at forums.xilinx.com) | 22:34 |
| nats` | they added a parameter to not pipe all puts through vivado core | 22:34 |
| nats` | should exploded soon | 22:38 |
| nats` | .. | 22:38 |
| nats` | bang | 22:38 |
| nats` | it was auto killed by linux :D | 22:38 |
| nats` | Block: 17 | 22:39 |
| nats` | StartI: 5844379 - StopI: 6188166 | 22:39 |
| nats` | inside interpreter 5844379 6188166 | 22:39 |
| nats` | WARNING: [Vivado 12-2548] 'get_pips' without an -of_object switch is potentially runtime- and memory-intensive. Please consider supplying an -of_object switch. You can also press CTRL-C from the command prompt or click cancel in the Vivado IDE at any point to interrupt the command. | 22:39 |
| nats` | get_pips: Time (s): cpu = 00:00:09 ; elapsed = 00:00:09 . Memory (MB): peak = 16285.488 ; gain = 510.996 ; free physical = 307 ; free virtual = 459 | 22:39 |
| nats` | /home/nats/Xilinx/Vivado/2017.2/bin/loader: line 179: 10787 Killed "$RDI_PROG" "$@" | 22:39 |
| nats` | so I guess a good mitigating solution would be to make an overlay either with an other tcl script or with a bash script to start block processing with good index | 22:40 |
| nats` | bash script seems coherent with the build process we use in fuzzer | 22:40 |
| nats` | uhhmmm I can test with 2017.4 I have it installed ! | 22:41 |
| nats` | nats@nats-MS-7A72:~/project/symbiflow/testing/072_ram_optim$ source /home/nats/Xilinx/Vivado/201 | 22:43 |
| nats` | 2016.4/ 2017.2/ 2017.4/ | 22:43 |
| nats` | (env) nats@nats-MS-7A72:~/project/symbiflow/testing/072_ram_optim$ source /home/nats/Xilinx/Vivado/2017.4/settings64.sh | 22:43 |
| nats` | (env) nats@nats-MS-7A72:~/project/symbiflow/testing/072_ram_optim$ vivado -mode batch -source split_job.tcl | 22:43 |
| nats` | let's try | 22:43 |
| nats` | I may have a 2018 install too | 22:43 |
| nats` | I should present the bill for the occupied hard drive to xilinx :) | 22:43 |
| litghost | Anyways, I'm okay with a bash based approach | 22:45 |
| litghost | You might want a two phase approach, where you identify the number of pips and then delegate to each vivado instance | 22:45 |
| litghost | Much like your interpreter approach | 22:45 |
| nats` | yep :) | 22:45 |
| nats` | and I was thinking each block to a different file | 22:46 |
| nats` | downhill_index.txt | 22:46 |
| nats` | downhill_$index.txt | 22:46 |
| litghost | ah sure, and then concat them | 22:46 |
| nats` | it can be easily merged after and would avoid to generate tons of multiGB text file on hardrive | 22:46 |
| litghost | FYI you could move ordered wires before https://github.com/SymbiFlow/prjxray/tree/master/fuzzers/073-get_counts and use that for the pip count | 22:47 |
| tpb | Title: prjxray/fuzzers/073-get_counts at master · SymbiFlow/prjxray · GitHub (at github.com) | 22:47 |
| nats` | uhhmm I suggest I do things step by step because i'm really new in the project :) | 22:47 |
| nats` | I don't want to make mistake and break things :) | 22:47 |
| litghost | sure | 22:47 |
| nats` | by the way I just realized WARNING: [Vivado 12-2683] No nodes matched 'get_nodes -uphill -of_object INT_R_X1Y149/INT_R.WW4END_S0_0->>WW2BEG3' | 22:48 |
| nats` | could it be a problem when you try to get_nodes on a failed match ? | 22:48 |
| litghost | that is fine, some pips are disconnected | 22:48 |
| nats` | you know usual not covered free on return ? | 22:48 |
| nats` | oky time to go to bed for me but i'll write the bash version tomorrow and test it before submitting it to Push request | 22:50 |
| nats` | and i'll so fix 072-074 fuzzer build | 22:50 |
| nats` | I already wrote it but can't test | 22:50 |
| nats` | good night | 22:51 |
| litghost | nats: Sounds good. As you go, it would be good to fix https://github.com/SymbiFlow/prjxray/issues/171 | 22:56 |
| tpb | Title: All output products should go into "build" dir · Issue #171 · SymbiFlow/prjxray · GitHub (at github.com) | 22:56 |
| litghost | Especially if you are adding more intermediates | 22:56 |
| nats` | litghost, that's what i'm fixing :) | 23:03 |
| nats` | that's because of that one i'm patching 072 because I have patch for 71-74 | 23:04 |
| nats` | 71 is merged but the other are dependant on 072 for testing | 23:04 |
| nats` | just efore going to bed I think I could start different interpreter in parallel for 072 so make a compromise to get more speed | 23:05 |
| nats` | talking tomorrow :) | 23:05 |
| nats` | good night | 23:05 |
Generated by irclog2html.py 2.13.1 by Marius Gedminas - find it at mg.pov.lt!