Sunday, 2019-03-17

*** tpb has joined #symbiflow00:00
duck2well, by 1) using pypy magic, 2) not using prjxray.db_cache, 3) not using threads in and also rearranging some code in build_channels, i could bring the build into the vpr stage in a LXC container with 2gb RAM and 1gb swap. it takes roughly 40 mins or maybe a hour00:11
*** OmniMancer has joined #symbiflow00:12
duck2honestly i don't think i can get away with vpr with this much ram but let's see00:17
duck2litghost: indeed, vtr uses pugixml which needs 2x the file size in memory to build a dom tree
tpbTitle: Incrementally reading very large files · Issue #166 · zeux/pugixml · GitHub (at
duck2in fact, rr_graph.xml would be a very neat json file but i guess that would be hacking too much02:06
litghostduck2: I think a zero parse format is worth investing in the future (e.g. captain proto or flatbuffers).  I have a VPR branch with flatbuffer support, but it turns out the flatbuffer support in python is actually really terrible :(03:01
*** _whitelogger has quit IRC03:46
*** _whitelogger has joined #symbiflow03:49
*** Bertl_oO is now known as Bertl_zZ05:27
*** i8hantanu has joined #symbiflow05:32
*** _whitelogger has quit IRC06:46
*** _whitelogger has joined #symbiflow06:49
*** i8hantanu has quit IRC07:41
*** blurSong has joined #symbiflow08:03
*** _whitelogger has quit IRC09:34
*** _whitelogger has joined #symbiflow09:37
*** _whitelogger has quit IRC09:58
*** _whitelogger has joined #symbiflow10:01
*** riha has joined #symbiflow10:33
*** riha has quit IRC10:36
*** i8hantanu has joined #symbiflow11:29
*** proteusguy has quit IRC12:45
duck2what is the problem with python's flatbuffers support? i looked at the tutorial and it does not seem worse than, say, javascript13:11
*** Bertl_zZ is now known as Bertl13:52
*** i8hantanu has quit IRC14:49
litghostduck2: It is implemented in python, using ctypes.  It is very very slow. It actually turns it to be faster to write out the rr graph in XML, and have VPR load the XML and write it back to flatbuffers.15:05
litghostduck2: It also doesn't support incremental writing, which may become important for larger graphs.15:06
*** OmniMancer has quit IRC15:22
*** _whitelogger has quit IRC15:28
*** _whitelogger has joined #symbiflow15:31
duck2i think converting to libxml2 SAX would be good in terms of memory usage and ease of impl, since the current code is suitable for sax: processes one element at a time15:43
duck2litghost: it's very sad. maybe pypy can help with that though, it runs pure python code quite fast.15:52
*** citypw has quit IRC16:26
*** kraiskil has joined #symbiflow16:34
litghostDuck2: yes, sax parsing was what I was implying above17:16
*** kraiskil has quit IRC23:04

Generated by 2.13.1 by Marius Gedminas - find it at!