Monday, 2019-05-06

*** tpb has joined #litex00:00
futarisIRCcloud_florent__ / Dolu1990 / daveshah: So I made the changes for ulx3s and tried compiling for 12F and 25F (using FOSS tools).  It fits, but I don't have access to a ulx3s to test it on, though. I think gojimmypi has a 12F though...03:38
keesjdaveshah: what server are you using? the one with dnsmasq or some tftpd?06:49
keesj dnsmasq is quite reliable in my experience06:50
keesjthe latter not06:50
daveshahkeesj: tftp-hpa, so the latter07:43
daveshahI'll try dnsmasq07:43
daveshahfutarisIRCcloud: Awesome07:43
daveshahHow are you planning to boot it (the ULX3S doesn't have Ethernet)07:44
futarisIRCclouddaveshah: SPI flash, much like arty does.07:44
daveshahAh, I didn't realise it had enough07:44
daveshahSome of the early ULX3Ss only have 4MB07:45
futarisIRCcloudFlash: Quad-SPI Flash IS25LP128F (4/8/16 MB for FPGA config and user data)07:45
futarisIRCcloudI suppose someone can load things over jtag or use the esp32.07:45
daveshahThe FOSS tools don't support the SPI flash either yet (need to sort out the MCLK primitive, and write some script to deal with the SPI over JTAG stuff)07:48
futarisIRCcloudflterm over ttyUSB serial should work at the very least.07:50
daveshahYes, might want to use something a bit faster than 115200 though07:51
_florent__it should work up to 3Mbauds07:53
keesj (my setup) I have two network card and one of them I configured using a static address. I am running ubuntu (that already has dnsmasq running). I start a second session with the config in the link (dhcp server and tftp boot). I am quite carefull to not play dhcp server in the office07:53
tpbTitle: Ubuntu Pastebin (at
daveshah_florent__: perfect, think that's the limit of the SPI chip the ULX3S uses too07:54
futarisIRCcloudYep. We could also get valentyusb running in the litex BIOS...07:54
_florent__we would also boot from SDCard with the ULX3S07:55
daveshahThat would be great07:55
_florent__i saw the discussion about that on twitter, i was also planning to add LiteSDCard boot support to LiteX BIOS07:55
daveshahGuess we could also have Linux SD access, even if just using GPIO for now to avoid custom drivers07:55
_florent__LiteSDCard is not resource optimal for now, lots of resources are lost on CDC (the PHY is running in another clock domain)07:57
_florent__we could have a minimal version that just runs with the sys_clk and remove this CDC07:57
daveshahIf you ran the link at 2x system clock, you could probably avoid the need for arch-specific DDR primitives too08:02
_florent__yes, that would be nice to have something vendor agnostic08:04
_florent__daveshah: i haven't yet looked how you integrated ethernet on your previous linux work, could you eventually provide me the linux patches, config, dts and directions on how to integrate it?08:06
_florent__for now we could probably just provide the linux patches and make buildroot apply the patches directly during the build08:07
daveshah_florent__: yes, that should work08:08
_florent__this avoid having to maintain a buildroot/linux kernel fork08:08
daveshahI'll look at the patches I made for 4.19 and see if they all still work on 508:08
_florent__ok thanks08:09
_florent__btw, i just tested rebuilding the versa5g target, it's working fine here too08:09
futarisIRCcloudlitex ethernet driver doesn't really work well in linux. I just e-mailed Joel earlier today...08:09
daveshahIf you are curious, it's pretty much all commits from e736fb4 on
tpbTitle: Commits · daveshah1/litex-linux-riscv · GitHub (at
futarisIRCcloudhe said "I saw that some people had done some further work on the networkdriver. Something I keep meaning to clarify was that the driver was work in progress - I was quite surprised that anyone could use it for anything more than sending a handful of UDP packets. As people found out, it hadn't implemented the reuse of memory, so every packet that came in would consume more and more memory until the system OOMd."08:10
daveshahI added a free to fix that08:10
daveshahI also fixed another panic bug08:10
daveshahIt was rock solid after that for a NFS rootfs with Yosys and nextpnr in it08:10
daveshahSee and
tpbTitle: liteeth: memory leak fix · daveshah1/[email protected] · GitHub (at
futarisIRCcloudI should just make a patch, and test it in buildroot (on arty).08:12
tpbTitle: Commits · futaris/linux-on-litex-vexriscv · GitHub (at
futarisIRCcloudOlofK has a ULX3S at ...08:19
_florent__futarisIRCcloud: i also have one and daveshah too08:23
futarisIRCcloudSo it looks like the esp32 doesn't have a driver for linux (esp8689), like the esp8266 had (esp8089).08:40
*** Dolu has joined #litex10:03
Doludaveshah : I added the yesterday specified option to relax the MMU translation timings for the D$. It improve timings by 6 Mhz in my case (unless that's seed luck ^^)10:42
DoluI used sbt "runMain vexriscv.GenCoreDefault --externalInterruptArray=true --csrPluginConfig=linux-minimal --singleCycleMulDiv false --dBusCachedRelaxedMemoryTranslationRegister true --prediction none"  on the enjoy-digital master head.10:42
DoluBut in all try i did, the critical paths names where completly eated by Yosys. Did you used a specific option/command to get them in the reports to sent me ? Or it was directly like this at the end of the synthesis logs ?10:45
daveshahDolu: there was a bug because a recent change in abc11:59
daveshah fixes it but isn't merged yet11:59
daveshahThen the net names should be more sensible (also make sure to look at net, not cell names)12:00
daveshah_florent__: I've added the liteeth driver as a kernel patch here:
tpbTitle: GitHub - daveshah1/linux-on-litex-vexriscv at linux_liteeth (at
daveshahIt isn't working yet however, as I can't figure out how to wire up the interrupt in the dts13:28
daveshahI think the external interrupt controller might not be being set up correctly, afaik nothing else is using it in Linux yet (timer is a special interrupt and UART is SBI)13:28
DoluLet's me know if you need help for debugging the RISC-V side of stuff. Also on change compared to previously, is that now VexRiscv has a specific external interrupt pin for the supervisor mode.13:35
DoluBy the past i don't know exactly how they tricked the things to get it working without it.13:35
daveshahPreviously Antmicro wrote a driver for the VexRiscv custom interrupt controller that used the CSRs13:35
daveshahI seem to remember tweaking how the interrupt was routed too13:36
Dolu--externalInterruptArray add the ExternalInterruptArrayPlugin plugin which is a none standard way of having 32 bits external interrupts in CSR13:38
DoluBut that's machine mode only13:38
DoluIf it can help i can realy easily create the same for the  supervisor mode.13:39
DoluActualy it is for the machine mode one13:39
tpbTitle: VexRiscv/ExternalInterruptArrayPlugin.scala at master · SpinalHDL/VexRiscv · GitHub (at
DoluOr we can have in this plugin a set of interrupt mask for both Machine and Supervisor mode.13:40
DoluAs far i remember, this none standard way of handeling a vector of interrupts in the CSR was made to be similar to the other softcore already presents in Migen/Litex13:43
daveshahYes, one of the issues is that stuff needs to work both in the LiteX BIOS and in Linux13:45
DoluSo, modifying, modifying the mask system to have maskMachine + maskSupervisor. And potentialy generating interrupts on both externalInterrupt and supervisorExternalInterrupt would be a good way of solving things ?13:48
_florent__Dolu: yes that could be a way to start getting things working13:54
_florent__daveshah: thanks for the liteeth patch, can't look now but i'll do a bit later13:55
DoluSo i go patch the ExternalInterruptArrayPlugin plugin ?13:56
_florent__Dolu: having the same interrupt interface for the supervisor mode than for the machine mode seems fine yes (if i'm understanding things correctly)14:06
DoluRight, it would use exactle the same physical 32 bits inputs pins14:11
DoluAnd internal in the CPU, you will get to set of interrupts mask, one for machine mode, one for supervisor14:11
DoluWhich would then be used to drive the two external interrupts flags of the cpu internaly, externalInterrupt (for the machine mode) and supervisorExternalInterrupt14:12
DoluI just have one question : Actualy, the pendings CSR flags is masked with the mask CSR    val pendings = mask & RegNext(externalInterruptArray)14:13
DoluIs that what you expect ? or when you read the pendings CSR, you expect to see all interrupts values, independently of the state of their mask ?14:13
_florent__Dolu: this seems fine, we don't want to see the status of interrupts that are not enabled, if you are not doing the mask in hardware, we would have to do it in software14:16
DoluSo i go forward with that14:20
_florent__thanks :)14:26
Doludaveshah _florent__ : It's updated and merged in the VexRiscv-Verilog repo. You just have to regenerate the VexRiscv.v for your needs. No addititional arguements is required14:33
DoluNice ^.^14:50
DoluSo, there would be probably a bit of waveform debugging to figure out why for instance, on that system, the boot time is that slow.14:50
daveshahThat system is SDRAM rather than DDR314:51
Doluok,What is the write policy of the L2 ? write-back or write through ?14:54
DoluIt is  write-back, cool.14:57
somlo_florent__: is this line: strictly necessary, given that it's immediately followed by "irq_setie(0)" ?16:17
tpbTitle: litex/boot.c at master · enjoy-digital/litex · GitHub (at
_florent__somlo: it will probably work without, but that's better to keep it (at least we know that the mask is cleared when jumping to new code)16:19
somloMakes sense, thanks! -- I was just studying the code, trying to figure out how to integrate the rocket PLIC with the least possible disruption :)16:22
somloIt looks like once the PLIC is initialized, it has MMIO bitmap registers for mask/unmask and  pending pins, so it should hopefully fit into the existing model, mostly. Substitute MMIO register read/write ops for csrr/csrw, but that's about it16:25
somlothe only funky part will be the isr() routine in bios/isr.c -- the PLIC has this thing where one "claims" an interrupt, then handles it, then notifies the PLIC of its "completion"16:27
somlobut that's a small routine, I can just #ifdef a rocket-specific one in its place16:28
keesjso WOW all this stuff amazing17:17
Doludaveshah, _florent__ : For the vector interrupt, notice the CSR id at
tpbTitle: VexRiscv-verilog/GenCoreDefault.scala at master · enjoy-digital/VexRiscv-verilog · GitHub (at
daveshahSo I think the Antmicro Vexriscv IRQ driver ( will need some tweaking, and integration with the new kernel, to get the Ethernet interrupts working17:29
tpbTitle: litex-linux-riscv/irq-vex.c at master · daveshah1/litex-linux-riscv · GitHub (at
DoluFrom a hardware point of view, just swapping the CSR ID should be fine17:39
DoluIt would then generate a regular Supervisor External Interrupt. I'm just not sure how the Supervisor External Interrupt is binded to that irq driver.17:40
daveshahYeah, in the old system the Linux startup code was hacked to jump to vex_handle_irq when SEI occurred17:41
daveshahNot sure how upstream Linux does this17:41
daveshahSeems like it calls handle_arch_irq
tpbTitle: linux/irq.c at master · torvalds/linux · GitHub (at
daveshahneed to work out where that goes17:44
*** Dolu has quit IRC17:45
daveshahaha, looks like we use set_handle_irq to set this function as a IRQ handler17:45
tpbTitle: linux/handle.c at master · torvalds/linux · GitHub (at
daveshah(afaik CONFIG_GENERIC_IRQ_MULTI_HANDLER is always set on RISC-V)17:46
*** Dolu has joined #litex19:05
*** Dolu has quit IRC19:10
*** Dolu has joined #litex21:29
*** Dolu has quit IRC22:19
*** Dolu1990 has joined #litex22:20
*** Dolu1942 has joined #litex23:13
futarisIRCcloudWow. So in a weekend we've gone from vexrisc-linux running in sim, to arty, then ecp5 versa, then ulx3s.23:13
futarisIRCclouddaveshah: Yep. CONFIG_GENERIC_IRQ_MULTI_HANDLER is always on in RISC-V.23:14
*** Dolu1990 has quit IRC23:17
Dolu1942That's great !23:18
Dolu1942But please, let's me know if performance are poor, area is too big, or if things are going wrong23:21
futarisIRCcloudmithro: Some discussion in #debian-riscv on OFTC / re rv32 debian23:41
tpbTitle: Snippet | IRCCloud (at
*** Dolu1942 has quit IRC23:57

Generated by 2.13.1 by Marius Gedminas - find it at!