Add support for flashing boards

[zguig52]

Hi guys,

I'm working on flashspi and board ECPIX5. I had first an issue with specific clock pin as described here: YosysHQ/prjtrellis#158

I finished by just removing the associated core clk pinout and have this finally:
# SPIFlash
("spiflash", 0,
Subsignal("cs_n", Pins("AA2")),
Subsignal("mosi", Pins("AE2")),
Subsignal("miso", Pins("AD2")),
Subsignal("wp", Pins("AF2")),
Subsignal("hold", Pins("AE1")),
IOStandard("LVCMOS33")
),
("spiflash4x", 0,
Subsignal("cs_n", Pins("AA2")),
Subsignal("dq", Pins("AE2", "AD2", "AF2", "AE1")),
IOStandard("LVCMOS33")
),
Is it the right way to do?

Second, now I would like to flash the FPGA config in it and saw that the make.py --flash is to be done in project linux on vexrisc. Have you a few guidelines/conclusions on how is planned to be implemented this function so I can try to do it?

[zguig52]

What I understood yet (for validation):

• loading to the flash the FPGA SOC needs first to program the FPGA with code to make a bridge and being able to access the chipset pins in programming mode (from the JTAG/FTDI link)
• ECPIX5 can be easily programmed with OpenFPGAloader (http://docs.lambdaconcept.com/ecpix-5/features/debug.html), with this command: openFPGAloader -b ecpix5 -f top.bit
• Using OpenOCD, there is an intermediate code input required to generate the bridge (flash_proxy_basename init property / await a .bit file)
• This code seems to be a good base to produce this intermediate bridge : https://github.com/quartiq/bscan_spi_bitstreams/blob/master/lattice_bscan_spi.py

[zguig52]

Did something working with:
1: changing the programmer type of board ecpix5 to OpenFPGALoader (litex_boards -> platforms/ecpix5.py):
def create_programmer(self):
return OpenFPGALoader("ecpix5")
2: adding the flash function to openfpgaloader to make its prototype compatible with all other models (litex -> build/openfpgaloader.py):
def flash(self, address, bitstream_file):
#address not used, for function prototype compatibility with other flash tools yet
cmd = ["openFPGALoader", "--board", self.board, "--write-flash", "--bitstream", bitstream_file]
self.call(cmd)
3. implementing flash option for linux vexrisc (make.py):
In class Board:
def flash(self, address, filename):
prog = self.platform.create_programmer()
prog.flash(address, filename)
[...]
In main:
if args.flash:
board.flash(address="0x000000", filename=os.path.join(builder.gateware_dir, soc.build_name + board.bitstream_ext))

[enjoy-digital]

Hi @zguig52,

thanks for looking at this. Linux-on-LiteX-VexRiscv is heavily relying on LiteX-Boards (https://github.com/litex-hub/litex-boards) and we should probably add the flash support there by adding a --flash to the targets that would Flash the bitstream in SPI-Flash (silmilar to the --load command that loads the bitstream in SRAM). For ECP5, we are already using OpenOCD to load FPGA bistreams, so if using OpenOCD is possible I would have a preference for this. Once supported in LiteX-Boards, we could reuse this in Linux-on-LiteX-VexRiscv.

[zguig52]

After some digging on the net, I see that there was a pull request related here:
quartiq/bscan_spi_bitstreams#7
It is also discussed here:
litex-hub/litex-boards#61

[zguig52]

I did some tests with current code for flashing through OpenOCD, with the svf file coming from @ilya-epifanov (from previous pull requests). I saw that he was also in charge of most of OpenOCD script associated.
I am stuck currently with this line from lattice/programmer.py: "flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32":

Debug: 135 2 command.c:143 script_debug(): command - ocd_command ocd_command type ocd_flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32
Debug: 136 2 command.c:143 script_debug(): command - ocd_flash ocd_flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32
Error: 138 2 tcl.c:1026 handle_flash_bank_command(): 'jtagspi' driver rejected flash bank at 0x00000000; usage: (null)
Debug: 139 2 command.c:143 script_debug(): command - ocd_command ocd_command type ocd_usage flash bank
Debug: 140 2 command.c:143 script_debug(): command - usage ocd_usage flash bank
User : 142 2 command.c:828 command_help_show_indent(): User : 143 2 command.c:865 command_help_show(): flash bankUser : 144 2 command.c:871 command_help_show(): User : 145 2 command.c:844 command_help_show_wrap(): bank_id driver_name base_address size_bytes chip_width_bytes
User : 146 2 command.c:828 command_help_show_indent(): User : 147 2 command.c:828 command_help_show_indent(): User : 148 2 command.c:828 command_help_show_indent(): User : 149 2 command.c:828 command_help_show_indent(): User : 150 2 command.c:828 command_help_show_indent(): User : 151 2 command.c:828 command_help_show_indent(): User : 152 2 command.c:844 command_help_show_wrap(): bus_width_bytes target [driver_options ...]
User : 153 2 command.c:828 command_help_show_indent(): User : 154 2 command.c:865 command_help_show(): flash banksUser : 155 2 command.c:874 command_help_show():
User : 156 2 command.c:828 command_help_show_indent(): User : 157 2 command.c:865 command_help_show(): flash listUser : 158 2 command.c:874 command_help_show():
User : 159 2 command.c:828 command_help_show_indent(): User : 160 2 command.c:865 command_help_show(): mflash bankUser : 161 2 command.c:871 command_help_show(): User : 162 2 command.c:844 command_help_show_wrap(): soc_type base_addr pin_id target

Did some tests also just to flash the svf file, by commenting the specific lines where defining the target and had errors on reset halt command:
Debug: 6070 2238 svf.c:1548 svf_run_command(): TDO read = 0x0200100
User : 6071 2238 command.c:544 command_print():
Time used: 0m2s118ms
User : 6072 2238 command.c:544 command_print(): svf file programmed successfully for 1963 commands with 0 errors
User : 6073 2238 command.c:687 command_run_line(): invalid command name "reset"

[trabucayre]

Maybe I'm wrong somewhere or not understood everything, but with an ECP5 (and unlike xilinx or intel) a bridge loaded in RAM in not required.
After sending an specific instruction the FPGA give a direct access to the SPI flash. This way is used by openFPGALoader and @gregdavill ecpprog to write the bitstream.
And, for at least, openFPGALoader, it's possible to write both firmware at the beginning of the flash and a binary file at an arbitrary location by using --offset.

[gregdavill]

@trabucayre Is correct the ECP5 JTAG logic contains a command that enables pass-through to the FLASH pins, so you don't have to load a bridge bitstream.

It is possible to use openOCD + a crafted SVF file. But it's quite slow, as the standard SVF protocol does not support conditional loops/branching. So after each page write and sector erase you really need to wait the maximum time as highlighted in your flash datasheet.

I have an example of python script that creates a flat SVF here: https://gist.github.com/gregdavill/4f9f536757966171ef974f98348bbacb

I also wrote a simple tool: https://github.com/gregdavill/ecpprog and I believe that openFPGALoader also works to perform FLASH writes through the JTAG interface with FT232H based programmers.

This approach is much faster because the busy flag in the flash is polled, so erase/write times are closer to typical values, rather than max.

[trabucayre]

I also wrote a simple tool: https://github.com/gregdavill/ecpprog and I believe that openFPGALoader also works to perform FLASH writes through the JTAG interface with FT232H based programmers.

@gregdavill true: openFPGALoader is able to perform FLASH writes with any JTAG compatible device

[trabucayre]

But it's quite slow, as the standard SVF protocol does not support conditional loops/branching. So after each page write and sector erase you really need to wait the maximum time as highlighted in your flash datasheet.

True again. But it's not limited to the flash: each time a wait until status bit is (un)set, SVF add a delay instead of polling for the bit.

[enjoy-digital]

Thanks @trabucayre, @gregdavill for the info/feedback. So let's forget OpenOCD for flashing on ECP5 then, @zguig52 we could switch the ECPIX5 to openFPGALoader or ecpprog and I'll do a pass on the different LiteX-Boards later to be sure we have flashing support for most of the supported boards.

[zguig52]

Hi, thanks all for the reply and support.

I did a quick survey on openFPGALoader capabilities yet WRT litex-boards.

Native supported boards:

openFPGALoader --list-boards
board name:    cable_name
acornCle215     
arty            digilent
colorlight      
crosslinknx_evn ft2232
cyc1000         ft2232
de0nano         usb-blaster
ecp5_evn        ft2232
ecpix5          ecpix5-debug
fireant         ft232
ice40_generic   ft2232
licheeTang      anlogicCable
littleBee       ft2232
machXO2EVN      ft2232
machXO3EVN      ft2232
machXO3SK       ft2232
nexysVideo      digilent_b
pipistrello     ft2232
qmtechCycloneV  
spartanEdgeAccelBoard 
tangnano        ft2232
ulx2s           ft232RL
ulx3s           ft231X
xtrx            

Supported FPGA:

openFPGALoader --list-fpga
IDCode      manufacturer  family         model               
0x81111043  lattice       ECP5           LFE5UM5G-25         
0x81112043  lattice       ECP5           LFE5UM5G-45         
0x81113043  lattice       ECP5           LFE5UM5G-85         
0xe12bb043  lattice       MachXO3LF      LCMX03LF-1300C      
0x100381b   Gowin         GW1N           GW1N-4              
0x10f0043   lattice       CrosslinkNX    LIFCL-17            
0x10f1043   lattice       CrosslinkNX    LIFCL-40-ES         
0x1111043   lattice       ECP5           LFE5UM-25           
0x1112043   lattice       ECP5           LFE5UM-45           
0x1113043   lattice       ECP5           LFE5UM-85           
0x129a043   lattice       XP2            LFXP2-8E            
0x12b5043   lattice       MachXO2        LCMXO2-7000HE       
0x12b9043   lattice       MachXO2        LCMXO2-640HC        
0x12bd043   lattice       MachXO2        LCMXO2-7000HC       
0x20f30dd   altera        cyclone 10 LP  10CL025             
0x2b150dd   altera        cyclone V      5CEA2               
0x3620093   xilinx        spartan7       xc7s15ftgb196-1     
0x362c093   xilinx        artix a7 50t   xc7a50t             
0x362d093   xilinx        artix a7 35t   xc7a35              
0x362f093   xilinx        spartan7       xc7s50              
0x37c4093   xilinx        spartan7       xc7s25              
0x900281b   Gowin         GW1N           GW1N-1              
0xa014c35   anlogic       eagle s20      EG4S20BG256         
0x1100581b  Gowin         GW1N           GW1NR-9             
0x110f1043  lattice       CrosslinkNX    LIFCL-40            
0x13631093  xilinx        artix a7 100t  xc7a100             
0x13636093  xilinx        artix a7 200t  xc7a200             
0x21111043  lattice       ECP5           LFE5U-12            
0x212e3043  lattice       MachXO3D       LCMX03D-9400HC      
0x310f0043  lattice       CertusNX       LFD2NX-17           
0x310f1043  lattice       CertusNX       LFD2NX-40           
0x41111043  lattice       ECP5           LFE5U-25            
0x41112043  lattice       ECP5           LFE5U-45            
0x41113043  lattice       ECP5           LFE5U-85            
0x44008093  xilinx        spartan6       xc6slx45            
0x612b2043  lattice       MachXO3LF      LCMX03LF-1300E      
0x612b3043  lattice       MachXO3LF      LCMX03LF-2100E      
0x612b4043  lattice       MachXO3LF      LCMX03LF-4300E      
0x612b5043  lattice       MachXO3LF      LCMX03LF-6900E      
0x612b6043  lattice       MachXO3LF      LCMX03LF-9400E      
0x612bb043  lattice       MachXO3LF      LCMX03LF-2100C      
0x612bc043  lattice       MachXO3LF      LCMX03LF-4300C      
0x612bd043  lattice       MachXO3LF      LCMX03LF-6900C      
0x612be043  lattice       MachXO3LF      LCMX03LF-9400C      

[zguig52]

Here the quickly done compatibility matrix (to be checked / validated / ...)

FPGACompatilbility.docx

[zguig52]

Best would be to have a one tool fit all, but not sure it will be possible. Maybe the best is to put a layer of abstraction in linux on vexrisc build / flash functions (as it seems to be already ?). Then only the list of requirements might be different depending on the board (openocd / openfpgaloader / ...). Related to OpenFPGALoader VS ecpprog, what are the major pro and cons that you would see related to each solution?

As I understood / saw, there are 3 main operations to be done:

• load gateware live (seems that OpenOCD do this quite good for all boards?)
• write gateware to flash when possible (seems that it is hard/inappropriate to do it everywhere with OpenOCD as appears in the discussion)
• write boot code to flash when possible

What do you think to keep the following operations and add this new one with this prototype:

• [NO CHANGE] board.load(1 mandatory parameter, the gateware file)
• [CHANGED] board.flash_gateware (1 mandatory parameter, the gateware file) / if specific offset is required to write the gateware, maybe this is managed internally by the board itself
• [NEW] board.flash_firmware (1 mandatory parameter, the boot.json?) / there is something related to the offset where to write that has to be managed automatically?

Other point I would like to discuss is about the build output files gateware names, I think it might be useful to add to the file name the configuration associated (or a subfolder for each conf) to the build to avoid always having to rebuild when parameters are changing (when changing the CPU count for ex, and so on).