Generate Graphviz representation without long runtime of 'hierarchy' command

[harnhua]

(Cross-posting from Stack Overflow due to lack of response there; seeking help from the community here)

I am using yosys to read a gate-level Verilog file, and subsequently output the design to a Graphviz dot file to visualize it. Measuring the time taken for the yosys commands used, it seems like parsing to an AST is much faster than generating the hierarchy.

For example, using a small 9.3MB Verilog design file,

• 'read_verilog -defer' takes 465ms
• 'hierarchy -auto-top' takes 42,961ms

For a larger, real-world design that I tried, the 'hierarchy' command does not end even after about 2 hours.

Question: Is it possible to output a Graphviz representation without running the 'hierarchy' command? I tried exploring the options of 'read_verilog' and 'hierarchy' to exclude unneeded operations, but cannot seem to find one that speeds up or omits the 'hierarchy' command.

Here is my Tcl script:

# source file
set time_parse_start [clock clicks -milliseconds]
yosys read_verilog -defer $::env(YOSYS_INPUT_VERILOG)
set time_taken [expr [clock clicks -milliseconds] - $time_parse_start]
puts "Time taken to parse: $time_taken ms"

set time_hier_start [clock clicks -milliseconds]
yosys hierarchy -auto-top
set time_taken [expr [clock clicks -milliseconds] - $time_hier_start]
puts "Time taken to derive hierarchy: $time_taken ms"

set time_output_start [clock clicks -milliseconds]
yosys show -viewer none -format dot -prefix $::env(YOSYS_OUTPUT_DOTPREFIX)
set time_taken [expr [clock clicks -milliseconds] - $time_output_start]
set total_time_taken [expr [clock clicks -milliseconds] - $time_parse_start]
puts "Time taken to output .dot file: $time_taken ms"
puts "Total time taken: $total_time_taken ms"

Executed with: YOSYS_INPUT_VERILOG=c10_default.vqm YOSYS_OUTPUT_DOTPREFIX=c10_default yosys test.tcl

Output:

1. Executing Verilog-2005 frontend: c10_default.vqm
Parsing Verilog input from `c10_default.vqm' to AST representation.
Storing AST representation for module `$abstract\eight_bit_uc'.
Successfully finished Verilog frontend.
Time taken to parse: 507 ms

2. Executing HIERARCHY pass (managing design hierarchy).

2.1. Finding top of design hierarchy..
root of   0 design levels: $abstract\eight_bit_uc
Automatically selected $abstract\eight_bit_uc as design top module.

2.2. Executing AST frontend in derive mode using pre-parsed AST for module `\eight_bit_uc'.
Generating RTLIL representation for module `\eight_bit_uc'.

2.3. Analyzing design hierarchy..
Top module:  \eight_bit_uc

2.4. Analyzing design hierarchy..
Top module:  \eight_bit_uc
Removing unused module `$abstract\eight_bit_uc'.
Removed 1 unused modules.
Time taken to derive hierarchy: 44377 ms

3. Generating Graphviz representation of design.
Writing dot description to `c10_default.dot'.
Dumping module eight_bit_uc to page 1.
Time taken to output .dot file: 431 ms
Total time taken: 45315 ms

End of script. Logfile hash: aed24130cb, CPU: user 45.01s system 0.23s
Yosys 0.34 (git sha1 4a1b5599258, clang 14.0.0 -fPIC -Os)
Time spent: 97% 1x hierarchy (44 sec), 1% 2x read_verilog (0 sec), ...

If this is not an intended use-case, please let me know as well.
I will gladly contribute any solutions back into the codebase.

[nakengelhardt]

So, the -defer flag means "don't do any of the work of interpreting the AST now, only do it on demand when hierarchy is run", so it's natural that all the time is then spent in the hierarchy command.

It is possible to omit hierarchy if you are fully aware of the limitations. The main limitation is that it is not in general possible to build the internal representation (netlist) of a verilog module until you know the value of all its parameters, due to things like ports with generic widths, or generate loops that create a different number of copies of some logic. So the main job of hierarchy is to find all instantiations in each module, look at the parameter values used for this specific instance, and traverse the AST to build the corresponding internal representation of the instantiated submodule for this particular set of parameter values. (It then also checks that the ports of the instantiated submodule match the ones used in the instantiation.) I assume that a netlist would be a flattened module with only one level of hierarchy, with the top module directly instantiating the architecture primitives (mostly LUT and FF cells if you're targeting an FPGA). So you could run just read_verilog without -defer, which will derive the module for the default parameter values right away - this is already what you want for your top module, as presumably a netlist will never be parametric. Then you could omit the call to hierarchy and call show right away. It will be interesting to see how much time this will already take on its own.

The other problem that you could run into with this is that show needs to know which ports on the submodules are inputs and which are outputs to place the connections on the left or right sides of the box. LUT primitives are usually parametric, with the truth table as a parameter, so each instance is a different derivation of the module - I'm not sure if it can pick up the port direction from the default derivation or whether it will not be able to find the module in the design and hence fall back to displaying everything as inout ports (if it doesn't know they'll all appear on the left side of the box, which makes the diagram hard to read).

[harnhua]

Thank you for your detailed and helpful explanation.
Since I would like to visualize instances as vertices and nets as directional edges, hence it seems that I need most of what "Generating RTLIL representation" does in terms of traversing the AST and building the internal representation.

As you suggested, I ran read_verilog without any arguments and skipped hierarchy to measure how long it would take.

Output:

 Yosys 0.34 (git sha1 4a1b5599258, clang 14.0.0 -fPIC -Os)

1. Executing Verilog-2005 frontend: c10_default.vqm
Parsing Verilog input from `c10_default.vqm' to AST representation.
Generating RTLIL representation for module `\eight_bit_uc'.
Successfully finished Verilog frontend.
Time taken to parse without -defer: 40964 ms

2. Generating Graphviz representation of design.
Writing dot description to `c10_default.dot'.
Dumping module eight_bit_uc to page 1.
Time taken to output .dot file: 430 ms
Total time taken: 41394 ms

End of script. Logfile hash: 5b822b80aa, CPU: user 41.27s system 0.17s
Yosys 0.34 (git sha1 4a1b5599258, clang 14.0.0 -fPIC -Os)
Time spent: 98% 2x read_verilog (40 sec), 1% 1x show (0 sec), ...

Comparing the different Tcl script runtimes,

Tcl commands	Runtime
read_verilog -defer + hierarchy	43,426ms
read_verilog	40,964ms

Follow-on question: Are there ways to "aggregate" or "simplify" netlists in the parsing commands, for example, skip or group instances of the same type so that not every single one of them is stored into the internal representation (and subsequently into the Graphviz representation)? Besides reducing runtime to parse large circuits, analytical complexity can be reduced as well.

Tcl script:

# source file
set time_parse_start [clock clicks -milliseconds]
yosys read_verilog $::env(YOSYS_INPUT_VERILOG)
set time_taken [expr [clock clicks -milliseconds] - $time_parse_start]
puts "Time taken to parse without -defer: $time_taken ms"

set time_output_start [clock clicks -milliseconds]
yosys show -viewer none -format dot -colors 29 -prefix $::env(YOSYS_OUTPUT_DOTPREFIX)
set time_taken [expr [clock clicks -milliseconds] - $time_output_start]
set total_time_taken [expr [clock clicks -milliseconds] - $time_parse_start]
puts "Time taken to output .dot file: $time_taken ms"
puts "Total time taken: $total_time_taken ms"

[nakengelhardt]

I think trying to deduplicate the internal representation would take even more computation time, when you are trying to reduce it (though it could possibly reduce memory usage). There's nothing more you can do here if you want to use yosys (theoretically you could try to optimize the frontend but I really do not recommend that, the code is very difficult to touch). If you don't need the yosys feature set but just a graph, you can try and see if any of the other open source verilog frontends can do it faster.

[harnhua]

Thank you for your insights.