Vram classes (#302)

* vram classes wip

* more progz

* smores

* fixins

* corrections

* grood

* clean

* fix test

* PR comments

CHANGELOG.md

@@ -1,5 +1,14 @@
 # splat Release Notes
 
+### 0.19.0: vram_classes
+
+* New top-level yaml feature: `vram_classes`. This allows you to make common definitions for vram locations that can be applied to multiple segments. Please see the [documentation](docs/VramClasses.md) for more details!
+  * Renamed `ld_use_follows` to `ld_use_symbolic_vram_addresses` to more accurately describe what it's doing
+  * Renamed `vram_of_symbol` segment option to `vram_symbol` to provide consistency between the segment-level option and the vram class field.
+  * Removed `appears_after_overlays_addr` symbol_addrs option in favor of specifying this behavior with `vram_classes`
+* Removed `dead` symbol_addrs option
+* A warning is now emitted when the `sha1` top-level yaml option is not provided. Adding this is highly recommended, as it prevents errors using splat in which the wrong binary is provided.
+
 ### 0.18.3
 
 * splat now will emit a `FILL(0)` statement on each segment of a linker script by default, to customize this behavior use the `ld_fill_value` yaml option or the per-segment `ld_fill_value` option.

disassembler_section.py

@@ -1,12 +1,9 @@
-import spimdisasm
-from util import symbols
-from typing import Optional, Set, Tuple
-from segtypes.segment import Segment
-from util import log, options, symbols
+from abc import ABC, abstractmethod
+from typing import Optional
 
+import spimdisasm
 
-from abc import ABC, abstractmethod
-from typing import Callable
+from util import options, symbols
 
 
 class DisassemblerSection(ABC):

docs/Adding-Symbols.md

@@ -83,14 +83,6 @@ Emits a symbol after the end of the data of the current symbol. Useful to refere
 rspbootTextStart = 0x80084690; // name_end:rspbootTextEnd
 ```
 
-### `appears_after_overlays_addr`
-
-TBD
-
-### `dead`
-
-TBD
-
 ### `defined`
 
 Forces the symbol to be defined - i.e. prevent it from appearing in `undefined_syms_auto.txt` should splat not encounter the symbol during the symbol detection phase.

docs/Configuration.md

@@ -362,10 +362,10 @@ Determines the list of section labels that are to be added to the linker script
 
 Determines whether to add wildcards for section linking in the linker script (.rodata* for example)
 
-### ld_use_follows
+### ld_use_symbolic_vram_addreses
 
-Determines whether to use "follows" settings to determine locations of overlays in the linker script.
-If disabled, this effectively ignores "follows" directives in the yaml.
+Determines whether to use `follows_vram` (segment option) and `vram_symbol` / `follows_classes` (vram_class options) to calculate vram addresses in the linker script.
+Enabled by default. If disabled, this uses the plain integer values for vram addresses defined in the yaml.
 
 ### ld_partial_linking
 

docs/VramClasses.md

@@ -0,0 +1,59 @@
+# vram classes
+
+Version 0.19.0 introduced `vram_classes`, a new top-level yaml section that can be used to help reduce duplicated data in your yaml and more clearly organize its memory layout.
+
+## Introduction
+Before vram classes, you might have had something like this in your yaml:
+
+```yaml
+- type: code
+  start: 0x4269D0
+  vram: 0x802A9000
+  vram_symbol: battle_move_end
+  subsegments: ...
+- type: code
+  start: 0x4273B0
+  vram: 0x802A9000  # notice same `vram` and `vram_symbol` for both segments
+  vram_symbol: battle_move_end
+  subsegments: ...
+```
+
+Having to duplicate the vram address and vram_symbol properties for potentially dozens of hundreds of overlay segments is tedious and pollutes your yaml with repeated information that can become out of sync. Enter vram_classes!
+
+```yaml
+- type: code
+  start: 0x4269D0
+  vram_class: maps
+  subsegments: ...
+- type: code
+  start: 0x4273B0
+  vram_class: maps
+  subsegments: ...
+```
+
+Here, we are telling splat that both of these segments use the `maps` vram class. We are now effectively pointing both segments to the same source of information. Now let's look at how vram classes are defined:
+
+## Format
+
+```yaml
+options:
+  ...
+  ld_use_symbolic_vram_addresses: True
+  ...
+vram_classes:
+  - { name: maps, vram: 0x802A9000, vram_symbol: battle_move_end }
+```
+
+`vram_classes` is a top-level yaml section that contains a list of vram classes. You can either define them in dict form (as seen above) or list form. However, for list form, only `name` and `vram` are supported (`[maps, 0x802A9000]`). If you want to specify other options, please use the dict form. The fields supported are as follows:
+
+- `name` (required): The name of the class
+
+- `vram` (required): The vram address to be used during disasembly. If `ld_use_symbolic_vram_addresses` is disabled or no `vram_symbol` or `follows_classes` properties are provided, this address will be used in the linker script.
+
+The following properties are optional and only take effect if `ld_use_symbolic_vram_addresses` is enabled:
+
+- `vram_symbol`: The name of the symbol to use in the linker script for this class.
+
+- `follows_classes`: A list of vram class names that this class must come after in memory. If we added `follows_classes: [apples, bananas]` to our above vram_class, this would make all `maps` segments start at the end of all `apples` and `bananas` segments.
+
+The internal linker script symbol name that is chosen for `follows_classes` is the name of the class followed by `_CLASS_VRAM`. You can override this by also specifying `vram_symbol`.

segtypes/common/textbin.py

@@ -106,13 +106,9 @@ def split(self, rom_bytes):
                 self.rodata_sibling.write_asm_contents(rom_bytes, f)
 
     def should_scan(self) -> bool:
-        return (
-            options.opts.is_mode_active("code")
-            and self.rom_start is not None
-            and self.rom_end is not None
-        )
+        return self.rom_start is not None and self.rom_end is not None
 
     def should_split(self) -> bool:
         return (
-            self.extract and options.opts.is_mode_active("code") and self.should_scan()
+            self.extract and self.should_scan()
         )  # only split if the segment was scanned first

segtypes/linker_entry.py

@@ -1,10 +1,8 @@
 import os
 import re
-from dataclasses import dataclass
 from functools import lru_cache
 from pathlib import Path
 from typing import Dict, List, OrderedDict, Set, Tuple, Union
-from segtypes.n64.palette import N64SegPalette
 
 from util import options
 
@@ -57,17 +55,6 @@ def write_file_if_different(path: Path, new_content: str):
             f.write(new_content)
 
 
-def segment_cname(segment: Segment) -> str:
-    name = segment.name
-    if segment.parent:
-        name = segment.parent.name + "_" + name
-
-    if isinstance(segment, N64SegPalette):
-        name += "_pal"
-
-    return to_cname(name)
-
-
 def get_segment_rom_start(cname: str) -> str:
     if options.opts.segment_symbols_style == "makerom":
         return f"_{cname}SegmentRomStart"
@@ -124,7 +111,7 @@ def get_segment_section_size(segment_name: str, section_type: str) -> str:
 
 
 def get_segment_vram_end_symbol_name(segment: Segment) -> str:
-    return get_segment_vram_end(segment_cname(segment))
+    return get_segment_vram_end(segment.get_cname())
 
 
 class LinkerEntry:
@@ -208,7 +195,7 @@ def add(self, segment: Segment, max_vram_syms: List[Tuple[str, List[Segment]]]):
         self.entries.extend(entries)
         self.dependencies_entries.extend(entries)
 
-        seg_name = segment_cname(segment)
+        seg_name = segment.get_cname()
 
         for sym, segs in max_vram_syms:
             self.write_max_vram_end_sym(sym, segs)
@@ -264,7 +251,7 @@ def add(self, segment: Segment, max_vram_syms: List[Tuple[str, List[Segment]]]):
         self._end_segment(segment, all_bss=not any_load)
 
     def add_legacy(self, segment: Segment, entries: List[LinkerEntry]):
-        seg_name = segment_cname(segment)
+        seg_name = segment.get_cname()
 
         # To keep track which sections has been started
         started_sections: Dict[str, bool] = {
@@ -329,7 +316,7 @@ def add_referenced_partial_segment(
         segments_path = options.opts.ld_partial_build_segments_path
         assert segments_path is not None
 
-        seg_name = segment_cname(segment)
+        seg_name = segment.get_cname()
 
         for sym, segs in max_vram_syms:
             self.write_max_vram_end_sym(sym, segs)
@@ -391,7 +378,7 @@ def add_partial_segment(self, segment: Segment):
         self.entries.extend(entries)
         self.dependencies_entries.extend(entries)
 
-        seg_name = segment_cname(segment)
+        seg_name = segment.get_cname()
 
         section_entries: OrderedDict[str, List[LinkerEntry]] = OrderedDict()
         for l in segment.section_order:
@@ -512,8 +499,11 @@ def _write_symbol(self, symbol: str, value: Union[str, int]):
     def _begin_segment(
         self, segment: Segment, seg_name: str, noload: bool, is_first: bool
     ):
-        if options.opts.ld_use_follows and segment.vram_of_symbol:
-            vram_str = segment.vram_of_symbol + " "
+        if (
+            options.opts.ld_use_symbolic_vram_addresses
+            and segment.vram_symbol is not None
+        ):
+            vram_str = segment.vram_symbol + " "
         else:
             vram_str = (
                 f"0x{segment.vram_start:X} "
@@ -547,7 +537,7 @@ def _begin_segment(
     def _end_segment(self, segment: Segment, all_bss=False):
         self._end_block()
 
-        name = segment_cname(segment)
+        name = segment.get_cname()
 
         if not all_bss:
             self._writeln(f"__romPos += SIZEOF(.{name});")
@@ -600,7 +590,7 @@ def _end_section(self, seg_name: str, cur_section: str) -> None:
 
     def _write_linker_entry(self, entry: LinkerEntry):
         if entry.section_link_type == "linker_offset":
-            self._write_symbol(f"{segment_cname(entry.segment)}_OFFSET", ".")
+            self._write_symbol(f"{entry.segment.get_cname()}_OFFSET", ".")
             return
 
         # TODO: option to turn this off?

segtypes/n64/palette.py

@@ -6,6 +6,7 @@
 from util.color import unpack_color
 
 from segtypes.n64.segment import N64Segment
+from util.symbols import to_cname
 
 if TYPE_CHECKING:
     from segtypes.n64.ci import N64SegCi as Raster
@@ -60,6 +61,9 @@ def __init__(self, *args, **kwargs):
                         f"Error: {self.name} (0x{actual_len:X} bytes) is not a valid palette size ({', '.join(hex(s) for s in VALID_SIZES)})\n{hint_msg}"
                     )
 
+    def get_cname(self) -> str:
+        return super().get_cname() + "_pal"
+
     def split(self, rom_bytes):
         if self.raster is None:
             # TODO: output with no raster