Accessing Memory in Grue

I need to talk about how to access memory so I can read values from it. And to do that I have to talk a little about a Z-Machine area of memory called the header. In this post, I’ll start getting information about a zcode program by reading the header stored in the memory of the zcode itself.

Fair warning: this post is going to be the heaviest so far on the actual implementation, thus there will be more code here than shown in previous posts.

A Little on the Header

First, let’s talk about that header part. The Z-Machine specification says:

By tradition, the first 64 bytes are known as the “header”.

Later on in the specification, I came across this:

The header table summarises those locations in the Z-machine’s header which an interpreter must deal with.

Well, since an interpreter must deal with them, that sounds pretty important. So I now know that I will want to read some information from this header for Grue to operate and that means reading from memory. In this case, I’m referring to the zcode memory or the “story state” that I talked about in the memory and state posts.

Reading from Memory

Turning again to the Z-Machine specification, I read this:

There are three kinds of address in the Z-machine, all of which can be stored in a 2-byte number: byte addresses, word addresses and packed addresses.

Two of those make sense from when I talked about bits and bytes. The “packed” concept is something that required a bit of digging into on my part and I won’t focus on that in this post. So let’s consider what we have so far. I have this simple logic in place in my App module:

zcode_file = open(program, "rb")
Memory(zcode_file.read())

And the Memory module looks like this

class Memory:
    def __init__(self, data: bytes) -> None:
        self.data: bytes = data

I have self.data containing the binary data of a zcode program. Since that binary data is a serialized state of the initial Z-Machine memory, I have to read the contents of that variable. From the specification, I know I have to read those three types of addresses from that variable. Thus, in terms of abstractions, I think having some helper methods to read each of those address types would be useful. But first, what am I going to read?

Read from the Header

The Z-Machine specification tells me:

The first byte of any story file, and so the byte at memory address 0, always contains the version number of the Z-machine to be used.

So here is what I created on the Memory class:

class Memory:
    def __init__(self, data: bytes) -> None:
        self.data: bytes = data

        self.version: int = self.read_byte(0x00)

    def read_byte(self, address: int) -> int:
        return self.data[address]

I created a helper method called read_byte and then I have a variable, version, that will store the byte at address 0. This should be the version of the zcode program. Or, put another way, the byte at this address tells Grue what generation of the Z-Machine the zcode program was designed to run under.

If you consider those oracle zcode programs I mentioned in the context post, each of those could be run and what they should return is a different version number. For example, zork1.z1 should return 1, whereas zork1.z5 should return 5. And this brings up how I want to test for this.

Testing the Memory

As a tester, I always want to get a basis for how I’m going to test early. What I ended up doing was creating a fixtures directory in my tests directory and adding some of those Zork 1 oracle files to that directory. Then I created a test_memory.py module that looks like this:

import pytest
from pathlib import Path
from grue.memory import Memory
from expects import expect, equal

FIXTURES = Path(__file__).parent / "fixtures"


@pytest.fixture
def z1_program():
    with open(FIXTURES / "zork1.z1", "rb") as file:
        return file.read()


@pytest.fixture
def z2_program():
    with open(FIXTURES / "zork1.z2", "rb") as file:
        return file.read()


@pytest.fixture
def z3_program():
    with open(FIXTURES / "zork1.z3", "rb") as file:
        return file.read()


def test_z1_version(z1_program):
    memory = Memory(z1_program)
    expect(memory.version).to(equal(1))


def test_z2_version(z2_program):
    memory = Memory(z2_program)
    expect(memory.version).to(equal(2))


def test_z3_version(z3_program):
    memory = Memory(z3_program)
    expect(memory.version).to(equal(3))

Here I just use the fixtures to pass the binary data of the oracles into the Memory module and check the version variable, which is in turn showing that my read_byte helper method is working as expected.

Arguably, rather than having the actual program files I could have mocked some zcode. But as development of Grue continues, my suspicion is that any such mock would grow complicated enough that it might as well just be the real thing. Given that the zcode files are relatively small, this seemed like a good tradeoff to make.

This is a good start! I can apparently read an aspect of the header, which the Z-Machine specification tells me is quite important for my interpreter. So I’m going to continue on.

More Header Data

The Z-Machine specification tells me this:

Story files are mechanically best identified by their release number and serial code, which are written into the header information at the bottom of Z-machine memory. The release number can be anything between 0 and 65535 but is usually between 1 and 100. The serial code can consist of any six textual characters but is usually the date of compilation, arranged YYMMDD: thus 970619 refers to June 19th, 1997.

This sounds like more stuff I can read from the memory and this means some more variables.

self.version: int = self.read_byte(0x00)
self.release_number: int = self.read_word(0x2)
self.serial_code: bytes = self.read_bytes(0x12, 6)

Notice the size of the release number mentioned in the specification? This means I have to treat that as a word (two bytes) and so I need another helper function for that. Likewise, the serial code is actually multiple bytes that have to be read, hence yet another helper function.

def read_byte(self, address: int) -> int:
        return self.data[address]

def read_bytes(self, address: int, length: int) -> bytes:
        return self.data[address: address + length]

def read_word(self, address: int) -> int:
        return (self.data[address] << 8) | self.data[address + 1]

Notice that each of those helper methods is reading from that binary data. I do have some tests in place to check this stuff.

def test_z3_release_number(z3_program):
    memory = Memory(z3_program)
    expect(memory.release_number).to(equal(88))


def test_z3_serial_code(z3_program):
    memory = Memory(z3_program)
    expect(int(memory.serial_code.decode("utf-8"))).to(equal(840726))

Here I’m just running the tests against one of the reference implementation zcode binaries. And, sure enough, if I open that reference program in someone else’s Z-Machine interpreter, like Frotz, I see:

So this tells me my overall logic is sound on how to develop and test. It also shows me that I’m correctly reading memory. If you want to follow along with the implementation of Grue as described in this post, you can do the following if you have cloned the repo:

git checkout v0.0.3

At this point, I felt it would be good to step back from the code for a bit and learn a little more about this memory that I’m reading. The next post will be a bit of a deep dive based on that learning.