What is mounting?

Up to this point, we’ve been looking at filesystems as a means of storing and accessing data, and we’ve discussed the filesystem interface in great detail, but we haven’t considered how to actually spawn a filesystem or how filesystems can interact with each other.

It should be no surprised that most operating systems have the ability to manage multiple filesystems at a given time. For example, most pen drives are formatted to FAT32 or NTFS, but linux systems, which usually use ext4 as their native filesystem, can still interact with data on these pen drives. This is possible in a sane manner thanks to the ability to mount a filesystem.

Mounting a filesystem generally refers to spawnning an instance of a filesystem, initializing it, and then assigning it a path on top of your existing file tree. Generally, in order to ensure that the paths you assign to a filesystem make sense, you need to have a pre-existing file or directory that you can set as the mountpoint. This is achieved either via the mount system call, or the mount utility.

With something like FUSE that operates in userspace, when mounted, a program is started in userspace which handles all the filesystem callbacks.

You can see an example of how we implemented mounting in our simulator at this file: /js/lfs.mjs.

Writing our own filesystem!

Way back in section 3 you wrote a mini filesystem that provided a single virtual file and implemented read and write for that file. Since then, you’ve hopefully learned a lot about how filesystems are structured and what the various callbacks provide, so let’s revist the topic of writing filesystems and write a simple filesystem that can actually be used to store data.

If you want to skip ahead to the end and try your hand at implementing your own filesystems that do interesting things, see the tutorial at /pages/writing-programs.

The filesystem we’ll be implementing will be an in-memory filesystem, meaning that all data will be stored in memory and when the filesystem process ends (or in our case, the page is refreshed or closed), all associated data will be lost. This is useful for implementing something like /tmp on most linux systems which store temporary files until the system is rebooted. On some systems /tmp is an in-memory filesystem.

Technically the toy filesystem we’ve been using so far is an in-memory filesystem, but it goes through the trouble of simulating a disk to enable the visualizations and make the simulator feel like a real UNIX box. We won’t apply such restrictions to ourselves here and we’ll take advantage of dictionaries and other javascript data structures to build a filesystem. Don’t worry if you don’t know javascript (I’m most certainly not an expert myself), we’ll guide you through the steps.

Let’s take a look at the default filesystem interface that we’ll be deriving our filesystem from.

Loading...

You can find the full details regarding each callback in /pages/filesystem-operations.html.

How do classes work in javascript?

The code above defines a class in javascript named defaultFS. To define a class, we can use the class keyword. You can use the this keyword to set and get properties of the class. The constructor of the class is just a special method named constructor. E.g. to define a class that has a constructor that takes in 1 parameter and saves it as a property:

class MyClass {
    constructor(x) {
        this.my_x = x;
    }
}

And now we can create a new instance of the class with:

var instance = new MyClass('a');
console.log(instance.my_x); // Will print 'a' to the consol

To add methods on the class we have two options, the first: add it to the class definition

``javascript class MyClass { constructor(x) { this.my_x = x; }

print_my_x() {
    console.log(this.my_x);
} } ```

Alternatively we can add it to the prototype of the class like so:

MyClass.prototype.print_my_x = function () {
    console.log(this.my_x);
};

This method is useful for either longer class definitions or splitting up a class across files. It comes from a slightly older syntax used to define classes in javascript. In this chapter, we’ll primarily be used the second, older, way of defining methods.

W3School has really good javascript tutorials if you want to learn more.


You can see that if we could somehow “inherit” from DefaultFS we’d get a free implementation of seek and stub methods for all callbacks that returns "EIMPL".

We can start deciding what functions we will actually be implementing. We don’t need to implement mount or umount (we already have a filesystem that implements almost nothing aside from mount and umount that will handle this for us). We can ignore ioctl, a filesystem operation that’s used to managed devices. We can use the default implementation of seek. Everything else has to be implemented by us.

We’ve provided several helper functions in /js/fs_helper.mjs and some helpful definitions in /js/defs.mjs.

To kick things off, let’s inherit the FS interface from DefaultFS and import in files that we’ll be using.

// Import definitions of various structs/constants
import {
    get_unused_ioctl_num, // function for generating ioctl numbers (we won't be using it)
    IOCTL_IS_TTY, // ioctl used by various shell utilities (we won't be using it)
    IOCTL_SELECT_INODE, // ioctl used by inodeinfo (we won't be using it)
    CONSTANTS, // Collection of constants (like permission flags)
    FileDescriptor, // Constructor for the object returned by open
    Dirent, // Constructor for the objects returned by readdir
    Stat, // Constructor for the object returned by stat
} from '/js/defs.mjs'

// Some useful filesystem-agnostic helper functions
import {
    split_parent_of,
    not_implemented,
    bytes_to_str,
    str_to_bytes,
} from '/js/fs_helper.mjs'

import {DefaultFS} from '/js/fs.mjs'

class MemFS extends DefaultFS {}

From here on out, we’ll be walking through the design/implementation of our filesystem MemFS. If you’d like to skip to the end and work on your own, scroll to the end of this section

Let’s start by designing how the file tree will be stored! Since we’re using a language like javascript, we have the ability to create objects and store references to those objects. This means that instead of implementing directories as regular files, they can instead be objects that store a list of pointers to other objects.

What do we need to store to describe a regular file (i.e. not a directory)?

  • At a minimum we need to store data. (We can use arrays of chars for easy use)
  • We’ve also learned that permissions are also useful to have.
  • Keeping track of the number of links let’s us implement hard links and unlinking
  • We want to keep track of atim, mtim, and ctim. (Make sure to initialize all time field to Date.now())

Let’s go ahead and implement that!

class File {
    constructor() {

    }
}
Solution 
class File {
    constructor() {
        this.num_links = 0;
        this.permissions = 0;
        this.data = [];
        this.atim = Date.now();
        this.mtim = Date.now();
        this.ctim = Date.now();
    }
}

Let’s move on to directories. A directory just needs to store a map of names to other Files or directories.

class Directory extends File {
    constructor() {
        super(); // Call the constructor of the parent class

    }
}
Solution 
class Directory extends File {
    constructor() {
        super(); // Call the constructor of the parent class
        this.files = {};
    }
}

Note that we make Directory inherit from File so that we can generically use the type File to describe both.

Now, we’ve layed out all the necessary components to setup the root of this filesystem. In the method below, create a property named root that points to a new, emptyDirectory. Make sure you set the permissions on the directory to 0o755 so that we actually have permissions to use the directory.

MemFS.prototype.setup_root = function () {

}
Solution 
MemFS.prototype.setup_root = function () {
    this.root = new Directory();
    this.root.permissions = 0o755;
}

Accordingly, we’ll now update the definition of MemFS to be:

class MemFS extends DefaultFS {
  constructor() {
    super(); // Call the DefaultFS constructor
    this.setup_root();
  }
}

Now, for most of the methods we’ll be working on, we’ll want to access the underlying File for a given path. Let’s make a method to find the File instance associated to a certain path. Start at the root directory and work your way down.

You may assume that all paths start with / and don’t have a trailing / at the end. You may also assume that all paths are absolute.

If the file doesn’t exist, return "ENOENT".

MemFS.prototype.find_file_from_path = function (path) {
 
};
Hints
  • When looking up “/”, return the root directory.
  • Given a string s = "/1/2/3", s.split('/') will return the array ["", "1", "2", "3"]
  • Looking up a string in a map, like this.root.files can be done like so: this.root.file["some string"]
  • If a string lookup fails, the return value is falsey - using it in a if is equivalent to using false. If it succeedes, it would return a File object, and all objects are truthy.
  • To check if a file f is a directory we can use f instanceof Directory.
Solution 
MemFS.prototype.find_file_from_path = function (path) {
    if (path == "/")
        return this.root;
    var parts = path.split("/");
    var curr_file = null;
    // remove the initial empty entry
    // Shift removes and returns the first element of the array
    parts.shift();
    curr_file = this.root;
    curr_file.atim = Date.now();

    while (parts.length) {
        if (!(curr_file instanceof Directory))
            return "ENOENT";

        curr_file = curr_file.files[parts.shift()];
        if (!curr_file)
            return "ENOENT";

        curr_file.atim = Date.now();
    }
    return curr_file;
};

This would be a good time to try writing some tests. Go back and copy all the blocks of code you’ve written so far into a text editor of your choice and save the file with the extension .mjs. You may need to do a bit of digging on how to write some barebones HTML, but try to setup an environment to hack in/test the code you’ve written so far.

Take a moment to look at fs_helper.mjs. Methods like split_parent_of will be really useful going forward!

Now, we can move on to some meatier functions, creating files and directories. Don’t forget to manage the num_links property!

MemFS.prototype.create = function (path, mode) {
 
};
Solution 
MemFS.prototype.create = function (path, mode) {
    var exists = this.find_file_from_path(path);
    if (!(typeof(exists) === 'string'))
        return "EEXISTS";

    var split = split_parent_of(path);
    var parent_dir = this.find_file_from_path(split[0]);
    if (typeof(parent_dir) === 'string')
        return parent_dir;

    if (!(parent_dir instanceof Directory))
        return "ENOTDIR";

    var newfile = new File();
    newfile.permissions = mode & 0o777;
    newfile.num_links++;
    parent_dir.files[split[1]] = newfile;
    return 0;
};

MemFS.prototype.mkdir = function (path, mode) {
 
};
Solution 
MemFS.prototype.mkdir = function (path, mode) {
    var exists = this.find_file_from_path(path);
    if (!(typeof(exists) === 'string'))
        return "EEXISTS";

    var split = split_parent_of(path);
    var parent_dir = this.find_file_from_path(split[0]);
    if (typeof(parent_dir) === 'string')
        return parent_dir;

    if (!(parent_dir instanceof Directory))
        return "ENOTDIR";

    var newdir = new Directory();
    newdir.permissions = mode & 0o777;
    newdir.num_links++;
    parent_dir.files[split[1]] = newdir;
    return 0;
};

Now we can implement readdir. readdir returns an array of Dirents (see defs.mjs). Just set inodenum to any number you want, since we don’t have any use for an inode number in this filesystem. To do so, we must keep in mind that we also need to provide the entries for . and ...

To list all elements of a map, use Object.getOwnProperty(map).

MemFS.prototype.readdir = function (path) {
 
};
Solution 
MemFS.prototype.readdir = function (path) {
    var file = this.find_file_from_path(path);
    if (typeof(file) === 'string')
        return file;

    if (!(file instanceof Directory))
        return "ENOTDIR";

    var entries = [
        new Dirent(0, '.'),
        new Dirent(0, '..')];

    for (let e of Object.getOwnPropertyNames(file.files))
        entries.push(new Dirent(0, e));

    file.atim = Date.now();
    return entries;
};

The next most trivial thing to implement would be link.

Solution

Naturally, after implementing link, we want to implement unlink. To remove an element from a map, use delete. For example, if this.root.files has an entry "newfile", delete this.root.files["newfile"] would remove the entry.

Don’t forget to decrement the num_links property.

Solution

Another easy one is chmod. All we have to do is replace the permission bits.

MemFS.prototype.chmod = function (path, permissions) {
 
};
Solution 
MemFS.prototype.chmod = function (path, permissions) {
    var file = this.find_file_from_path(path);
    if (typeof(file) === 'string')
        return "ENOENT";
    file.permissions = permissions;
    file.atim = Date.now();
    file.ctim = Date.now();
    return 0;
};

Next, let’s implement stat. To implement stat, simply return an instance of Stat from /js/defs.mjs. Stat takes in many parameters in it’s constructor. The order in which they should be passed in is specified by _stat_params.

Make sure you calculate the filesize by checking the .length property of the file’s data.

MemFS.prototype.stat = function (path) {
 
};
Solution 
MemFS.prototype.stat = function (path) {
    var file = this.find_file_from_path(path);
    if (typeof(file) === 'string')
        return "ENOENT";

    file.atim = Date.now();

    return new Stat(
        path,
        0,
        file.permissions,
        file instanceof Directory,
        file.data.length,
        file.atim,
        file.mtim,
        file.ctim);
    return 0;
};

Next up, let’s do truncate, which is extremely easy to do in javascript.

If the size passed in is greater that the length of the data, all we need to do is call .push("\u0000") on the file data repeated until it reaches the desired length.

slice can take 1 or 2 arguments; we’ll use the two argument variant. It takes a starting index and an ending index and returns a string containing the data between the start and end indicies. If the end index is past the end of the file, it does nothing.

Note that slice does not modify the original data, it just returns a new array.

Note that if the file does not exist, you don’t need to create it here.

MemFS.prototype.truncate = function (path, size) {
 
};
Solution 
MemFS.prototype.truncate = function (path, size) {
    var file = this.find_file_from_path(path);
    if (typeof(file) === 'string')
        return "ENOENT";

    while (file.data.length < size)
      file.data.push("\u0000");
    file.data = file.data.slice(0, size);
    return 0;
};

Now that we’ve gotten out feet wet with modifying data, how about implementing read and write? To that, we first need to implement open and close.

We’ll ignore implementing close and rely on javascript’s “garbage collection” to get rid of the file descriptors for us.

open returns a FileDescriptor as defined in /js/defs.mjs. While we don’t need a real value for inodenum, set inode to be the File (or Directory) corresponding to the provided path. Make sure to set fs parameter to this. Set the mode parameter to be flags.

note that the flags parameters will be a set of flags from /js/defs.mjs that start with O_ and are or‘d together. These flags are present as members of the CONSTANTS class. To find out if a particular flag, se CONSTANTS.O_APPEND is present, check the output of (flags & CONSTANTS.O_APPEND). If CONSTANTS.O_CREAT is passed in check if mode is truthy, if not, fail and return "EINVAL", if it is, call create with path and mode (make sure to ignore the error ‘EEXISTS’).

If CONSTANTS.O_APPEND is passed in, set the offset in the file descriptor to be equal to the size of the file.

If CONSTANTS.O_WRONLY is passed in, check if the file has write permissions set. If not, return "EPERM".

If CONSTANTS.O_RDONLY is passed in, check if the file has read permissions set. If not, return "EPERM".

If CONSTANTS.O_TRUNC is passed in and CONSTANTS.O_WRONLY is passed in truncate the file to size 0 first. If CONSTANTS.O_WRONLY is not passed in, return "EINVAL".

If CONSTANTS.O_DIRECTORY is passed in and the file is not a directory, return “EINVAL”.

MemFS.prototype.open = function (path, flags, mode) {
 
};
Solution 
MemFS.prototype.open = function (path, flags, mode) {
    if (flags & CONSTANTS.O_CREAT) {
        if (!mode)
            return "EINVAL";
        var error = this.create(path, mode);
        if ((typeof(error) === 'string') && (error !== 'EEXISTS'))
            return error;
    }

    var file = this.find_file_from_path(path);
    if (typeof(file) === 'string')
        return file;

    if (flags & CONSTANTS.O_TRUNC) {
        if (!(flags & CONSTANTS.O_WRONLY))
            return "EINVAL";

        var error = this.truncate(path, 0);
        if (typeof(error) === 'string')
            return error;
    }

    if ((flags & CONSTANTS.O_DIRECTORY) && !(file instanceof Directory))
        return "ENOTDIR";

    if ((flags & CONSTANTS.O_WRONLY)  && !(file.permissions & 0o200))
        return "EPERM";

    if ((flags & CONSTANTS.O_RDONLY)  && !(file.permissions & 0o400))
        return "EPERM";


    var fd = new FileDescriptor(this, path, 0, file, flags & CONSTANTS.O_RDWR);
    if (flags & CONSTANTS.O_APPEND)
        fd.offset = file.data.length;

    return fd;
};

Now, on to read. read takes in two parameters, a file descriptor and a Uint8Array. Our goal is to fill the Uint8Array with as much data as possible and return the number of bytes we actually filled in. Note that you can access the underlying File for a FileDescriptor via the .inode property.

To populate the array, try a for-loop and use the charCodeAt method of strings to get the byte values of chars in the data field. (e.g. "a".charCodeAt(0) will give us the ascii value of the character a). Make sure you start returning bytes in the data array from the offset specified in the FileDescriptor passed in, and that you update that file descriptor by the number of bytes read at the end.

If a read asks you to read past the end of the file, stop at the end of the file, and return the number of bytes read so far. As a specific case, if the offset is greater than or equal to the size of the file, return 0.

You can check the length of the buffer by reading it’s .length property.

MemFS.prototype.read = function (fd, buffer) {
 
};
Solution 
MemFS.prototype.read = function (fd, buffer) {
    if (fd.offset >= fd.inode.data.length)
        return 0;

    var bytes_read = 0;
    for (var i = 0; i < buffer.length; i++) {
        if ((fd.offset + i) >= fd.inode.data.length) {
            // we've reached the end of the file
            break;
        }

        buffer[i] = fd.inode.data[fd.offset + i].charCodeAt(0); 
        bytes_read++;
    }
    fd.offset += bytes_read; 
    return bytes_read;
};

The easiest way to implement write would be to first call truncate on the file to ensure we have enough space, then write a similar for loop to the one above, but instead copying data from the buffer to the file. To convert a byte to a character, use String.fromCharCode.

To call truncate remember that truncate takes in a path, and we can use fd.path to get the path from a FileDescriptor.

Remember to take the offset into account when calculating the final filesize of the file. Also note that we don’t want to shrink the file while writing (i.e. we should be able to only change a few bytes in the middle of the file without disturbing the rest of the file).

MemFS.prototype.write = function (fd, buffer) {
 
};
Solution 
MemFS.prototype.write = function (fd, buffer) {
    this.truncate(fd.path, Math.max(fd.inode.data.length, fd.offset + buffer.length));

    var bytes_written = 0;
    for (var i = 0; i < buffer.length; i++) {
        fd.inode.data[fd.offset + i] = String.fromCharCode(buffer[i]); 
        bytes_written++;
    }
    fd.offset += bytes_written; 
    return bytes_written;
};

MemFS