Further testing of File::LockDir and starting on PageArchive

I spent some time in this last set of work cleaning up some more stuff in PageArchive and File::LockDir.

First, I renamed the PageArchive::RCS class to PageArchive::Sequential, because it really doesn’t share much with RCS other than a file naming convention (name,version  to denote file version). It actually just stores each version of the file; no deltas, no branches, just a linear set of files for each version as it’s saved. 

This has the advantage of being so stupid that it’s hard to get wrong, but the disadvantage that there’s a lot of duplicated data. Casting my mind back to 2000, I think that the decision was “we want a wiki running as soon as possible, and we don’t think it’ll be that big. Let’s just get this going.” I think that we did eventually run into space constraints, given that the delete() and purge() functions are in there for cleaning up old versions. For now, we’ll leave this as is, but it definitely needs addressing down the line.

I started off with a tidy of the class, and committing that by itself to make it easier to track actual changes vs. whitespace fiddling.

Second, I came back to fatal() and note(); these were way more complicated than they needed to be. I was originally wrapping up the callbacks so that they’d look like native methods, and cute as that was, it was actually less straightforward than simply returning a sub ref, and then just using that directly:

    $archive->fatal->(“this is our message”)

I went though the code I had previously and cleaned it all up to match this spec instead. In addition, the default fatal() callback now uses carp(), so we see the error in the proper context instead of down inside the class where the error was found, but not where it happened.

The setError and getError method names weren’t consistent, so I renamed them to set_error() and get_error().

The dirhandle that the PageArchive class needs was being referred to via a direct lookup in the object hash, which is a code smell. Wrapped this up in a method too. Some other code was also factored out into methods, notably _dirname (setter/getter for the directory we’re using for the archive) and _page_in_archive (co-locating all of the “synthesize a name for a file in the archive from the object, a name, and a version). dh_reset was renamed rewind(), because it’s essentially a rewinddir() call. We retained the method to allow Win32 to close and reopen the dirhandle, as that’s the only way Win32 could do that operation in 2000. (It may have gotten better, but I don’t run Win32.)

The big change at this point was committing to Test2 for all my testing (thanks, Chad!). It in particular made testing multi-process locking far easier to do. I still have to fork and wait (a little), but it’s much less work than before, and I can do tests in both the parent and child processes.

PageArchive::Sequential was further reorganized to make the methods peculiar to it private; this gets me further toward creating a PageArchive interface — or perhaps an abstract base class — to make swapping PageArchive engines easier.

I’d been away for a bit, so I added the Dist::Zilla Cover class to allow me to easily run Devel::Cover and see how good my test coverage was. Not great; File::LockDir was only 50% covered, so I decided to focus on getting that closer to covered before doing anything else. nlock_state wasn’t tested at all yet, and I was definitely going to need that for PageArchive::Sequential, so I wrote basic tests to verify that it worked. Discovered a few bugs, most notably that I wasn’t using the carefully-maintained locking cache to prevent having to read the directory each time.

Now I was ready to start in on PageArchive::Sequential. The basic new() test was already done, but it was time to start testing the actual page manipulation functions. I started with put() and page_exists(), as those two are kind of the most basic. In the process, I realized that I hadn’t properly tested the “there’s no such directory”, “this is a file, not a directory”, and “I can’t write to this” cases. Added tests for those.

The put() function was very straightforward, since Storable was pretty much handling all of it. I just needed to make sure that the files I expected did end up in the archive directory, and that was easy with -e and PageArchive’s page name generator method.

I fixed my first TODO: remove the path crawl down the supplied path and change that over to make_path(). That simplified new() significantly.

I now started to get into testing File::LockDir in context, and there were some rough edges here. I think most of them were introduced by swapping File::LockDir’s implementation to be an object, but there were a few issues that were there before that I either never saw, or lucked out along the happy path and didn’t hit. 

One interesting aside: I had added Try::Tiny to PageArchive::Sequential…but I added it right after the use strict and use warnings, and not inside the package! Fortunately Perl::Critic pointed this out for me; the code otherwise would silently have executed both blocks…

While running the tests, I saw a lot more tracing output in the log (printed to STDERR), and wrapped up more of it in if $self->debug. This entailed adding debug() to PageArchive::Sequential, as it didn’t have it. 

I had used “lockee” to refer to the locking user, and besides being an esoteric word, it was also wrong. Changed this to locking_user to be accurate. In addition, while doing this cleanup, the tests showed me that I wasn’t being consistent in what was returned for locked vs. not locked; I had to reorder the code slightly and return different values to ensure that was working right. I also noticed that the owner file inside the locking dir was generated via concatenation and a magic string, so I moved that to a method and switched it to File::Spec->catdir.

I added extra checks to the locking tests in PageArchive::Sequential to verify that locking one version marked all versions locked; this was because I wanted to prevent someone fiddling with the “commit” history while someone else has the file locked. I verified it was still possible to read the old versions via get() while the page was locked — this is desirable as someone may want to look at the page or older versions while someone else is editing.

In the process, I found that max_version was throwing some weird errors, though it seemed to be working. I tracked this down to the readdir of the archive directory quite correctly returning the locking directory names as well as the page file names. Adding a grep to eliminate them cleaned that up. In addition, I rewrote the code that took the filenames and found the max version to be a full-up Schwartzian transform, so the code now looks like this:

    my @pages = grep { /,\d+$/ } $self->page_exists($name);
    return unless @pages;

     my @indexes = map  { $_->[1] }
                  sort { $b->[1] <=> $a->[1] }
                  map  { /^.*,(.*)$/; [$_, $1] }
                  @pages;

     return shift @indexes;

We take the filtered page list, extract the version with a pattern match, create an array of [$name, $version], sort that descending numerically, and then shift it to return the highest number. Note also that we catch the “there aren’t any versions of this” case as well.

Last, I tested page gets: can I get an arbitrary version? If I specify no version, do I get the most recent one? Do gets still work when the page is locked? If I specify a bad version, do I get the correct error? 

Then it was tests for delete() and purge:

• delete of a nonexistent version fails
• delete with no version deletes newest
• delete with explicit version deletes only that version
• purge of a nonexistent page works (silently)
• purge of an existing page deletes all versions

There’s an inconsistency between delete and purge; I think I’ll probably need to go more in-depth into App::WebWebXNG to figure out if this was purposeful, or just a brain fart.

Last test was for iterator(), which returns the highest revision of every file in the archive. (This would be trivial with a more modern SCM!) I did find another $^W = 0, which is a code smell…and no longer blocks the undef error I get in this code anyway. I removed it, changed the code that builds the hash recording the max version of each file so it does an exists check first, and everything now works.

That’s the breakpoint for today. File::LockDir is 96% covered, and PageArchive::Sequential is 97% covered; taking a quick look at the code, everything left mostly seems to be difficult-to-test error cases, so I’ll move on to App::WebWebXNG in the next “sprint” and see if anything bites me.