Category: Uncategorized

  • More adventures in metadata

    Despite the last set of changes, I still had problems with the iOS app losing its connection to the Azuracast websocket with no way for the code to easily see that had happened, so I dove into the code again, looking for alternatives. I think I’ve got a good solution.

    I’ve added Reachability to the websocket monitor; if I detect a network disconnect, then I force the websocket monitor to disconnect as well so that it is in a known state. When Reachability gets a reconnection signal

  • Flutter experiences

    TL;DR: Flutter builds are as much fun as Java and Scala ones, and you spend more time screwing with the tools than you do getting anything done. I don’t think I’m going to switch, at least not now.

    As I’ve mentioned before on the blog, I maintain an iOS application for RadioSpiral’s online radio station. The app has worked well and successfully; the original codebase was Swift- Radio-Pro, which works as an iOS app and a MacOS one as well (I have been doing some infrastructure changes to support Azuracast, as previously documented on the blog.)

    We do have several, very polite, Android users who inquire from time to time if I’ve ported the radio station app to Android yet, and I have had to keep saying no, as the work to duplicate the app on Android looked daunting, and nobody is paying me for this. So I’ve been putting it off, knowing that I would have to learn something that runs on Android sooner or later if I wanted to do it at all.

    Randal Schwartz has been telling me for more than a year that I really should look at Dart and Flutter if I want to maintain something that works the same on both platforms, and I just didn’t have the spare time to learn it.

    Come the end of May 2023, and I found myself laid off, so I really had nothing but time. And I was going to need to update the app for IOS 16 anyway at that point (the last time I recompiled it, Xcode still accepted iOS 8 as a target!) and I figured now was as good a time as any to see if I could get it working multi-platform.

    I started looking around for a sample Flutter radio app, and found RadioSai. From the README, it basically does what I want, but has a bunch of other features that I don’t. I figured an app I could strip down was at least a reasonable place to start, so I checked it out of Github and started to work.

    Gearing up

    Setting up the infrastructure Installing Dart and Flutter was pretty easy: good old Homebrew let me brew install flutter to get those in place, and per instructions, I ran flutter doctor to check my installation. It let me know that I was missing the Android toolchain (no surprise there, since I hadn’t installed
    anything there yet). I downloaded the current Android Studio (Flamingo in my case), opened the .dmg, and copied it into /Applications as directed.

    Rerunning flutter doctor, it now told me that I didn’t have the most recent version of the command-line tools. I then fell into a bit of a rabbit hole. Some quick Googling told me that the command line tools should live inside Android Studio. I ferreted around in the application bundle and they were just Not There. I went back to the Android Studio site and downloaded them, and spent a fair amount of time trying to get sdkmanager into my PATH correctly. When I finally did, it cheerfully informed me that I had no Java SDK. So off to the OpenJDK site, and download JDK 20. (I tried a direct install via brew install, but strangely Java was still /usr/bin/java, and I decided rather than tracking down where the Homebrew Java went, I’d install my own where l could keep an eye on it.

    I downloaded the bin.tar.gz file and followed the installation instructions, adding the specified path to my PATH… and still didn’t have a working Java. Hm. Looking in the OpenJDK directory, the path was Contents, not jdk-18.0.1.jdk/Contents. I created the jdk-18.0.1 directory, moved Contents into it and had a working Java! Hurray! But even with dorking around further with the PATH, I still couldn’t get sdkmanager to update the command-line
    tools properly.

    Not that way, this way

    A little more Googling turned up this Stack Overflow post that told me to forget about installing the command-line tools myself, and to get Android Studio to do it. Following those instructions and checking all the right boxes, flutter doctor told me I had the command-line tools, but that I needed to accept some licenses. I ran the command to do that, and finally I had a working Flutter install!


    Almost.

    When I launched Android Studio and loaded my project, it failed with flutter.sdk not defined. This turned out to mean that I needed to add

    flutter.sdk=/opt/homebrew/Caskroom/flutter/ 3.10.5/flutter

    (the location that Homebrew had used to unpack Flutter — thank you find) to local.properties. After that, Gradle twiddled its fingers a while, and declared that the app was ready. (It did want to upgrade the build, and I let it do that.)

    Build, and…

    The option 'android.enableR8' is deprecated. 
    It was removed in version 7.0 of the
    Android Gradle plugin. 
    Please remove it from 'gradle.properties". 

    Okay, I remove it.

    /Users/joemcmahon/Code/radiosai/.dart_tool/ does not exist.

    More Googling, Stack Overflow says Run Tools > Flutter > Pub Get. Doesn’t exist. Okaaaaaay.

    There’s a command line version:

    flutter clean; flutter pub get

    Deleted dart_tool, then recreated it with package_config.json there. Right!

    Back to Android Studio, still confused about the missing menu entry, and build again. Gradle runs, downloads a ton of POMs and

    Couldn't resolve the package 'radiosai' in 'package:radiosai/audio_service/service_locator.dart'.

    Looking one level up, in :app:compileFlutterBuildDebug, Invalid depfile: /Users/joemcmahon/ Code/radiosai/.dart_tool/flutter_build/bff84666834b820d28a58a702f2c8321/ kernel_snapshot.d.

    Let’s delete those and see if that helps…yes, but still can’t resolve
    radiosai. Okay, time for a break.

    Finally, a build!

    Another Google: I wasn’t able to resolve the package because I needed to pub get again.

    Module was compiled with an incompatible version of Kotlin. 

    The binary version of its metadata is 1.8.0, expected version is 1.6.0. Another Google. One of the build Gradle files is specifying Kotlin 1.6…it’s in /android/ build.gradle. Update that to 1.8.10, build…Kotlin plugin is being loaded, good. Couple
    warnings, still going, good.

    BUILD SUCCESSFUL

    Nice! Now, how do I test this thing? Well, there’s Device Manager over on the right, that looks promising. There’s a “Pixel 3a” entry and a “run” button. What’s the worst that could happen?

    Starts up, I have a “running device” that’s a couple inches tall, on its home screen. Hm. Ah, float AND zoom. Cool. Now I realize I have no idea how to run an Android phone, and I don’t see the app.

    https://developer.android.com/studio/run/emulator…nope. Beginning to remember why I didn’t like working in Scala… Gradle upgrade recommended, okay, and now

    Namespace not specified. Please specify a namespace in the module's build.gradle. 

    Specified, still broken…googling…This is a known issue –
    https://github.com/ionic-team/capacitor/issues/6504

    If you are using Capacitor 4, do not upgrade to Gradle 8.


    Yeah, I remember why I stopped liking Scala. git reset to put everything back…

    Execution failed for task:gallery_saver:compileDebugKotlin'. 
    > compileDebugJavaWithJavac task (current target is 1.8) and 'compileDebugKotlin' task
    (current target is 17) 
    jvm target compatibility should be set to the same Java version.
    Consider using JVM toolchain: https://kotl.in/gradle/jvm/toolchain 

    Fix android/app/build.gradle so everyone thinks we’re using Java 17, which uses a different syntax, ugh.

    Fix it again. Same for the Kotlin target too.

    'compileDebugJavaWithJavac' task (current target is 1.8) and 'compileDebugKotlin' task (current target is 17) jvm target compatibility should be set to the same Java version.

    This is apparently actually Gradle 8 still lying around after the (incorrectly) recommended upgrade. Removing ~/ gradle to nuke from orbit. Also killing android/.gradle.


    [Aside: I am used to using git grep to find things, and it is just not finding them in this repo!]

    Cannot read the array length because "" is null

    WHAT.

    Apparently this means that Gradle 8 is still lurking. Yep, the rm ~/.gradle/* didn’t remove everything because of permissions. Yougoddabefuckingkiddingme. Sudo’ed it, relaunched with the fixes I made above. App runs!


    However it stops working after a bit with no reason indicating why. Let’s stop it and restart. Stop button did not stop it; had to quit Android Studio.

    Well. Okay. This is not promising, but let’s see the benefit of using Flutter and check out if the iOS side works. Seems a lot more straightforward, though I’m not doing much in Xcode. cd iOS, launch the simulator (important!), flutter run…and we get the Flutter demo project. Looks like the IOS version wasn’t brought over from the Android side. Why did you even do this.

    Do we all remember that I wanted something that worked on both platforms? Gah.

    So I’m putting Flutter aside, cleaning up the ton of disk space all this extra infrastructure took up, and will maybe come back to it another time.

    But for right now, the amount of work involved is absolutely not worth it, and I’d have to write the damn thing from scratch anyway.

    Maybe I’ll run this through one of the LLMs and see if it can get me a common codebase as a starting point, but I am not sanguine.

  • So long, pemungkah@me.com

    That email address is now officially defunct.

    I created it back when I bought my iPhone 5.

    Years ago, it got leaked, and it has since been used for everything from someone in Canada’s VISA card to a bank account in Vietnam to some bozo’s Marriott account. (Hey David: ppppppppbt.)

    It got so bad that when Apple opened up creating Apple IDs with your own email, I did that, and essentially abandoned the me.com address.

    I used it for political mail for a while, but I’ve gotten disillusioned that letting every random person running for office send me begging letters does any good. (They’re never “I did this because that’s what you sent me to Congress to do”, but “MY OPPONENT HAS MONEY! SEND ME MORE!” — and most of the time it’s futile anyway.) Mostly it was spam, people using it as a test email (especially screw you people: use MailHog or something. How are you going to know if the mail you’re sending looks right?)

    I closed it today. Only had a couple things still left from the downloads I used it for, and I can reinstall them from my primary if I want them.

    A grand experiment, but I’m not sad to never have to deal with it again.

  • OCLP experience update: back to Ventura

    I’ve been running OCLP (the Open Core Legacy Patcher) on my 2012 MacBook Pro; recently I ran softwareupdate from the command line and accidentally upgraded to Sonoma from Ventura. The experience was definitely mixed.

    It handled it mostly okay for day-to-day work. Xcode 15.4 ran fine. Where I hit a problem, though, was when I tried running Azuracast under Docker. The machine ran insanely hot, so hot that it started throwing screen glitches. Rather than burn out my GPU, I elected to downgrade to Ventura. Here’s how that went. (Spoiler: a lot of toil.)

    Getting Ventura back

    The first step was to get Ventura back on the machine, This wasn’t particularly hard; I just needed to follow the standard OCLP procedure, but install to a new partition on my internal SSD. This cut the amount of space down by about another 200GB, but went well. I was able to install and have Ventura in good shape in a couple hours.

    Retrieving the data from Sonoma

    Here’s where we started having problems.

    I had hoped that I’d be able to use Migration Assistant to bring the data back from Sonoma to Ventura, but no dice. Migration Assistant looked at the Sonoma disk, said, “nuh-uh, I ain’t downgrading” and refused to even consider mounting the disk. This meant I’d have to port everything back from that disk to the new one manually.

    My first try was to rsync it over. This failed because now I didn’t have enough space to have two copies of the data. I deleted the data from the Ventura install and tried again. This time I created a service account with admin privileges, and copied ~/Library over from Sonoma. This didn’t seem to work either; most particularly iCloud login was broken.

    Fixing the broken copy

    After thinking about it a while, I decided that the problem was probably permissions. From the service account, I wiped the Ventura copy of my account again, and copied in two steps. First, I copied ~/Library over, then chown‘ed it to my user on the Ventura disk. I logged in as myself, set up iCloud, and all was good. Now came the question of moving the data without filling the disk.

    I was able to use rsync (from the service account again), but this time I added --remove-source-files and --ignore-existing to the command. This copied only files I didn’t already have on Ventura from Sonoma, deleting them as they transferred. After this finished, I logged in to my Ventura account, iCloud was okay, and all my files were back.

    I then rebooted into the installer again, removed the Sonoma partitions, and was ready to go.

    I’m now currently running Azuracast under Docker, and having it ingest the RadioSpiral tracks from my iTunes library. It’s running warm, but not hot, and no more screen glitching. I’ll probably leave it on Ventura unless someone else running the same machine gets good performance from Sequoia.

    And I can always run Linux if all else fails.

  • Leveraging an outage to build community and consensus

    We had our first extended outage at RadioSpiral this weekend, and I’m writing about it here to point out how a production incident can help bring a team together not only technically, but as a group.

    The timeline

    On Sunday evening, about an hour before Tony Gerber’s Sunday show, RadioSpiral went offline. Normally, the AirTime installation handles playing tracks from the station library when there’s no show, and it played a track…and then stopped. Dead air.

    The station has been growing; we’ve added two new DJs, doubling the number of folks who are familiar with servers, Linux, etc. Everyone who was available (pretty much everyone but our primary sysadmin, who set everything up and who is in the UK) jumped in to try to see what was up. We were able to log in to AirTime and see that it was offline, but not why; we tried restarting the streaming service, and the server itself, but couldn’t get back online.

    We did figure out that we could connect to the master streaming port so that Tony could do his show, but after that, we were off the air for almost 12 hours, until our primary sysadmin was up, awake, and had finished his work day.

    A couple hours of investigation on his part did finally determine that LetsEncrypt had added a RewriteRule to the Airtime configuration that forced all URLs to HTTPS; unfortunately it needs HTTP for its internal APIs and that switchover broke it. Removing the rule and restarting the server got us back on line, and our very patient and faithful listeners trickled back in over the day.

    Now what?

    While we’d not been able to diagnose and fix the problem, we had been chatting in the staff channel on the RadioSpiral Discord server, and considering the larger issues.

    RadioSpiral is expected to be up 24/7, but we’re really running it more like a hobby than a business. This is reasonable, because it’s definitely not making any of us money, at least not directly. (Things like sales of albums by our DJs, etc., are their business and not part of the station’s remit.) This means that we can have situations like this one, where the station could be offline for an extended amount of time without recourse.

    Secondarily, RadioSpiral is small. We have three folks who are the core of actual station operations, and their contributions are very much siloed. If something should happen to any one of the three of us, it would currently be a scramble to replace them and could possibly end up with an extended loss of that function, whether broadcast operations, the website, or community outreach and the app.

    So we started looking at this situation, and figuring out who currently owned what, and how we could start fixing the single points of failure:

    • Station operations are on an ancient Linux release
    • We’re running an unsupported and unmaintained version of Airtime. It can’t even properly reset passwords, a major problem in an outage if someone can’t get in.
    • The MacOS/iOS app is handled by one developer; if that person becomes unavailable, the app could end up deleted from the store if it’s not maintained.
    • The website is being managed by one person, and that person becomes unavailable…well, the site will probably be fine until the next time the hosting bill isn’t paid, but if there were any issues, we’d be SOL.
    • We do have documentation, but we don’t have playbooks or process for problem solving.
    • We don’t have anywhere that is a gathering point when there’s a problem.
    • We don’t have project tracking so we can know who’s doing what, who their backup or backups are, and where things are in process.
    • We don’t have an easily-maintained central repository of documentation.

    What we’re doing

    I took point on starting to get this all organized. Fixing all of the things above is going to take time and some sustained effort to accomplish, and we’re going to want to make sure that we have everything properly set up so that we minimize the number of failure points. Having everyone onboard is critical.

    • We’re going to move operations to a newer, faster, and cheaper server running a current LTS Ubuntu.
    • We’re going to upgrade from the old unsupported AirTime to the community-supported LibreTime.
    • We’re figuring out who could get up to speed on MacOS/iOS development and be ready to take over the app if something should happen that I couldn’t continue maintaining it. At the moment, we’re looking at setting up a process to bump the build number, rebuild with the most current Xcode, and re-release every six months or so to keep the app refreshed. Long-term we’ll need a second developer (at least) who can build and release the app, and hopefully maintain it.
    • We haven’t yet discussed what to do about the website; it is already a managed WordPress installation, so it should be possible to add one or more additional maintainers.
    • We are going to need to collect the docs we have somewhere that they can be maintained more easily. This could be in a shared set of Google docs, or a wiki; we’re currently leaning toward a wiki.
    • We need project tracking; there’s no need for a full-up ticketing process, at least yet. We think that Trello should do well enough for us.

    We have set up some new Discord channels to keep this conversation open: #production-incidents, to make tracking any new problems easier, and #the-great-migration, to keep channels open as we move forward in the migration to our new setup.

    Everyone is on board and enthusiastic about getting things in better shape, which is the best one could want. It looks good for RadioSpiral’s future. Admittedly we should have done this before a failure, but we’re getting it in gear, and that’s better than ignoring it!

  • Archiving papers: a strategy

    I’m helping a friend archive a lot of notebooks and papers that they’ve accumulated over several years of writing. They’d like to be able to travel, but are a little worried that not having any backup for all this work is risky; fires, floods, and theft do happen, so even a fireproof box isn’t a guaranteed backup.

    We’ve therefore been photographing the papers, page by page, and creating a 3-2-1 backup of all of the digital photos. After some experimentation, we’ve come up with a workflow that works very well:

    • Create a Photos library that is not the primary. (She has an art business and needs to be able to use her iCloud-synced Photos library without it getting cluttered up with hundreds of photographs of pages.) This is most easily done by holding down Option and launching Photos. When the “select the library” dialog comes up, create a new one.
    • Photograph the items on a second iCloud account’s primary Photos library. This automatically syncs them to that accounts iCloud Photos.
    • On the machine where the secondary Photos library lives, log into iCloud.com with the second account.
    • On that same machine, open Photos with the non-primary library. (Hold down the option key and open Photos to allow Photos to select the non-primary Photos library.)
    • As batches of photos are taken, wait for them to sync to iCloud, then on the iCloud.com page for the second account, download the batch to the machine where the secondary library lives.
    • Create a new album in that secondary library, and drag the new batch of photos into it.
    • Put a sticker on the notebook/folder, and write in an ID (A, B, C, etc.) and the date it was photographed last. This allows active notebooks to be archived safely. (You should also add a note on the last page scanned with the date and album ID so you can cross-check.)

    Photographing the cover of the notebook/the file folder the pages are in helps make sure that you keep different batches of photos separate. If you do this, it’s much easier to keep track of which pages belong in which album, and gives a better way to track back which things are done and which aren’t.

  • leetcode day 30 – I am not friends with slices: level-order tree

    Visit the nodes of a binary tree in level order: all nodes at the root level, left-to-right, all nodes at level 1, left-to-right, etc.

    I did try to solve this with slices and boy did I have trouble. I eventually stopped trying to use them and went to a list-oriented solution with a double closure.

    /**
     * Definition for a binary tree node.
     * type TreeNode struct {
     *     Val int
     *     Left *TreeNode
     *     Right *TreeNode
     * }
     */
    
    type MyList struct {
        head *MyNode
        tail *MyNode
    }
    
    type MyNode struct {
        Val int
        Next *MyNode
    }
    
    func levelOrder(root *TreeNode) [][]int {
        // Ideally, I want to end up with an array
        // of arrays of nodes, added in left-to-right order.
        getStash, helper := createClosures()
        helper(root, 0)
        stash := getStash()
    
        out := [][]int{}
        for i := 0; i < len(stash); i++ {
            out = append(out, []int{})
            thisList := stash[i]
            for scan := thisList.head; scan != nil; scan = scan.Next {
                out[i] = append(out[i], scan.Val)
            }
        }
        return out
    }
    
    func createClosures() (func() []MyList, func(*TreeNode, int)) {
        stash := []MyList{}
    
        var helper func(*TreeNode, int)
        helper = func(root *TreeNode, level int) {
                if root == nil {
                    // Nothing to do at this level
                    return
                }
                // Current node gets stashed at the end of the list for this level.
                // (*output)[level] is the slice to append to.
                // Add new node to list at this level
                if len(stash) <= level {
                    stash = append(stash, MyList{})
                }
                
                n := &MyNode{Val: root.Val}
                if stash[level].head == nil {
                    stash[level].head = n
                    stash[level].tail = n
                } else {
                    stash[level].tail.Next = n
                    stash[level].tail = n
                }
     
                // add the left and right subtrees at the next level down
                helper(root.Left, level + 1)
                helper(root.Right, level + 1)
            }
        return func() []MyList { return stash }, helper
    }

    This solves the problem, but it’s poor on memory (8th percentile) and runtime (26%). On the other hand, the list manipulation worked correctly the first time. Let’s try to speed it up. First let’s try to remove the lists.

    /**
     * Definition for a binary tree node.
     * type TreeNode struct {
     *     Val int
     *     Left *TreeNode
     *     Right *TreeNode
     * }
     */
    
    func levelOrder(root *TreeNode) [][]int {
        // Ideally, I want to end up with an array
        // of arrays of nodes, added in left-to-right order.
        getStash, helper := createClosures()
        helper(root, 0)
        return getStash()
    }
    
    func createClosures() (func() [][]int, func(*TreeNode, int)) {
        // This will be our output structure. As we go down the
        // levels, we add more []int slices to hold the next level
        // of nodes. Because we work left to right, we always add
        // new nodes at a level to the correct slice, and they always
        // are added to the right end of the slice, giving us the
        // output data structure we want.
        stash := [][]int{}
    
        // We have to declare a named variable to be closed over for
        // a Go anonymous function to be able to call itself.
        var helper func(*TreeNode, int)
    
        // The real meat of the process.
        helper = func(root *TreeNode, level int) {
                if root == nil {
                    // Nothing to do at this level
                    return
                }
                // Current node gets stashed at the end of the slice
                //  for this level.
                // stash[level] is the slice to append to.
                if len(stash) <= level {
                    // We have never accessed this level. Add a slice
                    // so appends will work. You CANNOT just assign to
                    // stash[level], because it doesn't exist yet and
                    // you'll get an out of bounds error.
                    stash = append(stash, []int{})
                }
                // Add this node's value to the end of the list at this
                // level.
                stash[level] = append(stash[level], root.Val)
    
                // add the left and right subtrees at the next level down.
                // Because we're tracking the level, we always append to the
                // right slice in the stash, which lives outside the recursion.
                helper(root.Left, level + 1)
                helper(root.Right, level + 1)
            }
        // The two functions the main function needs to call. Stash will be
        // exactly the needed data structure when helper finishes.
        return func() [][]int { return stash }, helper
    }

    That’s a significant improvement; 71st percentile runtime, 24th percentile memory. The problem I had the first time around was not properly managing the slices. To add an element to a slice after the current end, you must use append(), and I needed to do this for each level as I got there. The code to build the output structure from the lists was the learning experience I needed to get that part right in the rewrite.

    Still definitely more comfortable manipulating data structures. Let’s try removing the closure and see how that goes; we’ll just let scoping make the stash visible to the helper.

    /**
     * Definition for a binary tree node.
     * type TreeNode struct {
     *     Val int
     *     Left *TreeNode
     *     Right *TreeNode
     * }
     */
    
    func levelOrder(root *TreeNode) [][]int {
        // Ideally, I want to end up with an array
        // of arrays of nodes, added in left-to-right order.
        stash := [][]int{}
    
    var helper func(*TreeNode, int)
        helper = func(root *TreeNode, level int) {
                if root == nil {
                    // Nothing to do at this level
                    return
                }
                // Current node gets stashed at the end of the list for this level.
                // stash[level] is the slice to append to.
                // Add new node to list at this level
                if len(stash) <= level {
                    stash = append(stash, []int{})
                }
                stash[level] = append(stash[level], root.Val)
    
                // add the left and right subtrees at the next level down
                helper(root.Left, level + 1)
                helper(root.Right, level + 1)
            }
    
        helper(root, 0)
        return stash
    }

    Interestingly, this is actually worse according to leetcode; 68th percentile now (3ms instead of 1ms). Honestly I think we’re in the noise at this point. No change in the memory usage.

    I think if I’d gotten this in an interview I’d call the second or third solution good, and the first marginal. The rewrite time from the list version to the working slice version was only a couple minutes, and I never felt like I didn’t grasp the necessities of the problem.

  • leetcode month, day 2: Fenceposted by “valid parens”

    Today’s challenge is Valid parens. And it reminded me of several of the things that I don’t particularly like in Go.

    Problem statement boils down to “here’s a string of brackets: parens, square brackets, braces. Tell us if they match properly (true) or don’t (false)”. Algorthmically this is dead simple:

    Start with an empty stack.
    for each character in the string:
      if it's an open bracket, push it.
      if its a close bracket:
        if the stack is empty, return false.
        if the top of the stack isn't the matching open, return false.
        pop the stack.
    
    When we're done, return true if the stack is empty, false otherwise.

    In Perl or Python, I’d have a stack pointer, and move it up and down an array as I pushed and popped characters by altering the stack pointer and assigning to the appropriate point in the array. If the stack pointer went below 0, I’ve gotten too many close parens, and I return false.

    This approach doesn’t work well at all with Go, because Go arrays and slices Do Not Work Like That.

    To push onto an existing array, we need to use a := append(a, new). Just assigning to an index outside of the slice’s current boundaries will panic.

    We could still use the stack pointer to move backward, but then we’d have to have more complex code to decide if we need to append or just move the pointer. (Side note: doing it that way would probably use less memory — the working version of my solution used more memory than 90% of the other solutions). Instead, we just use the slice notation to pop the stack instead, with a := a[:len(a)-1].

    My original iterations with a real stack pointer failed miserably because I wasn’t properly tracking the length of the array, and was perpetually getting the stack pointer position wrong, causing the code to panic. It was only after completely discarding the stack pointer that I got it to finally work.

    Items of note:

    • I remembered I’d need to use range() to iterate over the string, but forgot that I would have to string() the characters I was getting so they were strings instead of runes. I wasted a good chunk of time trying to mess with runes before I realized that I needed string() instead. Runes are considerably more second-class citizens.
    • Still running afoul of using make() properly. Finally figured out the syntax to create an empty string array, but it took some Googling to get it.
    • I decided to make the code less complex by creating a pairs map that mapped left brackets to right ones. This meant I could check pairs[left] == right for whether I’d matched the left and right brackets. It also meant that if we ever added more bracket pairs in the future, it’d be be easier to implement them.
    • I got fenceposted by a[len(a)-1] accessing the last element, and a[:len([a)-1] dropping the last element. I naively expected that since a[len(a)-1] accessed the last element, I’d want a[:len(a)-2] to drop that last element, but that takes one too many, running me into another fencepost. Yes, it’s good that it’s symmetric, and now I’ll remember, but I definitely had to look it up to figure out both of them.
    • Forgot the check for “did we close all the opens”. I probably would not have missed it if I was making my own test cases, but it wasn’t in the tests. Note to self: watch out for poor initial test cases. There is an opportunity to add more, I think?

    So how’d I do overall? We saw I was in the bottom 10% in memory usage, boo, but runtime? “Beats 100.00% of users with Go”.

    I’ll definitely take that as a win, but it’s definitely obvious I need to keep practicing and getting these idioms back under my belt.

    func isValid(s string) bool {
        // We'll run through the string, doing the following:
        // - if we see an open bracket, we stack it.
        // - if we see a close bracket, we pop the stack. If the
        //   brackets match, we continue, otherwise we fail.
        // - when we run out of brackets, the stack must be empty or we fail.
    
        stack := make([]string,0)
    
        pairs := map[string]string{
            "(": ")", 
            "[": "]", 
            "{": "}",
        }
    
        for i := range s {
            char := string(s[i])
           _, ok := pairs[char]
           if ok {
               // left, push it
               stack = append(stack, char)
           } else {
               // right, stack empty?
               if (len(stack) == 0) {
                   fmt.Println("pop of empty stack")
                   return false
               }
               // Not empty. match?
               if (pairs[stack[len(stack)-1]] != char) {
                   fmt.Println ("mismatch")
                   // mismatched bracket, fail
                   return false
               }
               // Match. Pop stack.
               stack = stack[:len(stack)-1]
           }
        }
        // Check for "did we close all the open brackets".
        return len(stack) == 0
    }

    That’s one for today, and I need to clean the house, so I’ll call it a day.

  • leetcode month, day 1: Twosum and Add Two Numbers

    Well, these were fun, and definitely showed me I’m not ready to walk into an interview at the moment!

    Twosum

    Twosum is an easy challenge: you are given an array of numbers and a target value; your job is to find the only two numbers at different indexes in the array that sum to the target value and return those two indexes.

    The brute-force strategy just repeatedly walks through the array, trying to find the right pair of numbers. This ends up being O(n^2) because we essentially run over the upper triangle of a square of the elements of the array> If in the example below were looking for numbers that sum to 9, we’d run over the numbers four times finally finding it on the fourth pass, and doing 4+3+2+1 = 10 checks. (We can save operations by precomputing the second term in the sum: subtract the first search value from the target sum, and just compare that against the value in the array at that index instead of summing the two.)

    >1  2  3  4  5
     1 >2  3  4  5
     1  2 >3  4  5
     1  2  3 >4  5

    But the old Perl adage “when someone says search, consider a hash” comes in handy here. We can precompute exactly the number we want, so if we can make the values the keys of a hash (and we can, as we know that there is only one pair that sums to the target value), then we can make the indexes the values.

    We walk through the array twice:

    • The first iteration builds the hash by taking each value and inserting it as the key with the index as its value. And since we’re doing one iteration over the whole array anyway at this point, we can check each item as we insert it to see if it hits the target with the first item!
    • If we don’t luck out during the insert, then we iterate over items 2 to N, calculating the value we’d need to hit the target, and then doing a hash lookup to see if it’s there The hash lookup is O(1), and the pass over the array (including the load) is also O(1), so we’ve made a pretty good reduction in overall runtime.

    Things I’d forgotten in the last year of pretty much nothing but Scala:

    • The only loop structure in Go is the for loop.
    • var, not val. val is Scala, and as one tries for as much immutable data as possible in Scala, my fingers are trained to type val at every turn.
    • Go array initializer syntax. Square brackets, then the type, then the values in curly braces.
    • I remembered I needed to make() a map, but used Scala syntax instead of Go syntax trying to specify the types.
    • Go does not consider [2]int to be the same as []int. Fair.
    • Gotta have the parens for every function. Perl and Scala made me sloppy about that.

    Things discovered:

    • Adding the “try to catch it in the load pass” made a significant speed difference when the first term was at index 0.
    • Putting the first index in the array initializer for the result array instead of assigning it vastly increased the speed of the search loop — I went from the top 75% to the top 95% with that one change.
    • The hash solution uses more memory; I was at the 15th percentile on memory, but I’ll definitely take being faster than 95% of the other solutions as better.
    • I missed the “must be different indexes” restriction on my first pass; fortunately the second test case was for 6 with [3, 4, 2], and I got a test fail for [0,0].

    Here’s the final version. I do comment even when writing these because I can line out the code with comments and put the actual code itself in later. Writing down the assumptions helps clarify them as well (thanks loads, ADHD).

    func twoSum(nums []int, target int) []int {
        // Given that
        // - there is only one solution, then all entries must be unique
        // - therefore they can be used as keys to a hash
        // - this means we can iterate O(1) over the array for term 1
        //   and lookup term 2 in the hash, getting its index. No need
        //   for a linear search for it.
    
        // Fill the hash...and see if we can catch the solution
        // during this pass, since we have to do it anyway.
        locations := make(map[int]int)
        prematch := target - nums[0]
        for i := 0; i < len(nums); i++ {
            if i > 0 && nums[i] == prematch {
                return []int{0, i}
            }
            locations[nums[i]] = i
        }
    
        // scan the nums, looking for a match.
        // first match wins.
        for i := 0; i < len(nums); i++ {
            result := []int{i, -1}
            term2 := target - nums[i]
            j, ok := locations[term2]
            if ok {
                // Disqualify same-entry match. Missed this first time,
                // thank you test cases.
                if i == j {
                    continue
                }
                // Found
                result[1] = j
                return result
            }
        }
        // This is a "should not get here"; meant to show me if I've totally
        // blown the loop.
        panic("did not find a solution")
    }

    Add two numbers

    This one was recommended by leetcode as a “hot” problem, and it looked like fun, so I did it. It’s not in the Grind 75 list, but it’s a nice pointer manipulation problem, and I was a bit stale on pointers.

    The function is passed two linked lists; each list has an integer from 0-9 and a pointer to the next digit. The digits are presented in right-to-left order. The function is to take these two lists, and return a new list that represents the sum of the two numbers. Of course, the lists can be different lengths; if they are, then the list that runs out first is treated as if it were returning zeroes for the purposes of the sum.

    It was instantly clear that the challenge here would be maintaining the scan pointers for the two numbers and the result, and that trying to do bookkeeping for all of it in the loop would suck. So my approach was to construct a function to return a closure that would follow the rules for reading the numbers off the list:

    • If the scan pointer is null, return 0 and false
    • If the scan pointer is not null, return the node’s value and true, advancing the pointer to the next node.

    This meant that the list management was dead simple:

    first  := iterator(L1)
    second := iterator(L2)
    
    fVal, fState := first()
    sVal, sState := second()

    Each call to the first and second closures returns the next value to use in the sum; when both are false, the sum is complete.

    So all I had to manage in the loop was the scan pointer for the sum, and the carry from the addition operation: adding it in, clearing it, and recalculating it after the current operation.

    Things learned/relearned:

    • Had to remember when I needed * and when I didn’t for variable types.
    • The first run segfaulted. I realized that I’d mismanaged the scan pointer; I needed to save the root node, and then scan forward while doing the operations.
    • Once I fixed that, I found out I wasn’t clearing the carry. Whoops. Easy to fix.
    • The closures worked perfectly.
    • The “end the loop” and “restart the loop” keywords are break and continue. Trivial, but I had to go look it up.

    I did make one major mistake: I missed that the output was supposed to be another list, and started calculating the sum as an integer. This wasn’t too terribly off from the desired solution; I had figured out that I’d need the carry tracking, and I had a power variable that I was multiplying by 10 to switch columns in the output variable, but that probably wasted 5 or 10 minutes and might have made the difference between a pass and a fail in an interview.

    It was pretty easy to switch to the list building, but lesson hammered home again: be sure you read the problem statement correctly and know what the output is. Confirming with the interviewer that I got the details right is probably a good idea in a timed problem.

    **
     * Definition for singly-linked list.
     * type ListNode struct {
     *     Val int
     *     Next *ListNode
     * }
     */
    func addTwoNumbers(l1 *ListNode, l2 *ListNode) *ListNode {
        // The tricky bit with this one is going to be watching out
        // for the end of the list.
    
        // the dumbest and easiest way is to have a bool for each
        // list; when one list has no more entries, we set its bool
        // to false and return a zero value for it.
    
        // when both lists return a false value, we're done.
        first := iterator(l1)
        second := iterator(l2)
        var currentDigit *ListNode
    		var root *ListNode
    		carry := 0
    
        for {
            fVal, fState := first()
            sVal, sState := second()
    
            if (!fState && !sState) {
                // run off the end of both. Stop loop, finalize sum.
    						fmt.Println("Done")
                break
            }
            // At least one still returning a value. (The other returns 0
            // if there's no value.)
            // Sum the digits and the curent carry; if > 9,
            // take mod 10 and set the carry to 1. (no two
            // digits can sum to > 18).
            digitSum := fVal + sVal + carry
            carry = 0
            if digitSum > 9 {
                carry = 1
                digitSum = digitSum - 10
            }
            // Add space for a new digit, append it, continue.
            if currentDigit != nil {
                currentDigit.Next = &ListNode{digitSum, nil}
    		    currentDigit = currentDigit.Next
            } else {
    		    // Create and save root digit
                currentDigit = &ListNode{digitSum, nil}
    			root = currentDigit
            }
        }
        if (carry != 0) {
            // last addition had a carry we need to keep
            currentDigit.Next = &ListNode{carry, nil}
        }
        return root
    }
    
    func iterator(l *ListNode) func()(int, bool) {
        // Close over the pointer.
        p := l
        return func()(int, bool) {
            if p == nil {
                // Reached end. Keep returning 0 and false.
                return 0, false
            } else {
                // Capture next digit, advance pointer,
                // return next digit and true. If new pointer
                // is nil, we'll just return 0 and end-of-list signal
                // forever.
                v := p.Val
                p = p.Next 
                return v, true
            }
        }
    } 

    So how’d I do overall in comparison? Pretty fuckin’ good. I was in the top 95% on speed, and the top 91% in memory use. I suspect that managing all the bookkeeping in the loop might make it a tad faster (no call overhead for the closures), but at the cost of significantly more complexity. This solution is fast and small, and I’ll take it.

    Conclusions

    • My Go is pretty good. I didn’t spend a huge amount of time chasing bugs, and had it in a couple tries. I have a good grasp of the concepts and know mostly how to do things.
    • My Go is fairly stale. I don’t remember how to say things!
    • I had to look up a lot of things that I should have just remembered, like len() for array length (not .len, that’s Scala!). I need this practice!

    Tomorrow I’ll start off with Valid Parentheses.

  • Flexing the muscles: November leetcode challenge

    Most people do a novel for NaNoWriMo, and that’s a great thing.

    I am not a great fiction writer; maybe I’d be better if I practiced, but right now, writing code makes more money, so I’m going to spend the month of November working through the Grind 75 leetcode practice set.

    I would very much prefer a Perl job, but there just aren’t that many places now that really want someone whose primary is Perl. Python’s fine, but it’s mostly tied up in LLMs and AI at the moment, neither of which I actually find very interesting. (They seem to be more “build a model and hope it does the work you wanted” as opposed to “write the code to do the job you want done”, which I find much more satisfying.)

    I haven’t done much with Rust yet, and I think it’d be a fun language to work in, but I don’t see a lot of demand for it yet. Scala is fun to write but I’d rather jump out a window than use the JVM toolchain again. (This also crosses off Java, Dart, and Flutter for me as well.) I have not in general found C++ to be that much fun, and Javascript is what it is. I can write it, I just don’t enjoy it.

    So it comes down to two languages, really: Go and Swift. I enjoy coding in both (for different reasons), but at the moment I’m seeing more Go jobs than Swift ones, though that may just be my settings. Certainly when you need a Mac/iOS programmer, you need someone who at least understands Swift.

    So I’ll probably end up trying it with both, and seeing how it goes. Onward!