Willkommen auf Eva Winterschön's site für Erinnerungen und Reflexionen
Eva Winterschön

reflections on OSS, HPC, and Ai/ML engineering, with occasional considerations on Cognitive Neuroscience

    ...

    Episode 44: "Leaving ORD, minimalism yes please."

    At airport gate, waiting. Originally, a dawn departure.

    Hilton at ORD decides that rooms do not need the usual bathroom items, and prefers the full plastic half-assed attempt at a capsule pod coffee maker. Substantial displeasure occurs, events transpire, and now we’re rebooked on flight a few hours later — sleeping in for those hours and have been – words and thoughts – but brain still not really awake. Robin is finding coffees somewhere on the concourse.

    ...

    Episode 42: "We're leaving. Take only what you need to survive!"

    ETA, 2 hours till movers arrive, in yet another episode of… “Take only what you need to survive”, and they’re packing up everything else for transit. Countdown to California thereafter.

    Goodbye to another location. I suppose that I could say that Chicago was beneficial in the larger sense of existential purposes, and that much was accomplished, sure sure.

    However, and not in jest, nearly the entirety of those accomplishments could have occurred in an old military quonset hut located off-grid, deep in the vast stretches of the Northern Nevada desert, preferably near the Ruby Mountains. It’s far more beautiful out there, truly yes, but the pizza is not delivered, and Chicago does have very good pizza. 🤍

    a screenshot from the movie Spaceballs, in reference to the phrase, take only what you need to survive Auto-generated description: A serene mountain lake is surrounded by rocky cliffs and vibrant wildflowers under a clear blue sky. Auto-generated description: Snow-covered mountains with rugged peaks under a partly cloudy sky.
    ...

    Memories of a Younger Era — Tokyo Hiroshima Kyoto

    Memories of Japan are easy to get lost in, the time period was often a blur. Capsule hotels, bullet trains, insomnia, and several neuro-cognitive mods offering symbiotic boosts to perception. An immediate realization dawned upon landing, so many people seem to love tall western women, to be quite vocal in appreciation and awareness, and hey that’s me!

    Those were memories of a pure, simple existence, the joy of living in a beautiful foreign land with a wonderfully kind culture; one where I could not read a single sign, nor understand more than several helpful conversational phrases. A stranger in a strange land. A summer of intense philosophical reassessment, of isolationist meditation, with the majority of months spent traveling alone through four countries across two hemispheres. Circa 2017 feels so far away.

    Perhaps more travel photos will follow. I’m in a reflective mood today. 🩷

    #travel #photography #memories

    ...

    🌐 Nvidia Ai Cluster for Traveling 🌐

    A little travel cluster, all set for Ai engineering and development.

    • 1x Nvidia Jetson Xavier NX 16GB
    • 3x Nvidia Jetson Orin NX 16GB
    • 3x GoWin R86S 6005 + CX5 25G OCP 2.0
    • Various network hardware

    Setup in a Pelican Air case, ready for the flight!

    #ai #engineering #development #nvidia #cluster #jetson #travel

    ...

    🌐 Weekend Hardware Update 🌐

    Some additional reorganization time this weekend, all busy with the process of preparing the HomeLab for another interstate relocation. Having decided to repurpose some hardware to create a new router, this one formerly hosted infra VMs several years ago, but now it’s going to run a pair of “BSD Router Project” VMs for on-host H/A, and a bit of packet shuffling between hosts for RDMA on RoCE v2, iSCSI, and a few shared NFS mounts for compute hosts, along with the usual firewall and packet filtering, some Wireguard, OpenVPN, and Zerotier, NTP + PTP via GPS/GNSS.

    Here’s a quick rundown on the specs, aggregated from the parts shelves and “it’s not actively being used elsewhere” collection of enterprise hardware.

    #homelab #freebsd #router #networking #engineering #hardware #servers

    ...

    🗑️ Import Your Failures! 🗑️

    While using Soundcloud the other day, prompted by a little helper fellow which states:

    “Import your Spotify Playlists!”

    ok, sure. I’d like to use a single app for music if possible, and I prefer Soundcloud to Spotify in general. So, I click ok and authorize the apps' APIs to do their thing.

    It finishes importing. Great, back to work. Life goes on. Days later, that being five minutes ago, sitting at my desk… “time for some metal” and decide to play a near-daily favorite, “Heaven Shall Burn” compilation playlist which consists only of their music.

    Here’s what Soundcloud decided to import and use for the Heaven Shall Burn playlist (attached image). Look at the artist names, the album covers, the lack of sequential existence of tracks which should all be on the same several-albums. These aren’t even the same category of music, let alone the same band or albumns.

    Soundcloud, you have failed. This was an easy one; you have the API calls with the correct track and author IDs right there in the Spotify playlists to do the import calls on a 1:1 basis, but no, instead you’ve taken a giant pile of garbage and put it on my Library.

    Instead of “Protector” by Heaven Shall Burn, you give me "Protector" by Beyonce. Trust me, they are not equivalent. This is garbage, more enshittification of the internet by unchecked unvalidated coding styles which prioritize “we ship code always” over “we ship when code is ready”.

    Now I’m left with extra work to do, to clean up their mess of shitty playlists, and the tool imported A LOT of playlists.

    #music #streaming #steamingPileofSoftware #software #engineering #antipattern #trash #code #developers #enshittifcation #spotify #soundcloud #api #fail

    ...

    Memories of Harmonic Distortion, Descending Carson Pass

    While training for The Death Ride one year, so many years ago, I had quite a surprise on Highway 88 while descending Carson Pass in the Sierra Nevada. This was in the late season, the last week of October, a last minute plan with a best friend to get one last ride in before the snow storms arrived. On top of the usual physical challenges of climbing ~2,500' vertical up to 8,575 elevation, it was uncommonly cold.

    For the background, I loved endurance racing, and the many long, standing (not seated) hill climbs and mountain passes always were my favorite. Climbing the front-side and then heading down the back-sides of big mountain passes in the Sierra Nevada at 50+mph (80kph) was however not my favorite - but a good lesson in reaping what you sow. Thankfully I never crashed on those, despite having climbed all of the majors and several lesser traveled, many times over: Monitor, Ebbetts, Carson, Echo, Rose, Yuba, Donner, Tioga, Brockway, Daggett, Luther, Pacific Grade Summit, and famously adjacent Geiger Grade going to Virginia City (a common morning ride before work). All of my crashes were on mountain and cyclo-cross, so I was certainly not immune.

    The closest I came to a high speed get-off was on Carson Pass (highway 88), back in 2001. I was riding a new set of criterium race wheels that were ultra-lightweight, perfect for climbing up the pass to the summit, but during the descent speed-wobbles had set-in around the 45mph point and continued through to 54mph, starting directly after the first major right-hand curve.

    While arc’ing the bike inside the turn, standard approach to manage centrifugal force, and then leaning it back upright as the road straightened, harmonic distortion set in, rippling faster and faster through the bike’s frame, up into my body, blurring my vision. Sometimes going faster will sort out the wave, but not this time, and by 50mph the flexing nearly tore the wheels apart from a combination of light-weight hubs and spokes/rims not having sufficient torsional strength to sustain the pressure involved with the induced coefficient-of-friction and centrifugal force.

    This was the kind of turn with speed, which one can see racers engage with on the Tour, which require a rear-side body-torque movement to snap/push the frame out of its desire to continue through the curve and resume going straight - too much downward force started to turn the wheel into an slight oval. This same body movement is used in MotoGP and motocross racing, as it’s an essential skill for powering through a curve while pushing physical limits. The feeling was like that of an al-dente noodle, if one were to imagine being thrown down a hill while trying to keep everything straight and rigid - just stiff enough to not immediately die, but very close to dying really quickly. Wearing a helmet in that case wouldn’t have saved my life, as the pass is surrounded by granite walls and boulders and steep drops.

    It took an entire mile of deceleration, achieved through pulsing the front and rear brakes at a 30/70% ratio (like anti-lock brakes on a car, but without the computer), to safely slow down enough for the distortion to resolve. If you don’t pulse the brakes then the pads can overheat and melt, contributing far worse conditions into the mix. By the ~30mph point I had recovered enough control over the bike to gently guide us to a turnout several miles down the pass and then dismount. A lot of adrenaline that day, the intensity of focus and attention to the senses required was certainly a high-point at my youngish age (early 20s).

    My friend, a far better racer than I’d ever become, could only laugh at the scene. It wasn’t either of our first experiences with harmonic distortion, and what else is there to do but laugh and get back on the bike? Away we went, continuing down the steep but fast and straight-smooth road to Woodfords where we parked the car.

    Well anyway, I had those wheels relaced with bladed titanium spokes and hardened steel nipples tensioned with a digital torque meter (not “feeling the torque by hand”), and never used them for hill climbs ever again.

    As for The Death Ride, they no longer include Highway 88’s Carson Pass on the route, mostly due to safety.

    References

    #cycling #memories #theDeathRide #training #endurance

    ...

    💻 Mozilla Thunderbird - The Failure of a Once Great Client 💻

    Thunderbird, wtf are you doing where you need 83% CPU plus ~50GB (virt) and 20G (res) of RAM allocated?

    The entire mailbox isn’t even 1GB in total size. Yet here it is, cranking out all of this nebulous processing 24/7… literally 24/7 it’s running loads like this for months now. Generally I ignore it because the workstation has a modern EPYC with 32 cores and 128GB of RAM, so it’s not completely crippling my usage… but sometimes it does, and sometimes I want that 50/20GB split to not be around, and I want those clock cycles and L1/L2 cache doing something useful.

    This issue has persisted over several major versions, it’s not a new issue. I’ve gone through all of the usual steps, removing all themes and plugins, etc. Nothing fixes the problem, and there’s nothing useful in its logs. Uninstall and reinstall, use on different computers, use with different server settings, use on different domain accounts, it doesn’t matter… Thunderbird always runs itself into failure mode eventually.

    I don’t want to dtrace it, I don’t want to strace it, I don’t want to deal with this at all because email is not complicated enough to warrant such investment of my engineering time or focus.

    I hate what Thunderbird has become, and I’ve been using it since version 1.0 way back in the 2000s. I’ve used it on all of the operating systems upon which it’s been supported. This has become absurd, the slow death of a once great product.

    You’re an email client. You handle IMAP. This resource usage is unacceptable.

    #opensource #mozilla #thunderbird #email #slow #debugging #programming #linux #freebsd #desktop

    ...

    4am Dev Box - GPU & RDMA Hardware Shuffle

    Some photos from a bit of 4am dev box maintenance earlier this morning, focused on hardware reallocation and consolidation to maximize PCIe lane utilization.

    This box will host one VM using full PCIe passthrough on the GPUs and 50% NPAR for the NIC ports, and four smaller VMs. Storage for the VMs will be host-level ZFS on the two P3608, split up as several zvol allocated to the smaller VMs for a RoCE-v2 storage micro-cluster, replication to off-host network based storage also using RDMA.

    The GPU compute VM will be running our agentic-consensus LLM models, continuing development into deeper waters prior to running on the bigger hosts in colocation.

    • 4x Ampere A4000 GPUs
    • 2x 4TB Optane P3608 NVMe (2x 2TB per card)
    • 4x 128GB (512GB) Optane NVDIMMs + volatile 128GB of RDIMMs
    • 64 Thread Xeon 8370C, 2.8Ghz (Hyperscaler SKU)
    • 4-port 25GbE QLogic NIC, QL41234, using RoCE v2 RDMA

    #nvidia #gpu #ai #engineering #llm #development

    ...

    💾 Updates on Remote Access to POWER9 💾

    Some quick progress notes for a few FreeBSD and OpenZFS devs, which will be accessing one of my POWER9 systems.

    See photos for visual reference.

    • Separate L1 and L2 domain for all network access, zero-shared DMZ, unconnected to the rest of the RFC99 lab lack
    • Mikrotik router has been temporarily slapped into place on the side of my lab rack, directly in back of the Talos II system (dual socket, 144 threads of fun!), using PoE for the main link
    • PiKVM v3 is being cabled up for a jump box, will connect to the Mikrotik switch ports
    • Talos II’s OpenBMC will be connected to the Mikrotik switch ports, offering iKVM and ipmitool SoL terminal, which will be accessible via the jumpbox
    • Wireguard tunnels will be allocated on the Mikrotik router for remote users
    • APC PDU for remote power cycling (hard reboot style) of the Talos II is accessible via SNMP v3 authenticated command

    Later today on the “Bhyve Production Users” meeting, I’ll go over those notes.

    @dexter@bsd.network

    #freebsd #openzfs #power9 #engineering #opensource #networking #linux

    ...

    🩷 Good Morning Stormy Spring Weather 🩷

    #chicago #morning #photos #stormy #sleepy