Staring at this board, mentally converting PCIe lane count to device requirements, only to consider that yet again…

It’s 2025. We’re still using the PCIe slot standard for most motherboards, even on enterprise boards. Who’s doing this right, at least marginally so?

ASRack (brain, please grow up and stop making elementary jokes) has been putting four physical x16 slots on their “Deep mini-ITX” series of server boards, to which are added two x8 lane Slim-SAS and two x4 lane OcuLink headers. These can be used for a variety of SFF fan-out connection purposes.

Extensions like this include:

• multi-drive backplane headers for hot-swap 2.5", 3.5", U.2, U.3 bays
• additional M.2 slots using a SFF backed PCB
• conversions from one standard controller to another, PCIe to SATA3 or SAS3
• adding external ports for SAN and DAS expansions
• additional physical slots in x4, x8, or x16 - just mount those style connectors in an adjacent or remote chassis zone via PCIe 1:1 extension or PCIe switch

There are likely other examples, and I’m always interested in finding more.

#servers #baremetal #linux #freebsd #motherboard #engineering #hardware

The vast and varied subject of Women’s Safety Online is a topic which impacts many facets of daily life, including emotional and physical well-being. This post is not a primer on the concepts, rather it offers a short timeline to explain why I am no longer on LinkedIn or Twitter social networks for reasons of personal safety and privacy.

A Tenuous Timeline

1) Some years prior to a decade ago, I joined LinkedIn, and a small handful of years ago I joined Twitter.

2) Along the way a serious incident occurred, resulting in PTSD from a bodily assault; the details of which require privacy. During associated therapy a HIPAA violation occured, eroding my trust in the medical establishment.

3) Periodically I would receive messages on LinkedIn from recruiters at the company where that man worked. I did not want to be reminded of the events, and would never work at the same company. These external intrusions into trauma memories were not healthy and they needed to stop.

4) Concerns evolved which necessitated telling that company to remove me from their recruiting database and never message me ever again.

5) The company removed my profile, and they asked if I wanted to be involved with an investigation, which I refused for the unfortunately common reasons:

• I did not want to relive events, especially after the HIPAA issue
• I did not want to discuss those memories with yet another stranger
• I did not want to become more of a target for speaking up

It’s a terrible catch-22, and yet it’s nothing new. This person needed to be prevented from ever contacting me or knowing anything about my life, requirements which the company could not ensure.

6) For safety reasons, I changed my legal name, then I moved, and started a new job. Then I enabled a service which monitors for identity theft issues and personal-info data leaks. Other precautions were made. The whole process is time consuming, emotionally disruptive, and fiscally burdensome.

7) Years pass. In the interim, LinkedIn’s “people you might know!" feature will routinely recommend that I connect with people who reminded me of that trauma.

• The feature cannot be disabled and profile settings do not help. Blocking people and profiles does not sufficiently help. Their algorithm always does this.

8) Years pass. Identity theft issues become a greater risk. Someone on Twitter steals my profile, impersonating me to sell web3 tokens or whatever. I report it, and Twitter demands my government ID - which I refuse to upload.

• Preferring a human interaction, I offer to bring my ID to the Twitter office and show it to someone in person, but no - they want the digital copy to keep on file, which will totally never be leaked in a data breach.
• I refuse to upload the document. They lock my account. I have not bothered since.

9) Years pass. A few weeks ago, I’m running a social media account audit and notice that LinkedIn has two unknown logins from another country - and I’ve never been there, nor connected to an ISP or VPN or service related to that location.

• I revoke the logins, yet they reappear a week later.
• My account has 2FA enabled, yet the auth logs do not show it being used.
• Who is behind these logins? What amount of personal info and connections and messages do they have access to?

10) After seeking help from LinkedIn’s support system, they now want my government ID, just like Twitter.

• They want a third party service (Persona) to scan it, and surely this will never be leaked (sure sure).
• They assure that this third party is very safe, that they only give LinkedIn redacted info (yet Persona keeps the full info?)
• Surely I should trust their entire system. I do not trust their system, and I will not upload my ID for it to be stolen or have any of the data used in some way to harm me.
• The result is similar to Twitter, my account is restricted with no access.

End Results and Oh Well…

So, goodbye LinkedIn, goodbye Twitter.

I cannot place my trust in these faceless social media corporations, the ones which never assume responsibility for data leaks, data theft, or the sale of demographics and usage meta-data. The common refrain of, “we’re telling you that we redacted your personal information”, means nothing substantial from these organizations.

These sites profit extensively from user data - we are their product for sale, and when that happens it can lead to extremely unsafe conditions, particularly if that data can be used for location tracking, enabling access which leads to physical violence, stalking, harassment, or other detriments by which women too-often experience the online world.

Sure, I could upload my ID and “just trust them”, but that would not make me safer; it would invalidate the entire purpose of protecting my body, my mind, my future. Handing them my trust, when they have routinely proven that they are not trustworthy, would render-pointless every hardship and effort towards safety and security which I’ve been required to endure since those events so many years ago.

By prioritizing my personal safety in both of these cases, the end result is that I am no longer able to engage and connect with people from my work life via the two most common business-centric social networks. The alternative is no alternative at all; choosing not to be safe is not a choice.

I know what it’s like to have my choice taken from me, and I never want that to happen again.

The Q4-2024 through Q2-2025 cycle has been one of remarkable instability for Fedora kernels and systemd breaking-changes. F40 through F42 have offered more crashes, more kernel panics, and more core dumps than any triple-set of versions since its dawning into existence - and I’ve been running this distro for dev-only boxes since the first release.

In the worst of circumstances, Fedora notified me of a firmware update, which was a hands-off type of “reboot to install” EFI blob which bricked my Thinkpad X1 Nano laptop just last week. That’s an expensive failure and one which has no simple resolution (many hours, many iso to img to efi/cap conversions, and maybe soon a WSOP-8 NAND reprogramming).

I’ve been all-in on the RedHat infra train since 1999, having used it at global backbone providers, satellite infra providers, hpc clusters, media conglomerates, gaming networks, streaming services, cloud providers and hyperscalers, and just about everything in-between… and while I have my qualms on certain technical aspects and corporate strategy over those years, I still consider it one of my favorite Linux distos (top five at least), and it’s my first choice for enterprise Linux. So, I’m not just hating on the ecosystem here, I’m a vested user who’s been disappointed too many times by the direction

Unfortunately, with Fedora, I stand behind my position that Redhat has been too hands-off in conducting standards requirement alignment, in requiring a base level of stability upon which serious engineering development can be done, without having to clean up other people’s messes every few weeks.

It’s just not a great dev platform distro anymore, and yet the org markets the name as if it were a valid OS for all manner of production uses. IoT use? Never, no thanks. Edge devices? Never again. Security? Forget about it.

It’s not a “bleeding-edge issue” - as that’s Rawhide’s territory where stability of any sort is not expected. The evolving issue with Fedora is that it is uncommonly unstable for a “dev env”, and one which ships far too much untested code and pushes the equivalent of “crowd source fail-tests” where they expect the users to “just tell us if it doesn’t work”. Their release engineering and load-testing seems to occur with zero multivariate matrices sequences for all manner of performance, validation, and functional tests. I suppose they consider unit testing to be “full coverage”, but it is not. Happy to be corrected here, though I will remain disappointed and avoid any further deployments or firmware updates from Fedora.

#linux #fedora #releng #engineering #software #enterprise #rhel #distros #displeasure

Some quick progress notes for a few FreeBSD and OpenZFS devs, which will be accessing one of my POWER9 systems.

See photos for visual reference.

• Separate L1 and L2 domain for all network access, zero-shared DMZ, unconnected to the rest of the RFC99 lab lack
• Mikrotik router has been temporarily slapped into place on the side of my lab rack, directly in back of the Talos II system (dual socket, 144 threads of fun!), using PoE for the main link
• PiKVM v3 is being cabled up for a jump box, will connect to the Mikrotik switch ports
• Talos II’s OpenBMC will be connected to the Mikrotik switch ports, offering iKVM and ipmitool SoL terminal, which will be accessible via the jumpbox
• Wireguard tunnels will be allocated on the Mikrotik router for remote users
• APC PDU for remote power cycling (hard reboot style) of the Talos II is accessible via SNMP v3 authenticated command

Later today on the “Bhyve Production Users” meeting, I’ll go over those notes.

• Thursdays at 10am PST
• Recordings: www.youtube.com/@callfort…
• More info: callfortesting.org

@dexter@bsd.network

#freebsd #openzfs #power9 #engineering #opensource #networking #linux

I love a lot about the linux world, and there’s a lot of positive aspects to the kernel itself, which has brought a lot of joy over the years, with kernel module development, debugging, tuning, analyzing… specifically for my career, but also on a personal level. For one quarter of a century I’ve been using Linux and BSDs in many different flavors, but there’s an accelerating trend in the corporatized linux space which has been quite concerning.

The marketing trends used by The Big Names over the past decade remind me far too much of Cold War propaganda, of the lies from literal communists which stained mine and many other’s youthful years. Those lies fell apart in 1989 and we all hoped that would be the end of it… but of course not.

The Perception

Das Kollektiv der Gemeinschaft! Wir sind für Sie da! Schließt euch den Reihen des Proletariats an und vernichtet den Menschen!

Translated, loosely

The collective of the community! We are here for you! Join the ranks of the proletariat and destroy the Man!

I do not want your communism. I do not want your pseudo-cooperatives. I do not want your kollektiv of anything.

We’ve had our community this whole time, and the corporate influence was supposed to be secondary. We had to fight and push to get OSS into the corporate sphere for decades, and sure enough, people take that for granted.

There used to be an alignment where OSS was to “be against the corporate overloads of closed-source”, about “freedom of choice and implementation”, about “fair use licenses”. This was in opposition to Microsoft, Oracle, ATT unix, SCO unix, etc. But we all know what happened. The big names bought their way in, pumped up all the money making schemes, and now we see this trite marketing where billion dollar companies get to role play like they care about the communities which WE built over the past three+ decades.

Don’t believe me? Go look at the recent Fedora website redesign. Look at Canonical. There are many examples and one can find them easily by perusing the Distro-Watch list of a 100+ remakes.

I’m in no way against corporate interest and benefit when it’s ethical, when it’s up-front and honest about its intentions. Though they are not my favorites of the world, at least Redhat and Oracle don’t try to pull one over on the user with their main sites – it’s obvious that they are there for corporate players and for profitability, which is fine. I’ve paid for their licenses, for their hardware, many times in many ways, from personal use to license structures while operating a startup, so I’m not anti-corporation in the overall sense.

However, this approach to “we’re all in this together” fake messaging is tiresome. The irony of Fedora being owned by Redhat, who is owned by IBM; draw whatever conclusions one wants there, but they are all managed by different sectors.

So then, do we really need to be sold on the community aspect of what is inherently a community of developers working together for a shared goal? No, we do not, those are supposed to be intrisic and inherent qualities without need of mention.

Who does it better? Here’s a non-exhaustive short list from the top of my mind.

• Gentoo (https://www.gentoo.org)
• Chimera (https://chimera-linux.org)
• SUSE (https://www.suse.com)
• Void (https://voidlinux.org)

Don’t get me going on the BSDs… they are perfect and I love them, so it’s a totally different conversation. 💗

These are the topics that I’ve been spending time with, on no specific schedule, with no intended intention, often for debugging or optimization, sometimes purely for curiosity, and often in-between other tasks throughout the day and night, so they might as well get some blog posts for discussion. All of the topics have already been written via configs, scripts, list-docs, and automation tooling, but no formal blog posts just yet.

1. Ongoing sprints in “The EPYC Epic, Desktop Cooling Round III, Software Settings”
2. An Enterprise Engineering Desktop w/ FreeBSD: hardware, software, configs & optimizations
3. Terminal Living, Adventures with Shells, and Why I stopped using Bash after 22 years (sh, csh, tcsh, fish, nu, xonsh)
4. Not the Biggest, nor Smallest: Porridge Bears settle on a heat sink for EPYC cooling
5. Aerodynamics and Turbulent Efficacy: stfu Supermicro chassis fans
6. Xorg Always Works: FreeBSD with the Big Three GPU brands, running 4K 240Hz 1000R on AMD, Intel, Nvidia
7. FreeBSD Performance Analysis: machdep, c-states, turbo, and timecounter shootout
8. Lazy Cores, Hot Nights: CPU idle algorithm optimization, core pinning, and NUMA balancing

#FreeBSD #Engineering #Linux #Systems #Desktop

Data-At-Rest Encryption is a common topic, but one which is often misunderstood. Having led engineering and architectural efforts involved with hardware standards for encryption compliance, including detailed analysis of performance impacts while operating in accordance to service level agreements, here are some bits of discussion on the SED/Opal implementation.

TL;DR

• SED/Opal TCG spec’d drives are doing just fine. I trust it more than LUKS2 by a long shot.

Detailed Read

I have no issues trusting NAND and SAS firmware for the drive controllers that I’ve personally validated as part of a hardware acquisition process for global fleets which require federally mandated encryption.

Encryption compliance auditing is an involved process, as is the hardware validation process for identifying potential impacts to production storage performance workloads, and then there’s adapting encryption compliance requirements to an org’s procurement and provisioning pipelines, inc supply chain attack mitigations.

All of those concerns apply to hardware shipped outside of the country into export-regulated zones that have tight controls over encryption compliance.

Having been in charge of those workflows necessarily by job role, I can attest that it’s not exactly roses and rabbits and chocolate teddy bears every day of the week during audit season, but it is rigorous and it does create secure data persistence across geopolitical zones and high-risk locales*.

USA Enc Regs

• DoD: www.nsa.gov/portals/7…
• NIST, Encryption for End-User Devices: nvlpubs.nist.gov/nistpubs/…
• HIPAA Encryption Overview: www.hipaajournal.com/hipaa-enc… HIPAA Req via DHHS: www.ecfr.gov/current/t…

LUKS2 CVEs

• www.cvedetails.com/cve/CVE-2…
• www.cvedetails.com/cve/CVE-2…
• www.cvedetails.com/cve/CVE-2…

For non-USA regs there are similar standards for modern first-world countries, though I trust the US’s military industrial complex far more than anything from anywhere other than Germany (and to a lesser extent, also Japan).

Exo-Border Encryption IRL

• My favorite story for that example are systems in several Taiwan datacenters which were seized by the CCP during the past N years. They didn’t want to cut the power, given the facility’s status and criticality, so the systems continued to run unabated until their dead-stage.

Effectively this created a situation of situationally enfoced uptime. As long as any system was online it was fine, but no maintenaces were possible, and any drive failures remained forever-failed as no person was authorized to replace any hardware (even a simple hot-swap drive bay flip). Most of those systems are still online, vastly exceeding any manufacturer specs or expectations of SLA. When shipping sensitive data and secure systems around the world one must expect the worst, and that issue certainly wasn’t the worst, at least the data is still live.

Combat zones are an entirely different thing, and one can look to the Special Forces requirements for their portables to see how encryption standards work during active live-fire scenarios. It’s often not enough to have encrypted drives; there must also be provisions for hot-swap/ejection of the hardware from its chassis, auto self-destruct features, and similar controls which seek to mitigate the less common edge-case concerns (torture, etc).

Original Conversation This conversation began as part of a comment on my Mastodon account: mastodon.bsd.cafe/@wintersc…