WretchedGhost's Tech Blog and Rants

I just broke my ZFS pool today. It wasn’t from the fact that I was running out of space and had earlier in the week just added a 5th drive to my 4 drive raidz1 pool, so this knowledge was fresh. Nor was it from the fact that I was going to be gone for the weekend and I wanted to get it done before I left. Neither was it from me not being able to sleep well this night and woke up at 1:30 in the morning and said, “I can’t sleep so lets add that 6th drive to the pool today.”

Here is a short post on how I setup my rooftop tent on 1" rigid conduit and some spare U-bolt and saddles which grip the flat edge of the bed of my truck. Normally a rack would cost $800 or more but with this setup it can be free if you know some electricians who would normally have thrown some of the pipe away, all the way up to a little over $100 buying the hardware yourself.

I recently came upon a new program for myself that simplifies notifications and integrates with checkmk. This program is called ntfy and it, coupled with my Linode instance, provides me with a means to get notifications of all of my servers, tailscale approvals, SSH logins, and more virtually instantly via ntfy’s notification software which seems to be more reliable than email and faster than text messages.

First lets begin with checkmk.

Change Case

Alt+U - Make word uppercase from cursor to end of word

$ sudo systemctl start nginx
                        ↑ cursor here
# Press Alt+U
$ sudo systemctl start NGINX

Alt+L - Make word lowercase from cursor to end of word

Alt+C - Capitalize word from cursor to end of word

Due to issues with the latest sleep states in laptops I’ve had to resort using hibernation or S4 as the strata for all my Linux laptops:

1. Lenovo X1 Carbon Gen 8 (intel 10th gen CPU)
2. HP Gram 17z90n (intel )
3. Dell XPS 9315 13 i5

My older laptops like the System 76 Darter Pro before they moved to CoreBoot systems and Dell XPS 9380 still functions quite well in S3 state, not requiring hibernation.

sed (Stream EDitor) is a super simple program that can be used to replace text for another in a file. Let’s first view the contents of a file I made called greetings.txt with cat.

My intention is NOT to create a clickbait title, but I have found a re-invigoration within me with regards to data - or in layman’s terms, data hoarding. I have always had around 8-10TB of data between videos, family pics and videos, and some games (not my entire GOG or Steam library yet, but maybe one day). All of this takes up precious space on my HDDs.

Physical Media Renaissance

I’ve been in a mode where I head to my local second-hand DVD store and purchase movies and shows at great prices. Ripping them with my purchased copy of MakeMKV - mostly to stop the nagging and reinstalling required to “renew” the free license - has caused my media storage space to grow exponentially.

Show All Available Shortcuts

$ bind -p

This shows every single shortcut that bash knows about. There are hundreds.

Ctrl+X, Ctrl+E - Open current command in your default editor (usually vim or nano)

Article: PSA be aware of the difference between localhost and 127.0.0.1

(hic sunt localhost is a play on words from the old Hunt-Lenox-Globe which coined the phrase “Here be dragons” or “hic sunt dracones” in Latin.)

I was bit in the butt by a bug that sprung up at me over the weekend. It really wasn’t a bug, but something that did sting a little bit. Over the weekend I set up a new VPS with Linode where its only job was to be my central always-running syncthing server in the cloud. Upon first setting it up I was able to reverse SSH tunnel to it using my standard SSH config and keys using this command:

If you’re spending any amount of time in the terminal, learning shortcuts can potentially save you hours in the future. Most people know the arrow keys and backspace, but bash has decades of shortcuts built in that almost nobody uses.

I’ve been using Linux since 2011, and I’m still discovering new ones. Here are the ones I try to use on a daily basis, plus some obscure gems.

My server, cprox, as I call it, was set up with only 64GB of storage on the root partition. I don’t know what I was thinking at the time, but this caused issues only a week after setting up the new server.

Within this time, the OS, packages, and other things Proxmox does, consumed 86% of the drive. This made things run slowly and caused the command line to freeze sometimes, which was very weird since the OS was on mirrored SSDs - though which is exactly what allowed me to do this resizing live.

My ranger config can be found here

Here is a quick write up of a program that I have heard of before and installed but never found it necessary in my work flow. I have and still am happy to move around in the command line like I have been for the past ten years plus but now after finding a good reason to try it out this time since I have been moving around a lot more in the command line from writing blogs to notes to novels, and everything in between, I felt like my work flow wasn’t as optimized nor as speedy as it could be.

Why Use Vim Plugins?

In the Vim Basics series, we covered vanilla vim and how powerful it is right out of the box. And honestly, vanilla vim can handle 90% of your daily work as a sysadmin. But sometimes you want that extra 10% - better syntax highlighting, file navigation, or quality of life improvements that make your workflow even smoother.

The key is not to go overboard. I’ve seen people install 30+ plugins and end up with a slow, bloated vim that defeats the purpose of using a lightweight editor in the first place. As a system admin who jumps between servers constantly, I keep my plugin setup minimal and portable.

Building on the Basics

Now that we know how to open a file, edit it, move around in it, and save it, let us now look into how we can tweak it. My post to part 1 of vim basics can be found here: https://blog.lanlocked.xyz/post/vim-basics-part-1/. Part 2 can be found here: https://blog.lanlocked.xyz/post/vim-basics-part-2.

In this part, we’ll cover visual mode, text objects (which will change how you think about editing), and then dive into customizing vim with your .vimrc configuration file.

Welcome Back

In Part 1, we covered the basics of opening vim, understanding normal and insert modes, and moving around with the hjkl keys. If you haven’t read that yet, I’d recommend starting there as we’ll build on those concepts.

Now it’s time to tackle what might be the most infamous aspect of vim: how to actually save and exit the program. We’ll also dive into some useful editing commands that will start to show you why vim users are so passionate about this editor.

Getting to know vim

If you are looking for a way to get more efficient at editing configs and text then vim is what you are wanting. Even at its most basic interface with no added plugins, vim can help you manipulate a file with its efficient modes and commands.

vim is not the easiest to learn, especially for beginners where often newbies trying it out for the first time find they are not even able to exit the program. But, after some little quirks have been understood, you may soon ask yourself how you lived without it.

Editor’s Note: When I first published this article, the https://audible-converter.ml/ shortcut method still worked. As of August 2023, that site is down. The method below using RainbowCrack still works perfectly.

Why You Should Care

As Amazon continues to expand their collection of books, music, and audiobooks, licensing disputes with publishers and authors will continue to cause problems. What this means for us as users is that we could lose access to media we purchased on Amazon’s platform. This has happened before and will happen again when you least expect it.

I first dipped my toes into using Synology though their top-notch NVR system. I had eight cameras at the time around my house and it worked very well for myself and my family, though paying $40-$50 a license per camera can get expensive fast. I didn’t really use it for its other features for years until a fellow sysadmin introduced to me the functionality that it had offered him as he did large project like setup NVR and NASes for the city and school systems. It intrigued me enough to accept his proposal of bringing in a Synology 4U rackmount server into my main datacenter.

I am the new owner of a hand-me-down Chromebook that to me, now has a new reason to exist. No longer is it tied to Google and their tracking and ads policies. Rather it is its own system now, complete with Arch Linux/EndeavourOS on i3-wm.

I first purchased this laptop for my oldest child who required something for online school during the Covids and I landed on the Lenovo S340-14. Inside it has a Celeron N4000 dual core/dual hyperthread CPU, 4GB of RAM, 64GB of storage, and a 14 inch TN screen at 1920x1080 and 220 nits. It worked fine for her during that time but now that its basically a paperweight now that she has my System76 i7 Dart Pro gen 1 laptop ((I need to make a review on about it one day) which was before they stared using coreboot and is 15.6" rather than either 14" or 16" they have now, but that is a story for another day).

I have been using MikroTik (my-crow-tick or me-crow-teek), or however you want to pronounce it, for several years now and I just keep coming back to using it. MikroTik, for the uninitiated, can be quite a difficult system to learn and use. There are quite a bit of gotchas that can bite you along the way and even now, some things can still trip me up if I’m not paying attention to what I’m doing. Read below under Final Thoughts for the great benefits that I have found from using MikroTik and for those TLDR enthusiasts.

Even though this device is not meant to be running Linux, I’m going to be tough during this review.

Setup process

It’s extremely easy. Enter in the script, read what its outputting, click a few keys, voila, you have a running Asahi Linux system that dual-boots with MacOS. Ok, its not as simple as that but one through ten where 10 is installing Gentoo its like a two. The only gotchas that occur are when you don’t read the final output and your system goes into an infinite boot loop which is remedied by simply resetting and holding the power button until it tells you it is booting into config mode.

In my previous article of https://blog.lanlocked.xyz/data-hoarding-in-an-rv/, I documented how I was using a Synology DS1520+ as my NAS. After fighting with it and the main array of weird issues Synology desires to implement, which I won’t get into here, I have recently sold the setup and gone with a much more robust system, or systems, in my RV.

I have built three new server systems, but first lets start with what hardware I am currently running.

I wanted to show a little firewall rule that I setup that will allow an entity to attempt to login via SSH or Winbox and have them locked out for 21 days after so many attempts. You can tweak this to allow for more attempts and or have them locked out indefinably. This might be considered a tar pit for those that know that term except this is a manual tar pit that acts a little differently than the one MikroTik has built-in.

Updating gitea is a simple as replacing the binary in /usr/local/bin/gitea.

0. Stop the gitea service

$ sudo systemctl stop gitea

1. Move the current binary from gitea to gitea.old just in case things break.

$ sudo mv /usr/local/bin/gitea /usr/local/bin/gitea.old

2. Use wget to get the the current version: example -> https://github.com/go-gitea/gitea/releases/download/v1.20.2/gitea-1.20.2-linux-amd64. This is not the latest version, this is just an example.

cd ~/Downloads && wget https://github.com/go-gitea/gitea/releases/download/v1.20.2/gitea-1.20.2-linux-amd64

3. Change the name of the file from gitean-x.x.x-linux-amd to gitea

$ mv gitea-x.x.x-linux-amd gitea

I have basically gone this whole entire setup just to get synchronization on AntennaPod and my phone/devices. Is it worth it? Not really as you can do the same with exporting the backup the official AntennaPod to another phone or use an OPML file to do the same for other podcast clients, but its a fun project.

Now that we have Nextcloud up and running now we can install the GPodder Sync app.

After much research and trial and error I have found the gpodder implementation to be very terrible. The only known way to get synchronization across devices is to setup a Nextcloud server and install the plugin/addin gpodder. The other ways that I have tested but could not get any to work due to broken links and or not enough information are with micro-gpodder-server and gpodder2go. Both failed miserably.

The official link/website and how-to from gpodder.net called mygpodder has a broken link and requires various weird ways to get it to work using sqlite3, cargo, npm, etc. The cargo program is run in .exe making me think they have this running on windows and not on Linux.

I tried to use sed to rename filenames but found it rather cumbersome. sed is great when used to edit the content of a file which you can find some examples I created here, but I found this handy program that can do the same but for filenames instead.

Installed on my Arch, BTW, system I already had the program called rename which is not the same as the one we need called perl-rename. To install it I just ran it using pacman -S perl-rename. From here I was then able to rename a whole ton of files that I had scraped from the internet, cough cough youtube, quite easily.

Editor’s Note GrapheneOS has placed the Pixel 4 line in legacy extended support which is great news as I believe many people complained and or threw money at the GrapheneOS creators.

Unfortunately, my Pixel 4 XL that I have been using with GrapheneOS for around two years now is End-of-Life. All the Pixel 4 range except the Pixel 4a(5G) will no longer be getting any extended support nor security updates as of Oct 1, 2023. This sucks as the Pixel 4 XL has been the best phone I have ever used and I do not want to give it up. I will be looking for alternative ROMs to be installing but finding one that offered all the great things GrapheneOS does is going to be hard.

Credit Where Credit Is Due

I used a lot of https://old.jamesthebard.net/archlinux-and-u2f-login/ config but then tweaked it and added a litle more explination as his entry was from 2017~.

Let’s Start the Rant

I have always wanted a way to get away from always having to type in password to login or use sudo. There are security concerns by allowing this type of login, one is the fact that anyone who possesses my key could then login and run as sudo, also is the factor that with some tweaking you can make it only allow for key entry via u2f :lock:. A third way, which is the most secure way, is to have it where you must type in the password and possess the u2f hardware key to login.

Due to the higher complexities of running a modern website or blog securely or rather, for making it easier for people via CloudFlare and the like, I have found that when one of those main sites are down, again like CloudFlare, sites that rely on their DNS redirecting and whatnot, go down too. Many blogs that I follow are starting to jump on that band-wagon which is annoying since it can be quite a bit of time for a site to return to working order whether it was on CloudFlare’s side or the blog owner. Regardless of this I have looked into ways to have backup copies of their sites while this is still an option.