This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Linux Topics

1 - Archlinux Installation

Bios Settings

  • set the BIOS clock to GMT time zone
  • turn all virtualization optiona on

Keyboard Settings

Connect to W-Lan (TODO)

Disks

Create LUKS

cryptsetup luksFormat /dev/nvme0n1p2
cryptsetup open /dev/nvme0n1p2 cryptroot

use lsblk to check if cryptroot is created

Create and mount Filesystems

see also: https://wiki.archlinux.org/index.php/EFI_system_partition#Format_the_partition

mkfs.ext4 /dev/mapper/cryptroot
mount /dev/mapper/cryptroot /mnt
mkfs.fat -F 32 /dev/nvme0n1p1
mkdir /mnt/boot
mount /dev/nvme0n1p1 /mnt/boot

Create Swap File (did not do this)

see also: https://wiki.archlinux.org/index.php/Swap#Swap_file_creation

  • TODO: Link zu Ubuntu Seite
dd if=/dev/zero of=/swapfile bs=1M count=20480 status=progress
chmod 600 /mnt/swapfile
mkswap /mnt/swapfile
swapon /mnt/swapfile

To resume from swap see: https://wiki.archlinux.org/index.php/Power_management/Suspend_and_hibernate#Hibernation_into_swap_file

Select Mirrors

nano /etc/pacman.d/mirrorlist

Install Packages

pacstrap /mnt base linux linux-firmware
pacstrap /mnt nano gnome cryptsetup networkmanager
pacstrap /mnt efibootmgr dosfstools gptfdisk

# gen and check fstab
genfstab -U /mnt >> /mnt/etc/fstab
less /mnt/etc/fstab

#Chroot
arch-chroot /mnt

#Time zone
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
hwclock --systohc

nano /etc/locale.gen
- remove comment for de_DE UTF-8
- remove comment for en_US UTF-8
locale-gen
nano /etc/locale.conf

# see: http://www.gentoo.org/doc/en/guide-localization.xml#doc_chap3
LANG="en_US.UTF-8"
LC_TIME="de_DE.UTF-8"

nano /etc/vconsole.conf
KEYMAP=de-latin1-nodeadkeys

nano /etc/hostname
- enter hostname

nano /etc/hosts

127.0.0.1	localhost
::1		localhost
127.0.1.1	myhostname

nano /etc/mkinitcpio.conf

HOOKS=(base udev autodetect modconf block keyboard keymap encrypt filesystems fsck)


- see https://wiki.archlinux.org/title/Dm-crypt/System_configuration#mkinitcpio
- added after "block" -> "keyboard keymap encrypt"
- no `consolefont`

mkinitcpio -P

passwd

# install systemd-boot
bootctl install

systemd-boot does not accept tabs for indentation, use spaces instead.

nano /boot/loader/entries/arch.conf

title Arch Linux linux /vmlinuz-linux initrd /initramfs-linux.img options cryptdevice=UUID=701fea32-236d-49fd-a0f9-cb9f07c94703:cryptroot root=/dev/mapper/cryptroot rw


use cat to add UUID to file:
use the UUID of the base encryped partition ! NOT cryptroot

blkid >> /boot/loader/entries/arch-uefi.conf

- see https://wiki.archlinux.org/title/Systemd-boot#Adding_loaders
- see https://wiki.archlinux.org/title/Dm-crypt/Encrypting_an_entire_system#Configuring_the_boot_loader

nano /boot/loader/entries/arch-uefi-fallback.conf

nano /boot/loader/loader.conf

#timeout 3 #console-mode keep default arch.conf console-mode max editor no timeout 1


bootctl update

exit
cd /
umount -R /mnt
reboot

# create user
useradd <username> -m -s /bin/bash
passwd <username>

#gnome aktivieren
systemctl enable gdm.service
systemctl enable NetworkManager

reboot

# check heyboard layout at login!

# Gnome Settigs
- Keyboard
  - add German
  - remove en
- Mouse & Touchpad
  - Natural scolling aus
  - Tap to Click an
- check "Region & Language"
  - alle settings prüfen und synchronisieren
  - both tabs -> Language: English, Formats: all Deutschland
- Power
  - Automatic Suspend - on battery power und Plugged in aus
  - show percent


- als root ausführen: `localectl set-x11-keymap de`


# microcode
# https://wiki.archlinux.org/index.php/Microcode
pacman -S amd-ucode
# add in boot config
initrd  /cpu_manufacturer-ucode.img

# fstrim
systemctl enable fstrim.timer

# swappiness
# https://wiki.archlinux.org/index.php/Swap#Swappiness
nano /etc/sysctl.d/99-swappiness.conf

vm.swappiness=10

# sudo
pacman -S sudo
EDITOR=nano visudo
%wheel      ALL=(ALL) ALL
gpasswd -a <username> wheel

nano /etc/mkinitcpio.conf
MODULES=(amdgpu)
mkinitcpio -P

# enable aur
pacman -S --needed base-devel
nano /etc/makepkg.conf

Lenovo ThinkPad T495x specific

Mask this to avoid Boot Error

also see: https://wiki.archlinux.org/index.php/Lenovo_ThinkPad_T495s#Backlight

systemctl mask systemd-backlight@backlight:acpi_video0.service

2 - Archlinux on Hetzner Cloud with Btrfs

# boot rescue system
# ssh into rescue system

# download current bootstrap image from https://mirror.chaoticum.net/arch/iso/latest/
wget https://mirror.chaoticum.net/arch/iso/latest/archlinux-bootstrap-<date>-x86_64.tar.gz
wget https://mirror.chaoticum.net/arch/iso/latest/archlinux-bootstrap-<date>-x86_64.tar.gz.sig

# check PGP fingerprint: 0x54449A5C

# check signature
gpg --keyserver keyserver.ubuntu.com --keyserver-options auto-key-retrieve --verify archlinux-bootstrap-<date>-x86_64.tar.gz.sig

# --numeric-owner preserve UID and GID of files in case existing Linux system uses different numbers than Arch
# see https://wiki.archlinux.org/title/Install_Arch_Linux_from_existing_Linux#Method_A:_Using_the_bootstrap_tarball_(recommended)
tar xvfz archlinux-bootstrap-2023.01.01-x86_64.tar.gz --numeric-owner

# hack - see https://wiki.archlinux.org/title/Install_Arch_Linux_from_existing_Linux#Downloading_basic_tools
mount --bind root.x86_64 root.x86_64

./root.x86_64/usr/bin/arch-chroot root.x86_64
echo 'Server = https://mirror.chaoticum.net/arch/$repo/os/$arch' > /etc/pacman.d/mirrorlist
pacman-key --init
pacman-key --populate archlinux

# refresh package lists
pacman -Syyu

# install nano
pacman -S nano

# clean disk
blkdiscard -f /dev/sda

# fdisk
fdisk /dev/sda
# Disklabel type: gpt
# Device     Start      End  Sectors  Size Type
# /dev/sda1   2048     4095     2048    1M BIOS boot
# /dev/sda2   4096 39999487 39995392 19.1G Linux filesystem

# install Btrfs tools
pacman -S btrfs-progs

# Btrfs filesystem
mkfs.btrfs /dev/sda2
mount -o compress=zstd /dev/sda2 /mnt
btrfs subvolume create /mnt/@
btrfs subvolume create /mnt/@home
btrfs subvolume create /mnt/@snapshots
btrfs subvolume create /mnt/@var_log
umount /mnt
mount -o compress=zstd,subvol=@ /dev/sda2 /mnt
mkdir /mnt/home
mount -o compress=zstd,subvol=@home /dev/sda2 /mnt/home
mkdir -p /mnt/var/log
mount -o compress=zstd,subvol=@var_log /dev/sda2 /mnt/var/log

# what about
# /var/cache/pacman/pkg or just /var/cache/pacman
# /var/lib/portables
# /var/lib/machines

# more mount options https://wiki.archlinux.org/title/Security#Mount_options

# pacstrap with -M option:
# -M is to "Avoid copying the host’s mirrorlist to the target."
pacstrap -G -M /mnt base grub linux linux-firmware openssh nano btrfs-progs

genfstab -U /mnt >> /mnt/etc/fstab
nano /mnt/etc/fstab
# remove all space_cache and subvolid entries

arch-chroot /mnt
ln -sf /usr/share/zoneinfo/Europe/Berlin /etc/localtime
hwclock --systohc
nano /etc/locale.gen
# uncomment en_US.UTF-8 UTF-8
locale-gen
nano /etc/locale.conf
# add LANG=en_US.UTF-8
nano /etc/vconsole.conf
# add KEYMAP=de-latin1
nano /etc/hostname
# add hostname

# Btrfs
nano /etc/mkinitcpio.conf
add btrfs to MODULES

mkinitcpio -P

# set root password
passwd

# install GRUB
# use /dev/sda not /dev/sda1 or /dev/sda2
grub-install --target=i386-pc /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg

# systemd-networkd
nano /etc/systemd/network/20-wired.network
# add
[Match]
Type=ether

[Network]
DHCP=yes

# enable network
systemctl enable systemd-networkd
systemctl enable systemd-resolved

# enable sshd
systemctl enable sshd

useradd mike -m -s /bin/bash
passwd mike

# logout & reboot
exit
exit
reboot

Post Install

Pacman

nano /etc/pacman.conf
# uncomment Color, VerbosePkgLists and ParallelDownloads

# init pacman
nano /etc/pacman.d/mirrorlist

pacman-key --init
pacman-key --populate archlinux

# refresh package lists
pacman -Syyu

Install snapper

pacman -S snapper
snapper -c root create-config /
btrfs subvolume delete /.snapshots
mkdir /.snapshots
# edit /etc/fstab and mount /@snapshots to /.snapshots
mount /.snapshots
# reboot and see if all is mounted

# first manual test snapshot
snapper -c root create --description test01

# check if snapshot is stored
ls -ls /.snapshots

# install snap-pac to let pacman automatically use snapper to create pre/post snapshots
# install grub-btrfs to be able to boot from snapshots
pacman -S snap-pac grub-btrfs

# the grub main menu needs to be generated to make a menuentry for the snapshots sub menu
# see https://github.com/Antynea/grub-btrfs#-usage
grub-mkconfig -o /boot/grub/grub.cfg

# enable daemon script that automatically updates the grub menu when it sees a snapshot being created or deleted
# see https://github.com/Antynea/grub-btrfs#systemd-instructions
systemctl enable --now grub-btrfsd

Other

# enable periodic TRIM
# see https://wiki.archlinux.org/title/Solid_state_drive#Periodic_TRIM
systemctl enable --now fstrim.timer

Other Post Install Todo

3 - Btrfs

Tasks

  • check free space: btrfs filesystem usage <path>
  • allocation of block group types: btrfs filesystem df <path> (supports --format json)
  • device stats (io errors): btrfs device stats <path> (supports --format json)
  • show subvolumes: btrfs sub list <path>

Check redundancy (after a replaced disk)

Commands

balance

replace

Other Commands

# create raid1
mkfs.btrfs -m raid1 -d raid1 /dev/sda1 /dev/sdb1

# mount
mkdir /mnt/btrfs
mount -o compress=zstd /dev/sda1 /mnt/btrfs

# scrub
btrfs scrub start /mnt/btrfs
btrfs scrub status /mnt/btrfs

# show filesystem
btrfs filesystem show

# pull one drive and scrub + show

# replace disk 1
btrfs replace start 1 /dev/sdc1 /mnt/btrfs -f
btrfs replace status /mnt/btrfs

# device stats
btrfs device stats /mnt/btrfs

4 - Disk and Partition Management

Useful Commands

  • get disk UUID (for fstab): blkid
  • filesystem check: e2fsck -vf /dev/<disk>
  • delete all filesystems on disk: wipefs -a /dev/<disk>
  • delete all GPT and MBR data structures: sgdisk --zap-all /dev/<disk>

Mount

  • mount disk or patition: mount /dev/<disk> /mnt
  • mount everything defined in /etc/fstab: mount -a

Backup with dd

  • https://wiki.archlinux.org/title/Dd
  • backup a disk (or snapshot) to a file: dd if=/dev/<disk> bs=64K status=progress | gzip -c > <backup_name>.img.gz
  • backup a disk (or snapshot) to a file (with parallel gzip): dd if=/dev/<disk> bs=64K status=progress | pigz -c > <backup_name>.img.gz
  • restore a backup to disk: gunzip -c <backup_name>.img.gz | dd of=/dev/<disk>

LVM

Creation Commands

  • create the pv and the vg in a primary partition and not a bare disk
  • create physical volume: pvcreate /dev/<partition>
  • create volume group: vgcreate <volume_group_name> /dev/<partition> (same <partition> as for the physical volume)
  • create logical volume with fixed size: lvcreate -L <size>G <volume_group_name> -n <logical_volume_name>
  • create logical volume with percentage of free size (100 for remaining size): lvcreate -l <size_in_percent>%FREE <volume_group_name> -n <logical_volume_name>
  • format with ext4: mkfs.ext4 /dev/<volume_group_name>/<logical_volume_name>

List Commands

  • list physical volumes: pvs
  • list volume groups: vgs
  • list logical volumes: lvs
  • details about volume groups (including allocated and remaining space): vgdisplay
  • details about logical volumes: lvdisplay

Snapshots

  • https://wiki.archlinux.org/title/LVM#Snapshots
  • create snapshot: lvcreate --size <size>G --snapshot --name <snapshot_name> /dev/<volume_group_name>/<logical_volume_name>
  • revert to state of snapshot: lvconvert --merge /dev/<volume_group_name>/<logical_volume_name>
  • delete snapshot: lvremove /dev/<volume_group_name>/<snapshot_name>

Recover from Image of Snapshot

  • create snapshot of disk to recover
    • that snapshot must be a bit larget than the disk
    • if it is smaller or same size this might happen at merge time: Unable to merge invalidated snapshot LV vg1/restore_test
    • “[…] a small amount of the space you allocate to the snapshot is used to track the locations of the chunks of data, so you should allocate slightly more space than you actually need […]” - see https://linux.die.net/man/8/lvcreate
  • use dd to copy image to snapshot
  • merge snapshot

Enlarge a Logical Volume

  • https://wiki.archlinux.org/title/LVM#Resize_physical_volume
  • https://wiki.archlinux.org/title/Resizing_LVM-on-LUKS
  • add size and resize filesystem in one command: lvresize -L +<additional_size>G --resizefs <volume_group_name>/<logical_volume_name>
  • set new size and resize filesystem in one command: lvresize -L <additional_size>G --resizefs <volume_group_name>/<logical_volume_name>
  • change size in two steps:
    • add size logical volume: lvresize -L +<size>G <volume_group_name>/<logical_volume_name>
    • set new size of logical volume: lvresize -L <size>G <volume_group_name>/<logical_volume_name>
    • resize filesystem: resize2fs /dev/<volume_group_name>/<logical_volume_name>

Other Disk Tools / Commands

  • see disk activity
    • Debian: apt-get install iotop
    • command: iotop -o -P -d 2
  • remove Kubernetes filesystems from df: df -x overlay -x tmpfs -h

5 - File & Disk Tools

Remove Things

  • remove all comments from file: sed -e '/^#/d' <input_file> > <output_file>
  • remove all blank lines from file: awk '!/^$/' <input_file> > <output_file>

Split a File

This will not split single lines:

split --line-bytes=<size> <file_to_split>

The size argument is an integer and optional unit (example: 10K is 10*1024). Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,… (powers of 1000).

Un-Split a File

cat <file_to_split>-part-* > <file_to_split>

Remove Duplicates

sort <file> | uniq -u > <new_file>

Delete Disk

shred -vn 1 /dev/<device>

Checksums

  • calculate checksum: sha512sum <file>
  • store checksums for multiple files: sha512sum * > sha512sum.txt
  • check checksums for multiple files: sha512sum -c sha512sum.txt

6 - File Compression

tgz

  • compress
    • standard: tar cvfz <target_file>.tgz <source_files_or_dir>
    • fast: tar -I pigz -cvf <target_file>.tgz <source_files_or_dir>
    • fast with best compression: tar -I "pigz --best" -cvf <target_file>.tgz <source_files_or_dir>
  • view files in archive: tar -tf <filename>.tgz
  • test archive
    • standard: gunzip -t <filename>.tgz
    • fast: pigz -t <filename>.tgz

zip

  • test a zip archive: unzip -t <filename>.zip

bz2

  • unpack: bzip2 -dk <filename>.bz2

gzip

  • test .gz files: gzip -v -t <file>.gz

7 - Linux Commands

User Management

Hardware

  • get info about GPU: lshw -C display
  • get infos about CPU: cat /proc/cpuinfo
  • get infos about CPU temperature and fans
  • infos about USB: lsusb
  • details about USB: lsusb -vvv

Proxy Handling

SSH through a Proxy (to an EC2 instance in this example)

ssh -i <key_file>.pem <user>@<target_host_or_ip> -o "ProxyCommand=nc -X connect -x <proxy_ip>:<proxy_port> %h %p"

SCP through a Proxy (to an EC2 instance in this example)

scp -i ~/.ssh/<key_file>.pem -o "ProxyCommand=nc -X connect -x <proxy_ip>:<proxy_port> %h %p" <file> <user>@<target_host_or_ip>:

Special

  • output to terminal and file: command | tee <filename>

Rotate Terminal

  • to the right: echo 1 | sudo tee /sys/class/graphics/fbcon/rotate_all
  • to the left: echo 3 | sudo tee /sys/class/graphics/fbcon/rotate_all

Administration

  • reboot with timer (5 minutes) and message: shutdown -r +5 "<message>"

Bash Aliases

alias ls='ls --color=auto -h'
alias la='ls -lA'
alias ll='ls -l'

8 - NixOS

Packages

ZFS

9 - Screen

Commands

  • open a session with name: screen -S <session_name>
  • list sessions: screen -ls
  • resume detached session: screen -r <session_name>

Recover a ’lost’ Screen Session

If your remote connection crashed while you had an open screen session you have to detach it before you can reconnect. You do it this way:

Reattach a session and if necessary detach it first.

screen -rd <session_name>

Rename a session

To rename a session do:

  1. Attach to the session in question
  2. Press ctrl + a
  3. Type :sessionname <your_session_name> – yes, the first colon is needed there, no extra spaces
  4. Type enter

10 - SSH

Commands

  • generate default RSA key: ssh-keygen
  • generate Ed25519 key: ssh-keygen -t ed25519
  • tell ssh to use password and not key auth: add option -o PreferredAuthentications=password

Config File

Default Config

The default config looks like this

Host <hostname> <ip>
   HostName <ip>
   User <username>
   IdentitiesOnly yes
   IdentityFile ~/.ssh/<sec_key_filename>

Using a Jump Server

If you have a jump server <jump_server_hostname> write a normal config for both servers and add this to the config of the target server: ProxyJump <jump_server_hostname>

If you also want to connect through an HTTP proxy add the proxy config lines to the config of the jump server.

Connect through HTTP proxy

For Linux add this:

   ProxyCommand nc -X connect -x <proxy_ip>:<proxy_port> %h %p
   ForwardAgent yes

For Windows (git bash) add this:

   ProxyCommand connect -H <proxy_ip>:<proxy_port> %h %p
   ForwardAgent yes

SSH Tunnel

11 - SSL

Create self-signed SSL Certificate

# generate private key
openssl genrsa -des3 -out server.pass.key 2048
# or
openssl genrsa -des3 -out server.pass.key 4096

# remove passphrase from Key
openssl rsa -in server.pass.key -out server.key
rm server.pass.key

# generate csr (certificate signing request)
openssl req -new -key server.key -out server.csr

# generate self-signed certificate
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
or
openssl x509 -req -days 36500 -in server.csr -signkey server.key -out server.crt

Create Client Certifcate that can be used by Firefox and other Browsers

openssl pkcs12 -export -in client.crt -inkey client.key -out client.p12

12 - Systemd

Commands and Locations

  • custom services are stored here: /etc/systemd/system
  • analyse security of service unit: systemd-analyze security <service_unit>
  • restart a service: systemctl restart <application_name>.service
  • show logs of a service: journalctl -u <application_name>.service

List Services

  • list enabled services: systemctl list-unit-files --state=enabled
  • list running services: systemctl list-units --type=service
  • list failed services: systemctl --failed
  • list timers: systemctl list-timers

List all since boot

  • all boot cycles: journalctl -k -b all
  • current boot : journalctl -k
  • last boot: journalctl -k -b -1
  • two boots prior: journalctl -k -b -2
  • and so on

Journal

  • delete old journal files (ony keep last 2 days): journalctl --vacuum-time=2d

13 - Ubuntu

Package Management

  • clean cache: apt clean
  • list installed packages: apt list --installed
  • list manually installed packages: apt-mark showmanual
  • remove package with dependencies: apt-get <remove/purge> --auto-remove <package_name>
  • list installed packages from a specific source: aptitude search "?origin (<package_source>) ?installed"

Update the System

# fetch list of available updates
apt update

# list packages that can be upgraded
apt list --upgradable

# install some updates - do not remove packages
apt upgrade
# or
# install updates - also remove packages if needed
apt full-upgrade

# remove any old packages no longer needed
apt autoremove

.deb Files

  • get info about a .deb file: dpkg -I <package_name>.deb
  • list Content of a .deb File: dpkg -c <package_name>.deb

Configuration

Commands

  • vheck ubuntu version: lsb_release -a

Install Ubuntu as a VirtualBox Guest

This describes how to install Ubuntu as a VirtualBox guest system to do experiments with docker or other stuff.

Basic installation

  • download a server version of Ubuntu
  • the current LTS version might be a good idea
  • do normal installation

Install XFCE

To be able to copy and paste to and from the guest system you need a desktop environment like XFCE. XFCE is small compared to GNOME and the minimal installation does not contain LibreOffice and other stuff you do not need.

  • install tasksel: sudo apt install tasksel
  • execute tasksel: sudo tasksel
  • select Xubuntu minimal installation
  • press tabulatur and select Ok with return key
  • reboot

Install VirtualBox Guest Additions

Install Docker

To install docker do not install the package called docker. Docker is a “System tray for KDE3/GNOME2 docklet applications”. The package you need is called docker.io

  • install with: sudo apt install docker.io
  • start with: sudo systemctl start docker
  • stop with: sudo systemctl stop docker
  • enable (always start at boot time) with: sudo systemctl enable docker

14 - ZFS

Commands

  • list drive ids: ls -l /dev/disk/by-id/
  • show status
    • list zpools: zpool list - shows raw capacity, to see real capacity use zfs list
    • see status of zpool: zpool status
  • mirror handling
    • create mirror: zpool create tank mirror <vdev_1> <vdev_2>
    • add mirror: zpool add tank mirror <vdev_1> <vdev_2>
    • remove mirror: zpool remove tank <mirror-id> - get mirror-id with zpool status
  • delete pool: zpool destroy tank
  • dataset handling
    • create dataset: zfs create <nameofzpool>/<nameofdataset>
    • list datasets: zfs list