Skip to content
On this page

Amazon AWS New Setup

Manual Setup

Fedora

  • readme_backup_aws_git.md
  • todo checkout Cloudflare Zero again
  • cloudify

Must Haves

  • [ ] zsh / oh-my-zsh
  • [ ] setup ssh keys
  • [ ] pnpm / node / nvm
  • [ ] nginx / certBot

Usecases

  • [ ] DB / Docker / Redis
  • [ ] pihole
  • [ ] VSCode
  • [ ] Python / Conda (Mamba)
  • Puppeteer / Chromium
  • GDM? / X11? / Wayland?
bash
ssh dm5 #ec2-user
sudo dnf update #
sudo dnf install tree git zsh -y #

# todo move and inject:
# Delete if exists / continue
sudo dnf update
#sudo dnf install zsh tree git curl -y # curl might not work...
# curl already exists
sudo dnf install zsh tree git -y && \
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" && \
git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions

sed -i 's/plugins=(git)/plugins=(git zsh-autosuggestions)/' ~/.zshrc
exec zsh -l


sudo dnf install util-linux-user -y #chsh
chsh -s $(which zsh)

# todo also find and merge
ssh-keygen
ssh-keygen -t ed25519


## add aliases / using code i guess would be easier


###
# todo ssh key to repos / or copy one with passphrase

# save workspace

dev utils

bash
df -h
dnf list installed
cat /etc/os-release |  grep ID_LIKE

Timezone

East Coast

bash
sudo timedatectl set-timezone America/Toronto
bash
sudo timedatectl set-timezone America/New_York

AWS Calculators

https://aws.amazon.com/ec2/instance-types/

Nginx

bash
# Repeated in Nginx - Debug
dnf search nginx
dnf search --showduplicates nginx-core

sudo dnf install nginx -y #
sudo systemctl start nginx # 
sudo nginx -T
sudo nginx -t

## Nginx Status
systemctl status nginx #
systemctl start nginx #
nginx -t&&systemctl reload nginx # test before reload
systemctl reload nginx #
systemctl stop nginx #

systemctl is-enabled nginx #
systemctl enable nginx #
systemctl disable nginx #

# Folder struct
tree /etc/nginx/
nginx -t
nginx -T

tree /usr/share/nginx/ # recommend making folders here
tree /usr/share/nginx/html

tree /var/log/nginx
tail -f /var/log/nginx/error.log
tail -f /var/log/nginx/access.log

# try default.d or conf.d
# cd /etc/nginx/default.d # sites-aval?

sudo su #
cd /etc/nginx/conf.d # sites enabled? has to be .conf
cat > /etc/nginx/conf.d/cfw-old.conf # then certbot is wrong user... not using root is annoying
# old school way? should just use ln -sf /etc/nginx/sites-available.d/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf
# old school way? should just use ln -sf /etc/nginx/sites-available/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf
## official it's sites-available and sites-enabled, but i don't see that in the conf anymore
http://ec2-54-86-4-35.compute-1.amazonaws.com/


# maybe some nginx shenigans?





# ahh old sshd 8.7
# yes / no; various of without-password / prohibit-password
sudo sshd -T | grep -i permitrootlogin



# for gitjc... use gobert real quick to rsync over?
# or just use vscode as root?

sudo code -r /etc/nginx/conf.d/sameple.com.conf






# https://snapcraft.io/docs/installing-snap-on-fedora
## didnt work
# https://eff-certbot.readthedocs.io/en/stable/install.html
# https://certbot.eff.org/instructions?ws=nginx&os=pip

# todo document these steps...
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh -b

~/mambaforge/bin/conda init bash
~/mambaforge/bin/conda init zsh
python3 -m pip install --user pipx
python3 -m pipx ensurepath


# zsh manually for opnsense
# https://forum.opnsense.org/index.php?topic=25063.0

# if conda installed to bash instead of zsh (prefered)
~/.local/bin/pipx ensurepath
nano ~/.zshrc
export PATH=$HOME/.local/bin:$HOME/bin:/usr/local/bin:$PATH

# https://pypa.github.io/pipx/docs/
# pipx uninstall-all
pipx install certbot-nginx --include-deps

# didnt have to do this as root..
# sudo certbot --nginx
certbot --nginx

# todo Set up automatic renewal

echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

sudo /opt/certbot/bin/pip install --upgrade certbot certbot-nginx

Crontab

bash
crontab -l

crontab -u tstark -l

# root 
sudo crontab -e

ls /etc/cron.*
ls /etc/cron*
ls /etc/cron.d
cat /etc/crontab # system file
# cat /etc/cron.*
tree /etc/cron.*
ls /var/spool/cron # doesnt exist

# editor
EDITOR='code' crontab -e

# create users command... / or just use ec2-user
sudo certbot renew --dry-run
The command to renew certbot is installed in one of the following locations:

/etc/crontab/
/etc/cron.*/*
systemctl list-timers
# Timer
/etc/cron.monthly
systemctl status sshd
systemctl list-timers --all # shows inactive timers

## timer tut
# https://advancedweb.hu/how-to-write-a-simple-systemd-timer/

ls -l /etc/systemd/system/ |grep "timer$" #
ls -l /etc/systemd/system/ |grep "target$" #

nginx here

## https://documentation.suse.com/smart/systems-management/html/systemd-working-with-timers/index.html
# bit bettert tut
* check chpt

certbot --version

# sudo not found
certbot renew --dry-run

sudo /root/.local/bin/certbot renew --dry-run
# uprgade would just be thorugh pipx
# interesting pipx ignores conda? with --user i guess? and not through conda?
# or without the --user flag?

Timers testing?:

bash
[Unit]
Description=Syncthing monitoring

[Timer]
OnBootSec=5m
OnUnitActiveSec=1h

[Install]
WantedBy=timers.target
#####
[Unit]
Description=OpenBSD Secure Shell server
Documentation=man:sshd(8) man:sshd_config(5)
After=network.target auditd.service
ConditionPathExists=!/etc/ssh/sshd_not_to_be_run

[Service]
EnvironmentFile=-/etc/default/ssh
ExecStartPre=/usr/sbin/sshd -t
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/usr/sbin/sshd -t
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartPreventExitStatus=255
Type=notify
RuntimeDirectory=sshd
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target
Alias=sshd.service

move away docker

bash
# https://docs.docker.com/desktop/install/linux-install/#kvm-virtualization-support
sudo modprobe kvm
sudo modprobe kvm kvm_intel
sudo modprobe kvm kvm_amd
sudo kvm-ok

NVM Install Shell

bash
#!/usr/bin/env bash
# https://github.com/nvm-sh/nvm?tab=readme-ov-file#about'
# check for latest version
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
nvm install --lts
nvm use --lts

PNPM / Node

  • was so easy
  • nvm still better for installing node
bash
# as root:
curl -fsSL https://get.pnpm.io/install.sh | sh - #
# might need to use bash or / zsh
# install node 18
# https://pnpm.io/cli/env

# for some reason pnpm has to be installed as root?
# so annoying in that regards until they fix it
# nvm still seems better?
# pnpm hangs?

# nvm works and doesnt install to root
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
nvm install --lts

pnpm install-completion zsh

/home/ec2-user/.local/share/pnpm/pnpm env use --global lts
pnpm env use -g-lobal lts
pnpm env list


# probably better to not use root? only for file access?
## let's try that again with a fresh install later
## or just add write access to the user? sudo rsync -u?

pnpm i -g pm2
pm2 completion install
# pm2 init #?
pm2 startup #copy the command

# maybe use build? iunno

Nginx

bash
bash -l
zsh -l
cd /tmp
code cfw-old.conf # code /tmp doesnt work for w/e reason
cat /tmp/cfw-old.conf #
rsync -uv /tmp/cfw-old.conf gitjc:/tmp #

ssh gitjc 'sudo cp /tmp/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf&&sudo nginx -t&&sudo systemctl reload nginx' #

sudo cp /tmp/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf&&sudo nginx -t&&sudo systemctl reload nginx #

sudo systemctl start nginx #
sudo systemctl enable nginx #
sudo systemctl is-enabled nginx #

sudo systemctl status nginx #

sudo nginx -T

sudo cat /var/log/nginx/error.log
sudo cat /var/log/nginx/access.log
sudo cat /var/log/nginx/access.log |grep -i code

dig tmp.passionarte.ca +short
dig ec2-54-86-4-35.compute-1.amazonaws.com +short
54.86.4.35 # temporary public ip
dig v.codeforwings.com +short
curl -k http://tmp.passionarte.ca/ #
curl -k http://tmp.passionarte.ca/index.shtml #
curl -k https://tmp.passionarte.ca #
curl -k https://tmp.passionarte.ca/index.shtml #
curl -k https://tmp.passionarte.ca/index.shtml #

curl -k http://ec2-54-86-4-35.compute-1.amazonaws.com
curl -k http://ec2-54-86-4-35.compute-1.amazonaws.com/index.shtml
curl -k http://ec2-54-86-4-35.compute-1.amazonaws.com/index.shtml |grep -iw passionarte

curl -k https://v.codeforwings.com/ #
curl -k https://v.codeforwings.com/index.shtml #

sudo tail -f /var/log/nginx/access.log #

sudo tree /var/log/nginx/
sudo systemctl restart nginx #
sudo systemctl status nginx #



http://54.86.4.35:3001
http://54.86.4.35:3002


# debug:
# mkdir -p ~/tmp
# rsync -u /tmp/cfw-old.conf gitjc:tmp #

* still 187 mib left with pm2 etc

Goal

  • setup dev image? maybe make ami
  • setup dev db
  • make more sense to just pay for codespaces in the future
  • migrate static websites out as well
  • docker?

Potential folder struct?

bash
/var/log
/var/spool (to process)
/var/tmp

/usr/bin: Common location for user commands.
/usr/lib and /usr/lib64: Libraries for the binaries in /usr/bin and /usr/sbin.
/etc: Configuration files.
/usr/local: Locally installed software (often those compiled from source).
/opt: Optional or third-party software.

Domain name

linux groups commands

bash
getent group
# list users in group
getent group users
getent group adm
getent group sudo
# deb like

# chmod a+x same as chmod +x
chmod u+x 
chmod g+x 
chmod o+x 

chmod a+x 
chmod +x 

# get file permissions
## numeric permissions (octal notation)
stat -c '%a' /opt # 755
stat -c '%A' /opt # drwxr-xr-x

# for mac
stat -f '%A' /opt # drwxr-xr-x

Filesystem Hierarchy Standard (FHS)

md
/opt: This directory is reserved for the installation of add-on (optional) application software packages. A package placing files in the /opt directory would typically use a subdirectory that is named after the package or the provider of the package. For instance, you might have /opt/myapp for a custom application named "myapp".

/srv: This directory contains site-specific data that is served by the system. This could be web pages for a web server, files for an FTP server, or other data that is served by applications. The structure within /srv is left up to the system administrator but should be organized in a manner that reflects the purpose and the service being provided.

Link / Junction Mnemonics

md
* i.e. windows works like cp. and ln -s is reversed in a way...
* verbally it it's create link At/From to Target/Destination. which makes sense since it's `link -> target`
| System/Tool  | Command         | Parameter 1       | Parameter 2         |
|--------------|-----------------|-------------------|---------------------|
| Unix-like    | `ln -s`         | `"target_path"`   | `"link_name"`       |
| Unix-like    | `cp`            | `"source_path"`   | `"target_path"`     |
| Unix-like    | `cp`            | `"source_path"`   | `"destination_path"`|
| Windows      | `mklink /J`     | `"link_name"`     | `"target_path"`     |
| Sysinternals | `junction`      | `"link_name"`     | `"target_path"`     |

Java JAR Sample BFG

  • git big file cleaner
bash
tree ~/.local/bin
unlink ~/.local/bin/bfg
# sudo echo '#!/usr/bin/env java -jar' | sudo tee bfg.sh
cd /opt
sudo wget https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar
sudo apt install -y default-jre # 180 mb lol
sudo echo '#!/bin/sh' | sudo tee bfg.sh
sudo chmod +x bfg.sh
sudo echo 'exec java -jar /srv/shared/downloads/bfg-1.14.0.jar $@' | sudo tee -a bfg.sh
./bfg.sh --version
ln -s /opt ~/.local/bin/bfg
bfg --version

bfg --delete-files badfile.txt myrepo.git