Amazon AWS New Setup
Manual Setup
setup the instance through wizard
elastic ip
- Check speed / latency / Drive etc.
setup ssh config
Fedora
- readme_backup_aws_git.md
- todo checkout Cloudflare Zero again
- cloudify
Must Haves
- [ ] zsh / oh-my-zsh
- [ ] setup ssh keys
- [ ] pnpm / node / nvm
- [ ] nginx / certBot
Usecases
- [ ] DB / Docker / Redis
- [ ] pihole
- [ ] VSCode
- [ ] Python / Conda (Mamba)
- Puppeteer / Chromium
- GDM? / X11? / Wayland?
bash
ssh dm5 #ec2-user
sudo dnf update #
sudo dnf install tree git zsh -y #
# todo move and inject:
# Delete if exists / continue
sudo dnf update
#sudo dnf install zsh tree git curl -y # curl might not work...
# curl already exists
sudo dnf install zsh tree git -y && \
sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" && \
git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
sed -i 's/plugins=(git)/plugins=(git zsh-autosuggestions)/' ~/.zshrc
exec zsh -l
sudo dnf install util-linux-user -y #chsh
chsh -s $(which zsh)
# todo also find and merge
ssh-keygen
ssh-keygen -t ed25519
## add aliases / using code i guess would be easier
###
# todo ssh key to repos / or copy one with passphrase
# save workspace
dev utils
bash
df -h
dnf list installed
cat /etc/os-release | grep ID_LIKE
Timezone
East Coast
bash
sudo timedatectl set-timezone America/Toronto
bash
sudo timedatectl set-timezone America/New_York
AWS Calculators
https://aws.amazon.com/ec2/instance-types/
Nginx
- https://fedora.pkgs.org/37/fedora-x86_64/nginx-core-1.22.0-4.fc37.x86_64.rpm.html
- https://awswithatiq.com/how-to-install-nginx-in-amazon-linux-2023/
bash
# Repeated in Nginx - Debug
dnf search nginx
dnf search --showduplicates nginx-core
sudo dnf install nginx -y #
sudo systemctl start nginx #
sudo nginx -T
sudo nginx -t
## Nginx Status
systemctl status nginx #
systemctl start nginx #
nginx -t&&systemctl reload nginx # test before reload
systemctl reload nginx #
systemctl stop nginx #
systemctl is-enabled nginx #
systemctl enable nginx #
systemctl disable nginx #
# Folder struct
tree /etc/nginx/
nginx -t
nginx -T
tree /usr/share/nginx/ # recommend making folders here
tree /usr/share/nginx/html
tree /var/log/nginx
tail -f /var/log/nginx/error.log
tail -f /var/log/nginx/access.log
# try default.d or conf.d
# cd /etc/nginx/default.d # sites-aval?
sudo su #
cd /etc/nginx/conf.d # sites enabled? has to be .conf
cat > /etc/nginx/conf.d/cfw-old.conf # then certbot is wrong user... not using root is annoying
# old school way? should just use ln -sf /etc/nginx/sites-available.d/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf
# old school way? should just use ln -sf /etc/nginx/sites-available/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf
## official it's sites-available and sites-enabled, but i don't see that in the conf anymore
http://ec2-54-86-4-35.compute-1.amazonaws.com/
# maybe some nginx shenigans?
# ahh old sshd 8.7
# yes / no; various of without-password / prohibit-password
sudo sshd -T | grep -i permitrootlogin
# for gitjc... use gobert real quick to rsync over?
# or just use vscode as root?
sudo code -r /etc/nginx/conf.d/sameple.com.conf
# https://snapcraft.io/docs/installing-snap-on-fedora
## didnt work
# https://eff-certbot.readthedocs.io/en/stable/install.html
# https://certbot.eff.org/instructions?ws=nginx&os=pip
# todo document these steps...
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh -b
~/mambaforge/bin/conda init bash
~/mambaforge/bin/conda init zsh
python3 -m pip install --user pipx
python3 -m pipx ensurepath
# zsh manually for opnsense
# https://forum.opnsense.org/index.php?topic=25063.0
# if conda installed to bash instead of zsh (prefered)
~/.local/bin/pipx ensurepath
nano ~/.zshrc
export PATH=$HOME/.local/bin:$HOME/bin:/usr/local/bin:$PATH
# https://pypa.github.io/pipx/docs/
# pipx uninstall-all
pipx install certbot-nginx --include-deps
# didnt have to do this as root..
# sudo certbot --nginx
certbot --nginx
# todo Set up automatic renewal
echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null
sudo /opt/certbot/bin/pip install --upgrade certbot certbot-nginx
Crontab
bash
crontab -l
crontab -u tstark -l
# root
sudo crontab -e
ls /etc/cron.*
ls /etc/cron*
ls /etc/cron.d
cat /etc/crontab # system file
# cat /etc/cron.*
tree /etc/cron.*
ls /var/spool/cron # doesnt exist
# editor
EDITOR='code' crontab -e
# create users command... / or just use ec2-user
sudo certbot renew --dry-run
The command to renew certbot is installed in one of the following locations:
/etc/crontab/
/etc/cron.*/*
systemctl list-timers
# Timer
/etc/cron.monthly
systemctl status sshd
systemctl list-timers --all # shows inactive timers
## timer tut
# https://advancedweb.hu/how-to-write-a-simple-systemd-timer/
ls -l /etc/systemd/system/ |grep "timer$" #
ls -l /etc/systemd/system/ |grep "target$" #
nginx here
## https://documentation.suse.com/smart/systems-management/html/systemd-working-with-timers/index.html
# bit bettert tut
* check chpt
certbot --version
# sudo not found
certbot renew --dry-run
sudo /root/.local/bin/certbot renew --dry-run
# uprgade would just be thorugh pipx
# interesting pipx ignores conda? with --user i guess? and not through conda?
# or without the --user flag?
Timers testing?:
bash
[Unit]
Description=Syncthing monitoring
[Timer]
OnBootSec=5m
OnUnitActiveSec=1h
[Install]
WantedBy=timers.target
#####
[Unit]
Description=OpenBSD Secure Shell server
Documentation=man:sshd(8) man:sshd_config(5)
After=network.target auditd.service
ConditionPathExists=!/etc/ssh/sshd_not_to_be_run
[Service]
EnvironmentFile=-/etc/default/ssh
ExecStartPre=/usr/sbin/sshd -t
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/usr/sbin/sshd -t
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartPreventExitStatus=255
Type=notify
RuntimeDirectory=sshd
RuntimeDirectoryMode=0755
[Install]
WantedBy=multi-user.target
Alias=sshd.service
- https://certbot.eff.org/lets-encrypt/osx-nginx
- [ ] cloudflare / client certs
- 54.236.197.32 - drawtomusic.com
- 34.102.136.180 - jasonchan.app
move away docker
- https://docs.aws.amazon.com/linux/al2023/ug/install-docker.html
- just use native for now
bash
# https://docs.docker.com/desktop/install/linux-install/#kvm-virtualization-support
sudo modprobe kvm
sudo modprobe kvm kvm_intel
sudo modprobe kvm kvm_amd
sudo kvm-ok
NVM Install Shell
bash
#!/usr/bin/env bash
# https://github.com/nvm-sh/nvm?tab=readme-ov-file#about'
# check for latest version
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
nvm install --lts
nvm use --lts
PNPM / Node
- was so easy
- nvm still better for installing node
bash
# as root:
curl -fsSL https://get.pnpm.io/install.sh | sh - #
# might need to use bash or / zsh
# install node 18
# https://pnpm.io/cli/env
# for some reason pnpm has to be installed as root?
# so annoying in that regards until they fix it
# nvm still seems better?
# pnpm hangs?
# nvm works and doesnt install to root
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
nvm install --lts
pnpm install-completion zsh
/home/ec2-user/.local/share/pnpm/pnpm env use --global lts
pnpm env use -g-lobal lts
pnpm env list
# probably better to not use root? only for file access?
## let's try that again with a fresh install later
## or just add write access to the user? sudo rsync -u?
pnpm i -g pm2
pm2 completion install
# pm2 init #?
pm2 startup #copy the command
# maybe use build? iunno
Nginx
- remember to check the ssl on the cloudflare...
- needs flexible... not strict or anything
- https://dash.cloudflare.com/896d9011dd172ea4845b35581d07eb92/passionarte.ca/ssl-tls
bash
bash -l
zsh -l
cd /tmp
code cfw-old.conf # code /tmp doesnt work for w/e reason
cat /tmp/cfw-old.conf #
rsync -uv /tmp/cfw-old.conf gitjc:/tmp #
ssh gitjc 'sudo cp /tmp/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf&&sudo nginx -t&&sudo systemctl reload nginx' #
sudo cp /tmp/cfw-old.conf /etc/nginx/conf.d/cfw-old.conf&&sudo nginx -t&&sudo systemctl reload nginx #
sudo systemctl start nginx #
sudo systemctl enable nginx #
sudo systemctl is-enabled nginx #
sudo systemctl status nginx #
sudo nginx -T
sudo cat /var/log/nginx/error.log
sudo cat /var/log/nginx/access.log
sudo cat /var/log/nginx/access.log |grep -i code
dig tmp.passionarte.ca +short
dig ec2-54-86-4-35.compute-1.amazonaws.com +short
54.86.4.35 # temporary public ip
dig v.codeforwings.com +short
curl -k http://tmp.passionarte.ca/ #
curl -k http://tmp.passionarte.ca/index.shtml #
curl -k https://tmp.passionarte.ca #
curl -k https://tmp.passionarte.ca/index.shtml #
curl -k https://tmp.passionarte.ca/index.shtml #
curl -k http://ec2-54-86-4-35.compute-1.amazonaws.com
curl -k http://ec2-54-86-4-35.compute-1.amazonaws.com/index.shtml
curl -k http://ec2-54-86-4-35.compute-1.amazonaws.com/index.shtml |grep -iw passionarte
curl -k https://v.codeforwings.com/ #
curl -k https://v.codeforwings.com/index.shtml #
sudo tail -f /var/log/nginx/access.log #
sudo tree /var/log/nginx/
sudo systemctl restart nginx #
sudo systemctl status nginx #
http://54.86.4.35:3001
http://54.86.4.35:3002
# debug:
# mkdir -p ~/tmp
# rsync -u /tmp/cfw-old.conf gitjc:tmp #
* still 187 mib left with pm2 etc
Goal
- setup dev image? maybe make ami
- setup dev db
- make more sense to just pay for codespaces in the future
- migrate static websites out as well
- docker?
Potential folder struct?
bash
/var/log
/var/spool (to process)
/var/tmp
/usr/bin: Common location for user commands.
/usr/lib and /usr/lib64: Libraries for the binaries in /usr/bin and /usr/sbin.
/etc: Configuration files.
/usr/local: Locally installed software (often those compiled from source).
/opt: Optional or third-party software.
Domain name
- can have different domain name. interesting
- https://192.168.1.1/services_dhcp.php?if=lan
linux groups commands
bash
getent group
# list users in group
getent group users
getent group adm
getent group sudo
# deb like
# chmod a+x same as chmod +x
chmod u+x
chmod g+x
chmod o+x
chmod a+x
chmod +x
# get file permissions
## numeric permissions (octal notation)
stat -c '%a' /opt # 755
stat -c '%A' /opt # drwxr-xr-x
# for mac
stat -f '%A' /opt # drwxr-xr-x
Filesystem Hierarchy Standard (FHS)
md
/opt: This directory is reserved for the installation of add-on (optional) application software packages. A package placing files in the /opt directory would typically use a subdirectory that is named after the package or the provider of the package. For instance, you might have /opt/myapp for a custom application named "myapp".
/srv: This directory contains site-specific data that is served by the system. This could be web pages for a web server, files for an FTP server, or other data that is served by applications. The structure within /srv is left up to the system administrator but should be organized in a manner that reflects the purpose and the service being provided.
Link / Junction Mnemonics
md
* i.e. windows works like cp. and ln -s is reversed in a way...
* verbally it it's create link At/From to Target/Destination. which makes sense since it's `link -> target`
| System/Tool | Command | Parameter 1 | Parameter 2 |
|--------------|-----------------|-------------------|---------------------|
| Unix-like | `ln -s` | `"target_path"` | `"link_name"` |
| Unix-like | `cp` | `"source_path"` | `"target_path"` |
| Unix-like | `cp` | `"source_path"` | `"destination_path"`|
| Windows | `mklink /J` | `"link_name"` | `"target_path"` |
| Sysinternals | `junction` | `"link_name"` | `"target_path"` |
Java JAR Sample BFG
- git big file cleaner
bash
tree ~/.local/bin
unlink ~/.local/bin/bfg
# sudo echo '#!/usr/bin/env java -jar' | sudo tee bfg.sh
cd /opt
sudo wget https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar
sudo apt install -y default-jre # 180 mb lol
sudo echo '#!/bin/sh' | sudo tee bfg.sh
sudo chmod +x bfg.sh
sudo echo 'exec java -jar /srv/shared/downloads/bfg-1.14.0.jar $@' | sudo tee -a bfg.sh
./bfg.sh --version
ln -s /opt ~/.local/bin/bfg
bfg --version
bfg --delete-files badfile.txt myrepo.git