Compare commits

...

2 Commits

  1. 304
      talks/foss-united-apr-2021.html
  2. 230
      talks/foss-united-apr-2021.md
  3. BIN
      talks/img/cycle.jpg
  4. BIN
      talks/img/isp-monitoring-2.png
  5. BIN
      talks/img/isp-monitoring.png
  6. 78
      terraform/.terraform.lock.hcl
  7. 9
      terraform/modules/caddy/providers.tf
  8. 12
      terraform/modules/cloudflare/records.tf
  9. 2
      terraform/modules/doggo/conf/doggo.nomad
  10. 9
      terraform/modules/doggo/providers.tf
  11. 9
      terraform/modules/gitea/providers.tf
  12. 9
      terraform/modules/joplin/providers.tf
  13. 9
      terraform/modules/monitoring/providers.tf
  14. 9
      terraform/modules/pihole/providers.tf
  15. 9
      terraform/modules/restic/providers.tf
  16. 9
      terraform/modules/shynet/providers.tf
  17. 4
      terraform/versions.tf

304
talks/foss-united-apr-2021.html

File diff suppressed because one or more lines are too long

230
talks/foss-united-apr-2021.md

@ -0,0 +1,230 @@
---
theme: dracula
paginate: true
marp: true
size: 4K
footer: Hydra Repo: [git.mrkaran.dev/karan/hydra](https://git.mrkaran.dev/karan/hydra)
---
<!-- _class: lead -->
# Self Hosting 101
FOSS United - April 2021
*@mrkaran*
---
# `whoami`
👨‍💻 Writes YAML at Zerodha
📈 Interested in Monitoring and Observability systems
📓 Blogs about things I find interesting
🧩 **Self hosted enthusiast**
![bg right 66%](./img/cycle.jpg)
---
# Why (I) Self Host
- Break from the Big Tech Co
---
# Why (I) Self Host
- Break from the Big Tech Co
- Own your data
---
# Why (I) Self Host
- Break from the Big Tech Co
- Own your data
- No lock ins for data which is critical
---
# Why (I) Self Host
- Break from the Big Tech Co
- Own your data
- No lock ins for data which is critical
- Chance to contribute to OSS
---
# Why (I) Self Host
- Break from the Big Tech Co
- Own your data
- No lock ins for data which is critical
- Chance to contribute to OSS
- Experiment and learn
---
# My Setup
## Servers
- DigitalOcean Droplet (2vCPU, 4GB RAM, blr1 Region)
- 1 * RPi 4 Node (4GB RAM)
- 1 * RPi 4 Node (2GB RAM)
---
# Infra and Deployments
- Ansible
- Terraform
- Nomad + Consul
---
## Ansible
- Boostrap the server
- Harden SSH. User, Shell setups.
- Install `node-exporter`, `docker`, `tailscale`.
---
## Terraform
- DigitalOcean infra
- Droplet
- Firewalls
- SSH Keys, Volumes, Floating IPs etc.
- Cloudflare DNS
- `mrkaran.dev` hosted zone
- DNS Records in ^ the zone.
---
## Nomad + Consul
- Single node cluster.
- Runs every workload (mostly) as a docker container.
---
# Services I run
- Pi-hole
- Gitea
- Joplin Sync Server
- Shynet
- Firefly III
- Nextcloud
- `doggo` (Shameless self plug)
---
# Monitoring
- Grafana
- Prometheus
- Telegraf to collect home ISP stats
- Ping Input plugin
- DNS Input Plugin
---
![bg 90%](./img/isp-monitoring.png)
---
![bg 90%](./img/isp-monitoring-2.png)
---
# Networking
- Tailscale for Mesh Network
- Based on Wireguard VPN.
- Authenticated sessions only.
- Expose services on RPi easily without any static IP.
---
# Networking
- Caddy as a proxy for all services.
- Running 2 instances of Caddy.
- Private: Listens on Tailscale Interface.
- Public: Listens on DO's public IPv4 Interface.
- Automatic SSL with ACME DNS challenge
- Built my own image: https://github.com/mr-karan/caddy-plugins-docker
---
# Storage
- DONT use RPi for storage.
- Atleast not with SD cards.
- Newer RPis can boot off SSDs.
- Enable snapshots for volumes provided by cloud provider.
- Use separate DB instances for different applications.
---
# Backups
- Restic
- Periodic Job in Nomad.
- Single vault with everything inside `/data`.
- All applications mount inside `/data` folder.
- Upload to Backblaze B2.
---
# Security
- If it should not be public facing, don't expose to WWW.
- Prefer to use a VPN or mesh network instead of IP whitelists.
- Tighter Firewall rules otherwise.
- Pi-Hole, Gitea, etc Admin interfaces must always be protected with strong passwords.
- Wonder how many `admin/admin` Grafana instances are out in open.
- Or worse, no auth. Looking at you Elasticsearch.
- Periodic **updates** to App and OS.
---
# Takeaways
- Don't overthink. Pick something really simple (like Pi-hole) and host it.
- You'll feel pretty happy about it.
- Don't blindly copy/paste this stack.
- Took me 2 years of constant iteration and experimentation.
- KISS.
---
# Resources
[r/selfhosted](https://www.reddit.com/r/selfhosted)
- Incredible, beginner friendly wiki: https://wiki.r-selfhosted.com/
[github.com/awesome-selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted)
---
<!-- _class: lead -->
# Thank You
## Questions?

BIN
talks/img/cycle.jpg

Binary file not shown.

After

Width:  |  Height:  |  Size: 813 KiB

BIN
talks/img/isp-monitoring-2.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 133 KiB

BIN
talks/img/isp-monitoring.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 108 KiB

78
terraform/.terraform.lock.hcl

@ -2,20 +2,22 @@
# Manual edits may be lost in future updates.
provider "registry.terraform.io/cloudflare/cloudflare" {
version = "2.18.0"
version = "2.20.0"
hashes = [
"h1:DcHmM5XmjwrCCPqIdBRvYDrth5hjk6ldjsS94szjjJc=",
"zh:4121da1ad26081552e3a648e94c89df96e246d50c6e307fe5eba586664691de0",
"zh:4212865eeb42f491d3409f1b9edbb508dbc781a12144c4cb157a8057965144fb",
"zh:4965fad90d5caf7917e0f7617d76a5d3419ca3f003e408f6e58af5e53f20b1ba",
"zh:59d5dedb2b9c9b0a3fc5ad07fce4b1aefeaef5229dbe510e7f0f9f99bbb448aa",
"zh:6746bfa2cfe6005b64286ccf9fcc5b25d1dc29d1448fe9b4f9acf7d3f7f05f79",
"zh:78dd4811b35ea04f0ab11a0c7c600e8fe7f30e7645d8fc60d1d02272fa85568b",
"zh:c7a7adf710bbf686d879825428f9ba92ec35fcb44742ffac5ea9b9538c43a19a",
"zh:cd8827681b957a9a28cb8139414fd8430f228ff736be251c32ae26d8b146bfad",
"zh:cf398859858618b5569b2ad3e84ee5550836b70083b1e7bcf3ba8398ff06e247",
"zh:eb11da2096aea02c792dfe1a5e3605e711102401d03a722b8ae16223245e7f70",
"zh:fc8d289e98dfa3e846b2c737cfdd821a6e1836a062e5453265d1dbb1e35433f1",
"h1:hq912gbF5V9qU6O6TpIjDecKCOpZ3s/o0HIZAIESlw8=",
"zh:1cc439dcf2bb64fbc4b701e0345d47e32c488ebf3c5149c4b2a5da5b1f9b8dc5",
"zh:2b244c594bf674d7f3d939f75daae6bc68732f62603c179d63c8c16b4cd5c248",
"zh:3210c5e6bae1595f231f5e0a479c305b97192d5393b7e6650c6d3647f382041f",
"zh:332f9ac6aa5c18e2df20f177f59cd68ec4c4a46b30e46d7a0af37fffeaed90ca",
"zh:3e304aca15904cde3cf60e23de3422d4dbd84f1b527f587b7d6009f722d8645e",
"zh:5a4ff4512ed12fde14d27ff8ac2e76d3bbea2e39ccc12f0720292c0ebe4a3bf8",
"zh:71dd3268b5e9cb50831254451a6c2b4a3da4383c852b9e37b19ba94b63555532",
"zh:841dfe03377d38667541359c7b5857ff48df4d1cee12f63daadc5837e4ca1974",
"zh:8fa108c7d8b9e1edcdc2aa8be18a054d5e00c67137e29f27c16dc5e2fdcf76b7",
"zh:a38823be68ed1d0aed2e0051d459d150372edff396d3d108725bc889b91cd41f",
"zh:a6c4f12bb0ab72f381eff0172790b00ab35fd6f7f14c60f56a1d67a0bb3225ea",
"zh:bcc00547d0c6ccefe277aa16f5764d4621b0fe76579ab683bb88df192f107c12",
"zh:ca3922155eb365402b8b661c32fb7f9324fecfab7d627810fb9a425fa44ec534",
]
}
@ -41,36 +43,38 @@ provider "registry.terraform.io/digitalocean/digitalocean" {
}
provider "registry.terraform.io/hashicorp/http" {
version = "2.0.0"
version = "2.1.0"
hashes = [
"h1:PZBhWKq5UZDP0sv8lbR7D29dYcpnPrQXiXzunA/+zI4=",
"zh:066b5f4f80013dcc7d94d7d4b5537607c74f93e8a414ac993709d138a36d7db6",
"zh:41c1916d44c263828d39eeb12ab0bea8507e4e41af6f3376cb3f2b93158e35a6",
"zh:78a429752ae9c3587a4ccf106ea17a89918732511f45f99ec905d014f88a3e88",
"zh:8be68e5e4f095a090534594d1e0e08a8ff76638fc94cb38e5e38a683b8cb62ec",
"zh:b00a3d86b86ac07347cd3d89f8bbb966686d897a7c37a18a69b715e8e01a4728",
"zh:b9aab522396439716fb2e2cacf97c4083835fd0f8ea15b95d0a754578253c66c",
"zh:c475bfacb7da999ff5a652680e7a7969b63c1e994462936a1ac7b98708e4ea1f",
"zh:d85f228d5fb4d4197972939d27a00cc6f48309a3d562de44eade347c70f02f83",
"zh:d91ece204cd2aec195dc7e0d1a620ceb19dcc2a64072af134fc9e49f233abb3e",
"zh:faa5c9801b6686872908fb55d5feb4952168dd53e0dadec0962768e24b93116a",
"h1:HmUcHqc59VeHReHD2SEhnLVQPUKHKTipJ8Jxq67GiDU=",
"zh:03d82dc0887d755b8406697b1d27506bc9f86f93b3e9b4d26e0679d96b802826",
"zh:0704d02926393ddc0cfad0b87c3d51eafeeae5f9e27cc71e193c141079244a22",
"zh:095ea350ea94973e043dad2394f10bca4a4bf41be775ba59d19961d39141d150",
"zh:0b71ac44e87d6964ace82979fc3cbb09eb876ed8f954449481bcaa969ba29cb7",
"zh:0e255a170db598bd1142c396cefc59712ad6d4e1b0e08a840356a371e7b73bc4",
"zh:67c8091cfad226218c472c04881edf236db8f2dc149dc5ada878a1cd3c1de171",
"zh:75df05e25d14b5101d4bc6624ac4a01bb17af0263c9e8a740e739f8938b86ee3",
"zh:b4e36b2c4f33fdc44bf55fa1c9bb6864b5b77822f444bd56f0be7e9476674d0e",
"zh:b9b36b01d2ec4771838743517bc5f24ea27976634987c6d5529ac4223e44365d",
"zh:ca264a916e42e221fddb98d640148b12e42116046454b39ede99a77fc52f59f4",
"zh:fe373b2fb2cc94777a91ecd7ac5372e699748c455f44f6ea27e494de9e5e6f92",
]
}
provider "registry.terraform.io/hashicorp/nomad" {
version = "1.4.13"
version = "1.4.14"
constraints = "1.4.14"
hashes = [
"h1:Qca2Y2GlKPN+YRjLS4m95HSHyJIqokwrl2AjpYhnQt0=",
"zh:2406f842b2c5c70fceb5715b4dfc3ffe47630072816ecfb047b7f11169cd04fb",
"zh:318575fecf8c77ea9d9c99198ec2df9d8c35ad5a3d53674d92dc6bdce5598d4d",
"zh:3379c8466e0ba8f865ac2f0d909879e088e02559f527b0a45f6d9790fc7a13b5",
"zh:6209427c15d6bb1ff327613d8e261758f0c1abf5d8045b2fe985d6546333b4bc",
"zh:8c159fe5a9c2e12f831ac3e847ec9007e42d116ba4b8adc53c93998446d0e36d",
"zh:90bc5ea082ff0400b698df4f9d70ad82d8f85d21653b341c229a477aba196bf5",
"zh:a0c9c7fe2a0f024365a0e94894d074f61ab5f0db89092eeb538ba9b12ff0b9b9",
"zh:b35293b9fbacca3a3ef772658d977ddc7061c94e4b460623b184293e8fc8ebb4",
"zh:c5fbd8c0639a9b421f92f29268707ac6b16ae008b477256f4aac89d7f14c2f1d",
"zh:d4a8cfcb867fc24ab400340a07c06a62e913317d2d20961c0b6a4f4578af6cb5",
"h1:GxsjoJKg/PWeXYzpzoONBQiaGnY+bPEDDD+BsEDgc8Q=",
"zh:036cc8e0c1c6c2f91573149910eca29a7107b3415536eabeb2581861525da64a",
"zh:1414e2deb87af66a47e44ab5472b4606294cf511722beae2c0a3680041d66635",
"zh:623184a22b347fa5b696d3fbee35f5bff9ed30fbc8b067715c52b6300d655789",
"zh:7a026a57148a7c2e8a08a83c3641898911a7d9998c38eb2c6ca634107ccf49f9",
"zh:87d34e879284453b2ac825f8bb9c88c85027d404b1b9fa445ec97b519dfa59cb",
"zh:90591119307c2f3dd15a6a78964731689444fb1ce3d393eddf83e05a2f187b80",
"zh:b2cbf5e4d4f2d500804e7f1968b3fd2cebd4b164ccf76d7cb2c99ed1eb23957e",
"zh:d5f19ab3d0d172be8af098bb62b47667c632af736c60d1acab0fc1c31dbbcb99",
"zh:ee5f7f75a642eed607d4824b5888e4aacfc4dd435d54d9523d8f8165695d52a1",
"zh:f6300309339221a5f0863bec32d96b38a8e545c5a87b43c5bb8c65d2ff0492ed",
]
}

9
terraform/modules/caddy/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

12
terraform/modules/cloudflare/records.tf

@ -46,6 +46,18 @@ resource "cloudflare_record" "website" {
}
resource "cloudflare_record" "talks" {
zone_id = cloudflare_zone.mrkaran_dev.id
name = "talks"
type = "CNAME"
ttl = "1"
proxied = "false"
value = "eager-albattani-dc3cb1.netlify.app"
}
resource "cloudflare_record" "notes" {
zone_id = cloudflare_zone.mrkaran_dev.id

2
terraform/modules/doggo/conf/doggo.nomad

@ -28,7 +28,7 @@ job "doggo" {
}
config {
image = "ghcr.io/mr-karan/doggo-api:v0.3.7"
image = "ghcr.io/mr-karan/doggo-api:v0.4.0"
ports = ["http"]

9
terraform/modules/doggo/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

9
terraform/modules/gitea/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

9
terraform/modules/joplin/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

9
terraform/modules/monitoring/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

9
terraform/modules/pihole/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

9
terraform/modules/restic/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

9
terraform/modules/shynet/providers.tf

@ -0,0 +1,9 @@
terraform {
required_providers {
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

4
terraform/versions.tf

@ -13,6 +13,10 @@ terraform {
source = "cloudflare/cloudflare"
}
nomad = {
source = "hashicorp/nomad"
version = "1.4.14"
}
}
required_version = ">= 0.14"
}

Loading…
Cancel
Save