Keeping a list of public notes, that I found interesting or useful in my day to day work.

Content

• Exposing local server to Internet

Exposing local server to Internet

Cloudflare makes cloudflared available for tunneling purpose.

• Install by sudo pacman -Sy cloudflared
• Run to expose a http server running on port 3000 by cloudflared tunnel --url http://localhost:3000
• We get a public url like: https://pollution-lmao-occasions-dracony.trycloudflare.com

Content

• Why?
• 100 ft View
• Project Setup
• Scraping Information
  • Figuring out the API
  • Parsing Information
• Telegram Bot Setup
  • Creating the bot and channel
  • Sending message
• Deployment
• Conclusion

Why?

My whole family has applied for new passports, and I want to know the application status. But every time I open the website, I feel like giving up on life.

What could be worse, you say? Well, doing it four times.

Now, I use Check Application Status option on the homepage. It's an optimization by storing File Numbers and Date of Births in my BitWarden extension. That helps avoid having to fill the Captcha and logging in multiple times. It still hurts, albeit a bit less.

I abhor the 2 minutes, I spend each time I have to check the status. So, I'll spend next 5 hours setting up an automation script to notify me every 3 hours.

100 ft View

I will need the following:

• A job running every 3 hours
• A way to fetch passport application status
• A way to notify me

This is how it's going to work:

• At the onset of each 3rd hour, run a script
• Script will scrape the information from passport status webpage
• Parse out required information from the scraped page
• Send a notification to a chat app (whatsapp or telegram)

Project Setup

I start by defining the deps.edn file in the project root. This is the beginning of the project. deps.edn will contain the project dependencies and configuration for build and release.

Which dependencies will I need?

• To scrape information
  • HTTP Client
  • HTML Parser and Selector
  • JSON encoder and decoder to convert from clojure data structures to JSON
• To send messages to Telegram
  • Again the same dependencies, except HTML Parser
• For development a REPL server library

deps.edn file looks like the following:

; deps.edn
{:paths ["src"]                                                     ; Where the source code lives
 :deps {clj-http/clj-http {:mvn/version "3.13.0"}                   ; HTTP Client
        org.clj-commons/hickory {:mvn/version "0.7.5"}              ; HTML Parser and Selector
        cheshire/cheshire {:mvn/version "5.13.0"}}                  ; JSON encoder/decoder
 :aliases {:dev {:extra-paths ["env/dev"]                           ; dev environment config
                 :extra-deps {nrepl/nrepl {:mvn/version "1.3.0"}}   ; nrepl server development
                 :main-opts ["-m" "nrepl.cmdline"]}                 ; Options for main function in `dev` alias
           :prod {:extra-paths ["env/prod"]}                        ; Prod config
           :passport {:exec-fn passport-status/execute!}}}          ; Alias to execute actual flow

Now I can start the REPL server by executing the following command in the terminal:

$ clj -M:dev

As an output of the command, I'll get the nrepl-port that I can use to connect to the REPL server from the editor. I can also use the .nrepl-port file created by the above command.

I also create the following directory structure:

➜  automations git:(master) ✗ tree .
.
├── deps.edn
├── env
│   ├── dev
│   │   └── config.clj
│   └── prod
│       └── config.clj
└── src
    ├── passport_status.clj
    └── telegram.clj

4 directories, 5 files

• src/passport_status.clj contains all the logic related to passport status fetching
• src/telegram.clj contains all the logic to interact with telegram

Scraping Information

To find out how to extract passport application information, I manually go through the process.

Figuring out the API

• Go to the Passport Website FrontPage
• Close the annoying popup.
• Oh, I see a Track Application Status button on the left.

• When I click on it, they ask to provide File Number and Date of Birth, and type of service.
• When I fill that info and click Track Status. I notice the following request being sent in the browser's network tab (Copied in the curl format):

curl 'https://passportindia.gov.in/AppOnlineProject/statusTracker/trackStatusInpNew' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
  -H 'Accept-Language: en-US,en;q=0.9,hi-IN;q=0.8,hi;q=0.7' \
  -H 'Cache-Control: max-age=0' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -H 'Cookie: JSESSIONID=<*******************>' \
  -H 'DNT: 1' \
  -H 'Origin: https://passportindia.gov.in' \
  -H 'Referer: https://passportindia.gov.in/AppOnlineProject/statusTracker/trackStatusInpNew' \
  -H 'Sec-Fetch-Dest: document' \
  -H 'Sec-Fetch-Mode: navigate' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'Sec-Fetch-User: ?1' \
  -H 'Upgrade-Insecure-Requests: 1' \
  -H 'sec-ch-ua-mobile: ?0' \
  --data-raw 'apptUrl=&apptRedirectFlag=&optStatus=Application_Status&fileNo=<********>&applDob=<DD/MM/YYY>&rtiRefNo=&diplArnNo=&appealNo=&appealDob=&action%3AtrackStatusForFileNoNew=Track+Status'

Ahah, can I just send it over again and get the same response? Hell yeah. When I execute the above command on terminal, I get an HTTP response. (Thank God, for no Captcha)

Perfect. Now, let's also try without all those unnecessary headers and data in request body. Turns out we just need to send the request body:

{
  "optStatus": "Application_Status",
  "fileNo": "ABC**********",
  "applDob": "DD/MM/YYYY",
  "action:trackStatusForFileNoNew": "Track Status"
}

That really simplifies the problem.

Parsing Information

Let's start by fetching the response. This code goes in src/passport_status.clj.

(ns passport-status
  (:require [clj-http.client :as http]))

;; Data for each passport application
(def details
  [{:file-num "ABC234324234"
    :dob "15/09/1992"}
   {:file-num "ABC234324234"
    :dob "24/11/1987"}
   {:file-num "ABC234324234"
    :dob "10/01/1971"}
   {:file-num "ABC234324234"
    :dob "19/02/2001"}])

;; URL to fetch passport application status
(def ^:private url
  "https://www.passportindia.gov.in/AppOnlineProject/statusTracker/trackStatusInpNew")

;; Function that sends a POST request to the above URL
;; And fetches the actual HTML response
(defn- fetch-resp [{:keys [file-num dob]}]
  (http/post url {:form-params {:optStatus "Application_Status"
                                :fileNo file-num
                                :applDob dob
                                :action:trackStatusForFileNoNew "Track Status"}}))

The above code, should help me fetch the response from the server. Now it's time to parse the response and extract the structured information.

I am inspecting the web page source in the browser's developer tools. The information is in the following structure:

<div class="hd_blue">
<table>
  <tbody>
    <tr>
      <td>
        <table>
          <tbody>
            <tr>
              <td> <-- This is what we need
            <tr>
              <td> <-- This is what we need
            <tr>
              <td> <-- This is what we need

Hickory makes it super easy to select those elements with it's hickory.select functions.

(ns passport-status
  (:require [clj-http.client :as http]
            [hickory.core :as h]
            [hickory.select :as s]))

;; Omitted code

;; Selecting required data from hickory format
(defn- select-data [hickory]
  (->> hickory
       (s/select (s/follow-adjacent (s/class "hd_blue") (s/tag :table))) ; selects element with class `hd_blue` and moves to the adjacent `table` element.
       first                                                             ; Gets the first such element
       (s/select (s/child (s/tag :table)                                 ; Traverses the heirarchy shown in the previous code snippet
                          (s/tag :tbody)
                          (s/tag :tr)
                          (s/tag :td)
                          (s/tag :table)
                          (s/tag :tbody)
                          (s/tag :tr)
                          (s/tag :td)))
       (mapcat (fn [td]                                                  ; From here on, we are just extracting the text from the `td`s (Unintended)
                 (:content td)))
       (partition-all 2)
       (take 7)))

Telegram bot setup

Next step is to setup a bot for telegram, that will send up messages with data extracted in the previous steps. I chose telegram over whatsapp because of the simplicity of using its APIs.

Creating the bot and channel

I need a bot that authenticates me to use telegram APIs and a channel where the bot will send me messages. I can have the bot send messages to me personally, but I'll have it send to a channel, so that all the members of the channel can see the updates.

Telegram has a special bot that creates other bots. It's rightly named the @BotFather. Sending a /help message lists all the available commands to interact with it. Most important for us is the /newbot command. This command triggers a flow to create new bot and asks a few questions related to it.

• Name of the new bot
• A globally unique username that ends with bot

Once the bot is created, it sends a welcome message with the bot token and a link to chat with the bot. Bot token will be used to authenticate requests to the telegram API.

I'll also create a private channel and add the bot to it as an admin.

Sending message

Now, that I have the bot token and created the channel. I need the channel_id to send the message to the channel. It's hard to find it on the telegram app or website. The way we'll do it is by using getUpdates API endpoint. This endpoint returns the data that bot has received using the long polling method. The data contains the channel_id as well.

First step is to send a message in the channel I created, for the bot to receive it as update. So, I send a hi message in the telegram channel. To get the updates, I'll use the repl:

(require '[clj-http.client :as http])
(require '[cheshire.core :as json])

(def bot-token "<the-bot-token-from-previous-steps>")

(-> (str "https://api.telegram.org/bot" bot-token "/getUpdates")
    (http/get {:headers {:content-type "application/json"}})
    :body
    (json/parse-string))

; {"ok" true,
;  "result"
;  [{"update_id" 242976779,
;    "channel_post"
;    {"message_id" 8,
;     "sender_chat"
;     {"id" <chat-id>, "title" "Bot Devel", "type" "channel"},
;     "chat"
;     {"id" <chat-id>, "title" "Bot Devel", "type" "channel"},
;     "date" 1731041338,
;     "text" "hi"}}]}

The response is shown above. I get the <chat-id> from there, and save it for later use. Test sending the message by:

(def chat-id 1234434) ; Replace this with <chat-id>

(-> (str "https://api.telegram.org/bot" bot-token "/sendMessage")
    (http/post {:headers {:content-type "application/json"}
                :body (json/encode {:chat_id chat-id
                                    :text "Hello"})}))

I get a Hello text on the telegram channel. Ready to assemble the telegram.clj file.

(ns telegram
  (:require [clj-http.client :as http]
            [cheshire.core :as json]
            [config :as cfg]))

(def ^:private bot-url
  (str "https://api.telegram.org/bot" (-> cfg/telegram :bot-token)))

(defn- telegram-url [method]
  (case method
    :send-message (str bot-url "/sendMessage")
    :get-updates (str bot-url "/getUpdates")))

(defn- channel-id [channel]
  (or (get-in cfg/telegram [:channel-ids channel])
      (when (= cfg/env :dev)
        (get-in cfg/telegram [:channel-ids :default]))))

(defn send-message! [channel message]
  (http/post (telegram-url :send-message)
             {:headers {:content-type "application/json"}
              :body (json/encode {:chat_id (channel-id channel)
                                  :text message
                                  :parse_mode "HTML"})}))

config namespace comes from env/dev/config.clj file. The structure looks something like the following:

(ns config)

(def telegram
  {:bot-token "<bot-token>"
   :channel-ids {:default <chat-id>}})

(def env :dev)

env/prod/config.clj will have the same structure with prod related config.

The final passport-status namespace looks like the following, with some missing pieces and them integrated.

(ns passport-status
  (:require
   [clj-http.client :as http]
   [hickory.core :as h]
   [hickory.select :as s]
   [clojure.string :as str]
   [telegram :as tg]))

(def details
  [{:file-num "ABC234324234"
    :dob "15/09/1992"}
   {:file-num "ABC234324234"
    :dob "24/11/1987"}
   {:file-num "ABC234324234"
    :dob "10/01/1971"}
   {:file-num "ABC234324234"
    :dob "19/02/2001"}])

(def ^:private url
  "https://www.passportindia.gov.in/AppOnlineProject/statusTracker/trackStatusInpNew")

(defn- fetch-resp [{:keys [file-num dob]}]
  (http/post url {:form-params {:optStatus "Application_Status"
                                :fileNo file-num
                                :applDob dob
                                :action:trackStatusForFileNoNew "Track Status"}}))

(defn- select-data [hickory]
  (->> hickory
       (s/select (s/follow-adjacent (s/class "hd_blue") (s/tag :table)))
       first
       (s/select (s/child (s/tag :table)
                          (s/tag :tbody)
                          (s/tag :tr)
                          (s/tag :td)
                          (s/tag :table)
                          (s/tag :tbody)
                          (s/tag :tr)
                          (s/tag :td)))
       (mapcat (fn [td]
                 (:content td)))
       (partition-all 2)
       (take 7)))

(defn- format-message [data]
  (->> data
       (map (fn [[key val]]
              (format "<b>%s</b>: %s" (str/trim key) (str/trim val))))
       (str/join "\r\n")
       (str "<b><u>" (java.time.LocalDate/now)
            " " (-> (java.time.LocalTime/now)
                    str
                    (str/split #"\.")
                    first)
            "</u></b> \r\n")))

(defn ^:export execute! [& _]
  (->> details
       (pmap (fn [info]
               (-> (fetch-resp info)
                   (:body)
                   (h/parse)
                   (h/as-hickory))))
       (pmap select-data)
       (pmap format-message)
       (pmap #(tg/send-message! :infinite %))
       (doall)))

Understanding the above code is left as an exercise to the reader. From the passport-status ns, I just need to call the telegram/send-message! with the channel and formatted message data.

Deployment

Now, I need a way to setup a job that runs every 3 hours or so. I'll use an EC2 instance on AWS that I already use to host my personal projects.

First, I copy the project to the EC2 machine. Then, I use systemd timers for job scheduling because of its simple syntax and configuration.

I create two files for systemd config.

• passportstatus.service: keeps the config about what commands to run that will execute the passport-status/execute! function.
• passportstatus.timer: keeps the timer configuration, when to run the job and how often.

passportstatus.service

[Unit]
Description="Passport Fetch status"

[Service]
WorkingDirectory=/home/user/automation
ExecStart=/bin/bash -c "/usr/local/bin/clojure -X:prod:passport"

passportstatus.timer

[Unit]
Description="Run a job to fetch passport status."

[Timer]
OnStartupSec=5seconds
OnBootSec=5min
OnUnitActiveSec=24h
OnCalendar=*-*-* 09/3:00:00 # Runs every three hours starting at 9 AM.
Unit=passportstatus.service

[Install]
WantedBy=multi-user.target

Create softlinks to these files in /etc/systemd/system directory. And enable the passportstatus.timer unit by:

$ sudo systemctl enable passportstatus.timer

Conclusion

It was a fun little project, that would have taken a few days for me to finish, just a few months back. This exercise boosted my confidence in executing a project from scratch in clojure and getting to deployment. This is just the start of more automations, that'll come in the future.

Thanks for reading and stay tuned for more.

This was, until I lost someone close. Someone, I loved, respected and adored. My grandfather, one of the rare men that my heart has deep respect for. I never could fathom the pain, his departure broght. Although the pain was extreme, I had experienced it before in a different shade of black. It wasn't that of a death but a loss, nonetheless.

My grandfather, Mahipat Singh Patel, teacher, a man of few words and a strong character, left us in the morning of August 4th, 2024. His departure was painless, yet the journey to that point was filled with suffering. My heart breaks when I think about his last few weeks, yet I am thankful that he didn't suffer more. Dadaji, you are in a better place, I just miss your presence in my life. I already miss, just existing alongside you.

We spoke rarely, more rare were our meaningful conversations. You didn't express yourself, as much as I would want to understand you. You were quiet and quite consistent. A simple man, who realised the truth of his life. You tried to fulfill our wishes, without having any expectation.

You sat outside our home and watched people passing by. Some would wish you the time of day, others just pass and still others touch your feet. You would notice them passing by, responding to their wishes in return. After a while, you would return back to your thoughts. I miss these ordinary events of our lives.

I only have one regret. You never depended on anyone all your life. In the last 3 months, you completely gave off responsibility of your life to us, me in particular. I failed at that. I am sorry. You didn't deserve to go this soon. I miss you

You did more than your duties. You gave more than you had. You cared more than what was asked of you. I promise to follow your footsteps.

ॐ शांति! ॐ शांति! ॐ शांति!

Suffering is necessary to feel the joy from its absence. But what if suffering itself becomes joyful. We lose the incentive to overcome it.

We don't think about the end of the dark tunnel, the dawn after a dark stormy night.

We are content with the suffering itself because it's common among our peers. The good old sense of belonging wins over the will to rid ourselves of the pain.

That's why I prefer solitude. There's no other that I subconsciously mimic. There's no one I secretly love, envy or hate. I am an open book, when alone, albeit a messed up one.

A whole lot of us dislike change, including me. Mainly the ones that come unexpectedly, that we have no idea about.

eof

Welcome back, folks. It's been a while since the first post of this series came out.

Today's agenda is to install some basic plugins and configure them. While doing that, we'll learn some fennel and have our own config in parallel.

Why Plugins?

Plugins enhance the raw editor by adding features that the raw Neovim doesn't provide out of the box. In VSCode world, we call them extensions.

We need a plugin that makes it easy for us to install and use other plugins. These are called Plugin Managers. There are many of them in Neovim/Vim world like Packer, Vim Plug, Lazy et al. For the purpose of this post and because it's the new shiny thing, we will use Lazy.

plugins.fnl

Just to recap, after our last post, the config directory looks like the following

$ tree -L 3 ~/.config/nvim
├── fnl
│   └── user
│       └── core.fnl
├── init.lua
└── lua
    └── user
        └── core.lua

We are going to add the following plugins now:

• lsp-config - Language Server Protocol client for neovim
• nvim-treesitter - Allows to parse code into abstract syntax tree and use that in better syntax highlighting, code traversal et al.
• nvim-cmp - Autocompletion for neovim
• telescope.nvim - Fuzzy finder for neovim
• conjure - Allows you to hook into REPl servers
• nvim-paredit - S-expression motions made easy
• rainbow-delimiters - Colorful parenthesis
• autopairs - Allows working with pair characters easy

We'll put all the plugin installation related configuration in the plugins.fnl file. Let's start with installing Lazy, a lua based plugin manager for neovim.

;; fnl/user/plugins.fnl

;; Path to directory where lazy itself will be installed.
;; `(vim.fn.stdpath :data)` returns standard directory where nvim's data will be stored.
;; `..` is string concatenation in fnl.
(local lazy-path (.. (vim.fn.stdpath :data)
                     :/lazy/lazy.nvim))

;; `(vim.uv.fs_stat lazy-path)` returns nil if the `lazy-path` doesn't exist.
;; If the directory exists, it returns info about the directory.
(local lazy-installed? (vim.uv.fs_stat lazy-path))

;; We'll store plugins we need to install in this variable.
(local to-install [])

;; `setup` installs lazy plugin manager if not already installed, and also downloads the plugins added to `to-install`.
;; This is an exported function that is called from the parent module.
(fn setup []
  (when (not lazy-installed?)
    (vim.fn.system [:git
                    :clone
                    "--filter=blob:none"
                    "https://github.com/folke/lazy.nvim.git"
                    :--branch=stable
                    lazy-path]))
  (vim.opt.rtp:prepend lazy-path)
  (let [lazy (autoload :lazy)]
    (lazy.setup plugins-to-install)))

;; Exporting the setup function.
{: setup}

Let's import plugins.fnl in fnl/user/core.fnl and call the setup function.

;; fnl/user/core.fnl
(local plugins (require :user.plugins))

(fn setup []
  (plugins.setup))

{: setup}

In our init.lua, we need to call setup function of our core module.

-- init.lua
-- ...

require('user.core').setup()

Now, when we restart the neovim, the lazy plugin should be installed.

It is time for us to add the plugins we listed above.

;; fnl/user/plugins.fnl

;; ...
(local to-install
  [{1 :neovim/nvim-lspconfig
    :dependencies [{1 :williamboman/mason.nvim :config true} ;; Mason handles lsp servers for us and others
                   :williamboman/mason-lspconfig.nvim ;; default lspconfigs for language servers
                   {1 :echasnovski/mini.nvim ;; For lsp progress notification
                    :version false
                    :config (fn [] 
                              (let [mnotify (require :mini.notify)]
                                (mnotify.setup {:lsp_progress {:enable true
                                                               :duration_last 2000}})))}
                   :folke/neodev.nvim]} ;; neodev for lua neovim dev support.

   ;; Autocompletion   
   {1 :hrsh7th/nvim-cmp
    :dependencies [:hrsh7th/cmp-nvim-lsp]}

   ;; Fuzzy Finder (files, lsp, etc)
   {1 :nvim-telescope/telescope.nvim
    :version "*"
    :dependencies {1 :nvim-lua/plenary.nvim}}
   ;; Fuzzy Finder Algorithm which requires local dependencies to be built.
   ;; Only load if `make` is available. Make sure you have the system
   ;; requirements installed.
   {1 :nvim-telescope/telescope-fzf-native.nvim
    ;; NOTE: If you are having trouble with this installation,
    ;; refer to the README for telescope-fzf-native for more instructions.
    :build :make
    :cond (fn [] (= (vim.fn.executable :make) 1))}

   ;; Treesitter
   {1 :nvim-treesitter/nvim-treesitter
    :build ":TSUpdate"}
  
   ;; Conjure and related plugins
   :Olical/conjure
   :PaterJason/cmp-conjure ;; autocomplete using conjure
   :tpope/vim-dispatch
   :clojure-vim/vim-jack-in
   :radenling/vim-dispatch-neovim
   :tpope/vim-surround

   ;; Paredit
   {1 :julienvincent/nvim-paredit
    :config (fn []
              (let [paredit (require :nvim-paredit)]
                (paredit.setup {:indent {:enabled true}})))}
   ;; Paredit plugin for fennel
   {1 :julienvincent/nvim-paredit-fennel
    :dependencies [:julienvincent/nvim-paredit]
    :ft [:fennel]
    :config (fn []
              (let [fnl-paredit (require :nvim-paredit-fennel)]
                (fnl-paredit.setup)))}

   ;; Rainbow parens
   :HiPhish/rainbow-delimiters.nvim

   ;; Autopairs
   {1 :windwp/nvim-autopairs
    :event :InsertEnter
    :opts {:enable_check_bracket_line false}}])

Restart neovim, and we see Lazy UI installing the plugins added above.

Common configuration

Let's configure Neovim to solve some day-to-day pains eg. enabling relative line numbering, auto syncing system and neovim clipboards, some useful keymaps etc.

We'll create a general.fnl where we put all of this.

;; fnl/user/config/general.fnl

(fn setup []
  ;; disables highlighting all the search results in the doc
  (set vim.o.hlsearch false)

  ;; line numbering
  (set vim.wo.number true)
  (set vim.wo.relativenumber true)

  ;; disable mouse
  (set vim.o.mouse "")

  ;; clipboard is system clipboard
  (set vim.o.clipboard :unnamedplus)

  ;; Some other useful opts. `:help <keyword>` for the help.
  ;; For example: `:help breakindent` will open up vimdocs about `vim.o.breakindent` option.
  (set vim.o.breakindent true)
  (set vim.o.undofile true)
  (set vim.o.ignorecase true)
  (set vim.o.smartcase true)
  (set vim.wo.signcolumn :yes)
  (set vim.o.updatetime 250)
  (set vim.o.timeout true)
  (set vim.o.timeoutlen 300)
  (set vim.o.completeopt "menuone,noselect")
  (set vim.o.termguicolors true)
  (set vim.o.cursorline true)

  ;; Keymaps
  (vim.keymap.set [:n :v] :<Space> :<Nop> {:silent true})

  ;; To deal with word wrap
  (vim.keymap.set :n :k "v:count == 0 ? 'gk' : 'k'" {:expr true :silent true})
  (vim.keymap.set :n :j "v:count == 0 ? 'gj' : 'j'" {:expr true :silent true})

  ;; Tabs (Optional and unrelated to this tutorial, but helpful in handling tab movement)
  (vim.keymap.set [:n :v :i :x] :<C-h> #(vim.api.nvim_command :tabprevious)
                  {:silent true})
  (vim.keymap.set [:n :v :i :x] :<C-l> #(vim.api.nvim_command :tabnext)
                  {:silent true})
  (vim.keymap.set [:n :v :i :x] :<C-n> #(vim.api.nvim_command :tabnew)
                  {:silent true}))

;; exports an empty map
{: setup}

Plugin Configurations

It is time for us to configure each of these plugins to cater to our needs. These needs are plugin specific. We may want to add keymaps for commands we use often or tell Mason to make sure that particular LSP is always installed, or anything else that the plugin supports. These settings and configs can often be found on the plugin's github repository and neovim's documentation.

LSP configuration

;; fnl/user/config/lsp.fnl

(local nfnl-c           (require :nfnl.core))
(local ts-builtin       (require :telescope.builtin))
(local cmp-nvim-lsp     (require :cmp_nvim_lsp))
(local mason-lspconfig  (require :mason-lspconfig))
(local lspconfig        (require :lspconfig))
(local neodev           (require :neodev))

;; This function will be executed when neovim attaches to an LSP server.
;; This basically sets up some widely used keymaps to interact with LSP servers.
(fn on-attach [_ bufnr]
  (let [nmap (fn [keys func desc]
               (vim.keymap.set :n keys func
                               {:buffer bufnr 
                                :desc   (.. "LSP: " desc)}))
        nvxmap (fn [keys func desc]
                 (vim.keymap.set [:n :v :x] keys func
                                 {:buffer bufnr :desc (.. "LSP: " desc)}))]
    (nmap :<leader>rn vim.lsp.buf.rename "[R]e[n]ame")
    (nmap :<leader>ca vim.lsp.buf.code_action "[C]ode [A]ction")
    (nmap :gd vim.lsp.buf.definition "[G]oto [D]efinition")
    (nmap :gr (fn [] (ts-builtin.lsp_references {:fname_width 60}))
          "[G]oto [R]eferences")
    (nmap :gI vim.lsp.buf.implementation "[G]oto [I]mplementation")
    (nmap :<leader>D vim.lsp.buf.type_definition "Type [D]efinition")
    (nmap :<leader>ds ts-builtin.lsp_document_symbols "[D]ocument [S]ymbols")
    (nmap :<leader>ws
          (fn [] (ts-builtin.lsp_dynamic_workspace_symbols {:fname_width 60}))
          "[W]orkspace [S]ymbols")
    (nmap :K vim.lsp.buf.hover "Hover Documentation")
    (nmap :<C-k> vim.lsp.buf.signature_help "Signature Documentation")
    (nmap :gD vim.lsp.buf.declaration "[G]oto [D]eclaration")
    (nmap :<leader>wa vim.lsp.buf.add_workspace_folder
          "[W]orkspace [A]dd folder")
    (nmap :<leader>wr vim.lsp.buf.remove_workspace_folder
          "[W]orkspace [R]emove folder")
    (nmap :<leader>wl
          (fn []
            (nfnl-c.println (vim.inspect (vim.lsp.buf.list_workspace_folders))))
          "[W]orkspace [L]ist folders")
    (nvxmap :<leader>fmt vim.lsp.buf.format
            "[F]or[m]a[t] the current buffer or range")))

;; This binding keeps a map from lsp server name and its settings.
(local servers
       {:clojure_lsp            {:paths-ignore-regex :conjure-log-*.cljc}
        :lua_ls                 {:Lua {:workspace {:checkThirdParty false}
                                       :telemetry {:enable false}}}
        :fennel_language_server {:fennel {:diagnostics  {:globals [:vim]}
                                          :workspace    {:library (vim.api.nvim_list_runtime_paths)}}}})

(local capabilities
       (cmp-nvim-lsp.default_capabilities (vim.lsp.protocol.make_client_capabilities)))

(fn setup []
  (neodev.setup)
  (mason-lspconfig.setup {:ensure_installed (nfnl-c.keys servers)})
  (mason-lspconfig.setup_handlers [(fn [server-name]
                                     ((. (. lspconfig server-name) :setup) 
                                        {: capabilities
                                         :on_attach on-attach
                                         :settings (. servers server-name)

                                         ;; This is required because clojure-lsp doesn't send LSP progress messages if the `workDoneToken` is not sent from client. It is an arbitrary string.
                                         :before_init
                                         (fn [params _]
                                           (set params.workDoneToken 
                                                "work-done-token"))}))]))

{: setup}

Autocompletion configuration

;; fnl/user/config/cmp.fnl
(local cmp (require :cmp))

(fn setup []
  (cmp.setup {:mapping (cmp.mapping.preset.insert 
                         {:<C-d>     (cmp.mapping.scroll_docs -4)
                          :<C-f>     (cmp.mapping.scroll_docs 4)
                          :<C-Space> (cmp.mapping.complete {})
                          :<CR>      (cmp.mapping.confirm {:behavior cmp.ConfirmBehavior.Replace
                                                           :select false})
                          :<Tab>     (cmp.mapping (fn [fallback]
                                                    (if
                                                      (cmp.visible)
                                                      (cmp.select_next_item)

                                                      ;; else
                                                      (fallback))))
                          :<S-Tab>   (cmp.mapping (fn [fallback]
                                                    (if 
                                                      (cmp.visible)
                                                      (cmp.select_prev_item)
                                                      
                                                      ;; else
                                                      (fallback))))}                        
                         [:i :s])
              :sources [{:name :conjure}
                        {:name :nvim_lsp}]}))

{: setup}

Paredit configuration

;; fnl/user/config/paredit.fnl

(local paredit      (require :nvim-paredit))
(local paredit-fnl  (require :nvim-paredit-fennel))

(fn setup []
  (paredit.setup {:indent {:enabled true}})
  (paredit-fnl.setup))

{: setup}

Rainbow delimiters configuration

;; fnl/user/config/rainbow.fnl

(local rainbow-delimiters (require :rainbow-delimiters))

(fn setup []
  (set vim.g.rainbow_delimiters
       {:strategy {"" rainbow-delimiters.strategy.local}
        :query    {"" :rainbow-delimiters}}))

{: setup}

Telescope configuration

;; fnl/user/config/telescope.fnl

(local telescope  (require :telescope))
(local builtin    (require :telescope.builtin))
(local themes     (require :telescope.themes))

(fn setup []
  (telescope.setup {:defaults {:mappings {:i {:<C-u> false :<C-d> false}}}})
  (telescope.load_extension :fzf)

  ;; keymaps
  (vim.keymap.set :n :<leader>? builtin.oldfiles
                  {:desc "[?] Find recently opened files"})
  (vim.keymap.set :n :<leader>fb builtin.buffers {:desc "[F]ind open [B]uffers"})
  (vim.keymap.set :n :<leader>/
                  (fn []
                    (builtin.current_buffer_fuzzy_find (themes.get_dropdown {:winblend 10
                                                                             :previewer false})))
                  {:desc "[/] Fuzzily search in current buffer"})
  (vim.keymap.set :n :<leader>ff builtin.find_files {:desc "[F]ind [F]iles"})
  (vim.keymap.set :n :<leader>fh builtin.help_tags {:desc "[F]ind [H]elp"})
  (vim.keymap.set :n :<leader>fw builtin.grep_string
                  {:desc "[F]ind current [W]ord"})
  (vim.keymap.set :n :<leader>fg builtin.live_grep {:desc "[F]ind by [G]rep"})
  (vim.keymap.set :n :<leader>fd builtin.diagnostics
                  {:desc "[F]ind [D]iagnostics"})
  (vim.keymap.set :n :<leader>fcf
                  (fn [] (builtin.find_files {:cwd (vim.fn.stdpath :config)}))
                  {:desc "[F]ind [C]onfing [F]iles"})
  (vim.keymap.set :n :<leader>fch builtin.command_history
                  {:desc "[F]ind in [C]ommands [H]istory"})
  (vim.keymap.set :n :<leader>fm builtin.marks {:desc "[F]ind in [M]arks"}))

{: setup}

Treesitter configuration

;; fnl/user/config/treesitter.fnl

(local configs (require :nvim-treesitter.configs))

(local ensure-installed [:clojure :lua :vimdoc :vim :fennel])

(fn setup []
  (configs.setup {:ensure_installed       ensure-installed
                  ;; Add languages to be installed here that you want installed for treesitter
                  :auto_install           true
                  :highlight              {:enable true}
                  :indent                 {:enable  false}}))

{: setup}

Gluing everything together

Let's actually call setup methods of all these modules in our core.fnl.

;; fnl/user/core.fnl
(local plugins          (require :user.plugins))
(local u-general        (require :user.config.general))
(local u-telescope      (require :user.config.telescope))
(local u-treesitter     (require :user.config.treesitter))
(local u-lsp            (require :user.config.lsp))
(local u-cmp            (require :user.config.cmp))
(local u-paredit        (require :user.config.paredit))
(local u-rainbow        (require :user.config.rainbow))

(fn setup []
  (plugins.setup)
  (u-general.setup)
  (u-telescope.setup)
  (u-treesitter.setup)
  (u-lsp.setup)
  (u-cmp.setup)
  (u-paredit.setup)
  (u-rainbow.setup))

{: setup}

And that should be it. I know, this blog is long and contains a lot of code. Most of it should be self explanatory once you get familiar with the fennel syntax and vim docs.

The configuration is inspired by kickstart.nvim. More extensive configuration can be found on my dotfiles.

If there's a bug somewhere in here or you have a doubt, feel free to open an issue here.

You may gently ask for the meaning of PDE, what the fuck does that mean?

Well, you don't have to be so rude. It's an abbreviation for Personalized Development Environment, coined by TJ, Neovim's marketing head.

Why Neovim?

You know, same old bullshit like speed, muscle memory, extension of your body/mind, spiritualism, nirvana, moksha and so on. Nothing serious. Jokes aside, the real reason was seeing how fast Prime was, in giving up on clojure.

Why Clojure?

Because I love my high functioning parens. I can lisp down, 100s of reasons why, but that's beyond the scope. This emotion is hosted on a solid and reliable foundation. I am sure, a dynamic and immutable relationship is what keeps us tied together so strongly. Please forgive me, for my love for punning (nil?). IYKYK.

init.lua

Neovim is a professional's tool. You need to deserve it, earn its respect, you know? Like Thor and his hammer.

Or you can just install neovim and get started. At first sight, it doesn't look like anything more than a trap that you can never get out of (trust me you can :q!). And no, that was not my failed attempt at putting an emojee.

Create an init.lua file in $HOME/.config/nvim directory. This will be the entrypoint for your neovim configuration. It's similar to the main method in a java/c/cpp projects, an entry-point for the program to run.

Go ahead and add a print("Hello, world!") to the file. Now, when you run nvim, you should see Hello, world! at the bottom-left of your screen. Congratulations, for your first configured neovim instance.

Leader and LocalLeader

Just like how each country needs good leaders to function properly, neovim is no different. You should define a leader and localleader according to your convenience. People usually choose <Space> or "," as their leader. I'll go ahead with "," as that is what I am used to.

But wait, what is the purpose of the leaders, you ask? Well, the main reason is the "plugins". Plugin writers are not aware of how you have configured your editor. They can't arbitrarily setup keybindings in their plugins, as they may conflict with your bindings.

Neovim API exposes options via vim.g, vim.o, vim.opt, vim.bo and vim.wo. We can control behaviour of Neovim by setting these options. vim.g is used for global scoped options. We can read more about them in Neovim's help text using :help vim.g or :help vim.o etc.

Let's set our globally scoped leader and localleader.

-- init.lua
vim.g.mapleader = ','
vim.g.maplocalleader = ','

NSFW: This article goes deeper into how leader/localleader is helpful.

We'll use Fennel as our config language

Although, Neovim officially supports lua as its configuration language, we will use Fennel. Because, we love our parens. And also, we like to struggle and learn.

Fennel transpiles into lua, so we need a Neovim plugin to transpile our fennel files into lua. Olical's nfnl does exactly that. We will update our init.lua to download nfnl. We will also add nfnl's path to neovim's runtime path, so that it can find nfnl plugin.

-- init.lua
-- ...

local nfnl_path = vim.fn.stdpath 'data' .. '/lazy/nfnl'

if not vim.uv.fs_stat(nfnl_path) then
  print("Could not find nfnl, cloning new copy to", nfnl_path)
  vim.fn.system({'git', 'clone', 'https://github.com/Olical/nfnl', nfnl_path})
end

vim.opt.rtp:prepend(nfnl_path)

require('nfnl').setup()

We define a local variable nfnl_path. It holds the directory path where we will download nfnl.

• vim.fn.stdpath 'data' returns neovim's data directory.

  :help stdpath to know more about standard paths.

• ".." is lua's string concatenation operator. So essentially, we get neovim's standard data path and append /lazy/nfnl to it. We'll store nfnl here.
• vim.uv.fs_stat(nfnl_path) returns information about file/directory. If the file doesn't exist, it returns nil.
• vim.fn.system allows us to execute shell commands. We can also use vim.system(...) instead. They are exactly the same.
• We ran git clone to download nfnl from its official github repository.
• Then we added nfnl_path to the neovim's runtime path.
• At last, we require the nfnl module and call setup on it.

Let us create a directory to store our fennel config files. And also a core.fnl file inside that directory.

# Creates the directory
mkdir -p $HOME/.config/nvim/fnl/user

# Creates the file
touch $HOME/.config/nvim/fnl/user/core.fnl

Let's add a simple Hello, world! print form in our core.fnl.

;; fnl/user/core.fnl
(println "Hello, world! This is fennel config!")

When we restart our neovim instance, nothing happens. We should have seen a hello world message. We're missing a key nfnl config.

nfnl looks for a .nfnl.fnl file in a directory for configuration about which files to compile and how. Create a .nfnl.fnl file with empty config in $HOME/.config/nvim directory.

echo {} > $HOME/.config/nvim/.nfnl.fnl

When we restart our instance again, we still don't see anything printed. Well, there are a couple of things pending.

• Fennel files haven't been transpiled yet. nfnl transpiles fnl files on save by default.
• In our init.lua, we haven't required the user.core, so nvim doesn't know about it just yet.

On opening core.fnl, a prompt says that .nfnl.fnl is not trusted. Press a to allow (mark it as trusted). This is because nfnl by default won't compile files in a directory unless we specifically allow it to. Once we allow and save the core.fnl file, by using :write command, a new file lua/user/core.lua gets generated with transpiled lua code corresponding to the core.fnl.

Let's require user/core module in init.lua

-- init.lua
-- ...
-- append this to the bottom of init.lua file.
require('user.core')

Now, when you restart neovim, it greets you with:

Hello, world! This is fennel config!

This post is aimed at a fairly technical audience. It expects familiarity with clojure, babashka, git, github, cloudflare and markdown syntax.

Prerequisites: Install babashka and git

As mentioned in my previous post, I had a wish to start writing in public for the last couple of years. Mostly to frame my own thoughts in a structured manner and to keep a public record so others can relate or criticize.

Recently I found the amazing quickblog, that helped me get up and running with my first post within two hours.

Without wasting any more word, let's get started.

Outline

In this post, we will

• Set up a blog repository using quickblog
• Host it on github
• Write a github action config to generate static files on push to main branch
• Set up a CD pipeline on cloudflare to deploy the blog
• Point a custom domain to the blog site

What is quickblog?

Quickblog is a light-weight static blog engine for clojure and babashka. It allows you to write your posts in markdown, provides live reloading and generates SEO optimized static files.

Setting up quickblog

Create a directory that'll hold your blogging engine and blog posts. I will call it blog. Inside blog directory, create a bb.edn file. The structure looks like the following:

➜ tree blog 
blog
└── bb.edn

1 directory, 1 file

Let's add quickblog as a dependency in our deps.edn.

;; deps.edn
{:deps {io.github.borkdude/quickblog {:git/tag "v0.3.6"
                                      :git/sha "d00e14b1176416b7d7b88e6608b6975888208355"}}}}

Replace the :git/tag and :git/sha with the latest ones in quickblog repository.

Now we will add a babashka task that hooks into quickblog api and exposes it as a command line tool.

{:deps {...} ;; folded deps map

 :tasks
 {:requires ([quickblog.cli :as cli])
  :init (def opts {:blog-title "<Your Blog Title>"
                   :blog-description "<Short Blog Description>"
                   :about-link "<Your Website Link>"
                   :twitter-handle "<Twitter Handle>"})
  quickblog {:doc "Start blogging quickly! Run `bb quickblog help` for details."
             :task (cli/dispatch opts)}}}

Replace all the options, with your own.

Execute bb quickblog help, it should print a detailed help text about using quickblog.

First blog post

Now that we have quickblog setup, we will generate our first post. Quickblog command line has a new subcommand that allows us to do just that.

bb quickblog new --file my-first-blog-post.md --title "My first blog post"

The directory structure after the last command should look like the following

➜  blog tree .
.
├── bb.edn
└── posts
    └── my-first-blog-post.md

2 directories, 2 files

Run a live reloading server using bb quickblog watch, it spins up a http server on localhost:1888 by default.

Write your blog post, initialize the blog directory as a git repository and commit your markdown changes. You would also want to add a .gitignore file that excludes some directories from committing like .work, .lsp and .clj-kondo. We will also exclude the public directory from the commit to keep the main branch clean.

Setting up github action

Via github action, we will

• Switch to deploy branch
• Generate the static files
• Commit and push them to the repository

For this, we will need to allow github actions write permissions on the repository.

In your repository, go to the Settings > Actions > General. In the Workflow permissions select Read and write permissions for now. Later, you can spericy more granular permissions documented here.

Create the github action config file: .github/workflows/deploy.yml.

on:
  push:
    branches:
      - main
env:
  TZ: Asia/Kolkata # Replace this with your time zone

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: git config
        run: |
          git config --global user.email "<Github Email>"
          git config --global user.name "<Github Username>"
      - name: Switch to deploy branch
        uses: actions/checkout@v4
        with:
          ref: deploy
          fetch-depth: 0
      - name: Merge main into deploy
        run: git merge origin/main
      - name: Setup babashka
        uses: just-sultanov/setup-babashka@v2
        with:
          version: 1.3.186 # Update this babashka version to latest
      - name: clean
        run: bb quickblog clean
      - name: build
        run: bb quickblog render
      - name: git
        run: |
          export CURRENT_TIME=$(date)
          git add public
          git commit -m "deploy @ ${CURRENT_TIME}"
          git push origin -u deploy

Replace the placeholders for github email and username, and other parameters. Apart from that the file should be self explanatory.

We will keep it in our local for now, and push it later to github.

Setting up cloudflare

Create a new cloudflare account if you don't have it already. Log in to the account and go to Workers & Pages in the left panel.

• We will create an application
• Go to Pages tab.
• And Connect to Git

A modal pops up, where you select your Github account and the repository to connect. You'll be redirected to github. Allow access to just the blog repository and continue.

Now we go to the next step, add the project name and set the Production branch to deploy and Build output directory to public. Click and deploy.

Cloudflare will consider changes on all the branches other than the Production branch, that we set in the previous step, as preview branches. This means that changes on these other branches will also trigger a build, consuming our precious build quota.

To disable the preview branches, we will go to the project we just setup -> Settings tab -> Build & deployment -> Configure preview deployments -> None -> Save. We are done with the setting up our cloudflare page. It should be visible at <project-identifier>.pages.dev.

Note: Create the main and deploy branches and push them to our github repository. This will trigger github action build, after which cloudflare page build and you should see you first blog up on the url mentioned above.

Setting up custom domain

Here I am assuming that you already have a domain bought and configured on cloudflare. If you don't, then there are a couple of platforms that I use to find and get best offer for domains:

• Namecheap
• GoDaddy
• BigRock
• Hostinger

Once you get a domain, transfer it to cloudflare by following these steps.

Setting up a custom domain for your page is simple. Open our page project, and go to Custom domains tab. Click on Set up a custom domain and follow the steps.

PS: There are multiple ways to host your blog, and probably most of them would be simpler than this setup. It was my first time setting up my blog, and can certainly be improved. Will post an update when I migrate to a better pipeline. A similar post: Quickblog by Anders Means Different

If you find an error or have a question, open an issue.

First off, a very happy new year. May you find love, happiness and joy this year.

Wedding

2023, for me, was life changing. In a good way. I got married to my beautiful wife, spent more time with family (thanks to my new job at Oliv), read a lot of books and healed from wounds of the past. I can not be grateful enough for this phase of my life.

The wedding brought my family closer. It was tiring to the core to the whole family, but enjoyable nonetheless. I usually stay away from the crowd and excess stimuli but bent these boundaries for the family, and it turned out fine. Like all the things in nature, the excitement and joy came back to the normal level. Now, there are photos and videos of these moments, that we will cherish for the next several years.

New Job at Oliv

I have an itch of experimenting and reading about new things, specifically in the technology industry. For the last couple of years, I wanted to read and learn about clojure - a dynamic, functional and lisp flavoured programming language. The idea of REPLs fascinated me from the very first moment I read about them. So, I went ahead and tried to build a project with it. I failed miserably, it was hard to alter my thinking to align with the principles that clojure stood for.

Later when dvd contacted me about joining Oliv in the mid 2023, I couldn't say no. One of the reasons was clojure based tech stack at Oliv.

I hopped on the clojure journey and haven't looked back. I've come to love clojure due to following reasons:

• REPL driven development
• Relatively small and growth driven community
• Open source orientation
• Down to earth moderators and builders

There are probably more, that I don't recall right now. Join the Official Clojurians Slack if you're interested.

Books

Reading, as a habit, began when I faced an identity crisis during my undergraduate studies. I was looking to improve myself, probably due to self-esteem issues. Since then, I have been continuously reading psychology, philosophy and self-help non-fiction. At a point of time, I was only reading the books and nothing else.

2023 was a moderate reading year. I bought 20+ books, finished 1 and still reading 4-5 books at a time. It's become a hobby more than anything else now. Reading is a stress releaver.

Following is a list of books I read in 2023.

• Courage to be disliked: A paradigm shift is rare, and this book did it in a way that hit close to home. It has a dialog based writing style and I could see myself in one of the two main characters. A beginner friendly introduction to Adlerian Psychology, that promotes the idea that all the problems we face are interpersonal relationship problems. We can solve these problems by helping others and seeing them as our comrades.
• The beginning of infinity: In my teenage and college years, I was fascinated, as well as, troubled by the fundamental questions about universe, our existence and the lack of their answers. This book doesn't answer these questions but adds more perspectives for us to think about them. Loved every chapter of it.
• Goal: A Process of Ongoing Improvement: This book gives an idea about how to prioritise our work and what goals should we strive for in it.
• Designing data intensive application: Well, what can I say except "read this instead of taking those system design preparation courses". An enjoyable read about how large systems work and their most fundamental pieces.
• Atomic Habits: Science and psychology based habit building. James Clear, the author, makes it an enjoyable read by adding short stories about small algorithms we can implement to build good and destroy bad habits.

It is not an exhaustive list of books I read in 2023. The above is the ones that I found most interesting and would recommend everyone to read.

Writing

I came across a few people in last few years that have inspired me to write. Writing also helps me arrange and sharpen my thoughts. Having a brain that is susceptible to distraction poses a different set of problems to the quality of conversations and thoughts.

I wanted to start writing but before that I needed to build a blogging platform with a great design and implementation in clojure. Unfortunately, I always procrastinated as it was no simple task.

Late in 2023, I was going through the mighty borkdude's socials and found the quickblog, a light-weight static blog engine for Clojure and babashka. I was writing my first blog within a minute. Super grateful for this product. Thanks Michiel.

Goals for 2024

I am not one of those people who choose new year resolutions at the beginning of the year and as it passes, forgets about them.

On a high level, I want to work on my family and help them reach a place where they could focus on what's important (Story about this sometime later). Focus on my marriage and get the fundamentals right in terms of our plans and family. I would also like to improve and be cosistent in writing on this blog and reading the books that I couldn't finish in 2023.

These are the most important ones and I look forward to and wish you all have, a great new year.

Quote for the post

God give me serenity to accepts things I can not change, the courage to change that I can and wisdom to know the difference.