Kleinprojects

Testing HTMX 2 with Happy Dom

2026-01-22T00:00:00Z

I have a long running e2e test setup with Playwright that takes around 30 seconds to complete on my local computer and around two minutes in CI. Since I have adopted HTMX I am trying to find a better approach for testing HTMX interactivity without having to work in a real, slow browser environment.

The ~400 other unit tests execute in under a second - why can't I also run interactive tests via HTMX in under a second?

So I gave Happy Dom a shot as it's supposed to be faster then the other popular JSDOM alternatives. However, I hit some limitations in the process. First off, Happy Dom does not implement the XPathEvaluator that HTMX 2 needs:

ReferenceError: XPathEvaluator is not defined
      at <anonymous> (/.../public/libs/htmx-2.0.7.min.js:1:24685)

Since I was not using XPaths at all I mocked the XPathEvaluator implementation with a stub (see code appendix).

In the test itself, I just apply the html to the Happy Dom document like this:

document.body.innerHTML = html;

However, I faced a second issue with an TypeError: window[PropertySymbol.dispatchError] is not a function. error which turns out as a Happy Dom limitation due to the onclick listeners that are directly assigned in some places:

<button onclick="window.scrollTo({top: 0, behavior: 'smooth'})"></button>

I had to refactor those instances, in my case to hyperscript which looks like this:

<button _="on click go to the top of the body smoothly"></button>

And then it works! With bun the initial test ran in around 20ms and the existing Playwright test in around 2000ms making the non-browser version around 100x faster on first sight.

To not execute too many scripts I added an allow-list of scripts that I wanted to execute (see code appendix).

Conclusion

I am happy that I am able to test HTMX interactions without having to spin up a full browser. Going forward I hope this can guide someone into a better testing setup for HTMX applications.

I'll now wander off to migrate more tests into that new setup.

Code Appendix

This is the code for the stub:

class XPathEvaluatorPolyfill {
  createExpression(expression: string): XPathExpression {
    return new XPathExpressionPolyfill(expression);
  }
}

class XPathExpressionPolyfill {
  constructor(private expression: string) {}

  evaluate(
    contextNode: Node,
    type: number,
    result: XPathResult | null,
  ): XPathResult {
    // For now, return an empty result since HTMX primarily uses CSS selectors
    // and only uses XPath for advanced attribute queries
    return new XPathResultPolyfill([]) as unknown as XPathResult;
  }
}

class XPathResultPolyfill {
  resultType = 5;
  private index = 0;

  // Add required XPathResult properties (unused but needed for interface)
  booleanValue = false;
  invalidIteratorState = false;
  numberValue = 0;
  singleNodeValue = null;
  stringValue = '';
  snapshotLength = 0;

  ANY_TYPE = 0;
  NUMBER_TYPE = 1;
  STRING_TYPE = 2;
  BOOLEAN_TYPE = 3;
  UNORDERED_NODE_ITERATOR_TYPE = 4;
  ORDERED_NODE_ITERATOR_TYPE = 5;
  UNORDERED_NODE_SNAPSHOT_TYPE = 6;
  ORDERED_NODE_SNAPSHOT_TYPE = 7;
  ANY_UNORDERED_NODE_TYPE = 8;
  FIRST_ORDERED_NODE_TYPE = 9;

  constructor(private nodes: Node[]) {
    this.snapshotLength = nodes.length;
  }

  iterateNext(): Node | null {
    if (this.index < this.nodes.length) {
      return this.nodes[this.index++];
    }
    return null;
  }

  snapshotItem(index: number): Node | null {
    return this.nodes[index] || null;
  }
}

// Adding to global
globalThis.XPathEvaluator = XPathEvaluatorPolyfill;

Happy Dom setup as global registration:

// Whitelist of scripts that should be loaded during tests
// Scripts not in this list will be blocked to prevent errors from missing browser APIs
const SCRIPT_WHITELIST = ['htmx', 'hyperscript'];

// Helper to check if a script path matches the whitelist
const isScriptAllowed = (pathname: string): boolean => {
  return SCRIPT_WHITELIST.some(allowed => pathname.includes(allowed));
};

// App is a mocked implementation of the web server (in this case Hono with Bun)
const app = createApplication();

GlobalRegistrator.register({
  settings: {
    enableJavaScriptEvaluation: true,
    suppressInsecureJavaScriptEnvironmentWarning: true,
    disableCSSFileLoading: true,
    disableJavaScriptFileLoading: false,
    handleDisabledFileLoadingAsSuccess: true,
    fetch: {
      virtualServers: [
        {
          url: 'http://test.local/libs/',
          directory: './public/libs',
        },
      ],
      interceptor: {
        beforeAsyncRequest: async ({ request, window }) => {
          const url = new URL(request.url);

          if (url.pathname.startsWith('/libs/')) {
            if (isScriptAllowed(url.pathname)) {
              // Let virtualServers handle it
              return undefined;
            } else {
              // Return empty response to prevent script execution errors
              return new window.Response('', {
                status: 200,
                statusText: 'OK',
                headers: { 'Content-Type': 'application/javascript' },
              });
            }
          }

          // Build headers object from happy-dom's headers
          const headersObj: Record<string, string> = {};
          request.headers.forEach((value: string, key: string) => {
            headersObj[key] = value;
          });

          // Forward request to Hono app using URL string
          const honoResponse = await app.request(url.pathname, {
            method: request.method,
            headers: headersObj,
            body: request.body as unknown as BodyInit,
          });

          // Convert Hono Response to happy-dom Response
          const responseText = await honoResponse.text();

          const responseHeaders: Record<string, string> = {};
          honoResponse.headers.forEach((value, key) => {
            responseHeaders[key] = value;
          });

          const happyDomResponse = new window.Response(responseText, {
            status: honoResponse.status,
            statusText: honoResponse.statusText,
            headers: responseHeaders,
          });

          return happyDomResponse;
        },
      },
    },
  },
});

Experimenting with LLMs for Dynamic Game Content Generation

2025-09-28T00:00:00Z

AI generated blobs in the game

I recently built a proof of concept to experiment with large language models inside games, using a locally running LLM to generate dynamic content in real-time. The goal was to create a unique gaming experience where the game evolves based on player actions, with each level featuring procedurally generated content that responds to gameplay.

You can find the complete implementation in the llm-game repository.

The Core Challenge: Constraining Creativity

The fundamental challenge with integrating LLMs into games is finding the right balance between creative freedom and system stability. You need to constrain the AI within a specific frame so that a small, locally running LLM can generate content without breaking the game mechanics.

However, this constraint comes with a significant downside: reduced creativity. In my implementation, I limited all entities to circles to simplify collision detection and positioning logic. While this made the system robust, it also made the generated content somewhat monotonous - everything was just circles with different colors and sizes.

If I would loosen it up too much, I run in the risk of the LLM hallucinating or producing wrong code which would mitigate the player experience.

How the System Works

The core game is limited to a Player, that is controlled with arrow keys, a target that the player has to reach and LLM-generated NPCs. When the player reaches the target the game generates a new NPC and thus gradually generating more and more game content.

For each level, the system generates one or more NPCs with properties. The interface is limited, to reduce errors with the usage of the small LLM. The behaviorCode is the only generated property that the game engine executes.

interface NPCData {
  position: { x: number; y: number };
  color: string;
  size: number;
  behaviorCode?: string;
}

Behavior code wrapped into a TypeScript template and directly executed:

return function(npc, context, deltaTime) {
    try {
        ${code} // LLM generated code is added here
    } catch (e) {
        console.warn('AI behavior error:', e);
    }
};

This function is executed every update, giving the LLM access to the npc, context, and deltaTime variables.

Conclusion

Entity constraints (like only drawing circles), while necessary for system stability, severely limited visual variety and variety in behaviors. Everything generated was fundamentally the same shape, just with different colors and sizes. Those generations could have been done by a simpler generation algorithm. Next time, I would thus explore more fields where LLMs would really excel (like text and story generation).

For me, the experiment reinforced that successful AI integration requires careful architectural decisions and robust fallback systems. The idea shows promise, but finding the right balance between creative freedom and constraints is challenging.

Announcing ... Impromat ... Improvisational Theatre Session Planning

2024-12-21T00:00:00Z

I am sitting in front of my paper based notebook. My shiny improvisational theatre notes stare at me. It's time again to jot down thoughts as I signed up to lead our next session. I am opening a blank page.

A goal.

"I need a goal for my workshop", I think.

Something about group mind? Building characters? Emotions?

"I need some games".

It's always the same.

"I am a bloody software engineer. I can make this process more efficient".

...

Three years pass and I am ready to introduce:

impromat.app - dead simple improvisational theatre planning.

Create Improv Sessions with impromat.app

Why an Improv app?

In most improv groups, one person facilitates an improv session. That person is responsible for providing structure, exercises, games and in rare cases, fun.

There is books on improv. There exist encyclopedias like improwiki, improvencyclopedia, improvressourcecenter. Improv groups posses knowledge and unique styles. Most of that are individual games and exercises that exist to have fun. They are labelled sometimes. For example, the improwiki website assigns tags to their elements. That eases the search for warmup, exercises, or games.

When you ultimately prepare and run an improv session you are lacking tools.

I formed a new improv group and ran sessions on "how to do an improv workshop" to allow participants the facilitation. I run these sessions very spontaneously because there are many games and exercises that I know and that I can put into the workshop. That spontaneous setup makes it hard for me to pass on that knowledge or at least the games that we play.

And how do you seek inspiration for you improv group? You have your known set of games, but how do you get new ideas? You could participate in external workshops, or hire an external workshop leader. You could read books, or browse some of the online sources.

Guess what. There is a solution to all of these problems: Impromat.

Impromat inspires you with unique and new games. Impromat provides you with a dead simple planning tool that allows you to plan and share workshops with participants or the community.

Introducing Impromat

Known improv sites like, Improwiki, improvencyclopedia or improvresourcenter served as the initial content for my personal planning. I soon began developing a small application around it to make the planning of workshops more efficent. Then, I opened it up for everyone to use.

There is no good way right now to have a good improv session planning tool available that covers improvisational theatre. Most people use Excel or their personal notes.

Moreover, there is no easy approach in sharing improv workshops to inspire others.

Impromat brings these two aspects together: getting inspiration through other sessions -> planning a new session -> inspire other sessions -> planning a new session...

Of course, everyone has their private space. You don't have to share what you don't want to share.

So, what the heck is it, how is it build?

Elements and collections are in the core of the application. Elements are exercises or games like Freeze Tag, Dutch Triangle, Association Circle, and so on. Collections contain elements in certain orders like workshops. They contain elements that an improv leader goes through in the course of the session.

Elements are tagged to make your life easier with finding suitable ones. If you plan a workshop around character building you could filter all elements by the character tag. Elements are the smallest set of information to describe an activity that the group or individuals can perform.

Currently, there is the concept of workshops, e.g. sessions that you will actually lead in an improv group. But you can also see workshops like playlists and the elements like songs and create collections of elements that fit to your planned show format around the meaning of life.

How can I use it?

You can access Impromat on any device with a browser. It supports mobile and desktop layouts so you can dive into improv when procrastinating other important work on your PC, sitting in the bus or walking unprepared into the room where you should lead the next improv session.

Over 1000 fun and boring improv elements are waiting to be opened on Impromat Elements. Browse them, search them, give them a pet. If you like them, they like you. If you don't like them, you have the power to make them better, or pick another one.

For planning your workshops you can start with Creating a Workshop. Here you can use our improv workshop generation engine or start with a plain collection to fill in your ideas.

Use your private space to start with your creation. Contribute to the improv community by publishing your workshop so improvisers can use your inspiration.

Future

There is some stuff planned down the road. But like a road is first just a path of gravel, Impromat is in its first iteration. Join to make Impromat an Autobahn:

If you are an improviser, give it a shot.

If you are no improviser, give it a shot.

If you have an opinion, share your thoughts and ideas.

If you want to develop, join with some coding.

If you want to be a tree, give it at shot.

If you just landed here and don't know why, give it a shot.

If you feel frustrated, happy or hungry, give it a shot.

Let us have some fun.

impromat.app

Documenting Decisions

2024-08-17T00:00:00Z

This sheet describes some of my experiences with documentation. In particular decisions that would impact technologies and teams. There exist several approaches, one of the is the documentation via Markdown in the ADR or MADR format.

ADR is an architectural decision record, while MADR builds on that an tries to widen the scope to all decisions that are made in a project. Eventually, it doesn't matter what kind of format you would use as long as you decide on writing down agreements and context for your problems.

Most Important Sections of a Decision

The most important sections of a decision record are:

Context and Problem Statement
Considered Options
Decision Outcome

Context and Problem Statement

When writing the decision record, the most important section is the description of Context and Problem Statement. This should describe the situation that led to the decision. Be clear and concise about the background and problem at hand.

Example:

When constructing the `o_employee` table, we realized that there are false entries. For example, it showed roles for certain engagements that were not added by the intended user. This sheet documents how the issue was debugged and the proposed solutions.

Considered Options

The considered options gives a background on all options that you evaluated or thought about. You do not have to go into detail about each option, but this is the chance of providing a brief overview.

Example:

- Improve the internal merging of engagements for the employee table.
- Allow users to flag false and confirm correct merges.

Decision Outcome

This section describes the decision that was made. If applicable, you can add an implication section that depicts consequences of the decision.

Example:

We decided to improve the internal merging of engagements. This will allow us to keep the system less complex because we would have to add additional logic for dealing with user input. The limitation is, that we might still miss on edge cases due to the complexity of merging engagements.

Conclusion

It can be very easy to document decisions but it is so often overlooked or de-prioritized. In my experience, writing down small decisions help a lot with improving team collaboration and communication by pulling out already taken decisions to avoid reoccurring conversations about tech choices.

References

Examples for Architecture Decision Records: GitHub repo with examples for ADRs
Markown Any Decision Record: approach to document all decisions (not only architecture decisions) in a consistent way

The Hardest Part of a Personal Project is the Release

2024-05-01T00:00:00Z

This article reflects my current struggles with releasing my personal project Impromat.

Goal of the application is to create a cool tool for the improvisational theatre community where you can easily plan and share improv workshops. There is already existing wikis like improwiki, learnimprov, or improvresourcecenter. However, they are focusing on individual game and exercise descriptions and not on more holistic workshops. For example, what elements should you pick if you would do an improv session with pure beginners around the topic of accepting and saying yes to each other ideas?

This is the value that Impromat.app brings: creating more holistic improv experiences through a collection of meaningful elements to create immersive improv workshops.

It started quite straight forward: fetch all elements from existing wikis (with respective licensing) and place them one after another in a list, which is then the description of a workshop. That works.

Now the question of questions: when is it actually finished? When do I feel comfortable with publishing the release number 1?

Potentially never.

Potentially never, when I do not decide on what to focus on.

There is this saying: to go fast, go alone, to go far, go together. And I find that partially true. I am developing really fast, but often I am just developing things that are completely irrelevant for the projects success. For example, switching the frontend framework into a newer version, or redoing functioning deployment pipelines to try a different approach.

I am developing very fast. Very fast into any direction. That direction could be wrong. I don't know, I am just developing.

This development and technically driven motivation used to me very fine for me. I am happy spending a couple of hours fixing something there, implementing something over here. However, I am now realising that I would like to show the stuff that I am doing to more people. It would be quite easy to show it to more people by posting my app in online forums or reach out to more friends to try it out.

However, it is extremely challenging for me to find that cut, to be like "now, this is a good state Impromat is in". That is, because I am finding details that do not add up. For example, designs that are not 100% accurate. Or that the website is a little bit too slow because a search request takes 600 milliseconds, but I want it to be under 100 so that it feels smoother. Then, on the other side I am grinding myself up in exactly those details.

I am getting lost in details because I am not 100% satisfied. And I do not have the definition of 100%.

Then there are different saying like the 80/20 rule: you need 20% of your time to get to 80% success but then 80% for the last 20%. I think I am somewhere there. Overall it works great. Overall it is good. I am just always finding new things I could improve or even implement. Nonetheless, I might be stuck in my own box so it is crucial to show what Impromat is to a wider audience now.

Am I stuck in my own box of ideas? I only know when validating my product with end consumers as a good product manager would tell me.

To address this, I made a plan:

I am going to release version 1 of Impromat to friends as soon as the basic flow of workshop creation works for visitors (e.g. what is the minimum what a user requires for creating a simple improv workshop?).
I am going to post a small announcement on Reddit to broaden the user base.

I even wrote a small strategy for this because I was reading an engineering management book from Will Larson.

Those thoughts were possibly just for me, but possibly also for others that face struggles with releasing their personal projects.

Impromat Strategy

2024-04-27T00:00:00Z

As I am currently reading An Elegant Puzzle by Will Larson, I am reflecting on the approach on writing strategies. I want to try out the approach of writing strategies with my personal application impromat.app, a tool for planning improvisational theatre workshops.

To test it out, I am going to tackle the challenge of me being the only developer in Impromat (because it's my hobby project) and me not finding the right cut to release a version 1.0 that I can then potentially market in Reddit or other improv communities.

Diagnosis

A diagnosis that defines or explains the nature of the challenge. A good diagnosis simplifies the often overwhelming complexity of reality by identifying certain aspects of the situation as critical.

I am the only developer in Impromat with a technical background, lots of ideas but limited time. I love implementing things and work on technical tasks but it is challenging for me to find the cut of when to end development and when to start marketing or generating a user basis.

The challenge is rather, that I often loose focus on the features that I planned or that I did not even plan those features but just dive into the first technical challenge that comes up. This leads to feature development that might sometimes be unnecessary for the success of Impromat as a product.

I am sometimes working on CI/CD because it is fun to optimize the deployment process. I work on framework optimizations and minor features that are fun to work with but not necessarily needed for the first version.

Guiding Policy

A guiding policy for dealing with the challenge. This is an overall approach chosen to cope with or overcome the obstacles identified in the diagnosis.

Stick to the features that are needed for the first version.
Show what you got to others, even if it is not perfect.
If it works, it is good enough for the first version. Do not optimize that.

Coherent Actions

Coherent actions designed to carry out the guiding policy. These are steps that are coordinated with one another to work together in accomplishing the guiding policy.

Define target group for version 1.0. Who are the users that I want to target with the first version?
Define features for version 1.0 to target the defined group.
Implement exactly those features. Not more.

References

Project Steering through Stakeholder Mapping

2023-10-14T00:00:00Z

AI Art: Company structures might seem complex and messy at first.

Steering new projects in a leading position can be challenging, especially in large companies where navigating relationships with multiple stakeholders is crucial. How can you guarantee that your project goals align with those of the company? And how can you ensure that the actions you take go in the right direction?

This article discusses an approach to stakeholder management that provides a better understanding of how to turn unknown environments into known dependencies by focusing on people.

Overview of Relevant People

At the core of an organisation are individuals. The first step to stakeholder mapping, therefore, is to gather the people involved in your project activities. You could get them from meeting minutes, the company organisational chart, or by asking other project members. It's important to collect their names, roles and project involvement. Everything that provides context to that persons background. For important people you may also want to collect pictures so that you recognise them when you bump into them in the office . You could get those pictures from the company profile picture or from business networks like LinkedIn (pay attention to data privacy - if in doubt, collect only what is really necessary). Even though we are going to prioritise those people try to focus on important individuals and neglect unrelated ones.

Sample for a raw collection (made-up names):

Mary: CEO
Robert: Project Lead of Project X
Susan: Project Lead of Project Y
Thomas: Software Engineer in Project X
...

Take this collection as an example: there are 2 different projects and the CEO Mary who is an important stakeholder to our project (in sake of the example). Choose a representation that is most suitable for you. You could also use a list, or directly a diagramming tool that you can use at a later stage for visualising relationships like this:

Collection of stakeholders

In the beginning of projects I usually rely on people that have been involved in the project already. Additionally, as I take extensive meeting minutes I extract relevant individuals directly from my notes.

Prioritising and Clustering

This section describes how the collection is stripped down to most crucial and relevant individuals. There exist various dimensions on which the prioritisation could be based on. Try to reduce the collection while retaining the most comprehensive overview. Take the following examples as inspiration:

If you have many different projects ensure to keep the most important person per project. Example: your list contains five people for project X and the project is not that relevant you write down the most important person from your point of view.
To represent decision hierarchy, prioritise by role and influence of that person. Example: people within a steering committee take budget decisions which has a direct influence on project extensions.
A personal relationship with individuals is very valuable. Example: you have worked with Thomas (from collected persons) in a previous project so you could exchange with him on a deeper level to get valuable insights into Project Y.

As mentioned, this is just an inspiration of dimensions and incomplete. Eventually, it is important for you to understand what your project currently needs: are you planning the next project extension? Then it is important to collect all decision makers for the budget. Are you planning on implementing an upcoming feature? Then it is crucial to prioritise people and projects that influence or depend on that feature.

In a recent project, I extracted over 70 individuals from meeting minutes associated with a particular project. After prioritisation, I reduced the list to 10. I used a combination of dimensions to ensure that the most important people and projects were captured.

Putting People in Perspective

This sections describes how the filtered collection is mapped into a dependency map of stakeholders. Take you favourite visualisation tool. If you are creating the map collaboratively ensure to have a tool that synchronises changes. Also think about maintenance: are you creating the map for depicting a one-time visualisation or do you want to continuously update changes potentially even within the team?

With people names and roles wrapped in ellipses and project scopes denoted by rectangles a basic visualisation of above collection looks like the following picture. Arrows with a short text describe the relation of your project to other projects. The description is based on your understanding of the relation and might be directly related to a project or a person. If you are unsure about the relation you can also omit them in an initial step and just place a directional arrow.

Basic stakeholder map

Mary as the CEO is sponsoring the project, while Our Project relies on the functionality of Project X and, Project Y is interested in utilising our project's features. Note that you might want to add profile pictures to recognise people if you bump into them.

I usually have different maps for different occasions with different people. One time I had to visualise dependencies across companies and thus introduced a company boundary. Over time I have also established a fixed set of figures inspired by the C4 modelling technique.

Conclude Understanding

Your relational stakeholder map can help you understand the decisions and importance of the people involved. For example, what action should you take to secure Mary's continued sponsorship? This would require a deeper understanding of her goals and interests in your project, which you could derive from the stakeholder map.

If you were only looking at titles, Thomas might not seem important in terms of project scope or governance, but you might have some questions about functionality that your project depends on, and Thomas might be the quickest person to answer (rather than going through Robert). Susan may contact you frequently if you fail to keep her informed of your next project delivery. To further define the scope and direction of the project, you could discuss the requirements with them.

Once you have mapped all the important dependencies, you should share the map with your team, or anyone else who you think will find the map useful in understanding the bigger picture. The map allows you to create a shared understanding of your current project dependencies. It allows you to create alignment by clarifying relationships with people (who knows whom?).

The stakeholder map you create is a living document. It is a snapshot of one dimension rather than a holistic understanding of the project. Use it as a tool to understand dependencies and create alignment. You will need to refine and update it as your project environment changes.

I usually just create shared stakeholder maps that everyone in the team can access and edit. Especially at the beginning of new projects or when there is a big change in scope, the mapping has helped a lot to get a common understanding of the project.

Conclusion

This article presents a stakeholder management method that involves gathering relevant individuals and visualising their interdependencies. The resulting map can aid internal project team alignment and facilitate understanding of project decisions and their impact.

It represents only one dimension of the overall project setup and requires effort to keep it up to date. In addition, there are many other perspectives on stakeholder management, such as approaches discussed in the Miro blog or related topics on staffeng.com.

For me the greatest strength of the whole exercise is the alignment with team members. When I present my drawings I have often gotten the feedback that this map really helped with understanding the bigger picture of projects and teams.

Improving the Navigation of Impromat.app with a Bottom Navigation Bar

2023-09-10T00:00:00Z

To collect information about user behavior, I observed individuals using Impromat.app. Quickly, I noticed, that one issue was the navigation. For instance, users were not clicking on the drawer icon located in the top-right corner in order to view all functions of the app (such as the ability to search for improv exercises and games, build a personal workshop, or manage individual exercises and games). Even worse, some mobile devices have a native feature that navigates back when swiping from left to right. This made it extremely difficult to navigate the app, as users were unable to open the drawer.

So, I sought a solution to improve the discoverability of Impromat.app and enable swiping from left to right to open a drawer on devices lacking the option. Below is a screenshot of the elements page which allows improvisers to search existing elements and exercises.

The screen displays Elements for searching exercises and games. The app's other features are concealed in the drawer (refer to below), which may lead users to believe that only "exploration," "liking," or "my library" are possible actions.

However, clicking on the drawer icon on the top right reveals the full functionality of the app.

Open drawer that shows all available sub-pages. Please note that the search function is only one of its components.

Initially, I had considered dropping web app support and exclusively backing the locally installed application to avoid browser incompatibilities. However, this solution would not resolve the app's discoverability.

Therefore, I researched other frameworks and applications and discovered the top-level navigation of tabs, which are utilized by apps such as Spotify and Google Play store.

The Material 3 design offers helpful documentation on its website. According to their suggestions, a bottom tabs navigation (referred to as a "navigation bar") is suitable for compact screens, while a side bar navigation is better for larger screens.

Given the mobile-centric approach of Impromat, the bottom navigation bar with tabs is an appropriate starting point. Here is a preliminary version of the bottom navigation bar:

Impromat with a navigation bar to reveal more functions and simplify user interaction.

Sections are divided into the three main areas:

Home: About information, news, profile, settings, legal notice, data privacy, etc.
Exercises & Games: Search improvisational theater elements that you can add to workshops and manage your own entries.
Workshops: Plan and share your improv workshops.

Wrap-up

The implementation of the new navigation approach is currently under development, but I already like it a lot for its easy app discoverability, conciseness, and ease of use (just one click instead of a swipe gesture). However, I am still determining the optimal high-level differentiation, which will depend on users' goals when they open Impromat.

In the future, I will integrate a more adaptive design following the guidelines of Material Design source. Currently, I must assess whether implementing these changes will enhance the user experience before proceeding.

Navigating an Uncertain Project Landscape: Finding Your Path When Direction is Unknown

2023-08-13T00:00:00Z

As a consultant, I work with clients in a variety of IT projects and roles. It is common to find oneself in a complex environment that one has to learn how to navigate. I have been reflecting on my learnings, particularly regarding projects that I am involved with at multiple levels, from management to writing code.

Find the right people to talk to

If you are starting a project with a high level of uncertainty, you will want to start by creating an overview. Gather all the available material and create a stakeholder or project overview that you can refer to when aligning priorities with your direct client. In hierarchical organisations, you will usually also find an organisational chart to help identify structures and project set-ups.

The following image shows a very rudimentary description of people and projects. The arrows show the connection between our project and the neighbouring projects (e.g. Project Pen or Project Flower). Normally I would write the name and role of the person in the boxes:

Of course, the diagram is fluid: new decisions may affect the connection to a project. New stakeholders might become more important. People might leave projects, or projects might be reorganised. Also, there is no "one way" to approach this. The chart can have different views. For example, I once had a situation where we had to manoeuvre our project through a very political situation, so we put green, yellow and red dots on people to keep track of our communication with individuals.

The main goal is to create transparency and easy alignment with your project team. This allows you to plan specific communication actions and helps you to understand the roles of existing projects. The goals of other projects (e.g. what might a collaboration look like?) may become clearer.

Find the right abstraction layer

A project is not one-dimensional. It is not just a presentation to management to get the budget approved. It is not just the technical implementation. And it is not just the product specifications. There are different ways of looking at the project scope and requirements. As I support on all layers, e.g. management, architecture and coding layer, I found it extremely important to find the most effective layer and stick to it.

You may define your own set of layers that help with leading questions. Here are some notes from my differentiations:

Product layer: What is the value we create? Who are our end users? Why should others use our product?
Management layer: Who will own the project in a year's time? What are our next milestones? How much budget do we need to continue?
Technical layer: What are our technical interfaces? What are the components of our architecture? What programming languages are we using?

Layers can vary. If I were to write more code, I would also get into a coding layer: what do we need to implement to prove our architecture? What code do we need to write to demonstrate our MVP functionality?

Thinking in layers allows you to reduce complexity and better assess what your project needs most right now. Is the value proposition clear to everyone? Is the budget realistic? Do we need to demonstrate feasibility through technical implementation? Questions like these can guide you to the next steps that will have the greatest impact and be the best use of your time. Conversely, it would not help to have the most visually appealing technical demonstration if none of your stakeholders understand the business value of the project.

Parallelise Streams

Do not bet on a single horse, I learned it the hard way: after the first iteration we proposed a continuation of our project to management and got funding for a project plan approved. However, our collaboration was based on only one project, on whose success we were apparently too confident, because that one project collapsed in the second week after the start of our 6-month project iteration (which we had based on collaboration with that project).

After a panicky search for our other project collaboration options, we ended up with seven parallel project streams again. This was too much for our three-person team. However, after some assessment, we ended up with a handful again, which was totally manageable and would spread our risk across multiple project collaborations.

We and I have learnt the hard way to parallelise project streams. This reduces risk, because project dependencies can always fail or lead in an unwanted direction. More options give you more freedom to steer and shape your own vision.

As discussed, we are taking about a project that is complex in the sense that business requirements are not yet defined. The end users have not yet been fully identified. So it is crucial to create transparency about what you are working on. At least this is how I would build trust for myself: keep others informed about what has happened during the week and share thoughts about it. Thoughts could be potential risks I see or certain priorities we should focus on (from my point of view).

I would write these notes once a week and share them with my client and my internal delivery coaching team (e.g. consultants supporting my engagement). I also found that the right channel was important. Initially, I sent them by email without getting much response. After a while, I switched to our chat tool and just chatted my notes, which worked much better. It gives people a chance to react and comment on my weekly notes. It also gives me confidence and feedback that the points I have raised and will raise are the right ones.

Fuzzy Search Considerations for Impromat

2023-08-07T00:00:00Z

Context

Currently, the search for element is implemented with the fuzzy search library fuse.js.
For a search, all elements of the database are loaded and then searched via fuse.js.
This provided an easy start BUT: the approach does not scale as all elements have to be searched for every call.

Technology Stack

The investigated options consider the current stack:

Postgresql as database
Prisma for db mapping

Thus, options span from Postgres native features to Prisma capabilities to stay with the existing stack.

Considered Options

Prisma Full Text Search
Postgres pg_trgm
Basic text search via LIKE %searchText%

Prisma Full Text Search

Prisma preview feature for searching through database fields via a search logic:

Good, because Prisma is the db abstraction layer (shift complexity to abstraction layer).
Bad, because the search is not fuzzy but is a set of static search rules.
Good, because it provides huge performance benefits over the current solution.

See Prisma documentation

Postgres `pg_trgm`

Postgres trigram extension for fuzzy search features:

Bad, because it requires raw SQL (mixing with Prisma as abstraction layer).
Good, because it is extremely performant as it works directly with Postgres.
Good, because it is fuzzy.

Basic text search via `LIKE`

Good, because it provides the fastest query time.
Bad, because search is not fuzzy at all.
Good, because it can be implemented via Prisma contains filter.

Decision

Still unclear, how important fuzziness of the search is.

Switch to easiest contains option.
Observe user experience without fuzzy search and verify it is still usable.
Switch to pg_trgm in the mid-term if Prisma is not supporting a fuzzy search.

Prometheus and Grafana with Dokku

2023-07-30T00:00:00Z

The following instructions describe how I have setup server-side monitoring for Impromat.app. The server is running Dokku for managing all services.

The monitoring primarily relies on:

Prometheus
Grafana
node-exporter
cAdvisor

It also sets up a data source for:

PostgreSQL

Tested with:

Ubuntu 22.04
Dokku 0.30.2

Based on this article with adaptions to make it work with dokku version 0.30.2.

Replace impromat.app with your own domain.

Install required plugins

sudo dokku plugin:install http-auth https://github.com/dokku/dokku-http-auth.git
sudo dokku plugin:install letsencrypt https://github.com/dokku/dokku-letsencrypt.git

Install Prometheus

Add dedicated network for prometheus services (see Dokku Networking):

dokku network:create prometheus-bridge

Create prometheus app:

dokku apps:create prometheus
dokku proxy:ports-add prometheus http:80:9090
dokku network:set prometheus attach-post-deploy prometheus-bridge

Create volume mappings:

sudo mkdir -p /var/lib/dokku/data/storage/prometheus/{config,data}
sudo touch /var/lib/dokku/data/storage/prometheus/config/{alert.rules,prometheus.yml}
sudo chown -R nobody:nogroup /var/lib/dokku/data/storage/prometheus

dokku storage:mount prometheus /var/lib/dokku/data/storage/prometheus/config:/etc/prometheus
dokku storage:mount prometheus /var/lib/dokku/data/storage/prometheus/data:/prometheus

Add prometheus docker start command:

dokku config:set --no-restart prometheus DOKKU_DOCKERFILE_START_CMD="--config.file=/etc/prometheus/prometheus.yml
--storage.tsdb.path=/prometheus
--web.console.libraries=/usr/share/prometheus/console_libraries
--web.console.templates=/usr/share/prometheus/consoles
--web.enable-lifecycle
--storage.tsdb.no-lockfile"

--config.file sets the location of the prometheus config path.

Create config file at /var/lib/dokku/data/storage/prometheus/config/prometheus.yml:

global:
scrape_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    scrape_interval: 15s
    static_configs:
      - targets:
          - "localhost:9090"
  - job_name: node-exporter
    scrape_interval: 15s
    scheme: https
    basic_auth:
      username: <username>
      password: <password>
    static_configs:
      - targets:
          - "node-exporter.impromat.app"
  - job_name: cadvisor
    scrape_interval: 15s
    scheme: http
    static_configs:
      - targets:
          - cadvisor.web:8080

Deploy prometheus:

docker pull prom/prometheus:latest
docker tag prom/prometheus:latest dokku/prometheus:latest
dokku git:from-image prometheus dokku/prometheus:latest
dokku domains:set prometheus prometheus.impromat.app

dokku letsencrypt:enable prometheus
dokku http-auth:enable prometheus USER PASSWORD

Validate running Prometheus at https://prometheus.impromat.app/targets (using respective USER and PASSWORD)

Create node-exporter

dokku apps:create node-exporter
dokku proxy:ports-set node-exporter http:80:9100
dokku config:set --no-restart node-exporter DOKKU_DOCKERFILE_START_CMD="--collector.textfile.directory=/data --path.procfs=/host/proc --path.sysfs=/host/sys"

dokku docker-options:add node-exporter deploy "--net=host"
dokku checks:disable node-exporter

Storage mounts:

sudo mkdir -p /var/lib/dokku/data/storage/node-exporter
sudo chown nobody:nogroup /var/lib/dokku/data/storage/node-exporter

dokku storage:mount node-exporter /proc:/host/proc:ro
dokku storage:mount node-exporter /:/rootfs:ro
dokku storage:mount node-exporter /sys:/host/sys:ro
dokku storage:mount node-exporter /var/lib/dokku/data/storage/node-exporter:/data

Deploy node-exporter:

docker image pull prom/node-exporter:latest
docker image tag prom/node-exporter:latest dokku/node-exporter:latest

dokku git:from-image node-exporter dokku/node-exporter:latest
dokku domains:set node-exporter node-exporter.impromat.app

dokku letsencrypt:enable node-exporter
dokku http-auth:enable node-exporter USER PASSWORD

Create cAdvisor

dokku apps:create cadvisor
dokku proxy:ports-set cadvisor http:80:8080
dokku config:set --no-restart cadvisor DOKKU_DOCKERFILE_START_CMD="--docker_only --housekeeping_interval=10s --max_housekeeping_interval=60s"
dokku network:set cadvisor attach-post-deploy prometheus-bridge

Storage mounts:

dokku storage:mount cadvisor /:/rootfs:ro
dokku storage:mount cadvisor /sys:/sys:ro
dokku storage:mount cadvisor /var/lib/docker:/var/lib/docker:ro
dokku storage:mount cadvisor /var/run:/var/run:rw

Deploy cadvisor:

docker image pull gcr.io/cadvisor/cadvisor:latest
docker image tag gcr.io/cadvisor/cadvisor:latest dokku/cadvisor:latest

dokku git:from-image cadvisor dokku/cadvisor:latest

Setup Grafana

Create app:

dokku apps:create grafana
dokku proxy:ports-add grafana http:80:3000
dokku network:set grafana attach-post-deploy prometheus-bridge

Storage mounts:

sudo mkdir -p /var/lib/dokku/data/storage/grafana/{config,data,plugins}
sudo mkdir -p /var/lib/dokku/data/storage/grafana/config/provisioning/datasources
sudo chown -R 472:472 /var/lib/dokku/data/storage/grafana

dokku storage:mount grafana /var/lib/dokku/data/storage/grafana/config/provisioning/datasources:/etc/grafana/provisioning/datasources
dokku storage:mount grafana /var/lib/dokku/data/storage/grafana/data:/var/lib/grafana
dokku storage:mount grafana /var/lib/dokku/data/storage/grafana/plugins:/var/lib/grafana/plugins

Add prometheus data source to /var/lib/dokku/data/storage/grafana/config/provisioning/datasources/prometheus.yml:

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    orgId: 1
    url: http://prometheus.web:9090
    basicAuth: false
    isDefault: true
    version: 1
    editable: true

Deploy Grafana:

docker pull grafana/grafana:latest
docker tag grafana/grafana:latest dokku/grafana:latest
dokku git:from-image grafana dokku/grafana:latest
dokku letsencrypt:enable grafana

Go to https://grafana.impromat.app. Login via admin/admin and set a new password.

Adding more data sources

PostgreSQL

Get database info from dokku via:

dokku postgres:info <db-service-name>

Add /var/lib/dokku/data/storage/grafana/config/provisioning/datasources/postgresql.yml:

datasources:
  - name: Postgres
    type: postgres
    access: proxy
    url: <Internal Ip>:5432
    user: postgres
    database: <database name>
    secureJsonData:
      password: <DB_PASSWORD>
    jsonData:
      database: <database name>
      sslmode: "disable" # disable/require/verify-ca/verify-full
      maxOpenConns: 0 # Grafana v5.4+
      maxIdleConns: 2 # Grafana v5.4+
      connMaxLifetime: 14400 # Grafana v5.4+
      postgresVersion: 1400 # 903=9.3, 904=9.4, 905=9.5, 906=9.6, 1000=10

See Grafana Data Sources Documentation

Thoughts on Blogging

2023-07-24T00:00:00Z

So, I have been reflecting on blogging because it has been on my mind for quite some time to get more active with it. I have just been wondering why I blog and what I want to write about.

The following are just some thoughts that I am collecting about blogging:

Personal vs. Professional

Personal blog is hosted on own website. Not necessarily bound to my current company or career defining topics. I could share anything that I personally identify with:
- Coding
- Improvisational Theatre
- Climbing
- Music
- Other random stuff
Professionally, I could write about things that are related to my current project or my current employer:
- Topics I want do a knowledge sharing session about
- Experiences regarding my roles in software projects
- Coaching
- Mentoring
Personal articles may be:
- Incomplete in respect to thoughts and written content
- Covering random topics
- Might be reflective loud thinking
Professional articles may be:
- Spellchecked
- Proof read
- Published and referenced in professional and social networks (e.g. LinkedIn, Medium, Reddit)

Draft to Release

I might start writing something without finishing it. Shall I still publish?

I should, because it might be motivating to publish stuff.
I should, because it is hard to know when an article is truly finished for it's "release".
I should not, because I would publish scattered information.
I should not, because it might also indicated that the article is not of a high priority.

Could be at work or dedicated meetups.
Good, because you can go into discussions and get feedback.
Good, because you can validate ideas.
Bad, because it is more time effort than just posting it (Really? A post might also take a long time.)
In general, I see it as crucial step towards effective knowledge sharing.

Find conferences to speak at

Good, because you get a wide reach.
Good, because talking and presenting is fun.
Bad, because it might be hard to find a conference that is a good fit.
Bad, because it needs the most effort.

Acting Authenticity

2023-05-02T00:00:00Z

Was conducting a public speaking workshop session. One of the exercises was a 2 minute improvised talk about a certain topic. As participants also wanted me to present I took it and improvised a two minute speech about something.

Was interesting to get feedback: ticking all boxes for all categories that we have listed for "voice". BUT feedback was also that I did not came across authentic at all. "Felt like an advertisement". I also wanted to make it feel like an advertisement but was wondering: how can I control when I act versus when I try to be authentic? Can one reach authenticity when performing an act on stage?

Still an open thought.

Next 13 and Mantine will not replace the Ionic framework for Impromat (yet)

2023-03-26T00:00:00Z

Since Next 13 got released in October 2022 I wanted to test the framework to see if it is a viable option for Impromat, potentially replacing the Ionic framework.

Main motivations are:

Customized styling with the styling framework Mantine.
Trying out cutting-edge technology with newest Server Side Rendering (SSR) improvements.

However, Impromat is not meant to be a website but it is targeting mainly mobile devices through an installable application. Technically, this is implemented via a Progressive Web App (PWA) to retain one code base for several target devices. Next 13 may offer great improvements regarding SSR and implements cutting edge patterns and newest React improvements. Nonetheless, one of the main questions remained: why should I implement a bleeding edge SSR framework, if I am going to wrap it into a PWA either way?

An additional argument for Ionic is its support for Android and IOS devices including a easy component framework that offers most app requirement needs out of the box. Switching technologies would imply a re-implementation of the whole styling framework and developing most of the components from scratch.

Nonetheless, for the future I might still consider a switch to Next 13 to try out latest technology and benefit from SSR that Impromat could leverage for an improved Search Engine Optimization (SEO). Right now switching costs would just be to high.

Reflection on making Impromat accessible for search engines

2022-12-13T00:00:00Z

The following is a note to myself to reflect on options for making a current personal project Impromat ready for search engines.

Background

Impromat is a single page application (SPA) and currently hosted on https://impromat.app. However, this comes with several issues:

It's challenging to deal with search engine optimizations (SEO). As this article from Google describes you can assume that crawlers cannot process JavaScript and thus cannot process SPAs.
In the future a login might be required. Then it's not possible to "just share" anymore, however, sharing workshops is still one core feature of the Impromat.

Possible Solutions

Combine SEO optimization within the application

This means, that the backend would have to implement some kind of server side rendering (SSR) to allow the delivery of pre-rendered HTML that search engines could parse. There is another article from Google that also mentions an,unfortunately discouraged, approach called dynamic rendering. With this approach the web server would send a different, SEO optimized result to web-crawlers.

Google also provides a very good discussion in this article about the options and their advantages and disadvantages.

Separate the application page from a SEO optimized landing page

Another approach would be the separation of the actual application that users access, and a separate landing page that can act as a representation within the world wide web. From SEO perspectives this would be an ideal case because a well optimized, dedicated web server could host the respective landing page and link to the Impromat application. However, this also adds another service and another application that needs maintenance.

Conclusion

Both approaches got advantages and disadvantages. I have to reflect on the goals in regards to SEO and future development of the Impromat.

Thoughts on Data Privacy for my free time project Impromat

2022-11-06T00:00:00Z

Just some thoughts on data privacy and GDPR

For my current project Impromat which is an app for planning workshops for improvisational theatre, I have spent the last couple of weeks getting into that topic of data privacy and GDPR. In a nutshell: it's kind of complicated for such a small free time project. However, I value data privacy and I am sometimes a little bit afraid of some lawyers that specials in finding websites without data privacy notices.

Nonetheless, there are already some good resources about GDPR and also some decent generates like the Datenschutz Generator of Dr. Thomas Schwenk. The author of the website was also interviewed in a podcast which helped me with my understanding of data privacy especially in regards to GDPR (just search for it on Spotify if you are interested). Especially the generator helped me a lot with the setup, even though, I had to change some text descriptions and some contract information with my hosting provider (e.g. the Impromat backend is hosted on Contabo).

Impromat is not a commercial application and does neither track nor collect any user specific data per default (would be far to complicated for me to set up :P ). However, there is also a login functionality so that users can synchronize workshops across devices. Through a Google login users can get access to that synchronization feature and workshop data gets transferred to the Impromat backend server and shared with devices of that user. This is where data privacy really gets interesting because according to GDPR there are some regulations that the application must fulfill in this case. For example: users must be able to retrieve all there stored data and they must be able to request deletion of their data. I am very happy that Impromat is currently in such a simple state that I have a 100% knowledge about what kind of data is stored where.

After all, thinking about all those data sets that this small application processes is already very valuable to me. I still think that GDPR is a step into the right direction to protect user specific data, however, it's far to complicated for hobby or free time projects that you only spend a couple of hours per week or even month on.

Using the version from package.json in React applications

2022-10-20T00:00:00Z

You want to display the application version from your package.json in your frontend React application, that is using react-scripts.

React scripts automatically picks up .env files and populates environment variables that start with REACT_APP_. Thus, for runtime use a .env file with:

REACT_APP_VERSION=$npm_package_version

This populates the version from the npm_package_version environment variable on runtime which is set by the NodeJs environment. However, this does not always work if you are building in different environments.

Therefore, for build time use a tool like genversion in package.json:

// ...
"scripts": {
  "start": "react-scripts start",
  "build": "genversion src/version.gen.ts && react-scripts build",
  // ...
}
// ...

And include the version in your application for example via:

export const environment = {
  VERSION: process.env.REACT_APP_VERSION ?? version,
};

Ignore the generated version file in your .gitignore.

With this approach the version gets populated if you start your react app through the .env and you ensure the version on a build via the genversion command.

Incremental User Interface Development

2022-08-21T00:00:00Z

Developing a cool user interface can be really hard. It starts with the "what does the user actually wants to do?"-challenge and involves questions like "where does does the user wants to access the application?". The answer is often that neither you nor your user really knows what they want.

Recently, I had to deal quite often with this "user interface challenge" and how to develop and effective approach to develop a cool app that solves exactly the problem a user is facing. Therefore, I tried to take a step back and observe my approach to frontend development.

Initially, I observed, that I would spend tons of time looking and reading through new UI frameworks that would fit my need. And then there is a difference between the (1) UI framework and the (2) frontend framework. (1) is a library for the actual styling, like Bootstrap, Evergreen, Onsen UI, ... and (2) is a library that helps with the implementation of logic and state, like Angular, Vue, or React.

So, for the frontend framework I would just pick whatever which is mostly React right now. The challenge is (1), the UI framework because there are sooo many different frameworks and lots of advantages and disadvantages. E.g. if you want to build a hybrid app you might use Onsen UI, if you use React components only you might want to use Evergreen, or Material UI, there are old frameworks that still persist like Bootstrap. Simply too many to evaluate all options before even designing.

So how could I avoid spending all that time reading through pros and cons for individual UI frameworks? The answer is simple:

Do not use a UI framework :)

To be more precise:

Do not incorporate a UI framework if you are still exploring what your application should actually do.

What does this mean?

It means starting with the most basic components that HTML has to offer. If you need an input, use <input>. If you need a card use a <div> with some border property set.

With this approach I am able to push the UI framework decision to the future as long as possible and can rather ship working prototypes that I can quickly evaluate with end users to build a better product and eventually choose the UI framework that fits.

Using the Raspberry Pi as a Syncthing Server

2022-05-08T00:00:00Z

Today I reworked my Obsidian infrastructure setup to use Syncthing (https://syncthing.net/) instead of GitHub so I have complete control over my data. In this post I would like to share my setup with you.

Raspberry Pi as Syncthing Main Server

The main idea is having the Raspberry Pi as the main Syncthing server that is always online. Through relay servers any other client is able to establish a connection to that main server. This is important as Syncthing would at least need two devices that it can sync files properly within real-time.

Let's dive into the technical setup.

Setting up the SD Card

Download Raspbian Imager from https://www.raspberrypi.com/software/.
That software writes the Raspbian OS onto your SD Card. Pick Raspberry Lite (no Desktop) and flash your SD Card.

To automatically connect to WIFI place a wpa_supplicant.conf into the SD card root folder with the following content:

country=de
update_config=1
ctrl_interface=/var/run/wpa_supplicant
network={
scan_ssid=1
ssid="WiFi SSID"
psk="WiFi password"
}

Additionally, place an empty ssh file into the root. This triggers an SSH login on the first boot.

Connecting to the Raspberry PI

Next, start the Raspberry with the prepared SD Card. It should automatically connect to your network.

To find the IP you might use an IP Scanner like Angry IP Scanner or if you have access to your router you can just see listed devices there. If this is your only raspberry pi in the network try to connecting to raspberrypi as ip address.

Connect to it via SSH:

ssh pi@<raspberry pi ip>

Username: pi
Password: raspberry

If you encounter troubles just connect to a display via HDMI and see what the logs show.

After the initial login change the default password to something safe! (or better: enable SSH key authentication only - I will not discuss this here)

passwd

Installing Syncthing

Syncthing is available as a Debian package at https://apt.syncthing.net/:

sudo curl -s -o /usr/share/keyrings/syncthing-archive-keyring.gpg https://syncthing.net/release-key.gpg
echo "deb [signed-by=/usr/share/keyrings/syncthing-archive-keyring.gpg] https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list
sudo apt-get update
sudo apt-get install syncthing

Run Syncthing:

syncthing

And note your device Id
This will print starting logs including the Syncthing ID of the raspberry. The line looks like this:

...
[XXXXX] 2022/05/08 16:17:15 INFO: My ID: XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX
...

Exposing the Syncthing Web Console

There is a CLI for Syncthing but it's much easier to maintain configurations via the web browser. Enable it in the ~/.config/syncthing/config.xml by changing the address attribute to a 0.0.0.0 value like this:

<gui enabled="true" tls="true">
 <address>0.0.0.0:8384</address>
 <apikey>xxxxx</apikey>
</gui>

Then, restart syncthing with:

syncthing

And access the web console at https://<raspberry pi ip>:8384.
Add a new user authentication here to avoid others within the same network of accessing your Syncthing settings. The web console will also prompt you to do this.

Starting Syncthing on Login

To run Syncthing on startup (e.g. when the Raspberry Pi powers up), we can edit the ~/.bashrcfile that gets executed on login of the pi user:

vi ~/.bashrc

and add the following line to start syncthing:

# ...
# Run Syncthing in detached mode (notice the &)
syncthing serve &

Note that if you login into the pi user the script will always execute. You can also configure a service to run syncthing with init.d or systemd if you want a cleaner execution. The mentioned approach is very simple and sufficient for my case.

Conclusion

This guide walked you through the basic setup of Syncthing on a Raspberry Pi.

You can now start your Raspberry Pi to automatically run Syncthing and access the web console to manage all devices and folders.

Consider adding a backup mechanism to your setup to make it even more robust.

Scoping a Hobby Project - Personal Learnings

2022-04-16T00:00:00Z

This post is meant to be a small self-reflection for a personal note taking application called Noteberry.

Noteberry is a hobby project I have been working on during evenings or weekends for the past months (think I started 1.5 years ago in around December 2020). It should have been "a toolbox for working with block-based linked markdown notes" (GitHub Repo). It was meant to be an application to deal with my local markdown notes.

However, I escalated a little too much in scope and built first of all an editor with VIM support to support easy editing. Then I continued developing a parser for markdown with wikilinks ([[wikilink]]) and so on... I realised, that there were already applications that would implement exactly the thing that I was trying to develop (like LogSeq, Obsidian or Neuron. And I also realised that they have either spent a very long time developing it or have a whole team behind it.

Recreating any of these applications wasn't my first intention when starting the project. I just wanted to deal with my local markdown files. However, over the past couple of months scope was increasing unexpectedly because my goal for Noteberry was not clear.

Despite of this, I learned a lot of technical things:

Work with syntax parsing and build a parser with Remark to parse markdown files (published a small article about it here)
Develop an editor extension with CodeMirror 6 to create a text editor with VIM support
Setup a monorepo with Lerna to manage and publish all packages from one repository
Setup a working CI/CD Flow with GitHub Pages to test publish without any more effort
Learn how the new NodeJS Ecma Script Modules (ESM) work with TypeScript to write modern and fast code

Nonetheless, I realised that I have not produced something that I can actually use. I have rather deepened my knowledge about the above topics.

For my future projects I am taking this question with me:

What is the problem that I am trying to solve?

And then will not directly solve it but rather look for existing solutions:

What solutions already exist to solve it?

I was neither aware of Obsidian nor LogSeq when starting out but the thing that I was trying to built was just too large to solve it with a one person show in a timely manner, so:

Can I produce results in a timely manner?

With these questions I should have a good foundation for scoping future (hobby) projects.

Nonetheless, development and implementing stuff should also be fun and I will not only create projects to solve specific issues. I will also keep that in mind :)

How Daily Notes Work in Networked Note Taking

2022-04-09T00:00:00Z

A daily note makes it simple to track thoughts, meetings, and events during a specific day.
If you create a link for a topic that you mention in multiple dailies you will get time based relationships like the follow picture illustrates:

Note that in this case there is a daily note for each day and one topic note that would get referenced with a [[Topic]] link.

However, there will not be only one topic that you write down during a day but you will write notes regarding various topics like the following illustration depicts:

This means: all three days mention Topic A, the 4th and the 5th link Topic B, and the 5th and 6th reference Topic C.

Let's see how this approach looks from different angles.
If we select Topic B we would see this:

Selecting the 4th of April would show something like this:

In a conclusion, with a daily you can keep track of your information flow over time. Specific topics evolve over time which is inherently depicted by daily notes that reference and contribute to new topics.

Linking Markdown Notes with TypeScript

2021-12-19T00:00:00Z

Linked note taking brings the advantage of connecting knowledge in an intuitive, connected matter as opposed to traditional, section-based note taking approaches. This approach gained traction through new networked note taking applications like RoamResearch, Obsidian, and Logseq. They pursue the concept of linked notes - that means that instead of dividing notes into sections, they are all in a flat directory and define structure by relations to other notes.

In this post we are using Markdown to stick to an easily editable text format without vendor lock-in. We could even drop these plain files into a Git folder for effortless backups and versioning. For linking markdown notes we are going to use the widely adopted Wikilink format (e.g. [[wikilink]]). These connect arbitrary documents without the need of specifying the exact location.

I did not find too many resources and guides on how you could implement algorithms by yourself to interlink Markdown notes with the power of [[wikilinks]]. That is why this post dives into an implementation to

Setup a base project (Source on GitHub) for the implementation
Read markdown files from your folder to have data to analyze
Parse the files to find all wikilinks for further mapping
Extract titles to that we can map our links
Build a master mapping table to quickly interlink our notes
Map links to document paths for an example on using our master mappings

I assume knowledge of TypeScript but it is not necessary in order to grasp the concept and process.

Objective: Building a basis to link any markdown notes through a mapping data structure that gives you direct access to path, title, and links of documents.

Setup

At first, we are going to setup our environment and base project. Ensure you have a recent installation of Node.js. If not head over to their download page grab the latest version and install it. We use the package manager yarn for dependency management. However, you could also install everything via npm if preferred. So, go ahead and install yarn.

npm install --global yarn

Let's create a new folder for our project and initialize an empty yarn project with default options.

You could also find all code on GitHub.

mkdir linked-markdown-notes
cd linked-markdown-notes
yarn init -y

Add required dependencies for running the project.

yarn add --dev typescript ts-node @types/node @types/fs-extra
yarn add remark-parse remark-wiki-link unist-util-visit-parents

We are using ECMAScript Modules (ESM), therefore add the "type": "module" option to your package.json and create a start script for running our project:

{
  "name": "linked-markdown-notes",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "type": "module",
  "scripts": {
    "start": "node --loader ts-node/esm src/index.js"
  }
}

Create a new tsconfig.json file for our TypeScript configuration:

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true,
    "paths": {
      "remark-wiki-link": ["./typings/remark-wiki-link.d.ts"]
    }
  }
}

As the code base of remark-wiki-link is JavaScript and the package ships no types we have to create a new type definition file and reference it in the tsconfig.json. For this we just create a new file under typings/remark-wiki-link.d.ts with the content

declare module 'remark-wiki-link';

Either use an already existing folder with markdown files or create a new test/resources folder that contains some sample markdown files. In this post, all examples and code will work with the latter one. If you want to follow it, the create 4 files in that folder:

test/resources/Dog.md

# Dog
There is a dog.

test/resources/Cat.md

# Cat
Cat plays with [[Dog]].

test/resources/Farm.md

# Farm
Farm has a [[Dog]], a [[Horse]], and a [[Cow]].

test/resources/Home.md

# Home
Home has a [[Cat]] and [[Dog]].

The content of the files make no real sense, but we will need these examples as a basis to test the [[wikilinks]]. Eventually, our folder structure should look this:

test/
  resources/
    Dog.md
    Cat.md
    Home.md
    Farm.md
typings/
  remark-wiki-link.d.ts
package.json
tsconfig.json

With our basic setup we are now ready to dive into the actual implementation.

Reading Markdown Files

Let's write some code for reading our files that we want to link. For that create a new file under src/index.ts as application entry. Add the following function to read markdown files of a specific directory:

import fs from 'node:fs/promises';
import path from 'path';

interface FileSystemDocument {
  path: string;
  content: string;
}

/**
 * Reads all markdown files of a directory.
 *
 * @param directoryPath Directory to read.
 * @returns Path and content of markdown files within the directory.
 */
async function readMarkdownFiles(
  directoryPath: string
): Promise<FileSystemDocument[]> {
  // read files in folder
  const dirents = await fs.readdir(directoryPath, { withFileTypes: true });
  // filter files from folder
  const files = dirents
    // only consider files that have a markdown extension
    .filter(dirent => dirent.isFile() && path.extname(dirent.name) === '.md')
    // return path of file
    .map(dirent => path.join(directoryPath, dirent.name));
  // read file contents
  const filesWithContentPromises = files.map(
    async (filePath): Promise<FileSystemDocument> => ({
      path: filePath,
      // read content of file
      content: (await fs.readFile(filePath)).toString('utf-8'),
    })
  );
  // resolve asynchronous file loading
  return Promise.all(filesWithContentPromises);
}

// call function with our test repository
const files = await readMarkdownFiles('./test/resources');
// prints results
files.forEach(({ path }) => console.log(`Read file path: ${path}`));

Run the code snippet and examine its result:

yarn start

It should have printed a list of markdown files in your test/resources folder:

Read file path: test/resources/Cat.md
Read file path: test/resources/Dog.md
Read file path: test/resources/Farm.md
Read file path: test/resources/Home.md

Nice. We are set with our files. Now we can really start with file parsing and linking...

Parsing Markdown Files

As a second step we have to find all wikilink occurrences that we can use for lining our files. Therefore, we are going to parse the Markdown content and analyze the syntax tree.

We use the library remark-parse for markdown parsing with the remark-wiki-link extension for parsing wikilinks of the format [[...]]. We start with the creation of a WikiLinkNode type that we need for the remark-wiki-plugin. Afterwards, we we create a SyntaxTree type we can use as the root type of our parser. Additionally, let us import visit - an utility to traverse the syntax tree - see tree traversal article in TypeScript for more information about that.

// ... other imports
import MDAST from 'mdast';
import UNIST from 'unist';
// gonna need visit for our next function
import { visit } from 'unist-util-visit';

interface WikiLinkNode extends UNIST.Node {
  type: 'wikiLink';
  value: string;
  data: {
    alias: string;
    permalink: string;
  };
}
type SyntaxTree = mdast.Root | WikiLinkNode;

Unist and mdast are syntax tree structures, which remark-parse uses for parsing markdown. In the next step, we add two more functions to parse all document links and build a mapping table from all documents and links. Let's start with a function to parse wikilinks from our syntax tree. For this we are going to use the visit utility we imported in the previous snippet:

function parseDocumentLinks(parsedDocumentContent: SyntaxTree) {
  const documentLinks: string[] = [];
  visit(parsedDocumentContent, 'wikiLink', wikiLinkNode => {
    const value = wikiLinkNode.value;
    const alias = wikiLinkNode.data.alias;
    // if a link contains a ":" the remark-wiki-link plugin would parse it as an alias
    // as we are not using the alias feature we re-construct the full link content
    // LIMITATION: this approach does not recognize links that have the same name and alias
    // e.g. name:name - keep that in mind when using this snippet
    const linkName = `${wikiLinkNode.value}${
      value !== alias ? ':' + alias : ''
    }`;
    documentLinks.push(linkName);
  });
  return documentLinks;
}

The function returns an array of wikilinks that it finds in the syntax tree. Now, we can add our function for building the document to map links:

function buildDocumentsLinkMap(files: FileSystemDocument[]) {
  // call the unified parser
  const documentParser = unified().use(remarkParse).use(remarkWikiLink);
  // end result
  const documentLinksMap: { [key: string]: string[] } = {};
  files.forEach(({ path, content }) => {
    // parse each content and put its links into the result map
    const parsedDocumentContent = documentParser.parse(content);
    documentLinksMap[path] = parseDocumentLinks(parsedDocumentContent);
  });
  return documentLinksMap;
}
// print the result
console.log('Document links map: ', buildDocumentsLinkMap(files));

Running our program will now yield something like this:

Read file path: test/resources/Cat.md
Read file path: test/resources/Dog.md
Read file path: test/resources/Farm.md
Read file path: test/resources/Home.md
Documents links map: {
  'test/resources/Cat.md': [ 'Dog' ],
  'test/resources/Dog.md': [],
  'test/resources/Farm.md': [ 'Dog', 'Horse', 'Cow' ],
  'test/resources/Home.md': [ 'Cat', 'Dog' ]
}

Voilà, we successfully parsed our markdown files, extracted links, and created a mapping table. With this mapping table we can easily match documents and links. The next sections discuss how we can implement that mapping.

Extracting Document Titles

Our links have to map against some sort of name or title. In our case, we are using the document name of each file as the matching target (you could also use e.g. the first heading in you markdown file). It is only important to create a map of titles that the documentsLinksMap can use for linking. Therefore, we add a new function for extracting document titles:

function buildDocumentsTitleMap(files: FileSystemDocument[]) {
  const documentTitleMap: { [key: string]: string } = {};
  files.forEach(({ path }) => {
    // extract document title
    const title = /([^\/]+)\.md$/.exec(path)?.[1];
    // verify a consistent mapping
    if (!title) throw new Error(`Could not extract title from path: ${path}`);
    if (documentTitleMap[title] !== undefined)
      throw new Error(
        `Title "${title}" already exists for path "${documentTitleMap[title]}".`
      );
    documentTitleMap[title] = path;
  });
  return documentTitleMap;
}

console.log('Documents title map: ', buildDocumentsTitleMap(files));

The regex ([^\/]+)\.md$ returns the title of a document and the first matching group contains the title. Additionally, the function contains two additional checks to guarantee uniqueness of titles. A run with yarn start now prints:

Read file path: test/resources/Cat.md
Read file path: test/resources/Dog.md
Read file path: test/resources/Farm.md
Read file path: test/resources/Home.md
Documents links map:  {
  'test/resources/Cat.md': [ 'Dog' ],
  'test/resources/Dog.md': [],
  'test/resources/Farm.md': [ 'Dog', 'Horse', 'Cow' ],
  'test/resources/Home.md': [ 'Cat', 'Dog' ]
}
Documents title map:  {
  Cat: 'test/resources/Cat.md',
  Dog: 'test/resources/Dog.md',
  Farm: 'test/resources/Farm.md',
  Home: 'test/resources/Home.md'
}

Well done, with the documentsLinksMap and the documentsTitleMap we can now interlink our markdown files.

Building Master Mappings

You might already wonder how we could actually visualize all the links by using the mapping tables. Thus, for our final mapping we create mappings based on our two original mapping tables and combine them all! They will enable any future algorithms to directly find mappings for links. Let us add a function that creates our master mapping tables:

function buildMasterMappings(
  documentTitleToPathMap: { [title: string]: string },
  documentPathToLinksMap: { [path: string]: string[] }
) {
  // create interim mapping tables to easily map entries
  const documentPathToTitleMap: { [path: string]: string } = {};
  Object.entries(documentTitleToPathMap).forEach(
    ([title, path]) => (documentPathToTitleMap[path] = title)
  );
  const documentTitleToLinksMap: { [title: string]: string[] } = {};
  Object.entries(documentPathToLinksMap).forEach(
    ([path, links]) =>
      // map document title to links
      (documentTitleToLinksMap[documentPathToTitleMap[path]] = links)
  );
  // return all mappings
  return {
    documentTitleToPathMap,
    documentPathToLinksMap,
    documentPathToTitleMap,
    documentTitleToLinksMap,
  };
}

console.log(
  'Master mappings: ',
  buildMasterMappings(
    buildDocumentsTitleMap(files),
    buildDocumentsLinkMap(files)
  )
);

A re-run of our project with yarn start will now yield

... (output from above)
Master mappings:  {
  documentTitleToPathMap: {
    Cat: 'test/resources/Cat.md',
    Dog: 'test/resources/Dog.md',
    Farm: 'test/resources/Farm.md',
    Home: 'test/resources/Home.md'
  },
  documentPathToLinksMap: {
    'test/resources/Cat.md': [ 'Dog' ],
    'test/resources/Dog.md': [],
    'test/resources/Farm.md': [ 'Dog', 'Horse', 'Cow' ],
    'test/resources/Home.md': [ 'Cat', 'Dog' ]
  },
  documentPathToTitleMap: {
    'test/resources/Cat.md': 'Cat',
    'test/resources/Dog.md': 'Dog',
    'test/resources/Farm.md': 'Farm',
    'test/resources/Home.md': 'Home'
  },
  documentTitleToLinksMap: {
    Cat: [ 'Dog' ],
    Dog: [],
    Farm: [ 'Dog', 'Horse', 'Cow' ],
    Home: [ 'Cat', 'Dog' ]
  }
}

Do You see where this is getting? Hence, we got mappings into all directions with all document titles, paths, and links. Now we just have to use it.

Mapping Links to Document Paths

We could utilize these master mappings in various use cases. To look into one in particular, we implement a function to map links to the respective file path:

(Note, that some links don't have a matching target document - our function considers this case as well)

// we build a global master mapping table
// so we can reuse it later
const globalMasterMapping = buildMasterMappings(
  buildDocumentsTitleMap(files),
  buildDocumentsLinkMap(files)
);

function findDocumentPathOfLink(
  masterMapping: ReturnType<typeof buildMasterMappings>,
  linkName: string
) {
  const linkPath = masterMapping.documentTitleToPathMap[linkName];
  return {
    // return true, if the path exists
    // if false, there is no document for the link name
    existing: linkPath !== undefined,
    path: linkPath,
  };
}

Eventually, let us print some results of that function:

console.log(
  'Path for link Home: ',
  findDocumentPathOfLink(globalMasterMapping, 'Home')
);
console.log(
  'Path for link Farm: ',
  findDocumentPathOfLink(globalMasterMapping, 'Farm')
);
console.log(
  'Path for link Farm: ',
  findDocumentPathOfLink(globalMasterMapping, 'Cow')
);

A run with yarn start now logs:

Path for link Home:  { existing: true, path: 'test/resources/Home.md' }
Path for link Farm:  { existing: true, path: 'test/resources/Farm.md' }
Path for link Farm:  { existing: false, path: undefined }

This case shows us how easy it is to utilize the master mapping to quickly find paths for linked notes.

Limitations and Outlook

This post covered a basic implementation for processing linked Markdown notes. Feel free to take it as an inspiration and let me know if you do something cool with it.

Finally, I want to close with some questions worth asking when looking at your map of notes:

Are all notes connected with each other, or are there outsiders? This might be an indication for lost notes.
What links do not have a mapping target? This would be links pointing to non-existent documents.
Which links point to a specific document (also known as backlinks)?
What notes are the most important ones? E.g. you could sort with the PageRank Algorithm

Enjoy working with your set of networked notes.

Cheers, Marco

Find code for this article on GitHub

Linking markdown files for better note taking

2021-11-28T00:00:00Z

Currently, I am investigating approaches to work with linked notes - markdown notes in particular. Linking notes is not a new concept but I found no application that meets my requirements to 100%. However, some applications start picking up these concepts like Obsidian or Roam Research.

Inspirations:

RoamResearch
[[Obsidian]] https://obsidian.md/
https://github.com/Jermolene/TiddlyWiki5
[[Typora Note Taking App]] https://typora.io/
[[Standard Notes]] https://standardnotes.com/features
Markdown backlink janitor https://github.com/andymatuschak/note-link-janitor

Resolve tag clobber

2021-02-08T00:00:00Z

If you got conflicting tags force pull the latest tags from your remote.

> git pull --tags origin develop
From github.com:name/repo
 * branch            develop    -> FETCH_HEAD
 ! [rejected]        0.1.177    -> 0.1.177  (would clobber existing tag)
 ! [rejected]        0.1.179    -> 0.1.179  (would clobber existing tag

Resolve with

git fetch --tags -f

Linux Cheat Sheet

2021-01-12T00:00:00Z

Check Kernel Version

uname -a

Check IP

ifconfig

ip addr show

Check disk space

df -ah

Work with services

service <service-name> status
systemctl status <service-name>

Check folder size

Disk use

du -sh <directory>

Check for open ports

netstat

Check CPU usage

ps aux | grep <process-name>

top

Mounting

mount /dev/sda2 /mnt

Check for existing

mounts

Auto mount in file

/etc/fstab

Manpages

Manual pages

man <command>

Permissions

Show permissions

stat file.txt

Change permissions

# denies file read for others
chmod o-r file.txt
# read for user, read for group, deny access to others
chmod u=r,g=r,o= file.txt

File Transformation

Use jq to transform JSON files.

cat logs.json | jq '.["@timestamp"] + "   " + .message' > logs.txt

Transforms extracts timestamp and message from JSON objects and prints them into logs.txt.

Practical Vim Command Cheat Sheet

2021-01-10T00:00:00Z

Surround with brackets

Select with visual mode.

Use

c()<Esc>P

Visual mode

Visual mode for lines (Visual Line mode)

Visual mode for characters

Leave insert mode

Ctrl-C or ESC

More movements

Insert line above

Insert line below

Paste above

Paste below

Hop through results: n reverse: N

Go to next occurrence: * reverse: #

Horizontal Movement

Find and select character

f / F <character>

Find and not select character

t / T <character>

Example combo with d

df <character>

Delete until <character>

Delete and insert

Delete character

x / X

Delete character and enter insert mode

Delete line and enter insert mode

Vertical Movement

Move to top

Move to bottom

Move to middle

Search and replace

Search for all occurrences of foo and replace with bar after confirmation

:%s/foo/bar/gc

Open finder

:e <folder>

Jump between two last buffers

Ctrl + ^

Set mark with

m <letter>

Set global mark with

m <Shift + letter>

Splits

All under Ctrl + W are window controls.

Vertical

Ctrl + W v

Horizontal

Ctrl + W s

Equal split

Ctrl + W =

Resize horizontally

:resize 10

Resize vertically

:vertical resize 20

Vim for Firefox

2020-12-07T00:00:00Z

Tridactyl is a Firefox extension to enable vim key bindings for a more efficient browsing experience.

Go back to tutor

:tutor

Modes

Normal mode
Hint mode
- Enter with f
Visual mode
- Enter with v, ;h, /
- Search via s, S
- Exit via Esc, <Ctr-[>
Command mode
- Enter with :
- Exit with Esc, Enter
Ignore mode
- Pass through key events
- Toggle with Shift-Insert, Ctrl-Alt-Escape, Ctrl-Alt-Backtick, Shift-Esc

Normal mode

Scroll with h, j, k, l.

Go to top with gg.

Got to bottom with G.

Go back in history with H and forward with L.

Use . to repeat the last action.

Open new tab with t.

Open hinting mode to open tab in background use F (use f to open it in same window).

Focus first textbox on page with gi. Switch textboxes with tab.

Useful commands

Bring up list of tabs with b. Use Tab, Shift-Tab to cycle through them.

Open web pages:

w open in new window
o in current tab
t in new tab
Use capital letters to open current URL
s search
start search query with search engine like duckduckgo, or google.

Use yy to copy current URL.

Use p to open clipboard content as a webpage. Use P to open in a new tab.

Quickly search for source of a quote with ;p to copy a paragraph and P to search it

Use zi, zo, zz to zoom in, out, and to reset zoom.

Search with /. Navigate through matches with Ctrl-g, Ctrl-G.

Use Ctrl-v to send next keystroke to website.

To check what a binding is doing type :bind [keys].

Logging Exceptions with SLF4J

2020-12-01T00:00:00Z

Exception logging in SLF4J works without including it in the format:

logger.error("An exception occured", new Exception("my exception"));

As opposed to the {} format notation.

Go Cheat Sheet

2020-11-19T00:00:00Z

Go Lang Tour

This post is my cheat sheet for the (Go Lang Tour)[https://tour.golang.org].

Package main runs programs.

Imports

Factored Import Statements

import (
	"fmt"
	"math"
)

Exports

In Go, a name is exported if it begins with a capital letter.

Functions

func add(x int, y int) int {
	return x + y
}

You may omit types if they are equal for consecutive parameters.

func add(x, y int) int {
	return x + y
}

Multiple Results

func swap(x, y string) (string, string) {
	return y, x
}

func main() {
	a, b := swap("hello", "world")
	fmt.Println(a, b)
}

Naked return

func split(sum int) (x, y int) {
	x = sum * 4 / 9
	y = sum - x
	return
}

Variables

var c, python, java bool

Short declaration with :=

func main() {
	var i, j int = 1, 2
	k := 3
	c, python, java := true, false, "no!"

	fmt.Println(i, j, k, c, python, java)
}

Datatypes

bool

string

int  int8  int16  int32  int64
uint uint8 uint16 uint32 uint64 uintptr

byte // alias for uint8

rune // alias for int32
     // represents a Unicode code point

float32 float64

complex64 complex128

Conditions and Loops

For

sum := 0
for i := 0; i < 10; i++ {
    sum += i
}

For is Go's While

for sum < 1000 {
    sum += sum
}

Defer

func main() {
	defer fmt.Println("world")

	fmt.Println("hello")
}

Examples

Read in line:

func Scanln(a ...interface{}) (n int, err error)

Example

var name string
fmt.Scanln(&name)

Life, the Universe, and Everything

Read lines until we read 42.

package main
import "fmt"

func main(){
	// your code goes here
	var line string
	for fmt.Scanln(&line); line != "42"; fmt.Scanln(&line) {
		fmt.Println(line)
	}
}

A dead simple approach to using SASS in Eleventy

2020-08-16T00:00:00Z

I prefer going with as simple solutions as possible, avoiding the usage of third party packages that always introduce some risk into your project. You don't know if they are up to date if there are no tests included, you don't know if they are secure and what packages are attached to them if you don't check source code.

So, to include SASS within my Eleventy project I am using only the official SASS npm package and the officially included fs package of node to write the transpiled files.

The .eleventy.js file contains only three lines within the configuration function to compile SASS into CSS:

const sass = require('sass');
const fs = require('fs');

function compileSass() {
  console.log('Compiling sass.');
  const cssContent = sass.renderSync({ file: 'styles/main.scss' });
  fs.mkdirSync('_site/css', { recursive: true });
  fs.writeFileSync('_site/css/main.css', cssContent.css);
}
fs.watch('styles', (event, filepath) => {
  console.log('Styles file changed', filepath);
  compileSass();
});
compileSass();

This a a very simple approach without asynchronous code or anything. The only package you need is the npm sass package. They even write that "renderSync() is more than twice as fast as render()" (SASS npm package), so you might consider if its worth for you introducing parallel execution.

Important note to fs.watch

This script does not work when publishing to GitHub pages automatically as the fs.watch command will block input. To unblock if Eleventy isn't run with the --serve flag we can only enable file watching if we discover that flag like so:

const isServing = process.argv.includes('--serve');
if (isServing) {
  // watch only files if we are serving
  fs.watch('src/styles', (event, filepath) => {
    console.log('Styles file changed', filepath);
    compileSass();
  });
}

Clearing the _site directory in Eleventy automatically

2020-08-16T00:00:00Z

If you want to ensure an always up to date version, with all files being removed you might want to clear your _site directory before each build. This needs no further packages then the already built in fs package of node. The function rmdirSync(folderPath, {recursive:true}) can accomplish excactly this.

Your .eleventy.js then looks like this:

const fs = require('fs');

// clear site on initial build
fs.rmdirSync('_site', { recursive: true });
console.log('Cleared _site folder');

Note that we do not export a function as you would normally do it. This is, because we want to run the folder deletion only once per build or serve!

That means, that this code does not block our subsequent building pipeline. However, when you run Eleventy with eleventy serve you might end up with unwanted files in the end. So always remember to run eleventy build before publishing your site and you will always get a clean version.

My journey to using Eleventy as a static website generator

2020-08-15T00:00:00Z

I started renting my first server something like 10 years ago back in school. Initially I merely used it to play around with some new technologies. While studying I launched my first websites including a small blog.

First I put together a website from scratch to post content into the world wide web. Then I tried out more advanced and feature rich systems like the ghost blogging platform. However, I always struggled with this one thing: updating the server.

I only published content and applications sporadically. Therefore, my server kept getting out of date and I killed it before trying to walk through a complex updating process. Often my blog posts and content got lost in that process.

So its now 2020 and again I killed my server as my last updates happened 2 years ago. Yes, initially I did backups and restored them on my new Ghost blog instance. However, now I am pursuing a more sustainable and lasting approach to keep stuff that I am working on online.

Of course I also considered hosted blogging platforms like Medium. Nonetheless, I prefer hosting my own stuff and learning new things in the process.

Static Websites

During my research of a better hosting alternative I stumbled across GitHub Pages that allows hosting of websites for free, with SSL encryption. After some investigation I considered this as a valid hosting option for my new website. Building a static website also had the benefit of changing hosting providers quickly as it is only a bunch of HTML and CSS files without the need of backend services.

Templating Websites

Building a website with merely HTML and CSS is a pain when you consider the long run. You might want to change some things and have to update layout and styles at lots of different places. Therefore I decided to go with a templating engine. Jekyll was my first consideration as it is a very popular framework for static website generation. It uses Ruby under its hood and after setup and some tests I quickly figured out that this is not the language I want to host my websites with.

Eleventy as Static Website Generator

As mostly work with front-end technologies like Typescript or Javascript I searched for an engine that uses excactly these technologies. That was when I discovered Eleventy, a static website generator built with Javascript and using the package manager NPM for building.

Getting used to Static Website Generators

As Eleventy is my first static website generator I have ever worked with, I started reading through the documentation and setting up a very simple website. There are lots of demo projects, however I decided to start from scratch to learn about that technology and all the features it has.

There are several things I already learned in the process: for example setting up basic fragments that can be reused in other templates, applying CSS, or building a basic post collection.

So I decided to build a new website from scratch. It should be minimal, show off projects that I am working on, and provide the capability of posting stuff from time to time. It should be easy to maintain (from a coders perspective) and last for a long time.

Kleinprojects

Testing HTMX 2 with Happy Dom

Conclusion

Code Appendix

Experimenting with LLMs for Dynamic Game Content Generation

The Core Challenge: Constraining Creativity

How the System Works

Conclusion

Announcing ... Impromat ... Improvisational Theatre Session Planning

Why an Improv app?

Introducing Impromat

So, what the heck is it, how is it build?

How can I use it?

Future

Documenting Decisions

Most Important Sections of a Decision

Context and Problem Statement

Considered Options

Decision Outcome

Conclusion

References

The Hardest Part of a Personal Project is the Release

Impromat Strategy

Diagnosis

Guiding Policy

Coherent Actions

References

Project Steering through Stakeholder Mapping

Overview of Relevant People

Prioritising and Clustering

Putting People in Perspective

Conclude Understanding

Conclusion

Improving the Navigation of Impromat.app with a Bottom Navigation Bar

The Navigation Bar Solution

Wrap-up

Navigating an Uncertain Project Landscape: Finding Your Path When Direction is Unknown

Find the right people to talk to

Find the right abstraction layer

Parallelise Streams

Create Transparency by Sharing Progress and Thoughts

Fuzzy Search Considerations for Impromat

Context

Technology Stack

Considered Options

Prisma Full Text Search

Postgres pg_trgm

Basic text search via LIKE

Decision

Prometheus and Grafana with Dokku

Install required plugins

Install Prometheus

Create node-exporter

Create cAdvisor

Setup Grafana

Login to Grafana

Adding more data sources

PostgreSQL

Thoughts on Blogging

Personal vs. Professional

Draft to Release

Find a group or community to share thoughts with

Find conferences to speak at

Acting Authenticity

Next 13 and Mantine will not replace the Ionic framework for Impromat (yet)

Reflection on making Impromat accessible for search engines

Background

Possible Solutions

Combine SEO optimization within the application

Separate the application page from a SEO optimized landing page

Conclusion

Thoughts on Data Privacy for my free time project Impromat

Using the version from package.json in React applications

Incremental User Interface Development

Using the Raspberry Pi as a Syncthing Server

Raspberry Pi as Syncthing Main Server

Setting up the SD Card

Connecting to the Raspberry PI

Installing Syncthing

Exposing the Syncthing Web Console

Postgres `pg_trgm`

Basic text search via `LIKE`