Hey, I'm Konur

KonurPapa.com


It's easy to be selfish in conversation. "Will this person shut up already so I can say what I want to say?"

This is not a way to have meaningful conversation. People are not as stupid as we think; they can see through our disinterest. Here are a few thoughts on what I try to cultivate in myself when talking with people:

I often remind myself that (most likely) I know very little about this person's viewpoint on the particular topic of conversation. Not only is it good to foster curiosity about how they see things so I can get to know them better, but I will look pretty foolish if I jump to conclusions and don't hear them out (this goes for most things -- be it confrontation, preferences, or otherwise).

My general rule of thumb is I want the other person to leave the conversation thinking, "wow, I like talking to Konur!" Even if we disagree on a subject, there's no reason the conversation can't be an enjoyable one. If the other person has a good impression about me when we leave the conversation, they will want to talk to me more in the future -- and more importantly trust will be built.

Trust is the bedrock upon which (possibly all?) meaningful interaction stems from. As such, this means that learning how to communicate in this way is one of the most vital skills one can learn.

An Approach to Conversation


I'm realizing that there's a lot that gets lost in translation between programmer and layman.

I've had a number of times when giving too many details to a boss/manager has burned me. In the past, I've tried to be thorough and supply all the necessary details:

> Well, it's not _totally_ successful, because there's `x` problem and `y` bug...

The problem is, this usually leads to panic and questioning. I can almost hear them saying, "Don't you know what you're doing?" or "Don't you realize by this deadline we needed to see good results?"

I'm not advocating lying, but I think the bigger issue here is that ultimately nobody else knows my workflow as well as me, and thus giving too many details often does more harm than good.

If I get too wordy and try to explain why it's not a _total_ success (but I knew `x` bug was going to show up and it's not a big issue), I often get a deer-in-the-headlights response. All they heard was:

> There's a problem; it doesn't work.

If I instead say that things work, and I just need more time to do some bugfixing & cleanup, suddenly it's a very different response.

> Konur, you've done it again! You are a scholar and a gentleman!

Well, maybe that's a bit of wishful thinking.

I wish that was what I heard every time.

---

Anyway... I'm finding it's best to supply few details upfront, and be very clear and articulate if prompted for more info.

Spooking managers with false positives (or negatives? since it's bad?) isn't beneficial for anyone. Sometimes people just need to hear "yep, everything works" even if things are still somewhat in disarray behind the scenes.

I think there's a happy medium between the two extremes.

In unrelated news, everything with my website works great.

In Defense of Dishonest Reporting


Ever wondered how spellcheck works? Perhaps if you've ever been angry by its incorrect rewording of whatever it was you wrote.

But consider for a moment how often it gets things _right_. How does it so often seem to know what word you meant to type, based on your typo?

My latest nerdy forray has been into the world of Levenshtein automata. Simply put, this is an algorithm for determining how similar a given string of text is to another string (probably from a list). Like the spellcheck example: how similar is the typo `stea` to an actual word in the dictionary?

> This is an algorithm for determining how similar a given string of text is to another.

In this example, there are some things to consider. Firstly, this "similarity" can be composed of 3 distinct logical operations: deletions, insertions, and replacements. In this case, `stea` could be matched within 1 operation to the words `tea` (a deletion), `stead` (an insertion), and `teas` (a replacement).

This is why spellcheck often offers more than one potential word as a typo solution, and the likelihood of `x` match is based on probability. This is also why some editors don't ask your permission and automatically correct some words; it's certain the word must have been a typo because the probability is so high.


## Beyond Spellcheck

I've been working on a project where I've put this concept to use (hence my deep-dive). I have an input item (or _items_), and I need to find the best match from an internal database of items.

Problem #1 is this input item may or may not be typoed. Problem #2 is this searching database is pretty big (a few thousand entries).

Obviously, the speed at which this processes is of utmost importance. I could manually iterate over every single index in the database until a match is found, which I know would _technically_ work. But if there are numerous input items, this will start to take a really long time:

| Processing Time | x100 Items |
| --------------- | ---------- |
| 0.5 seconds     | 50 seconds |
| 3 seconds       | 5 minutes  |

Once it starts taking even a fraction of a second longer, the processing time is going to multiply out exponentially by the number of items.

I proceeded to do a ton of research, and came across a few good resources. I'd be amiss not to share some of them; specifically, [this](http://norvig.com/spell-correct.html "How to Write a Spelling Corrector") article on the logic behind a spelling corrector, and [this](https://julesjacobs.com/2015/06/17/disqus-levenshtein-simple-and-fast.html "Levenshtein automata can be simple and fast") article explaining how Lenvenshtein automata works (with some Python examples as well).

The solution I eventually came up with is to first try direct key matching by converting all database entries into key/value pairs. Then something like:

```python
input_handler(item):
    return database.get(item) != null
```

This handles a pretty sizeable percentage (apparently people sometimes _can_ spell).

But what about typos? This is where I use Levenshtein automata (AKA "fuzzy" matching). There's a really helpful Python library called `fuzzywuzzy` (I think now it's called `thefuzz`) that has a bunch of built-in functions for handling matching operations. It also returns a probability score, which I use to sort more likely matches from less likely ones.

Here's another helpful article explaining how each of these functions work:

https://www.datacamp.com/tutorial/fuzzy-string-python

So here's my flow of logic:

- Split the database into categories, so we only have to iterate over database entries of the same type (I'll probably get into the details of how this part works in a future post).
- After the direct-key matching attempt, iterate over the entire database list of the same category as the input item.
- Check if the item shares any starting letters in common with the current database item; if not, skip it (no sense wasting time on fuzzy matching).
- Add this potential entry into a list, and rank the list based on similarity (using the built-in Python function `.intersection`).
- Finally, try to perform a fuzzy match on the top potentials.

The resulting system works accurately and runs pretty fast, averaging at about 0.5 seconds (from what I can tell... it's difficult for me to get a totally accurate metric for other reasons).

There are tons of other fun applications for Levenshtein automata; this is just my own example. Hopefully I've explained this in a way that's a little easier to understand and more accessible.

A good tool to have in the programmer arsenal!

Understanding Levenshtein Automata


I've posted various blog posts on my website over the past few years, but it used to be a Wordpress site and there was a lot of pointless bloat. Janky functionality in places, and there were always sections that looked nice on the outside but were heavily unfinished.

I'm a programmer... what am I doing?

As the old saying goes, the cobbler has no shoes for his family (or something like that). One day I got fed up, and decided it was time for a fresh start.

So here we are. My new website, stripped down to the basics and coded myself. It will get progressively more interesting as I put more effort into different aspects, but I think this is a good place to start.

My plan is to release both one-off, in-depth blog posts every now and again, but also to document my journey of learning over the years. It's always fun to get random deep-dives into particular subjects, and watch the progression. Hopefully it can be of some benefit to you, too.

And in time, who knows. I might evolve to more video content, online courses, or other fun stuff.

So here we are: a fresh start.

How exciting!


Hey, I'm Konur

An Approach to Conversation

In Defense of Dishonest Reporting

Understanding Levenshtein Automata

A Fresh Start