I'm a Big Boy Now

The Monk and the Vagrant

Mon, 15 Mar 2021 00:00:00 +0000

The Monk and the Vagrant

A wealthy man drove his car down the road. As he pulled up to an intersection, he noticed a man sitting with a sign indicating he was hungry and homeless.

“A ha!” The man exclaimed. “I will save this man! I have money, and all this man needs is a little money to turn his luck around!”

And so it was that the wealthy man gave the vagrant large sums of money. Being unlearned in the ways of using money wisely, after some time, the vagrant found himself without fortune and back at the same intersection.

A famous man drove his car down the road. As he approached the intersection, he too saw the vagrant.

“A ha!” The man exclaimed. “I will save this man! I have connections and fame, and all this man needs is promotion to turn his luck around!”

And so it was that the famous man introduced the vagrant to all of his friends. Being unlearned in the ways of social graces, after some time, the vagrant found himself bereft of the connections bestowed upon him. And so was he back at the same intersection.

A pious man drove his car down the road. As he approached the intersection, he too saw the vagrant.

“A ha!” The man exclaimed. “I will save this man! I have virtue and piety, and no deed is beyond my conviction. All this man needs is conviction to turn his luck around!”

And so it was that the pious man took the vagrant under his wing and introduced him to his virtues, strictures, ceremonies, and gods. Being unskilled in the ways of self discipline, however, it was not long again before the vagrant found himself back at the same intersection.

One evening, as the clouds in the sky became the envy of all of the colorful beings of creation, a mendicant approached the intersection on foot. He came upon the vagrant who had not noticed his approach. He observed the vagrant just lost in space staring at the sunset. After a moment, the vagrant realized that the monk was next to him.

The monk sat next to the vagrant unbidden, and offered the contents of his bowl: two partially eaten chicken wings. The two men savored the remnants together, and the vagrant told the monk his story.

“Thrice have I been saved, and thrice have I returned. It seems I cannot be saved.” The vagrant lamented.

“If thrice you have been saved, and thrice you have returned, then perhaps you are never to be saved.” The monk agreed.

“What ever shall I do?” The vagrant entreated, his eyes pleading.

“I suppose what you have always done. Sit at this corner and enjoy the splendor of the sunset.”

The vagrant let out a hearty laugh. “I suppose I shall,” he agreed.

The monk collected picked up his bowl, bowed to the vagrant and smiled. Then he was on his way.

Story-Oriented Programming

Fri, 25 May 2018 00:00:00 +0000

A World of Gremlins

Handle is a gremlin. Handle can make other gremlins. Handle tends to make gremlins in the likeness of what he imagines, and what he imagines always comes in pairs. Today Handle has borne two children: Reqla and Resnak. But he does not know the purpose of his children. He just exists to create them. Polys, the muse god, inspires Handle and Handle transcribes the inspiration with his creative ability. Upon creation, Handle sends them to a gremlin school, where the gremlin teacher Routin helps them find their way in the world.

Routin immediately understands why Resnak is important. He sees that Resnak’s whole purpose in life is to communicate Reqla’s purpose back to Polys. Whenever anyone tells Resnak something, he and Reqla dissolve into the aether. It’s very important that whoever talks to Resnak say the best possible thing to him, because Resnak is capable of talking to Polys directly and upon discerning Reqla’s value, will tell Polys so that Polys may experience the joy of understanding of his creation. But to him, the purpose of Reqla is less clear. He converses with Reqla to attempt to figure out how she can best serve Polys.

In general, when Routin is dealing with gremlin children, if he can determine their purpose, he will send them out to fulfill that purpose. Otherwise, he would tell the child who can speak to Polys, Resnak in this case, that he does not know the other child’s importance.

Today Routin discerned that Polys is playing a language game with himself, and, growing bored of the language in which Polys typically thinks, wanted to hear something in a different language for the joy of it. However, Routin doesn’t know anything about language, but he knows someone who does. He sends Reqla and Resnak to the one and only gremlin translation company.

When they arrive, they are met by the company foreman Logon, who knows just enough about language to know which member of the company would be the most suited. After a short conversation with Reqla, he knows just who Reqla should talk to get her message translated.

Let’s pause here and get a little more interactive.

Reqla and Resnak are currently with Logon. A translator who is capable of translating Reqla’s message is known to exist by Logon. What happens next? Should Reqla and Resnak both go to the translator? If they do, does the translator greet them, or does Reqla petition the translator? Upon translation, who tells Resnak the translation, Reqla or the translator? Should Resnak stay with Logon? If he does, who brings the translation back? Does the translator stay sequestered away (maybe he is very fat, and cannot easily move)? Upon return, does the message carrier tell Resnak directly? Or does he tell Logon who tells Resnak? Is there any value to telling the foreman first? Does the foreman need to negotiate something internally (or maybe externally, with his accounting buddy) before following through to tell Resnak? Unwinding back, maybe it’s more economical for the translator to be in the presence of all three.

No matter which question you choose to answer, they all beg the question of what line of reasoning is logically consistent with the contrived world. If you find out that translators are all kept in ivory towers and you can only shout up to them, and they back to you, then that’s going to preclude you from calling them into the foreman’s office.

If you have a background in web API development, you may realize this is a very thinly veiled anthropomorphization of a translation web service. In the jargon of what is typical of the domain: a request and response pair are generated by the underlying http server. A handler dispatches the request/response to a router which determines which sub handler to call. If it’s a translation request, the translation sub handler dispatches, based on language detection heuristic, to a translation service, which may not be colocated, but remotely accessed through sub request. Upon completion of the translation, the translation is sent back to the requesting entity through the response body.

Clearly the description of the service using jargon is much shorter than the story. And it’s tantalizing to think that the shortness is necessarily an asset.

Is it possible that the brevity is actually cryptic or opaque to the requirements of implementation?

The latter might be how I would actually describe the service to another developer if they asked about it. I certainly usually wouldn’t concoct a story like I did above (which took me like at least 30 minutes to ensure narrative consistency, and even the most basic palatability, which I am still not sure it contains, lol.)

In going through the exercise of actually making the description more verbose, and forcing my components be broken up as active sub processes (the gremlins) behaving more or less as humans might, I found it causes lateral considerations that tend to be just left out of typical “formal” systems “design.”

Narrative Reasoning

Human intuitive reasoning seems to be very closely linked to the concept of narrative, specifically narratives involving other human actors. While it is possible for humans to reason about things that seem to bear no clear resemblence to another human, such as a perfect geometric object like an isosceles triangle, this seems to be mostly secluded to the right side of the bell curve.

All more or less functional humans have the ability to reason about narrative. It’s quite literally the human way. Hidden in these myriad reasonings may be an emergent solution to a problem that can be leveraged from an otherwise average minded individual. This is similar to the difference between a strong classifier like a support vector machine (an analogy to a very smart person), and the adaptive boosted weak classifiers like chained decisions stumps (a crowd of average people). In the machine learning metaphor, the average person is not contributing a miracle solution despite his averageness, he is being leveraged to exert just the right pressure on the problem to yield a solution. This analogy in turn reminds me of the classical conceptualization of the wisdom of crowds. Some people will be smarter than the entire crowd. But the reality is that these individuals are incredibly rare, and anecdotally they seem to work well neither with others who are equally as intelligent nor with less intelligent individuals.

If we wanted to democratize reasoning about complex things, like systems, and specifically reasoning about interactions between things that give rise to this complexity, it would seem prudent to treat components directly as human agents, or carriers of intentionality, because this is something even your average person can reason about. Too often we attribute human intentionality to components of a system anyway when reasoning about them, so instead of mixing technical jargon with behavioral analogy, what if we just treated the components as little humans, or little gremlins, or lemmings or whatever. What if instead of couching speech about systems development in highly technical jargon, we told stories instead?

The Morality Argument

There exists a moral imperative concerning software and our world as well. Our world today is driven by software. Most of it is terribly written and really hard to reason about because of that, and by some miracle still works a good 80% of the time. But bugs are rampant even in solved problem domains. A lot of time these bugs come from the difficulty not in reasoning about how a single component operates, but in understanding the relationships and communication patterns between components. Breaking software into components is a good first step in eliminating bugs, and there are probably thousands of articles on why this is the case.

We need to write reliable software. People’s lives rely on it often, including my own. In order to fulfill this basic moral criterion we have to come to the realization that our current ways of reasoning about systems are inadequate.

A History of Decomposition

There are several historically prominent methods by which systems were composed to try to improve the ability to reason about them, a few examples being structured programming, 90s object-oriented programming, and the recent uptick in functional programming. Finally, cutting across all of these methods was the usage of static typing to attempt to automate some aspects of reasoning so humans did not have to do all the heavy lifting.

Previous to the 90s, the predominant way this was achieved was through the usage of structured-programming which broke down procedures into sub-procedures that followed a few heuristics in order to stay well-behaved. But sub-procedures could literally do anything, and because of this they could manipulate state that other procedures relied on in a way that was unpredictable, etc. From Wikipedia:

Structured programming is a programming paradigm aimed at improving the clarity, quality, and development time of a computer program by making extensive use of the structured control flow constructs of selection and repetition, block structures, and subroutines in contrast to using simple tests and jumps such as the go to statement, which can lead to “spaghetti code” that is potentially difficult to follow and maintain.¹

In the 90s, the manner by which problems were popularly broken down was what is still called “object-oriented programming.” Unfortunately, the person (Dr. Alan Kay) who originally envisioned this concept much earlier (70s) lost it to the morass of popular opinion of something else entirely. Endemic in this approach is the anthropomorphization of things often leading to convoluted design that sounds something like “the elevator object talks to the floor object to know if it needs to stop.” Wut..? Floors don’t talk. Neither do elevators. But we were told that if we look at the requirements document and find the nouns, those can be candidates for our domain objects. There are numerous criticism of what OOP eventually became.²

This became such a travesty in design that many people opted to leave it behind entirely for the repopularized holy grail of mathematical or functional programming. Instead of floors and elevators, we had vectors whose positions represent elevators and whose values represent floors, and we pass these vectors into analysis algorithms also informed by call button state vectors. These combine to produce a description of an action that will alter which elevator will stop at which floor.

Talking about math will always make you sound smart to those who don’t know any better. Actually knowing what you’re talking about can make you admirable. If you honestly are amongst the relative few people who know what they are talking about when the now aphoristic “a monad is just a monoid in the category of endofunctors” quote gets bandied about, you know we are few.

In all of these forms of programming, types have also been included to allow for differing levels of automated reasoning about a program’s correctness. It turns out that writing types that abstract object creation (classes) or types that describe abstract mathematical properties across data structures (haskell typeclasses) is actually somewhat of a hard problem if you are looking for true simplicity. Typed languages that are used by well meaning, but under educated, people tend to end up in type declaration explosion. Type A and type B may do 50% of the same things, because they actually are ad-hoc realization of some underlying type C, but discovering the underlying abstraction C requires more than just copying and pasting the 50% of the code that is shared. This type of ad-hoc inheritance then leads to extremely brittle code when A and B are coupled in ways they should not be.

One of Alan Perlis’ famous epigrams states the following, “It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.” This pithy statement is generally backed up by concepts that can be borrowed from abstract algebra. But finding this data structure that gives rise to 100 useful functions is sometimes very obvious (lists come to mind), and sometimes much more difficult (program I/O).

The Proposal

There is utilitarian, aesthetic, and performative value for making systems design crowd-sourceable.

There is moral value as well.

Components of a system should be structured in such a way as to be as human like as possible. By this I mean components should have a purpose or a direct responsibility they manage, their interactions should reflect or caricaturize what would be a meaningful interaction between two or more people. Breaking up a system in this manner allows any person familiar with the system to recount the systems behavior as the behaviors of individuals. Inherent in this is the ability for people outside of the technical space to reflect and reason about the system. This necessarily allows recruitment of more people to more realistically leverage something they have spent their entire lives doing: thinking about human relationships, as a way to scale solutions to system design problems. This is the democratization of systems design. This is story-oriented programming.

This is not an original idea. This is a reframing of a previous idea that allowed the idea to click in my own mind. James Coplein gave a talk³ at the GOTO 2017 conference where he asserted that the original conceptualization of OOP was that the ontology in the user’s mind was directly reflected in the ontology of systems design. If I see a dog on the screen, I should be manipulating a dog in code (to some degree). This immediately sheds light on the absurdity of some sorts of OO designs (the character walks into a wall object, and it’s the wall that tells the character it collided.) This idea started me down this train of thought.

An additional novetly of this particular reframing is its internal/external homoiconicity. In many of Alan Kays talks that have been put on YouTube, he talks about how his original conceptualization of scale was based on a model of something that has already scaled: the cellular model of the human body. The human body and brain are a stunning example of singular components in a deeply woven network working very well at very large scale. But no cell is really “smarter” than another cell in the sense we might talk about a human being smarter than another human.

This begs the question to me, if the human body is essentially a sort of distribution and democratization of life to the constituent cells, then other forms of democratization may be able to be leveraged to produce artifacts at scale. Dr. Kay proposed that democratizing tasks to components similar to cells in a computer system would give rise to scalable systems. This was bastardized into 90s OOP which has failed spectacularly in some cases. Rewinding back to that original concept, and reframing it slightly; if instead of cells, we conceptualized the system as a series of actors, gremlins, humans, whatever, engaged in a story, we leverage a reasoning technology for interaction that almost every human can contribute to. We scale out the problem of systems design in the very same way that we scale out the system itself.

Objections

In internalizing and fleshing out this argument, I came across a number of objections that I addressed internally, and would prefer to address point by point. These objections came in two varieties: aesthetic or pragramtic. Aesthetic objections are those that argue for preferential prejudice based on an arbitrary factor that can slide around (an aesthetic). Pragmatic objections are those that argue that the actual enactment of the suggestion is not possible due to social or psychological constraints.

Aesthetic

Reductio Ad Absurdem, the Slippery Slope, if we do it in one place we have to do it everywhere.

This is the argument that everything, including a value as simple as a number, should be an actor. I don’t know that I agree or disagree with this. I am presenting the aesthetic I use for discrimination.

The extant universe does seem to justify that communicative decomposition can go all the way down. We can (and do) talk about quarks exchanging color-charged gluons. Charged particles like protons and electrons exchange photons. These examples attribute human-like communication or intention onto what are otherwise extraordinarily simple phenomena.

There does exist a realm of phenomena that have no physical, semi or otherwise, referents. Products of formal thought systems, such as mathematics, do not exist in the physical world. This is the perfect line in the sand that can be drawn to figure out whether or not something should be modeled as an actor or a solid process.

Thus we would probably not say that the number 5 talks to another number 5 saying “plus.” Instead we would just use the lexical constructs of that formal realm. Much the same way that set theory has its own set of combinatorics, category theory theirs, and high school algebra theirs, these systems of thought form closed, self-consistent systems of reasoning in which no negotiation or other communication needs to happen.

A caveat to this is that care must be taken to understand that purity is easily violated. While you can talk about numbers and strings as if they are mathematical constructs (and thus do not require decomposition through communicative processes), should they come to violate their own purity (a mutable string, or a sufficiently large number such that decisions must be made for memory allocation), we once again enter the world of a mathematical notion with a physical referent, and the case can again be made for narrative decomposition.

More relevant to real-world scenarios is the example of an HTTP web server. The protocol specifies that bytes coming in conform to a specific shape. This shape is a string with further constraints on what is in the string. However, part of the specification allows for some aspects of this shape to be unbounded, such as the request body. In process, you can model a request as a string, and then treat it as if it’s a pure mathematical entity. That works until you run into the real world constraint of large request bodies that may consume all available memory, or for performance reasons, you don’t want to locate the request entirely in memory. This breaks the string abstraction and shows the request for what it really is, an interpretation of electrical signals over a wire. Treating it under this lower level view also reveals the temporal-spatial nature of a physical process. Thus, eschewing the string abstraction and instead looking at the request as an actor capable of negotiating its available bytes honors the underlying process, and more accurately allows navigation of the optimization concerns previously stated.

A fascinating union of these concepts is to first design a sub-system using narrative design, and then, if it’s internal communication concerns can be safely abstracted over, a library that treats the flows as data objects themselves can be created that populates the sytem and changes it in otherwise pure ways. This allows the programmer to flow back and forth between the option to use pure manipulations and narrative manipulations depending on the use case. Additionally, as is the case with business to business transactions: collections of subprocesses can treat other collections of subprocesses as a macro process for sake of abstraction over an agreed communication protocol.

Math is Simpler than Natural Language. Mathematical models simplify, natural discourse obfuscates.

There is an aesthetic consideration that communication should be accomplished in the simplest possible manner. Simplicity is often hard. When you discover something simple, it tends to be universal. The language of mathematics is simple, and because of its simplicity, it tends to be confusing. Yet there is room for a philosophical critique that the simplest language is the most universalizable in potential. This runs afoul of pragmatic concerns.

Humans are not born into this world understanding math. The first linguistic tool all humans learn is messy, full of exceptions, and irregular. It’s natural language: full of tone, nuance, and context. Above and beyond that, a young human must learn to strip away context and talk in terms of formal abstractions. Many simple never learn this skill above and beyond a certain level. Pragmatically speaking then, while mathematics, abstract or concrete, has the potential as a universal language, it also requires a level of commitment or intelligence to reach fluency in, and therefore precludes its candidacy for a democratic language.

A prime example of math establishing a generic language is in regards to communication and side effects in programming languages. Most of these sorts of effects, such as time, space, divergence, can be encapsulated using a notion from applied category theory called a monad. If you read anything about programming in aggregators like Hacker News, chances are you have run across this word. And maybe you’ve also run across functor, applicative, typeclass, composition, identity, densities, ends, cones, isomorphism, anamorphisms, catamorphisms, yoneda lemma, kan extensions… (where is that Kmett generator when you need it).

And here is the crux. It takes a long time for most people to be comfortable even hearing the language of category theory, let alone internalize it to the level required to make proper use of it or to communicate it to others. For a lot of people, the math is simply intimidating, language, symbols and all. They will never learn it.

But — almost morally speaking — why should they? Monads as they are applied in computer programming, are a formal mathematical construction that captures the idea of composing functions whose values are annotated with additional data. That last part is an opaque way of saying communication. Any function whose output is a monad is communicating something in addition to the result. That communication could be failure, it could be a log, it could be the description for communication with an external system. These communications can then be sequenced meaningfully using the monadic combinators. This is all to say that monads capture communication and treat it both abstractly, generically, and mathematically.

I have to explain that. I have to explain monads. Let that sink in. The majority of programmers know how to talk. They, more or less, understand that humans have relationships, and that relationships are navigated using communication. This is wired deep into us, and you don’t have to even be cognizant of it to leverage it. Why am I going out of my way to tell people, “hey, you already have a powerful technology at your disposal, let me show you how to talk about talking in a way that you can’t understand?” I don’t have an answer to this question. There is an aesthetic beauty to the abstractness of monads that appeal to analytical minded people like me, so I think I erroneously assume everyone else has the same aesthetics.

Finally, I think the true death-blow to this aesthetic desire is the argument that a mathematical representation of a problem is necessarily a premature optimization. It is an optimization away from the most accurate encoding to one that ignores what are considered irrelevant details. Murphy’s law puts a time limit on how long those details stay irrelevant though. If you get lucky the intersection of when the abstraction breaks, and when you have to further maintain the code does not exist. More often though, when you aggressively pursue casting solutions into abstractions that ignore too much, you will be bitten more quickly.

The function f(x, y) = x + y is perfect until x and y are not providable at the same time. Turns out there is a monad for that (future/stream). And are not locatable from the same source. Turns out there is a monad for that too (environment/reader). The number of people who understand monads really well are few. The number who understand monad transformers (composing these effects) are fewer. The implementation of monad transformers in languages without sophisticated type checkers is a nightmare. So when we picked this beautiful mathematical abstraction that we attempted to force into a new requirement domain suddenly it got a lot more complicated.

Pragmatic

In addition to these aesthetic considerations are a handful of pragmatic objections.

Silliness and Ego

One possible reaction to the concept of story oriented programming, that I experienced myself, was how humbling it was (I just felt silly when I relayed the gremlin story to a coworker out loud) to talk about components of a system as if I was telling a story to a child. In reflection, this is not unexpected. Components of a system at this time will not be a deep and nuanced human, so talking about them as anything more than something like “see spot run” will simply not happen unless aspects are attributed to the actors that they do not possess. But no matter how expected this seems, it is almost shocking in a way to experience a reframing of a problem that makes it sound like any child could solve it. That is exactly the point. To democratize a process requires that it be rendered into parts consumable by the lowest common denominator. A 5 year old mind makes a great exemplar.

Avoiding humility or feeling silly or stroking a big ego are not valid (moral) reasons to preclude something that may otherwise lead to progress in an area that is difficult. If you are a very intelligent person, and you find that you have a hard time letting go of the need to midas-touch everything you interact with, you may be contributing to the problem of difficult to maintain and difficult to reason about software.

Condescension

A second objection I considered is the pragmatic issue of transition. Due the democratizing nature of this decomposition technique, systems problems can be more easily explained to non technical people so they may offer their own narrative reasoning. However, it seems conceivable that in some organizations, if you were to hear someone who normally talks in jargonese suddenly tell you a story about something that sounds like it’s for children, it could come across as extremely demeaning or condescending.

This seems relatively easy to mitigate by either only practicing it with a subset of the culture who understands the value it brings, while educating outside of the culture, or just starting by espousing the benefits to non technical people (project managers, etc) and that it sounds the same no matter who is talking about the problem space. The language does not dumb down the exposition to the listener, it lowers the barrier of entry across the board.

Identity Dynamics

The final pragmatic consideration that came to mind is chained off a consideration about purity and impurity. Sometimes people defer to mathematics so they can wash their hands of intentionally human affairs. An equation may say nothing about time, but neither does it say anything about gender or race.

Prominent code bases have had contentious conversations around the usage of identifiers or concepts like “master/slave” as being exclusive of Black Americans, or people inappropriately referring to sexual concepts in code (0x8008135). It is a valid concern that departure from the extremely formal world of mathematics necessarily can encode language people take issue with into discussions about an otherwise unsentient system. In my own story above I only had one actor of female pronoun.

My response to this criticism is that these issues transcend the solution I am presenting. You are no less racist or sexist just because you practice colorblind or genderless programming. Microsoft’s racist Tay twitter bot is an indication that the systems we build extend and encode the memes of the systems that produce them. Precluding your system from these concerns implies that you preclude the concerns from the system producing it, that is, your team.

Conclusion

That is Story-Oriented Programming in a box and why you may want to try it. I will be frank though. The idea seems wildly radical to me, as I fall aesthetically in the camp of people who want something like math to be the lingua franca of computers. I have used this post to espouse a deeply counter-advocated position that indicts my own position as elitist and inherently unscalable in both the artifacts it produces, and the systems surrounding artifact production.

Maybe it resonates with you. If so, I encourage you to try it today. I will for sure be attempting to put these ideas into practice soon in my own personal projects. Then maybe I will be brave enough to bring these concepts into larger discussions at my company, or in productions systems. The implications of this approach are extremely far reaching, and so the change they would bring would be sweeping.

The human condition is narrative. It’s time to directly weave narrative into our software approach. It’s time for some story-oriented programming.

https://en.wikipedia.org/wiki/Structured_programming ↩
https://en.wikipedia.org/wiki/Object-oriented_programming#Criticism ↩
www.youtube.com/watch?v=ZrBQmIDdls4 ↩

Critique of "Out of the Tarpit"

Sat, 19 Nov 2016 00:00:00 +0000

I just finished reading the paper entitled Out of the Tarpit and I wanted to share some critiques I have of it.

My critiques are mainly philosophical and linguistic in nature. While I agree that complexity is a large contributing factor to the software problems today, I take issue with the way in which they specifically went about illustrating it. They separated things into essential state (and complexity) and accidental state (and complexity). They then bring up what I think is an incredibly dubious idea of “required accidental complexity.” If they were going to go this route they should have chosen a better word, such as ancillary complexity, rather than picking two words that have opposing connotations.

This is not even the main critique. The real problem is one that I don’t think the authors are even aware of, and is postmodern in nature. Take for instance something like communication. I think it’s arguable that communication support mechanisms are as important to semantics as the “platonic” essence of the communication itself. For example, if I were to communicate to you over a phone that dropped every other word, that would not convey the “essential” meaning or semantics. Therefore the support system is required for semantic reception. Similarly, if I spoke 1/10 of average speed, you would fail to receive the “essence” of the communication. Carrier and payload, surface and deep structure are both required for semantics. You simply cannot escape this. As a preview to later in my critique, even in the most pristine DSL, you still have to type keys into a keyboard onto a virtual document representing some sort of executable structure. This is a carrier. The question of how much carrier is required to convey a given deep structure is something that I think is not naively derivable from just the deep structure. I think they influence each other, this is the heart of the post modern critique of language.

To make the point concrete, I can apply the abstract critique to their approach. They argue that there should be a separation of concerns with regards to essential logic and state, and accidental logic and state. This is, I think, a fancy way of saying, have a safe language, and an unsafe runtime, or have a language and an interpreter. However, their own definition of accidental complexity is “if a user doesn’t know what it is, such as a thread pool or loop counter” then it’s accidental complexity. Then they go on to define a language for the communication of “essential state” that involves joins, unions, projections, etc from the relational algebra. Try as you might, I am pretty sure structuring programs as relational algebra necessarily means that my mother wouldn’t know what “project_away a field from a relvar” means. This is therefore accidental complexity (by their own definition) in a solution that was supposed to eschew it.

This reinforces my point that carrier and payload of a message are intertwined. You can focus on reducing carrier weight, to the lightest possible point, which would be the lisp/scheme style DSL + interpreter, and I think this is a smart approach. However, I think the differentiation of platonic intent and surface structure is dangerous to formalize, and I think their own approach bit them as a performative contradiction. They essentially argued restricting yourself from a general purpose language to a relational language plus a relational runtime. But that decision was arbitrary, and was most likely done because the author favors the relational model aesthetically. The relational model introduces extra structural concepts on top of the business domain. The only way to strip away as much “accidental” complexity as theoretically possible is to ensure that there is no syntax that doesn’t relate directly to the underlying structure being conveyed. In other words, if there is syntax, it is necessary it be related to the domain that is being conveyed. Union, join, project are the syntax of the relational algebra (or possibly a populist politician), and therefore convolute the message being sent in the same way a TCP handshake convolutes the message being sent (though TCP doesn’t interweave with its payload at all.)

I part with the authors in thinking that we should all program in a relational model for large scale applications, and given that I think that was the heart of their argument (even though they had a lukewarm closing that self-deprecated their argument), I can’t say I agree with the majority of the paper. Moreover, with regards to their discussion of complexity, I think they used a naive method of discussing complexity that itself complected (to borrow the term from Rich Hickey) the understanding of how to deal with complexity by not addressing it holistically (in addition to atomistically).

Shared vs. Individual Aesthetics and Over-Engineering

Sat, 22 Oct 2016 00:00:00 +0000

I recently finished The Goal and I am also in the middle of The Lean Startup and Out of the Crisis. This has me thinking about several issues with regards the relationship between workers and the companies who employ them. A recent article ends with the lukewarm assertion that “Unfortunately, there’s no universal answer for what always constitutes Over-Engineering and what doesn’t.”

I disagree with this.

There is no first order definition for over engineering, but there is a second order definition that I think fits the bill perfectly. First I will need to exposite a little context.

In The Goal, the end strategy ends up subverting the need for local processes to have efficiency in isolation for the goal of having the system at large have the highest efficiency (throughput). The Goal specifically treats the context of manufacturing, and at the end, the fictional plant manager receives a promotion to division manager. Unfortunately, the story mostly stops here. We don’t get to see the strategy of optimizing the performance of the division.

I spent some time thinking about the adaptation of this to software development, which seemed to be much more artisanal in nature, and thus seemed to thwart the classifications laid out in The Goal. However, after examination I think I know what’s happening.

It is very common for individual developers to argue over the best way to approach a given problem. I think there are two reasons for this. I think each individual developer wants to be as productive as possible, and, altruistically, each individual developer has reasons for believing that these approaches would improve the productivity of other developers. Or in a more Machiavellian sense, they could just not want to have to deal with any other approaches. Either way, there is a distinct desire to optimize the productivity of the person in isolation. Specifically, this optimization tends to be based in aesthetics. We have the language of aesthetics in several areas of software development. We have beautiful webpages, software craftsmanship and other similar language floating around the industry. There is nothing inherently wrong with these concepts, but I think they speak to the desire of the individual to maximize their sense of productivity through the ability to freely express what is necessary to solve a problem. This is the first step toward understanding what constitutes over engineering.

The Goal systematically demonstrates that performance of individual constituent parts of a system will not perform with 100% efficiency when the goal is to maximize the throughput of the system as a whole. Looked at another way, the aesthetics of the business, or the values of the business, which is generally to grow, maximize throughput, and thus sales, do not correspond to the aesthetics of the individuals who comprise the system.

The definition of overengineering is extremely precise in light of these two things. A component, or section of a system, or the system as a whole is overengineered when the work that contributed to creating that part or parts exceeded the contribution necessary increase to the throughput of the system as a whole.

One example could be the argumentation over language selection or technology stack which delays the delivery of an MVP. Another example could be excessive rearrangement of the architecture of a subsystem. As developers, we don’t want to introduce bugs, or have bad UX, or poor performance, or any number of personal concerns be attached to our work. Producing “poor work” in the name of business needs drives most developers insane, myself included.

This would seem to, at least in my experience, classify all attempts at software craftsmanship, and desire to produce something of quality as being on the precipice of over engineering. More than this, it would color nearly every engineer I know as guilty of over engineering everything. In reflection, I think this is actually accurate, and I will offer an analysis of the social conditions that create this pervasive atmosphere.

In nearly every environment I have worked in, including startup environments, as lead engineer or journeyman engineer, there was absolutely no connection between the work I did and the people who used it. In fact, there was little communication of the value add to customers, or the ability to reflect on the delivery of this value add. There is little socialization of business values with engineering, and when there is it is canned, artificial, and there is no quorum about business value and corporate value that intrinsically includes engineering.

This leads to a categorically Marxist dissonance concerning working to produce something you will not see the value of in either your own usage, or the vicarious usage of others. There is literally no ownership over the value produced. Instead it is traded for capital. However, developers are not production line workers. We work in a medium of expression, and a little bit of our soul goes out with each line of code we write. Giving this away is hard to justify at any price. So we built adjunct guild-like structures where we confer on best practices, and we develop determinations of personal efficiency, whether that’s ability to code for 12 hours straight, the ability to write code any developer can read, the ability to write the shortest possible code, etc. These aesthetics arise as a way of resolving the internal dissonance for trading soul for capital. But there is no reason that shared aesthetics, business aesthetics, cannot be established.

This is how you stop over engineering. You socialize and build consensus between business aesthetics and values with the people who are responsible for delivering on those aesthetics. Because, at least for me, if I can see the joy of someone using my horrendously buggy software, I will be more concerned with continuing to deliver them value than I will be with its performance, its relative correctness, its code beauty, or any other artificial metrics that I use to measure the cost of my soul that I spent to create it.

Over engineering is definable in relativistic terms. The culture of over engineering condenses from a culture of toxic capitalistic separation of values. The solution isn’t necessarily a Marxist revolution, but a simple humanistic evolution that realizes that producers of software are human, and have human contributions. When they see the value of their contributions, they will stop inventing artificial metrics that cause local optima and detract from the throughput of the system as a whole.

The Case For Lenses in Pragmatic Business Applications

Thu, 06 Oct 2016 01:14:00 +0000

So it occurred to me today, as I was explaining an example of how to use Ramda Lenses that the value that lenses provide is something that is simple, but not so immediately obvious. A lot of people do not understand why you would use lenses in a business application, and I have had trouble explaining why you might want to in my own dealings with other coworkers, over something like plain addressing or encapsulation in an object.

Put succinctly, lenses are a concept that abstract away data structure addressing, access, and modification. This is dry, so I will attempt to put it in pragmatic terms. The power that lenses bring to a project is the ability to refer to addressing logic in a generic way that allows for the replacement of the underlying data structure without modification to any consumer code.

An example is in order.

Let’s say you have a person represented in a fairly obscure structure, maybe a list:

const personList = ["John Smith", 25, "123 Flower Ln", "Dallas", "Texas", "75204"];

And now you want to get the state, well that’s easy, just reach for the 4th element:

console.log(`proc list: ${personList[4]}`);

Now your boss comes to you and says, good news! We can represent a person as an object now!

You look over the specification of person now, and it looks more like the following:

const personObject = {
  name: "John Smith",
  age: 25,
  address: {
    street: "123 Flower Ln",
    city: "Dallas",
    state: "Texas",
    zip: "75204"
  }
}

So you find and replace all instances of your list index with the following:

console.log(`proc object: ${personObject.address.state}`);

A few months pass, the team has been infiltrated by elitist functional purists who insist on using immutable structures; or maybe they just want full blown maps in ES5…who knows. Your boss now apprises you to the fact that the person objects look like this:

const personMap = Immutable.Map()
  .set("name", "John Smith")
  .set("age", 25)
  .set("address", Immutable.Map()
        .set("street", "123 Flower Ln")
        .set("city", "Dallas")
        .set("state", "Texas")
        .set("zip", "75204"))

So you once again try to find and replace all your instances like the following:

console.log(`proc map: ${personMap.get("address").get("state")}`);

Exhausted from the change, and irritated at the edge cases where you missed a few consumers, you set out to refactor the code. Trusty object encapsulation to the rescue:

////// object approach

//// person encapsulating list
class Person {
  constructor(person) {
    this.person = person;
  }

  state() {
    return this.person[4];
  }
}

// modified to

//// person encapsulating object
class Person {
  constructor(person) {
    this.person = person;
  }

  state() {
    return this.person.address.state;
  }
}

// modified to

//// person encapsulating immutable map
class Person {
  constructor(person) {
    this.person = person;
  }

  state() {
    return this.person.get("address").get("state");
  }
}

Now your consumer code looks uniform:

console.log(new Person(underlying).state());

But did you need to immediately jump to encapsulation? Could you be introducing incidental complexity by using objects over the underlying data structures? Potentially, when it comes to mutation especially, objects can hide mutation, and you know your elitist functional overlords will never go for that…what to do?

This is where lenses come into play. Lets take a look at the two lenses that have helpers first:

////// function lens approach

//// index lens
const stateLens = R.lensIndex(4);
const person = personList;

// modified to

//// object lens
const stateLens = R.lensPath(["address", "state"])
const person = personObject;

But what do we do for the case when we want to use a more exotic underlying structure? We construct our own lens:

//// exotic lens
const immutableGet = R.invoker(1, 'get');
const immutableSet = R.invoker(2, 'set');
const immutableAddressLens = R.lens(immutableGet('address'), immutableSet('address'));
const immutableStateLens = R.lens(immutableGet('state'), immutableSet('state'));
const stateLens = R.compose(immutableAddressLens, immutableStateLens);
const person = personMap;

And now our consumer code looks uniform:

console.log(R.view(stateLens, person));

Both objects and lenses provide the ability to abstract over data access. Unlike objects, lenses come with three useful functions: view (allowing you to see the value), set (allowing you to change the value immutably), and over (view, apply function, set). These three operations are most of the boilerplate that is abstracted over in classical objects, but you can get them for free with lenses.

Additionally, if you’ll notice, the lenses were built piecemeal and then composed together using regular function composition. This is an important additional benefit. Lenses (of the type in Ramda), are just functions, and they compose nicely with any other functions. The lensPath helper in the previous block abstracts over even this aspect. It could have been constructed using composition and Ramda’s lensProp function.

To revisit – lenses abstract data addressing, access, and modification in a uniform (non-idiosyncratic or boilerplate) way providing uniform consumer code and the ability to change the underlying structure without modifying consumer code, without the headache or boilerplate of classes.

All functioning code in JSBin

Intentionality and You: A Closer Look at Novel Evidence Revealing the Existence of True Intent

Fri, 10 Jun 2016 23:35:00 +0000

Intentionality. Intention. Intent.

The concept implies a relationship to free will, to the ability to realize ourselves in a meaningful, contextual way, independent of internal and external forces.

There has been a lot written about free will recently in the discussion relating to artificial intelligence, machine learning, and the overall theme of sentience relating to computers. In the school of philosophy I largely identify with, intentionality would be a prerequisite for sentience. Intention is the compulsive flow of that which is me into that which is not. It is action, expression, realization. From inside to outside, that is the key thing. I have my private thoughts that I experience, that are interior to me. I realize them through intent into the exterior world. Or as Ken Wilber snidely put it in Boomeritis: a sentient computer is one that would to want to kill itself.

The reason I am writing this post is to explore a set of phenomena that I think grants important contextual clues to the nature of intent.

Sleep walking. Sleep cooking. Sleep writing. Building things while asleep. Ambien has given us a wealth of people who perform amazing technical feats while asleep, and those who experience their friends and loved ones performing these particular feats.

But if you are asleep, there is a lack of conscious attachment to the behaviors. Behaviors independent of consciousness implies that consciousness is not reducible to behaviorism. However, the depth of ability in these behaviors imply that behaviorism is obviously a very large part of experience. So if people are capable of driving to work while asleep, and navigating traffic, then where does that leave intent? Is it as bleak as some people would have us believe? Is intent merely an ex post facto experience to help us cope with our behaviors?

No.

The reason I say no is because there is something distinctly missing from the experience of people who behave while sleeping, and that is rational filtration, reflection, and any sort of internal experience whatsoever. There is no ability to stop what is happening. And while awake, we can choose to stop driving, cooking, or anything else we are doing. But while asleep these impulses are realized regardless.

Which brings me to what I think is an important reframing of the concept of intentionality. That which we experience as intentionality is not positivistic or constructivist. We do not build an intent. Behaviors and impulses arise within us, and intent is the shape we give the expression of those behaviors and intents. This is an important difference, and aligns more with the concept of gating and thresholding at the neuronal level. Free will is not the ability to create something from nothing then, free will is the ability to choose from the myriad impulses that arise within us. Freer will is the existence of those impulses to begin with (after all, if you can only choose from raping and pillaging, I suppose you are going to rape or pillage). Filtration of impulses (intent) leads to patterns of behavior. Patterns of behaviors lead to neuronal rewiring for efficiency. Neuronal rewiring leads to habitual impulse in anticipation of stimulus/response.

So then we can say that free will is manifest in neuroplasticity. It is the exterior form of free will. The more neuroplastic we are, the more free will we have, because we can more quickly rewire our brains through our intent. In this case I am reducing “free will” to the ability to operate my mind and body in the manner I desire, I am not factoring in social contracts and cultural bindings.

Intent feels positive. I experience the decision to go to the bathroom, or to cook, or whatever. But I think a more mature understanding of intent is a navigator of a ship. Forces act on the ship (wind, current, etc), but it is the captain who learns to navigate the forces and provides structure around them to direct the ship. This means intent is actually negative, it is a gate, it is that which stops the impulse. And of course all things that are new are old. I leave this post with the following verse from the Tao Te Ching.

Tao Te Ching: Verse 11
We join spokes together in a wheel,
but it is the center hole
that makes the wagon move.

We shape clay into a pot,
but it is the emptiness inside
that holds whatever we want.

We hammer wood for a house,
but it is the inner space
that makes it livable.

We work with being,
but non-being is what we use.

———————————————

Do not identify with what is. Do not identify with what is not. The positivity and negativity of being and non being arise together. The negativity of intentionality arises with the positivity of behavior. The bends in the river are as much a part of the river as the water which rushes through it. You are neither yin nor yang, but the circle which encompasses both.

From Desolation to Dissolution

Fri, 08 Apr 2016 01:12:00 +0000

I originally wanted to be dramatic and write this post on some sort of logical interval (i.e. a year). However, recent internal events compel me to write this in the present. For those of you who know me at all, you know that a close friend of mine decided to end our friendship in June of last year (2015), in wildly dramatic fashion. This post is not a history of that occurrence, but a chronology of my personal transformation under what has been one of the most internally trying times of my life.

The nature of the friendship was codependent. I relied on him, and he relied on me, and it was more than just resourceful reliance, it was also emotional reliance. It was the classic “A” pattern codependency, and being that there were loved ones involved as well, it fulfilled the Karpman Triangle rather spectacularly. This was the backdrop for a really out of control interaction that led to him leaving.

In the wake of that, I was left wholly bereft. A person with whom I had irresponsibly fused some of my personal identity to had basically rage quit and taken his ball and went home. Except the ball was supporting my shanty-town of an identity, which subsequently came crashing down. It took about a month for me to regain full functionality (with therapy), two months to regain composure, and it was around three months that I started rebuilding my sense of self.

I would like to acknowledge several books (and the authors) that helped me rebuild this sense of self as a truly independent internal structure that was independent of any person except myself.

The Six Pillars of Self-Esteem: The Definitive Work on Self-Esteem by the Leading Pioneer in the Field by Nathaniel Branden
To Be a Man: A Guide to True Masculine Power by Robert Augustus Masters
Radical Honesty: How to Transform Your Life by Telling the Truth by Brad Blanton
Practicing Radical Honesty by Brad Blanton
Emotional Unavailability & Neediness: Two Sides of the Same Coin by Gabriella Kortsch
The Power of Your Heart: Loving the Self by Gabriella Kortsch
Consolations: The Solace, Nourishment and Underlying Meaning of Everyday Words by David Whyte

These books helped me reestablish a sense of “me-ness” that in the continuing months I would be able to subsequently build on. I would also like to acknowledge the work of Casey Capshaw and his work with the AMP podcast (which is on iTunes, but is now defunct), and Mike Hrostoski in his generous broadcast on Periscope. I highly recommend most of these resources to anyone who needs help rediscovering what it means to truly connect with yourself in a healthy and meaningful way (with obvious gender limitations with some of the resources). I cannot promise they will help you if you are in a codependent relationship, as it is easy to be distracted by codependency and develop a false sense of security.

The consumption and integration of the material of these books and resources took several months, and was delayed by two other different personal relationships that I was involved in. With the evacuation of these relationships from my life, I was left in a relative solitude. This was around month six of my reboot. This is about when I posted the post How Do You Value Your Time? In the months following that, using my new found sense of self, I was able to confidently find a female partner with whom I am happily in a relationship currently.

This brings me to about February of this year. Having always been a personal growth type, but having been distracted by rather poisonous and inhibitory situations, I found myself with a newfound security, but lack of underlying substance. There was a “me” but who was I? This is the more existential version of the question, not the foundational version. This led me back into a search of my youth, a deep exploration of personal spirituality.

In my teens, I was deeply inculcated in occult philosophy. I ultimately abandoned it when I couldn’t buy the “myths” that it was peddling anymore. After that I studied Ayn Rand objectivism, and from there Wilberian integralism. I moved into a “hyper” rational objectivist view of reality until about the time where I entered into friendship with this person. I had adopted Zen Buddhism as my dharma about a year before meeting the friend on whom this post is predicated. Zen allowed me to exercise a level of ontological abandonment (the epistemic fallacy). This lack of ‘certainty’ of knowing is partially what led to the events of my codependency.

Around February I received an email about a course from Ken Wilber on how to prepare for death (such as a sudden death scenario, not the Im-getting-old death scenario), created in concert with an author who was an expert on Tibetan Death Traditions. This was the little kick I needed to reexamine what had been, up to that point, a purely epistemological view of the world (a messy, disconnected hazy version of reality in which that which is known is all that is, never reflecting that which truly is, and in which that which is known is not ontologically secure in any way). I have not completed the course or accompanying book quite yet, but it provided enough motivation to revisit the Wilberian integralist point of view which I have always been fond of.

So I reread Sex, Ecology, Spirituality: The Spirit of Evolution and found the critique of a worldview I had adopted so completely moving (it is simply amazing how my cognitive development over the years facilitated a reading of the critique that I simply could not process at 21 years old) that I have moved back into a contemplative mode of existence for about the past 2 months.

Up until this point of my recovery, I had had a satori, which, in checking around, seems to correspond to the fifth bull. After a few months of death contemplation (a subject that I had avoided out of nihilistic dread before), and with the provocative words of SES created a frame in which I believe I have very recently had a realization of the sixth bull. And I can feel the pull of deeper and deeper, higher and higher transcendence calling to me.

And all this is to say, as the title so states, that I have moved from an utter sense of desolation in my solitude to an existence on the brim of spiritual dissolution. I can feel the pull of infinite causal awareness. It has already been something I am experiencing more regularly, and in my solitude now, I use the time to cultivate, like a gardener, a self-flow that reflects the infinite causal ground from which all things arise. I liken it to Kinhin, the Zen walking meditation, but instead of it being in a Zendo, I cultivate my Kinhin as a living meditation, flowing, interacting translogical experience in my solitude.

Tickled by the notion of a 140 characters as a poetic limit, I composed a series of poems on twitter that I want to make sure are not lost to the annals of my twitter ruminations, and so I will recount them here in order as a commemoration of everything I have been through.

Eros from me
Agape from you
But Phobos for thee
And Thanatos my rue
I pulled us up
You stretched us out
I held you tight
Then did you route

Our codependency
Sounds almost dialectic?
Fusion does not abide
You and I
Yet together we discovered
And then forgot
Our Original Faces

I’d like to say
That I am alone
Without you
But I remembered
My Original Face
So instead
I am just sad
That you haven’t
Remembered Yours

In the dark night
That shrouds our Sangha
The silence that
Rings so loud
I will hold a candle
So if you Remember
You can make
Your way Home

I write this story to serve as a demarcation of my progress, to commemorate and honor myself in the trials I have endured, and to hopefully serve as inspiration or possibly even a mild guide to other people facing a similar situation.

The world is illusory.
Brahman alone is real.
Brahman is the world.

If you are scared when you are alone, that is ok. You have simply forgotten that you are divine. Work everyday to remember your divinity, to remember the face you had before you were born. Work tireless to realize the world as your sangha, and never in your solitude will you ever feel alone again.

LambdaConf Controversy through an Integral Lens

Mon, 28 Mar 2016 16:21:00 +0000

When the LambdaConf decision first came to light, I followed a number of very lively, vitriolic, passionate responses on twitter denouncing the decision to permit Curtis Yarvin (a.k.a. Mencius Moldbug) to speak at the conference. The conference took the viewpoint that his contribution outweighed his beliefs, and furthermore, that his beliefs were irrelevant to the conference. Many people who are opposed to his attendance and speaking argued that you cannot separate a person’s contributions from his political beliefs, just because he is only talking about one at a specific time.

Yarvin wrote a rebuttal to people’s objections saying that what he believes, and what he will be presenting are different things, which is very similar to the conference’s own stated position.

I continue to follow it as it is currently developing right now.

This strikes me as a very contemporary struggle between the orange rational level and the green postmodern level. Orange (from which Yarvin and the LambdaConf organizers are arguing), is the first stage where concern for other humans manifests at all, coming out of the ethnocentric amber state which would be more willing to burn someone at the stake than to engage in any sort of discourse, even if that discourse were dehumanizing. To people at orange, there is some awareness of the height of this particular unfolding, and so they justly (for their level) argue that the achievement of a human should overshadow his particular beliefs. This is the very unifying force that drove orange out of amber to begin with, so of course it is going to seem extremely prominent as the foundation of any argument.

The opposition is largely coming from a green altitude, where there is a rejection of the fundamentally dehumanizing aspect of orange. At orange, if I make more money than you, it’s because I am just better than you. Not morally superior, but economically. If I am more beautiful than you, it’s because I was born with better genes. Orange is the level where rationalism reigns supreme to the point where the subject is nearly completely suppressed in an attempt to actualize through achievement, whether that achievement is “the way things are” (biological determinism) or “the way things came to be” (individual exceptionalism).

At green, there emerges a truth that while no human is better than any other, some humans are suffering more than others, and it is our duty as agents of inclusion to make sure that no human is left behind just because some other human succeeded. This is the positive aspect of green. At the extreme, green is still a tier one consciousness, and because tier one relies on setting extrinsic security boundaries in order to stabilize its emergence, green will roundly reject orange contributions. “You are unwelcome to speak about virtual machines because you think that slavery is ok.” However, this is categorically throwing the baby out with the bathwater. Green generally accepts this, and assumes that the baby was not worth saving any way if the bath water was too dirty.

This rejection, that all tier one consciousnesses harbor, first finds its intrinsic contradiction in green who is so obsessed with inclusion. But the exception to the inclusion is if you do not share the mindset of inclusion. So it intrinsically has an exclusive quality. As I stated before, this is part of the extrinsic insecurity of tier 1 stages, it is required for the stage to fully actualize. And at this point therein lies the issue.

At an integral level, and starting at tier two, the consciousness first begins to understand that these tier one parts of itself are intrinsically valuable. It understands that without orange, there is mental stagnation; without green, there is too much division. But tier two comes from a level of understanding that extrinsic security is not longer necessary. Instead, intrinsic security takes over (I, as a universal, am not threatened by the machinations of levels preceding me), and a drive toward actualization of all that has come before begins. Teal begins the process of weaving the carpet that eventually becomes the tapestry of the Kosmos to Turquoise.

And that’s where I spend a lot of my waking life, is at teal bordering on turquoise. I have access to turquoise consciousness, but I am not stably there yet. So I thought, in what I think is traditional teal fashion, how could we prescribe an integral solution to this problem. I used my intuition into turquoise (i.e. how would a group of turquoise people handle an orange speaker), and came to the conclusion that there really is no way unfortunately. If the conference is predominantly orange, it will push away a lot of green contributors. If the conference is green, it will outlaw people it sees as “too” orange. There is no reconciliation because both viewpoints believe themselves to be absolute. These viewpoints can be reconciled internally (as in, I can operate at green and orange in order to inform teal and turquoise), but they cannot be reconciled into a group.

And so that leaves us with both sides being true, but partial. Orange conferences will focus more on individual achievement, to the potential (not guaranteed) detriment of some class of people, in this case anyone who has been a present day target of the philosophy that Yarvin perpetuates. Green conferences will focus more on community, choosing only the people who are a part of that inclusion community to speak, at the risk (but not guarantee) of throwing the baby out with the bathwater. There is no reconciliation or solution to this problem is what my turquoise intuition tells me, because this is exactly how these levels are supposed to act. Yarvin is on the unhealthy side of orange, and I have seen some unhealthy green responses, but the gravity of both the orange and the green communities that are arguing seem to be doing exactly what they have learned to do thus far.

So therefore this controversy, this discussion, as passionate and lively as it is, is completely necessary and appropriate between these two levels. There is no “reconciliation” because this is how these levels relate, and it is the ties that bind, whether conflict or accord, that tie together the world in which we live.

The Interrelationship of Object Orientation and Functional Programming

Wed, 20 Jan 2016 20:13:00 +0000

I once was a zealot of object oriented programming. Then I was born anew into the fascinating mathematical world of functional programming. It has been my journey in the past six or so months to reconcile the two. I had the pleasure of conversing over email with Dr. Alan Kay to get his opinion on the matter, because as far as I can tell, he is still one of the most vociferous proponents of message passing and late binding (and why wouldn’t he be?) The following treatise will be my attempt to contextualize the two modalities so that they can be thought of as a single paradigm, and not two paradigms.

First I would like to start with a disclaimer. I have never been a really detailed oriented person. While I have a gift of memorization of certain details, most of the times, I focus on the patterns that connect, and so you will not find rigorous mathematical proofs or anything like that from me. You will instead find ideas that may not even have the formalizations yet to be borne out of proofs. This means I will hand wave, and I will hand wave hard. I realize that hand waving puts me at risk of missing important discriminatory details. Such is life, and I will revise my conceptualizations if my hand waving is contradicted.

Object Orientation

I think it’s important to talk about the strengths of this approach as were originally envisioned in the 60s (rather than the panacea in the 80s/90s).

There are uncertainties in the world. How do we address these uncertainties? We just hope for the best. Literally. Using a message passing attitude, we send messages to receivers from whom we should really not expect anything in return because we don’t know if they received the message or not. This is put to the extreme in languages like Erlang where all messages are sent asynchronously.

Some languages, most languages frankly, which claim to be object oriented, lead us into the false trap of synchronicity, or the idea that we can count on the return value being a meaningful acknowledgment from the receiver of the message we send.

Before I finish that thought, I would like to posit that in strict OO languages, like Ruby for instance, there are 3 meaningful orthogonal categories of objects:

value objects, which are immutable and increase the richness of the languages primitives by creating new primitives at a different level of abstraction
stateful objects, such as random number generators
external interface service objects, which help us communicate with the outside world.

It is possible that some objects become a conflation of the three, but I would argue that decomposition into one of these three types will help keep the system easy to think about (I’m looking at you, Active Record object).

The only object which you should send multiple subsequent messages are value objects.

list.map(&:capitalize).first[0...-1]

in Ruby is a good example of something that is the transformation of value, and not the communication of said value. Chaining these messages is tantamount to expressing a contained algebra, and not object communication per se.

The other two types of objects should be assumed to be stand alone processes which will communicate their states by themselves. While random number generation is somewhat fast, imagine you have a prime number generator. Each time you want to get the next prime, you will wait longer and longer. Thus the prime number generator should communicate its findings to a listener asynchronously.

Most understandings of single responsibility principle are deconstructive. Given a massive heaping mound of junk that is a giant method, we tease apart responsibilities until one remains. However, the preceding paragraph lays out a constructive framework for methods, namely that every method should have one calculation, and one-to-n asynchronous communications of that calculation. That is it. The moment you wait for a message to return and you switch on it, you have introduced temporal coupling into your method, and certainly it is now doing more than one thing (sending a message, and then deciding which next message to send).

def magic_string_manipulation(s):
  new_s = s.upcase.reverse
  @listener.manipulated(new_s)
end

Here is an example of a method with a single (compound) responsibility, it communicates the new value of s to a listener. Here is how that method can go wrong:


def magic_string_manipulation_bad(s):
  new_s = s.upcase.reverse
  printed = print_something(new_s)
  if (printed)
    @listener.manipulated(new_s)
  end
end

This example sends a message to what is ostensibly a string printer, and waits for a response. This subsequent code, the if statement, is now spatially coupled, and method arguably has three responsibilities which is to first print something, the structure of deciding of whether or not to send a message, and the actual code to send the message. The similar method

def magic_string_manipulation_alt(s):
  new_s = s.upcase.reverse
  print_something(new_s)
  @listener.manipulated(new_s)
end

Is an acceptable variant because the responsibility of the method is to communicate the new value to the interested parties, which happen to be two different objects. This is generalized to sending the message to n parties over a loop of observers. What makes this acceptable is that there is no temporal coupling. I can call print_something and manipulated in whatever order I want.

One calculation, one communication (to possibly n listeners). This makes a method with one responsibility.

As a final note, message passing does not require methods on objects per se. There are other notable constructions of message passing (such as CSP), which allow for the organization of code around a different atom than the method.

Functional Programming

Let’s switch gears for a moment and talk about the strengths of functional programming.

Functional programming has its foundation in mathematics. In most simple mathematics, there is no concept of time. You have what is called equational reasoning. An equation such as x = 5 is true now, and 5 minutes from now, etc. Leveraging this knowledge allows us to do decompositions of equations such as y = 5x² + 6x + 7 where, because of the commutativity and associativity of the addition operator over the real numbers, this can be deferred to 3 different processes (one to calculate each term). Many gains can be made when there simply is no concept of time.

Similar to the concept of equational reasoning is the concept of referential transparency. I can say y = 5x² + z + 7 where z = 6x and it’s exactly the same as the previous equation because, once again, there is no time. Whether I use z symbolically or 6x is irrelevant, and I can optimize computational resources based on this substitutability.

Finally, also in reference to the both equational reasoning and referential transparency is the concept of denotational semantics. Denotational semantics is essentially the idea that the semantics of the program can be inferred from the semantics of its constituent parts (denotations). If y is our program, we can say that the meaning of y is 5x² + 6x + 7. If we are also given x, then we can further denote the semantics of the program. This is an incredibly helpful model of computation as the compositional capabilities of purely denotative code are endless. If you can write every intent as a denotation, you can achieve infinite composition, which means infinite code reuse, which is the “gold standard” of some aspects of programming.

But you may notice that time and again, I have said that there is no time in these models, and that is true. Once time is introduced, the concept of denotational semantics and referential transparency start to get muddy. If x = 0 at time 0 and x = 5 at time 1, we cannot say what y equals universally. This is one aspect of time. The other aspect of time is communication. A system who merely calculates something but does not communicate the calculation fails to achieve anything at all. Communication is a hairy problem because we have to decide to what or whom we are communicating. If we communicate to another process we own, we can ensure nice denotational semantics. If we communicate with a process we do not own, we lose this.

Bureaucracy vs Democracy

Some useful models for the “at scale” conceptualizations of functional programming already exist at the social level. These can be helpful when understanding how OO and FP relate to each other, and when it could be appropriate to use either.

You may have noticed at this point that original object orientation, with its emphasis on message passing and late binding is very good at communication. It is the ultimate democracy when done right. You may have a few infrastructural units that ensure that messages are delivered, but other than that, the behavior of the system arises from the communication of the constituent parts. This is like a slime mold, no centralized structure, like a brain, but high intelligence (adaptability).

Functional programming, with its rigid and formal decomposition is the perfect hierarchy. The main program delegates to constituent calculations and behaves in a highly formal way. This is more like the army, built to do one thing, and one thing well, and not very adaptive to change. It is also like the governmental bureaucracy, a hierarchy of units that perform one task and delegate subtasks in the service of some set of goals (or one goal if you consider the “law” one goal).

Bureaucracies are good at representing closed systems, or systems in which all inputs are formalized (if not known in advance), and their transformations are formalized so the output is essentially a transformation or extrapolation of the input. Democracies are good at representing open systems, where inputs can change, and cannot be formalized, and decisions have to be made about new or different requirements as the system evolves over time and agents are introduced or removed.

I would like to give a nod to the fact that it is theoretically possible for bureaucracies to adapt to change by imposing a supervisory bureaucracy to manage the structure of the managed bureaucracy. This means that it is possible to manage “degrees of temporal freedom” in systems that otherwise represent closed systems. Similar to the concept of position, velocity, and acceleration, you can have data, transformations, and meta-transformations. Each level of abstraction on the transformation allows a degree of temporal freedom on the underlying transformation, which can allow a closed system to be more flexible. Which is to then say that functional programming systems can have temporal formalizations allowing them to change with time (and thus current conditions), but ultimately all functional systems require formalization, and formalizing at a meta-temporal level of abstraction can be quite difficult for even the most gifted person. One method of managing temporal freedom in this way in a functional way is through functional-reactive programming. When you have streams of streams, you then have the meta-transformational layer as well.

The Obligatory Monad Detour

The manner by which the richest functional systems communicate change is typically through the usage of a mathematical construct known as a monad. I’m going to hand wave the fuck out this next part so bear with me, this is mainly to communicate to people who have not studied category theory.

Imagine you have two worlds, a world of green objects, and a world of blue objects. If there is a mapping between these two worlds such that for every green object there is a blue object, and for every green relationship there is a blue relationship, you have a functor. Functors preserve objects and relationships from one world to another (world = category). Now imagine you have a second functor that maps blue back to green, but differently from the first, so the second functor is not the inverse of the first functor. Now you have a way to go from green to green (like a regular green relationship), but going through blue. This traversal and transformation, by way of the blue world is a formalization of the act of communication. In Haskell, it is through observation of this second “world” that actual communication with the real world happens. However, observation is not always required, sometimes the second world merely hides the accumulation of effects (such as in random number generation). But for either case, monads exist as a formalization of different types of effecting mechanisms.

Get ready for REAL handwaving here. Monads are similar to vector mathematics and vector calculus, in which you have n orthogonal dimensions that you are manipulating at the same time. A side effect can be modeled as a consequent orthogonal calculation (sometimes with no meaningful input or output) on a dimension orthogonal to the original one. A simple monad is like a two dimensional vector, and a transformer allows for n-dimensional vectors, which all communicate their respective side effects to the correct dimension.

This understanding is still a work in progress for me, as it does not directly translate intuitively to comonadic structures, but I believe it is sufficient as to be useful to anyone who understands it.

The Reconciliation

Alan Kay said somewhere that CLOS (Common Lisp Object System) was the closest thing he could think of at the time to his original conceptualization of message passing object orientation. Lisp lends itself particularly well to functional programming techniques, if not being a pure language like Haskell. In fact, SICP is taught in Scheme, a dialect of Lisp, and all of the major foundations of object orientation are introduced, if under other names. Additionally, Lisp’s strong history with homoiconic macros introduces a level of power and expressiveness that allows for symbolic manipulations, which help with adaptation to change over time with uncertainty. This is why Lisp is employed so often in AI and machine learning, because it can do symbolic manipulations with expressive ease.

This is not to say run out and use Lisp. The point is that Lisp is a multiparadigm language, and in some of its best forms embodies the strengths of both an object/message passing model, and a functional calculation model. Most object oriented languages excel at inter-object communication; dynamic OO languages excel at late binding. But communication is one half of the coin. Good strong calculation semantics are given from the world of functional programming, employing them to ensure that the values that are passed in messages are correct is something that suits it very well. Calculate using FP, communicate using OO. Or as a wise friend of mine once said to me, but it didn’t click at the time “OO shell, FP internals”.

If you are using a language like Haskell, you are at the somewhat forefront of mathematical formulations of communication. Math always lags phenomena because it attempts to formally describe it. OO systems can produce phenomena that math will not be able to describe formally for a very long time (neural networks are a good example). This limitation is not very limiting however, in that it is possible to build a good deal of systems that are required today without emergent behaviors. Additionally, there are some systems that are completely closed with well known constraints. You don’t want your brake system in your car or any sort of life support system to have “emergent” behavior. You want to know, formally, that it works, and FP heavy languages can give you that.

Now it’s time for the old, boring, played out platitude: in computation, when things are turing equivalent, there is no “right” way, only tradeoffs. Assess your tradeoffs and execute accordingly.

How Do You Value Your Time?

Fri, 15 Jan 2016 23:26:00 +0000

I basically started this on twitter, but I want to write something that I think everyone can take something away from. You charge your employer for your time. Maybe not as much as you like, so lets assume you charge your employer what you feel like your time with them is worth.

Now here is a value that you can loosely use to factor into decisions. When you spend time with people, you do it for free; you are roughly giving up the amount you charge your employer in order to spend time with some person. When you think of these people in your head, would you pay someone (or forgo) the amount your employer would pay you to spend time with them? If not, and you think you would spend less money to spend time with them, and you spend time with them anyway, you are saying that your time is less valuable than you charge strangers (your employer). This is magnified exponentially if you actually spend money on the person as well.

If a person cannot assess your value, or cannot communicate your value to you, that person is intrinsically creating a space in which they will not give you what you are worth in the time you spend with them. So don’t spend time with them. Over time, this assumption of non-worth changes your internal value scale of what you are worth, because they treat you like you are worthless, and you accept that. I am worth what I charge my employers at the very least, and all of you are worth what you think you should charge them.

I realize this analogy is a bit tenuous, but it’s just to get people thinking. When you are alone, you have all the power in the world to create a $100/hr or $200/hr or $300/hr experience. But you lose some of this control over your experience around others in order to relate to them. If you can create a $100/hr experience for yourself, your friends should at least be able to meet or exceed that. If you’d only give up $50/hr to spend time with them, you are saying to yourself that you are worth that instead.

For your sake make sure your experiences with others are of a higher value than what you would give up to deal with the people you deal with all day at work.