Software Engineers Should Standardize and Unionize (Rated R)

Prostitutes have organized against unfair treatment by society, according to several news sources 1, 2, 3, that police may be disallowed to have sexual relationship with a prostitute in order to arrest the prostitute in hawaii.

The example given in the article says that a police received a blow job from a prostitute (anything other than penetration), and then promptly arrests her for prostitution. Some how the prostitutes felt that this, among other treatments, were demeaning and wrong, and they organized and changed the law.

Software engineers should unionize and make this otherwise proud and difficult professional a respected one, one at least as respected as prostitution.

In one scenario, the manager performs his managerial duties of keeping a project slow or delayed by strategically introducing incorrect information or even outright lies or even blatantly changing documentation or code during some phases of the project and later altering it again to create incorrectness in some part of the software design or coding.

In another situation, the natural abilities of teammates are kept at bay by various nefarious means so that the “right” person, typically an ardent supporter of the manager who is either lacking the knowledge or experience or who just has flexible professionalism and are willing to put in whatever suites management irrespective of actual technical viability. (Sure they are all viable choices and we can write in X86 assembly too, just as long has his gang member knows it well enough)

There is a sorely missing industry standard for software engineering. There is nothing that stands for good (enough) software. The government cannot even put a “Engineer” stamp on software engineering as it can put on structural or civil engineers. Any foreign third grader could come to the US, slap a Mooc in his resume, and become a senior software engineer with rousing praise from others of similar situation a few years earlier… (same applies to domestic self-starters, but please DO NOT come to my home and put dog shit on my lawn, this is an attempt at a civil discussion of my concerns)

Look, guys, look at the Hawaiian prostitutes! They can see something when it is wrong. They stood up against a part of society that is wrong: those who take advantage of prostitutes by enjoying their service–including police man while trying to arrest them. (Of course the accusation is beyond that. Law enforcement actually are accused of soliciting bribery in the form of sexual service for favorable treatment) This blog does not comment on morality of individual state or nation states law enforcement practices. Instead it states a tautological fact: the law strongly discourages the sale and purchase of sexual service. And that should include law enforcement officers. It is an enforcement of something that society has agreed is not right.

Now, can we agree on things that are not right in software engineering?

I don’t imagine that your typical software engineering project my team does to be more complicated than building sky scrapers(think! What would/can president Trump do??):

  • market analysis and requirements
  • product requirement and specifications
  • product design
  • financing, resourcing labor and materials
  • designing and deploying project management
  • project implementation
  • quality assurance
  • completion
  • launch
  • maintenance
  • transfer of ownership
  • remodeling
  • demolition

Think of it! Buildings fall down at far lower rate than you getting sad whale at twitter, or even the once in a while sad robot from Google, when did you ever see a sad leaning Empire State Building?

Think of your salaries and stock options. Think about the quality of service You perform for your fellow man.

Think of the last SEV you caused.

Now think of that builder putting together that building, and the SEV of his work. (Or also, think of those high speed rails in China or Japan, or think of an airbus jumbo jet… there’s a lot of really awesome engineering around the world)

Now think of the government inspectors, zoning, city planning, the federal regulations, the building codes…

Now think of president trump who made some buildings all work.

Now think of the prostitute who demand social justice after being caught sucking a police officers dick. She wasn’t allowed the dignity of washing her hands until she was booked under camera! Think of the dehumanizing experience.

Now think of the last less-than-stellar line of code that you had to write with your hands under duress under recording and review by your company’s overseers, other than that last SEV.

I don’t know why these things mutually activate in my head. But let’s do a association test, do you really feel closer to one of those other engineers than the prostitute? What’s your MTTF?

But all of it could be “solved” if software engineers just had a little pride in their industry and take a stand against “what ever suites the business, the product manager, the engineering manager” or “agile means don’t think, just build, software things naturally and surely will break”

STOP THAT. It doesn’t have to be this way!

We can build something right–risking their exorbitant salaries and stock options of course. Once software engineers standardizes practice and unionize, we would kiss those big pay checks and ISO’s and RSU’s goodbye.

But we can get a button that says “you are a software engineer”

I know I know, a honest to goodness software engineer wouldn’t need a button to know that… and plus look at all the raving reviews on LinkedIn! Isn’t that button enough for you? (No, not really)

(It does not escape me that once we build a reliable car, the demand for or next model would be lower, so let’s build a car that have parts taking turn breaking right after warranted expire and still take on the lost revenue on maintenance…. darn I guess there are other kinds of engineers out there too…)

But we can do better.

There is a lack of objective software building standards. There is a lack of responsibility towards software engineering, instead there is presently only responsiveness for business or politics. There is no pride in this line of work outside of the number next to that $ (or your CTR, or your QPS, or your LOC… take your pick of business metrics)

😦

Let us take a stand and be a better engineering discipline!

Whence be Villainy (retro)

This blog post is being scheduled for far far in the future of actual events that inspired it. All statements are fictional:

At fraud convention, a pair of specialist come on stage and talks about a phishing scam being perpetrated by a desperate Greek. He tricks a stereo typical American broad into committing mail fraud for him and while he’s at it he gets her to buy some stuff for her on credit cards that unbeknownst to her were stolen. But in out experts hands, the bad Greek actor was captured by Interpol and stolen goods were recovered. The broad was slapped around until she smarts.

A few speakers later, another ring was disclosed. This time Asian crime ring. Huge syndicate got us in the back with a Chinese cleaver. We never saw it coming. Thankfully we have gathered the pattern of attack for future preventative care! Another heroic win, sort of.

Do you hear any problems with this? Maybe not if you are an American. We are used to being gouged or stolen from by those poorer, lesser developed, lesser democratic, not-chosen, further-from-God, faceless, moraless creature swarms. What’s next? We’re ready for Algerian prince, Latin American drug lords, or the polean Penguin?

Quickly commissioner, to the bat signal!

From whence shall our next major villain emerge? Where ever it is, it matters little as we will be ever vigilant and perpetually watching from our cave deeply buried under all that worldly wealth.

You better watch it! Evil doers and evil doer’s henchmen! Because we are watching you!

Gross do not read(retro)

By some random brain crossing I remember a conversation at my son’s first pediatrician visit at 24 hrs. I asked the doc why his sexual organs seems much smaller than when he had first emerged the womb with.

The doc explained that there is a lot of trauma, rubbing, squeezing, etc. during birth that causes the components to swell  as all human tissue due when squeezed Or rubbed excessively.

Okay, so…, this kind of means that his enlargements are due to rubbing on his mother’s vagina?And all natural births of men by women mothers experience the same.

Kinda gross. But it undeniably real.

Democracy is humanism

Been reading this book called Sapients. It is a fascinating overview of many important ideas. I found the title sake of this posting very interesting. But it sounds obvious after hearing it. Of course democracy, that which is based on input by its members, is founded on the believe that it’s members are of prime importance and most intelligent and deserving beings of the world.

Actually, I do wonder if this is all said in one breadth the same way throughout history.

Did man, in throes of desire for freedom and respect and political power, out of self interest, not settled on democratic rule as matter of experimentation and thereby selecting a government that worked, finally?

Some longs time later, did founding fathers of America all shared genuine concern for fellow man and then create the American government and those for which it stand?

Truthfully, today, would an argument of humanism sufficiently motivate us to create the same government? I feel that it is more likely to win the argument that it works (or at least doesn’t not work badly enough) than for people to respect humanity enough for that. It might be just me, but I do not see or feel the humanity in most people I see. At least not enough humanity that I would be willing to create a democracy out of trust and respect for their humanity and intelligence.

In realty, most of our organizations are not democracies. I think if you search your thoughts and count the number of people you trust your rights and valuables to, you will find yourself in lackluster company.

How much do we really believe in democracy? How humanistic is it? How do we understand it? 

I wish someone knows the answer… and may that entity be a human person.

New Contest II

Last time we considered my need for constructive contests. Recall that we have  now distinguished contest of an hypothetical nature and those contest with real world benefits beyond the purpose of the game. It was also a concern that some games are not fun.

Recently I mentioned to a coworker that brute-force solutions to games are so 20th century. Deep blue already proved it was possible. A large neural network is so 2010's, and alphaGo didn't just beat one world champion, it beat a lot of champions online as well. 

What about this? Let's have a competition that is played by human and computer together and that their enegy consumption is a divisor in the calculation of final score.

This is along the same line of thought from a post long ago before New Contest considerations: automated race car driving competitions where in addition to automotive restrictions that there will be CPU restrictions. Core limited to less than 2^16 cores, each running at no more than 4GHz, total power consumption is to be less than 1kw, etc.

So in a similar idea, one can imagine there be additional restriction on storage. The code the ANN model, the memory about current course and opponents, these all take memory space. So the contest could have an additional rule that there be a 256Gb limit to onboard memory, of any kind, hdd, ssd, cache, ram, GPU ram, etc. 

Suppose the contestants scored S in a scale where each score is real number originally associated with each player. and the highest score wins, the score used to determine the competition result can be S/C where represent the costs paid by competitors to win. For one example this could be the amount of code executed as measured by storage space or as measured by records kept on the CPU itself.

Therefore, in the post-AI years, we will have new contests: the winners must not only accomplish the objectives, they must also complete them efficiently.

TAS for Chomsky

Continuing the thoughts around expressing Chomsky’s ethical opinion regarding US military actions as expressed by the documentary that I watched during 2016-2017 break.

It became obvious that while propositional logic and set theory are comforting languages, they are not the most convenient when operationalizing an ethic. For example, there seem to be a need to distinguish these prescriptive targets: must do, must not do, may do, may not do.

\detokenize{must_do_actions}\subseteq \detokenize{may_do_actions}

\detokenize{must_not_do} = \detokenize{may_not_do}

It seems “may not do” is “not may do” due to common English use even though technically both expression should be subset relation. I.e. “You may not smoke” states “you must not smoke” instead of “you can choose not to smoke.” Although that imprecision is inconsequential to the current discussions.

So in fact the expression can be simplified by including do_not_do_ actions for each action in simple TAS. We must also impose a contemporaneous interpretation to a \in TAS as to mean a is an action viewed at some references moment, ostensibly now. 

Then, the predicate must\_do(a, p1, p2), reads “p1 must do a to p2” is expressed as ethical(a, p1, p2) \land \neg ethical(do\_not\_do\_a, p1, p2). may\_do is simply ethical.

may\_not\_do is \neg ethical
The need to restrict preposition to single moments in time is in retrospect necessary. All preposition can be sub-indexed with reference time: prescriptive proposition ethical_t means it is ethical at time t, while descriptive preposition do_t means something is done at time t. do_w(A,ally, axis) \implies ethical_u(A, axis, ally) \forall{w,u} \in WWII

That’s a mouthful. But at least we can avoid the pitfalls such as the “may not” fiasco we have in English.
(Disclaimer, I watched a thirdprty documentary about Chomsky. Some videoed statements were stitched together and I watched that. doesn’t mean I am writing to explain what he actually said or meant.)

The sweet spot between CSS and OSS

I’ve been hacking on a commercial database at work recently. Spent a good week of time querying the database for a good sample of a decade of the company’s historical data. The thing that really kills me is this 15 minute query sniper that has been around since before I joined the company–it exist on all data platforms we have ever had: MySQL, postgresql, hive, vertica, spark, Hadoop.

But in an ironic way, I am actually really glad that all this crap works at all! How recent was it that MySQL would just get stuck or run out of memory or some other hidden unknown problem? How recent was it when hive crashed? How recent was it when you had to query hbase for data? But all that is under control because the company managed to pay for a product that actually works. system-v (anonymize to protect the company) can actually handle the work load that we have doing funky 5-level deep subqueries, multiple mixed inner and outer joins, filters group-by’s, aggregations, string operations, and it never peeped a single complaint! Just runs until query is killed. Production etl was not impacted. Other analysts didn’t complain. All I got was when the query got big, it was killed. 

Of course this two dozen person team, director level senior management and all, and all those servers and licensing fee, and all those training classes, and maybe a few quarters of ramp up is more expensive than the two weeks I spent bringing up a spark cluster on an HP laptop and Dell server. My cluster, btw handled the same sized query fine on spark. I only had to upgrade the disks slightly from factory default.

What this illustrates is that closed source software is catching up with Open source software! This is the sweet spot where closed source is at parity in performance and feature when compared with OSS lacking only cheap install and maintenance. Everything works every where. All you choose is your price and reliability. This is where software should stay for ever! Any geek can code up any new algo in matter of seconds. Test it, launch it in the next release. Beat the CSS by a quarter or two to the market. Companies that chose not to use OSS has to wait the few quarters, but that is a choice that they now have! CSS actually must keep up with OSS to stay afloat. OSS is no longer the only choice for real features, performance and non-stupid implementations.

Competition is so awesome for consumers!

“Everybody Lies”

I’ll be posting about this book by Seth Stephens-Davidowitz in a year or two. It’s an interesting book for its time, May of 2017. It is brutally honest, without reservation for political correctness, but hard to distinguish stated facts and opinions, hard to distinguish statistically qualifiable statements and idle observations(and author tries to mark the lack of rigor, imho, by saying he doesn’t really use properties of the vodka test (Kolmogorov–Smirnov test) or otherwise his meaning regarding repeated mention of the vodka test is completely over my head! what? is it a useless test or what? That’s really old news that statistical tests are sometimes useless in the face of big data due to simple algorithms working really well. I also look forward to Gelman’s begrudged review and discussions about statistical aspects of his work) 

The book probably does the most to make me want to see his data and analysis. To check if his conclusions that (white) Americans are very consciously and obnoxiously racist(oh thank goodness, I thought I was so unfit for this modern society with my own frequent racist thoughts)…that interest in homosexuality is geographically uniformly distributed and limited to 5% minority population, that… Indian men often want to drink wives’ milk, …that all these hulking men and bobbing chicks in America are humping at rate less than half a dozen each fortnight, …that…….

Well let me finish reading the book and learn a bit more and post back in a few years.
And it would be really nice if this ability to look into humanity is preserved some how.

Reimagining Bad AI–what will humanity lose?

AlphaGo just beat the Chinese at 围棋。All the games, single person, 5 people, never came close. The Chinese team were gracious and kept up good spirit of collaboration and learning. Although, seriously, the Champ seems to speak his mind and probably that of many other people: It really sucks to lose, it doesn’t feel good, and who wants to play against an AI ?

There is some joy, I suppose for some people, to watch these pompous Chinese play their most treasured games and get beat badly. For some people, it is a disgrace that they just have to swallow whole. It is a game where you have the worlds best bang his head against an AI for 4 hours and just lose. Like these passionate people who are at top of their profession, giving it a real go, just lost to the computer. And then the computer pulled a Chinese move and quite professional competition at top of its game! Wow!

So, I guess AI is really closer than we thought. Seeing top pro of one competition decimated by AI immediately brings to mind all the other people who are also passionate and top of their game in a similar situation at their own games. Thinking of drivers, programmers, chefs, teachers, scientists, artists, … leaders. Will they all compete and lose to the AI in such heartbreaking defeat?

I used to make light of the situations and say that that AI has no chance against Hitler’s of the world. It cannot possess enough evil to out evil human kind. But after the last AlphaGo win, I do not even believe that. Think of the next allied forces banging their heads against an AI Hitler, then losing humanity to it.

Evil perhaps is not the only thing that begets evil–perhaps knowledge begets evil too?

Just think of your passionate and top-of-their-game financial advisor fighting against a machine. What about your political leader fighting against a machine? What about artists? The reality is setting in. What incentive will human have to do any of these things well? Why would a human improve himself in any thing if he can make a computer better at it with less human effort and sacrifice? 

Sure there is the counter argument that people run much faster marathons than the Olympians and that survived horses, boats, planes, segways and automobiles. But in reality, running survives because it is a nonessential and mot-really-competitive sport. Like, the scaling of the whole jogging-marathon thing, it doesn’t really scale. Not running has no impact on most people, and running isn’t what makes most people happy. And running usually produces more happiness standalone than winning at competition.

It seems, in the impending age of AI, we can still try to make some predictions about what will be eliminated and what will stay. If a human is making a rational decision then things that will stay are those that we need directly for physiological, psychological and social reasons.