Continuous Delivery Made Easy

After graduating college I wound up with a job at a large bank. There I had a great history lesson on how software used to be developed. Huge requirements documents. Six month development windows. Saturdays spent trying to meet arbitrary requirements set forth my a committee. It was painful.

After the first week I sat down with a senior developer who was supposed to show me how to access the mainframe (this was 2011). He looked at me and said, “What are you even doing here? You should get out while you can.” I laughed and didn’t think anything of it for three years. For those three years I suffered through some pretty rough times including some major health issues but just thought of what I did as just a four letter word: W-O-R-K. It wasn’t fun but it was money.

Suggested Books

As I work with more and more junior developers I often find myself suggesting different books that have helped mold me into the developer I am today. For better or worse these books have changed my perspective on the industry and keep me coming back for more. I love reading technology books and so it can often feel overwhelming when I try to compile a list. Nor do I like the “Best 10 Technology Books Everyone Should Read” posts because I find them cliche and not that helpful because it lumps books together in one pile rather than by category or interest. I’m hoping this post is a living one in which I will be adding and updating content as time goes on since I’m revisiting and reading new books each day.

Work and Team Breakdown

The breakdown of work and the relation it has within development teams has interested me for a long time. There is no question as to where the thoughts began but they have led me down a larger path of discovery. The journey began by reading The Phoenix Project by Gene Kim, in it he talks about the four types of work, how work should be scheduled and broken down into easily repeatable tasks much like a manufacturing plant would. This made sense to me, what is all work but a sense of easily broken down tasks and steps? The smaller the tasks the easier it is to complete and increase throughput.

See One, Do One, Teach One

In the medical industry there is a common practice among surgical students learning a new procedure which is “see one, do one, teach one”. Seeing one allows the students to study from another experienced surgeon, ask questions, and take notes completely focused on learning from a teacher. Doing one enforces the application of the learned material allowing for physical practice with guidance from a teacher on the best practices. Finally in teaching one you learn to explain what you did to another and relearn and reinforce the steps of a procedure.

Working Together While Working Separately

In my current job I’ve been helping lead a small team in the development of a new product. This is a coordinated effort across the company and been a challenge to some as we quickly switch between discovering our customer’s needs with developing them. We have a road map and a plan but translating this into actual work has been difficult. All of the reading I’ve done over the years has helped me develop plans and strategies in order to help organize a group of people around a given task in order to deliver. As with anything application ends up being more difficult than theory.

Joy of Creating

I enjoy my work. I find it fulfilling and the problem solving to be engaging. Yet at the end of the day it’s work. The drudgery of completing tasks, the politics, the deadlines, can take away joy. Software development isn’t unique in this area, almost all jobs have this fine balance between fulfillment and drudgery which is why people have hobbies outside of work. Softball teams and woodworking, painting and poetry, all contribute to balancing our lives and bringing us joy through leisure.

Why I Program

We believe that programming can be, and should be, an intellectually rewarding activity; that a good programming language is a power conceptual tool… - The Art of Prolog

Why do I program? I like the intellectual simulation of solving a problem or designing something. So much of the time as a developer you end up building the same thing over and over. API endpoint for this or a message processor on a queue for that. Maybe this is just the pattern I’ve fallen in over the years. There is room for improvement or building better services but that isn’t enough sometimes. Where I find joy is in solving the problems or learning something new.

Mighty Oak

There is something strange about the way we view ourselves within in jobs. On television there is always a leader who emerges from a difficult scenario or a person who saves a company single handed by making the right deal. But this isn’t how it works, or at least it shouldn’t. Throughout my career I’ve learned that anything of significance shouldn’t be done on your own. In order to build something significant you need to work as a team. Now, you may be the first person on a project or a person who comes up with the idea, but without a community to support you failure will be inevitable.

Technology Maven

In my career I have found myself collecting things. I like to collect programming books, links about new databases and patterns, talks about functional programming. This collection has become somewhat obsessive in some cases and led to the disorganization I’ve described in other posts. When I was starting out I was told that this was not good, that you could run around your entire career chasing “shiny things”. This is true if you are chasing the newest JavaScript library or a cutting edge language, but I felt that maybe this didn’t apply to me. I collected books about LISP and functional programming which weren’t new. I watched talks about Java patterns and concurrency models. Was I chasing shiny things? Was this a waste of time?

Getting Started

Last week I wrote about issues I was having getting things done and worrying about stagnating in my career. I took some time over the past week to disconnect, relax, and reflect about both this problem with “closing” and my career. Throughout my life I’ve been constantly rushing forward and in my career worrying about what will come next. This rushing and worrying always leads me into a mild depression that becomes difficult to overcome. Eventually I realize where I am and find out where I need to go. Sometimes it’s a job change, other times it’s a mentality shift, and in one case it took me flying half way around the world for work to start fresh. What I’ve learned is to do the best where you are at and things will move forward. Following the course of the river instead of paddling against it. What is more important is maintaining a feeling of exploration and hunger. In Becoming Steve Jobs the authors write:

Where I Am

I’m all over the place. I can see that by my notebooks. Each month I sit down and write down a list of goals I want to accomplish. This ranges from books to read to projects to complete to posts to write. All in all I typically manage to complete about 60-70% of what I set out to do. But lately I’ve not even come close. This past month my goal was to clean things up and organize and I accomplished nothing.

Rushing

I’ve always rushed things. In elementary school I rushed through activities to move onto the next thing. In middle school I rushed through classes to take High School classes early. I did the same thing in High School, taking college courses early on to keep moving forward. I couldn’t wait to get through college and “into the real world”. Only to find myself there thinking about changing jobs or going back to school. It would be easy to make the excuse that my brain just is always rushing fast, but I feel like that is an easy excuse. I also don’t think I’m alone in this and hope to find a way to solve this personally.

Simple Site

Over the past couple of months I have worked on trying to increase my flow of writing. So far I have failed at that. I don’t really have an excuse for it but upon reflection I thought that I could maybe make things a little easier on myself by simplifying my website.

I drew inspiration from a site I found called Tiny Projects and another post about building a digital garden. The problem with my sites have always been they are cumbersome and mildly complicated. This stems from the origin of having a “digital resume” or portfolio to show to others for job searches. There I tried to flex my developer muscles by writing complicated HTML, to building with Gulp.js and Pug, then moving to a Javascript framework, to using Hugo. Each step had it’s complications and were not maintainable.

Project Of My Own

Michael Pollan is one of my favorite authors. Each of his works follow a similar pattern. Define an overarching theme (gardening, cooking, eating), then define some components, go into the history and practice of each sub component. For example in his work A Place of My Own the overarching theme is architecture and some sub-components are location, design, material, etc. In the section on location he talks about Feng-Shui and energy flow of the earth to find the right spot for his house going into the history of Feng-Shui and providing examples of buildings like Falling Water as ideal locations. In the end its a great example of personal project exploration.

Language Ergonomics

There are a ton of programming languages. Each language then holds a specific set of values that lend itself to solving certain problems. Too much time is spent arguing over which language is best and for what reason instead of looking at which languages match can help you solve a problem or match your values.

This last point is why I have spent a lot time exploring new languages.

Early in my career my manager had told me to focus on a depth and breadth strategy for developing in my career; go deep on one technology but explore others to see what is out there. And so I set off to become a better Java developer while still exploring languages and technologies that I came across. Soon I learned about containers, learned how they worked, how they were made and what languages worked best to create lightweight containers. This is where I started developing in Go. Over time I grew to love developing in Go and moved on from Java.

Engineer Mathematician

When I was in college I intented to major in Mathematics. Computer science was supposed to be my minor and support my interests in math. In my first semestor I took a formal math class and was completely lost. Meeting with my professor I discovered that my interests were in his terms “messy math” and not “pure math”. Dissalusioned with this notion of being interested in “messy math” I decided to shift my focus to Computer Science which I felt would allow me to reason through problems in the “messy way” I liked.

On Writing

Throughout my career I’ve always had side projects to help me learn, to practice, to sharpen my skills. Sometimes they were relate to things I had read or ideas that had crossed my mind. Whatever the reason for working on the project it typically ended up being benefitial for my personal growth. Unknowingly I was establishing a practice of a kata, doing an activity in order to refine and hone your skills.

Brave New World

Having just finished Aldous Huxley’s Brave New World, I was struck at the predominance of theme of desensitization. In a culture today where people have become desensitized to the world around them, I worry about its impact on our society. I’m not saying we’ve entered into a dystopian world but these types of books are meant to be hyperbole in order to draw attention to larger concerns expressed by the other or conclusions drawn from the reader.

Be Good to Yourself

Anything that gives you joy or happiness that is entirely for yourself. I have always struggled with self care because it always felt selfish… which is what it is supposed to be. Taking something for yourself can feel awkward at first while you look at all the things that need to be done or the expectations others may have for you. I have always struggled with the need to be the perfect father, husband, son, and employee. With this expectation of myself I left little room for self care and became frustrated, depressed and disengaged in all aspects of my life. All from a lack of self care.

Lambda Terraform

In a previous post I wrote about how to deploy an application in a container. The gist of the post talked about how containers provide a layer of abstraction so that your applications can run on any machine. This is great for local development and large scale applications. However, in order for them to run in a production environment they more than likely are required to run on some sort of runtime or server which can provide some additional cost overhead.

Growing As a Developer

Becoming a better developer does not always mean being an expert in a programming language. Nor does it mean being the smartest person in the room. More often than not you will find that you aren’t an expert and you aren’t the smartest person. Work is more than just programming, it’s understanding and developing products that people will use and to do that it requires more than just a rock star programmer, it requires a person who is has the abilities to help their company succeed.

New Year

I doubt anyone subscribes to my blog or reads it regularly but do those who tune in may have noticed that I haven’t been updating as much recently. For a while I was on a good pattern of posting once a week or at least every other week. These posts where planned and structured and part of an on going process of learning and self discovery. But that changed. Where I was once learning and writing about technology and felt inspired each step of the way, I soon found that both the projects and writing became strained and obligatory instead of inspired.

Cloud Team Thoughts

When I started developing Microservices I felt like I had been missing out on a huge part of the modern technology world. It turned out that I was right. Over the past few years I’ve been trying in my mind to figure out how modern development teams run effectively. What I discovered was that there are tons of people out there trying to figure out the same thing. Many of the things I’ll write about are patterns that have been around for more than a decade, yet still many companies don’t seem to adopt them. In the end building a team in the “cloud era” has changed the landscape and we can find that each change we see in infrastructure, development practices, project management, and testing perpetuates further advancement and change.

Go Docker

So you’ve built an app that runs on your machine. Great. Now what?

Obviously software is built to be used and today most of that software is being run on servers in the cloud. But sometimes it’s not. Sometimes it might be running on some old hardware in the basement of a company or maybe another developer needs to use your code as part of integration.

Whatever the case may be it would be nice to package your application in such a way that it’s guaranteed to work no matter what environment it is in.

Agile Career

There is a very important product that you have complete control over: you. Your skill set, your personality, your overall career is completely within your control. Those are all features of your product and it’s on those pieces that you need to iterate and improve.

Good products are successful often because they have good product managers. It is up to you to decide how you fit into the world and what people are demanding of you. Product managers help take “market interest” and turn it into deliverable features that enhance the product. Sometimes these features aren’t in demand but will aid in the overall quality of the product.

Go Pipeline

One of the most powerful things about Go is the ability to run parts of your application concurrently. This allows the application to run in parallel at times (depending on the number of cores) or continue to run while waiting for a blocked process. There are a number of concurrency patterns that allow you to structure your applications to get a performance boost.

Recently I stumbled across the “Pipeline” pattern and was able to use it to transform incoming data and aggregate the results. Imagine a water treatment plant. Dirty water comes in, clean water comes out. The water goes through several phases before it is clean. First you may filter out large chunks of sediment, then smaller chunks, until you are screening microscopic pieces of debris. Then you want to put it through a series of chemical cleanings and finally test the water quality. Water flows through these various phases. At no point is water stopped until the first step is fully complete. Instead the water is processed as it flow ultimately becoming clean.

Zero Dollar Idea

People enter into technology thinking they can pursue an idea that will make them a lot of money. If you read startup stories you will find that most start with an idea, evolve into a business, become broke, rebound and make millions…. or end up being broke and learning a lot.

I’ve read these stories but I am not a risk taker. Maybe I just don’t believe in my ideas as much as others or lack the confidence to pursue them. However, I like to see things come to life. Will it work? I don’t know. I know how to build it just not how to sell it.

Go Mock Testing

You should always write tests. To fully understand how to write production level applications in any language you should be able to write unit tests. Some people go to the extreme of doing Test Driven Development (TDD). TDD advocates that you should write tests before you write the code. This can be a good approach when trying to write code that can be easily tested. Yet I find that TDD is a lot like agile; a lot of people say they do it but in reality they do some sort of hybrid version. In the end it boils down to eating your veggies before your steak. Some people love veggies, just like people love writing unit tests, but most people find satisfaction in running their code and seeing it work.

Go Hex Arch

In an earlier post I wrote about how to structure Go programs to be leveraged by both a FaaS and a standalone application. This simple application showed the power of what a Hex architecture can do by limiting the need to rewrite logic when an external resource has changed. In that example we looked at changing communication between an HTTP call from an AWS Lambda to a direct application call.

Brain Expansion

I have a terrible memory. It’s not because I don’t pay attention; there is just too much on my mind. The barrage of incoming texts, work emails, news alerts, and ideas creates chaos. Disconnecting is not an option all of the time, especially for those in the technology field, so what can you do?

It used to be when a computer was running slow or crashing the best solution was to add more RAM, expand its memory. You can’t just open someone’s head and add more memory. Nor do I believe that “brain exercises” can help. Instead, what I have found is the only way to handle the title wave of distraction is to create a map, establish boundaries, and think out loud.

Serverless to Server

Functions as a service (FaaS) have really become a staple in most development circles. They provide little infrastructure overhead in the development of microservices and are often very lightweight. Yet FaaS face their own sets of challenges when it comes to costs, response times, and observability.

I believe that architectures will evolve over time. In some cases you may never want to change your FaaS but there may be cases that you’ll want to move it to it’s own application. The problem is you may want to go back or you may not want to rewrite all of the business logic injected into that FaaS. So what do you do?

Catching Up

I felt that I had a pretty good grasp of what the technology world was like when I graduated from college. After all I just built a series of programs for my classes (of varying quality), none of which had real world applications. I had built systems using live APIs like Twitter and used cool frameworks like Google Web Toolkit. So in the end I felt pretty confident in my abilities even though I had no exposure to the industry.

Understanding Serverless

To be honest, I’ve had a rough time understanding how FaaS fit into my vision of a microservice architecture. It made sense that they were microservices themselves, especially when they are event driven: put something in a bucket, then do something, maybe do something else and so on. But what I didn’t understand was why build a serverless function if you could build an API.

In the end I realize that it was how I had learned to develop microservices that was the crux of my issue. I have always been a very detail oriented person. I like to write the code, package it, provision my server, deploy my code to the server, add the routing, etc. All of this obviously takes time, overhead, and an understanding of the underlying systems.

Accelerating Development

The main goal of every startup is to make money. Every part of the organization should be driven to create money for the company. This means creating products that users want to buy while decreasing the cost to produce the product all in a timely manner.

In a technology startup the product to deliver in many cases is code. The beginning phases of this process involve a lot of late nights, rapid development, and manual processes. As the company grows and the product becomes more complex these practices becomes untenable and error prone. Consequently the speed of development slows and the ability to respond to the market diminishes.

Consumer Driven Contracts

In the past I have written about what a microservice architecture looks like and some of the tradeoffs that come with it. One of the biggest gains this architecture brings you is the ability for many people to be working on different projects at the same time. Inevitably the services you are writing will be used or you may need to consume someone else’s API to get your work done.

This is easy right?

Clean Repositories

When reading Clean Architecture by Robert Martin, I came across a chapter of the same name. In it the author describes this “idealized” architecture structure where business and application logic are at the center of the diagram with outward rings surrounding it describing interactions with various frameworks and IO devices.

This architecture can also be known as Hex Architecture or Ports and Adapters Architecture, all having a similar design where core business logic is independent of the IO interfaces surrounding it allowing for architectures that are independent of frameworks, ui, and databases.

Narrative Code

Intro

In starting this series, I was confronted with the dilemma of which area of programming should I start with? Testing? Data Modeling? Language Types? All of which are very important topics and areas that any good developer should know. However, I felt strongly that the best area to start was not with code at all but writing.

As a developer we tend to write our code for one person, ourselves. Yet most of the time our code can live longer than the tenure at a job. Or even worse, you are forced to revisit code you wrote years ago and have no clue what it does.

Programming Is a Craft

This is a response to the following article Programming as craft

Programming is a craft. I will never claim to be a master of the craft but to say that programming cannot be considered a craft overlooks many elements of the profession. Sure, groups of developers aren’t building artisan furniture or designing magnificent cathedrals but none of these crafts were perfected overnight and relatively speaking programming is in its infancy.

The notion that programming is not a craft stuck me in an odd sense and I have been wrestling internally with its contents for a few weeks now. This response is not a critique of the authors points but rather a different perspective reflective of a personal journey that I’ve had in my career as a developer. I hope to address a few of his criticisms and point out some possible overlooked areas of the profession.

Dumb Phone Eulogy

I am one of the few people who’ve probably traded in their smart phone for a dumb phone. By dumb phone I mean a phone that pretty much just makes calls and texts. For over eight years now I’ve scorned the smart phone market because of the increasing negative effects I feel it has on people. The concept of being constantly connected always presents itself as a great thing but often leads to isolated interactions with people because they are busy checking their Facebook or Twitter feeds or playing some new game.

Sagas

My last couple posts have been about developing and designing cloud based applications. In the What Are Microservices blog post I wrote about the importance of having small shippable applications that can easily scale depending on demand and are decoupled from other services to allow for faster development and independent deployments. In A Tangled Mess we explored how to decouple the systems even more by publishing events as they occur and having other systems react in appropriate ways.

A Tangled Mess

The goal of microservices is to produce independent systems that have loose coupling. This allows for the ability to manage and release upgrades without impacting other applications. However, many first implementations of microservices make the developers believe that the only way to communicate with another API is through a client. Thinking that obviously this is part of the whole process, they design many APIs that rely on other APIs and the chain goes on and on, and in some cases back to the original API for more data. As you can imagine this becomes a tangled mess.

What Are Microservices

At the beginning we all start building an application. Then some part of it takes off and before you know it you are throwing new pieces on the original application until it becomes unweildy. Thus is born the Monolith. In time your system gets large, cumbersome, and hard to maintain. Then you hear about microservices (or maybe you started that way because it was in fashion when you started development) and start building them. But in developing microservices we can often forget about what they are and why we are doing it. It’s not because it’s the cool thing to do nor is it the easiest thing to do (quite the opposite) but it’s a smart thing to do.

Another Tech Blog

Who needs another technology blog? Aren’t there enough out there? It seems like it’s a right of passage for someone to create one but to who’s benefit? This has been the third or fourth (maybe fifth) attempt for me to start some sort of blog each with their vision.

Yet in the end they sit in a corner of the internet wasting away helping no one. All blogs feel that they need to solve the world’s problems, get one million hits, and then the authors can quit their day jobs.