Stoplights can really disrupt the flow of traffic, causing long queues at intersections that seem to favor the wrong movements. These traffic patterns vary over the course of the day. When roadways have a series of intersections, it’s ideal to have the heaviest traffic load hit green lights rather than stopping. This alleviates congestion and saves energy.

My team worked with a university that designed an algorithm to optimize the performance of traffic intersections dynamically. We ingested telemetry from the computers at intersections, fed it to our backend, processed the data into optimal traffic plans, and then pushed the plans back out to the intersections. Our tests demonstrated measurable reduction in congestion.

It’s a familiar situation. You need to add a new software service. You spin up a new git repository, add boilerplate you’ve added in a dozen other repositories, then get coding. Once coding, you want a piece of code from that other project, so you have to consider how that previous code can be used in multiple places. Maybe you start building shared libraries, and maybe they are, once again, living in their own git repositories. All the boilerplate repeats, even the CI/CD pipelines.

There’s a better way. I led an effort to move to a monorepo - a git repository that holds the source for multiple services and shared libraries.

The benefits include reusable libraries, a common CI/CD pipeline, tooling consistency, and reduced boilerplate. There are drawbacks, such as more complex and slower builds. Yet the sharing of code by itself makes the whole endeavor a great success!

With the rollout of Docker not that long ago, a revolution started in software. Since that moment, how we deploy software has forever changed.

While working on my final project at CDK Global, I was part of the team that pioneered the use of Docker for production systems. I learned the software and started teaching others, holding lunch and learns and sharing source code.

Kubernetes emerged as the dominant platform for container orchestration. My current team deploys backend and remote services to Kubernetes, and I’ve become comfortable using and building Helm charts. The Helm CLI is the last run command in the build pipeline.

Containerization is also useful on the desktop. It allows desktop testing of server-like environments without installing local software. Unit tests can run in the same OS as the production environment. It’s possible to provision resources for local testing such as databases, message queues, and caches. Fluency with Docker or Podman with some BASH coding allows quick creation of integrated environments on the laptop!

I worked for many years on a product that allowed customers to account for and charge their clients for printing. The cost parameters were paper quality, sheet size, and sheet count. However, the most expensive component of printing is ink and toner.

I headed a project to capture a digital copy of every print job, render it in memory, and perform ink requirement calculations on the image. Cost recovery could now relate to colorant consumption rather than just counting sheets of paper. The feature was successful in differentiating our print cost recovery solution from the competition.

Extract, transform, and load operations periodically move data from one system into another, changing the shape mid-flight to fit the target system.

I worked on one of my favorite ETL projects at CDK Global, where sales leads started in internal systems. Sales teams wanted to use Salesforce.com to manage leads. The system I developed used the Salesforce API to send leads into that system. When the team made edits in Salesforce, the information was fed back into internal systems.

The takeaway was that the legacy system still provided much value in terms of analytics, and the sales team had their preferred tool to manage customers and leads. Integrating two systems was a win/win!

I’ve had the opportunity to work with many forms of databases, from traditional stores like Postgres, to NoSQL stores like Cassandra, to data lakes. I’ve leveraged Kafka to implement command query responsibility segregation (CQRS) architecture. I’ve written stored procedures, SQL functions, and scripts to provision databases.

Having worked with many data storage systems, perhaps the biggest challenge of all was performing calculations on cyclical time series data. SQL aggregate and window functions allow grouping data by field values, but it is not straightforward to aggregate cyclically along a timeline.

In the use case I faced, I needed to aggregate records where a traffic signal field showed green, yellow, red, then started over. Given a small budget and no ability to use time series add-ons, I wrote custom aggregate functions that would locate transition boundaries in such data. A solution still required a long common table expression (CTE) query, but it was now possible to aggregate on cyclical attributes.

Conway’s law states, “Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” In other words, your product can resemble that efficient standup in which everyone appeared on message and working toward a shared goal. Or, your product can resemble the rambling zoom call where most of the team is off camera and fumbling with phones.

The difference between the two teams is leadership driving engagement. Good leadership listens. It inspires. It creates a safe space for ideation. It organizes. It checks back with the team rather than issuing fire and forget, vague commandments to go do the work.

In software, leaders must be technically adept. They should choose productive time over meeting time when possible. They use tooling to improve transparency and team alignment. Most of all, they put people first.