Software Engineer Checklist

When I started my career as a software engineer, I didn’t have any map or list of skills that are necessary to succeed. I’ve spent too much time on unimportant stuff, a far too little on crucial things. If you know at least something about software development - you can create web service in Django, or build microservice in Spring, crate some machine learning model, or write Spark Job - and you want to advance your career to the next level, this blog post might be helpful for you.

Before we start, I have good and bad news.
The good news is that at some point you will have to unlearn what you have learned. If you learn something, you are becoming blind to other things that you can learn. Remember, you are a creative person; your mind should be free and open for new waters. Do not be afraid to forget stuff.

The bad news is that you will probably never stop learning.
Over time, some things became similar, and we use similar design patterns in different contexts. Even that something is similar, it is not an excuse to stop learning. You need thousands of hours of practice and writing software to be good at it. Besides spending many years writing software, I know that I have deficiencies in some areas that I want to improve.

Disclaimer

I’ve tried to build this competence area map to be as universal as possible. Nevertheless, I realize that backend software engineers found it applicable, whether other specializations may found it less useful.

If any of those concepts are new for you, do not try to learn everything in one week. Learning about software engineering is like going down the rabbit hole. You think that you get it, but then you realize that there is another level of abstraction or other stuff to discover. If you are a person who likes following the rabbit holes, this guide may be for you.

Original idea

It all started with writing down the desired competency list for myself a few months ago. In the same way, many of us do a medical examination, and I wanted to do my professional skillset examination. I did it in Google Sheets. The total number of rows that I’ve written was 50 - each describing skill, knowledge area, or competence that I want to build or improve in myself.

Hard technical skills

Hard technical skills are a necessity when you are a software developer. While I was writing down technicals skills, I divided them into three groups:

  • General. Those are universal skills that benefit you for many years. Understanding them is an important thing.
  • Support. Although those skills are not necessary to do your job rightly, by having those skills, you gain a deeper understanding and be able to propose more suited solutions.
  • Trendy. The IT profession is susceptible to trends. Today we use that technology, but tomorrow we may use different.

General technical

General technical skills are skills that are not going to be outdated promptly. On the other hand, practicing those skills consumes much time, and in my opinion, you need years to practice them, rather than months.

  1. Algorithms. I often get alarmed whenever I hear that someone is stating that “Learning algorithms is a waste of time.” Studying algorithms is the essential thing that you can do when you learn computer science. Algorithms are everywhere. Algorithms are often quite different from one another. By studying more and more algorithms, you may develop different solutions for the problems which arise during a regular job. I have to admit that I learn mostly from my experience. Studying algorithms is an eye-opener for new approaches on how to solve things. If I hadn’t studied algorithms, I would propose worse code, simply because I wasn’t familiar with some techniques.

  2. Data structures. Each data structure is designed to arrange data to suit a specific purpose. Sometimes we want to find data quickly or store quickly. Knowing fundamental data structures, like Hash Maps or Trees, is a necessity. Also, it is vital to be familiar with more advanced data structures like probabilistic data structures, immutable data structures, or when we talk about BigData - distributed data structures.

  3. Programming. Can you write code in your primary programming language without an IDE (Integrated Development Environment)? Can you do that using only a text editor and compiler? Without knowing language syntax, builtin libraries, data structures, fundamental data types, it is impossible to be productive. Although I use IDE on my daily basis, it provides me little to none help when it comes to solving simple and primitive code mistakes.

Programming languages

  1. Framework. Can you write code without using StackOverflow? My primary programming framework is Spring Framework. Things like exposing an API, fetching data from a database, calling other services using HTTP Client, writing unit tests, are natural for me because I am familiar with this framework. If you need to Google everything you need to do and copy-paste the solutions - especially simple tasks - you are probably not familiar with your framework.

Stack overflow

  1. Architecture. It is easy to understand, maintain, and develop a system with clean and transparent architecture. There is no silver bullet here. You have to know at least a few architectural styles to communicate your ideas, and to understand others. You have to understand the basic concepts. When someone is saying to you, “Event sourcing is ideal for this problem.” you need to be on the same page to discuss details of that architectural style.

  2. Database. In the initial steps of designing a new system, a database might be a nonrelevant detail to your application architecture. Sooner or later, the database is an element that cannot be ignored. Embrace the polyglot database style. Analyzing database queries, managing indexes, configuring partitioning, and replication is a piece of must-have knowledge. There are far too many different databases on the market to know each well, but you have to know one database at least well and know when to use which database.

  3. Design patterns. If your primary language is object-oriented, you have to know basic design patters (reading a book about design patterns may help). Besides that, there are also more rules, like SOLID, KISS, DRY. There is also DDD and CQRS, which are more like architecture styles. Use those patterns, but do not overuse them.

  4. Coding styles. There are different approaches on how to write code. There is TDD (Test Driven Development), Test First Approach, Pair programming, writing PoC (proof of concept), or writing scripts for one-time use. Each of these styles aims to achieve different things. Try different things and know the advantages and disadvantages of different approaches.

  5. Concurrency, no wonder, is the scariest topic for many developers. It’s hard to test, debug, and reason. Not only you have to understand the behavior of concurrent programming in your language, but you also need to understand your platform well. For example, take JVM. At first glance, you download it, and it works. For me, this is a typical example of a rabbit hole. You have to investigate the Java Memory Model, Garbage Collection, Just-in-time compilers, bytecode. You can not study it in just one day. It may take you weeks to investigate all this stuff.

It is a hard thing to do

  1. Tests. There are many different tests that you can write for your code: Unit tests, Integration tests, End to End tests, Chaos testing, mutation testing. It is easy to write test code, but the tricky part is maintaining the test codebase in vital conditions. Test code should be treated with the same importance as your production code. Remember that if you have robust and reliable tests, you can rewrite your production code without any bother.

Testing

  1. CI and CD. I don’t pay much attention, whether it is Jenkins, Bamboo, or GitLab CI. Pick one, and know it well. Understanding why we do Continues Integration or Continues Development is an important aspect. The tool does not matter. The right configuration to achieve the right goals is crucial.

  2. Security. This a vast topic. Essential is asymmetric cryptography (public/private key), authentication methods, access delegation methods - OAuth. On the other hand, you have to know different vulnerabilities (like SQL Injection or Cross-Site Scripting). Then it is essential to realize what types of malware there are and what are the root causes of exploits. And last, you have to know your technology stack, to design and build safe applications.

  3. Network communication. The first thing to know is “8 fallacies of Distributed Systems.” It helps you to understand the behavior of a network application and characteristic of network communication. Then, digging deep into the network stack helps you understand network communication via modern protocols and debug problems, which sooner or later happen.

  4. Rapidity. Practice, practice, practice until you achieving fluency in general programming. After a few years in software development, you should be able to swiftly develop a new application, fix a bug, or re-engineer a complete system. Anyone can probably write any system, given no time limit.

Supporting

Supporting skills are skills that are not your core competence. Instead, knowing those things helps you see the complete spectrum of the Software Engineer Toolbox. Although you do not have to be an expert in each aspect, you have to be familiar with each aspect, and be able to develop something in other areas.

  1. Distributed systems. Truth be told, I don’t think that I will ever write a system that runs on a single computer. Because we are not able to build faster CPUs and single machines in general, it is reasonable to host your services on commodity class machines or in a cloud in a shared environment. We all need to understand how to design, develop, and maintain a distributed system.

  2. Statistics and math. I noticed that I often use mathematical statistics concepts, which I learn in School. It is Percentiles, standard deviation, quartiles, mean, distribution. I use it all the time, for example, in analyzing service response times or working with almost any kind of data.

  3. BigData. It is good to have some experience with Hadoop, Spark, Job Scheduling mechanism (like Airflow), Stream processing. In the world where we treat data as a bar of gold, you have to know how to work it. A few years ago, it was hard to find someone doing big data, but now I think it is a necessity.

Dan Ariely

  1. Web . A full-stack developer is a most wanted developer. In my opinion, people claiming to be full-stack developers, most of the time, prefer frontend or backend. It is good to understand both worlds. If you prefer backend, you shouldn’t be afraid to change the frontend and vice versa, but hardly I see experts in both words.

  2. Machine Learning. If you are a Software Engineer, machine learning is an entirely different career path. Machine learning models are becoming more and accessible, so you will only benefit from knowing how machine learning models are built or how to train and work with an artificial neural network.

  3. Unix. Why do I recommend Linux or Mac for a Software Engineer? Because of Unix Philosophy. For example, text Processing Toolset. You may state that a Software Engineer should write software and should not work with raw files. Far too often, I see a use-case for tools like sed, awk, or even simple grep does the job. That and many other tasks are simple in Unix.

  4. DevOps toolbox. In your company, your infrastructure for services is probably ready and configured, and there is a dedicated person to run and manage that thing. Unfortunately, problems are unavoidable. Knowledge of how and where your services are running may be crucial when there is some failure.

  5. Site Reliability Engineer mindset. By this, I understand how to release, monitor, manage an emergency, and all things that you can read in a free by Google Engineers Every software developer who has an application on production should embrace those concepts.

  6. Computer architecture. Do you know a computer or server architecture? Do you know about CPU caches, RAM, or network bandwidth? Do you know the limitations of hard disks? It is essential to understand bare metal machines and their physical limitations. It is also good to know about computer construction. Nowadays, we tend to go to the shop and buy a notebook. It might be an unusual experience to build a PC from scratch by yourself.

Trendy

In this section, you have to answer what is trendy right now. Take a look and write your answers. Do not be sentimental about past technologies. It is not a confession.

Gotta catch ‘em all!

  1. Core programming language: ……..
  2. Main framework: ……..
  3. Monitoring: ……..
  4. CI/CD: ……..
  5. Build tool: ……..
  6. Asynchronous and non blocking communication: ……..
  7. Version Control: ……..

Some choices are clear to me and rather stable - Git as a version control system. Your primary programming language probably influences other choices. That is fine, as soon as you are aware that you are not working in obsolete and deprecated technologies.

My programming language of choice is Java (but I have a remarkably pleasant experience with Kotlin), with Gradle as a built tool and git as version control, with project reactor. I know that the project reactor is ugly. Keep in mind that the topic of asynchronous and non-blocking communication is continuously changing. At the begging it was CompletableFuture, then it was RxJava. Now it is a choice between Project Reactor and Kotlin Coroutines.

People

Technical skills are not everything. Software Engineers work in teams, so communication skills are as relevant as the ability to code.

  1. Communication. For me, excellent communication is not about speaking or giving a great presentation. Speaking may be a monologue. Dialogue is when you listen actively to others. Never assume that you know other person intentions or goals. Paraphrase and ask for clarification.

  2. Be authentic. Do not pretend someone you are not.
    Language: Business and technical. Communication is tricky. Business people tend to see profits and results, unlike technical people, who tend to see problems and potential bugs. You have to learn how to communicate your ideas to different groups of people.

  3. Respect. It is a magic glue that holds teams together. We are all equal, so we should treat everyone in the same way. Our workplaces should allow expressing ourselves freely. You have to do it professionally and cut down any insult or aggression.

  4. Willing to help. Giving useful information or advice to your teammate is a great way to build a relationship. Soon you realize others became willing to help you in the moments when you need it.

  5. Give and receive feedback. Providing high-quality, fact-based, structural feedback is hard. What is harder? Receiving feedback. No one likes to be criticized, and far too often we make it personal. When we make it that way, we became defensive. Thus we do not accept the feedback and remove any chance to improve ourselves.

Product

We - software developers - do not make software just for fun. We do not use design patterns because someone at the conference said so. Our product should be the most useful for our clients, with the right features; hence we built maintainable software that is easy to extend.

  1. Focus on product, not tasks . It is easy to forget about the product when working with some issue tracker. Task after task, swiping from TODO column to DONE. Stop for the moment and look for business value gain in those tasks. If there is little to none product improvement in your tasks, there is something wrong. On the other hand, if there are only business tasks and no time for maintenance (or technical debt), there is also something not right.

  2. Propose new features. Although stakeholders or business people are good at business, they are not experts in technology. I’ve noticed that the best ideas and solutions came from technical people - developers, designers, product owners. Having a good idea is one thing, but having the courage and charisma to propose your ideas is another thing. If you have good ideas, you also have to communicate them effectively.

  3. Public relations. There is going to be a time when your system will not be working correctly or will be down. It may be due to deployment of the new version of the system, hardware or network failure, wrong configuration. No matter what was the cause, you have to clean up the mess, and you have to it professionally. This process usually has three steps:

    a) Fix the problem. Make the system healthy again.
    b) Repair potential damage.
    c) Introduce changes to avoid similar emergencies.

What is essential during this time is communication. You have to communicate what happened, what was the impact when the solution is going to be deployed. In the end, writing a blameless postmortem is a thing that helps the whole organization to learn and improve.

  1. Predict (next) requirements. You have written and deployed your first service from scratch. Soon after that, there are new requirements to develop in that service. There are features often omitted (sometimes on purpose) in the first version of the service and always required in the next iterations. I find that the most popular cases are:

    a) Reporting and analytical module.
    b) Authorization mechanism.
    c) System readiness for A/B testing.
    d) Data search features.

You have to predict or at least talk about those common features at the beginning of the project. Discussing those may help you better plan and organize the design of your system.

Learning

During my time at university (5 years at Warsaw University of Technology), the most important thing which I learned was learning how to learn. It is vital to know your possibilities in self-learning. Most of the things that I know are from self-study. It is apparent that I had and still have mentors that point me in a direction, but I’ve to learn by myself. Below are 6 things that I use to learn new things.

About learning

  1. Books. My average peace of reading is about 15 books a year. It is not much. I’m trying to read books that suit me well. I use Goodreads to find new inspiration. You can see books which I read here

  2. Blogs. There a lot of useful blogs to read online. I’ve discovered many exciting blogs from my friends, who shared with me interesting articles. I’ve also added those blogs to my RSS Reader: Feedly so that I can stay up to date.
    Podcasts and vlogs are modern forms of blogs. I can listen to them on the bus on the headphones. It is the most convenient form for me.

  3. Conferences (offline). I like traveling to conferences. I’ve to admit that my attitude toward the conference changed. At the beginning of my career, I was eager to listen to every conference talk. Now, most value for me is meeting with people, discussing novelties and trends.

  4. Conferences (online). Nowadays, conference talks are available online - on YouTube. Because you can speed up or pause the video, I prefer this form of watching conference talks.

  5. Workshops. I love attending workshops. During the workshop, you have a unique ability to focus on a problem, experience new tools on hand. You can discuss the unknowns with the mentor.

  6. Knowledge sharing. And by this, I do not mean to be a rock star speaker or world-famous blogger. It is about helping your colleagues every day, about small things. Whenever you are giving a piece of small advice, you often make sure that this is the right and correct, you often do some research and discover new things. Maybe someone else has a different point of view, and you can learn something new. The code review process is a great place to start. Famous polish speaker Jacek Walkiewicz said: “Kto przewiezie innego człowieka swoją łodzią na drugi brzeg, sam też tam dopływa.” My translation is: “Who will carry another man with his boat to the other side, he also arrives there.” You have to notice if you support someone else, you also support yourself.

Motivation

What motivates you? Are you desirous to wake up early and code, or maybe each day in the office is a nightmare for you? For me, if I weren’t a geek, I couldn’t be a software engineer.

  1. Look what is under the hood. No matter what project you do, sooner or later, something goes wrong, and going deep is inevitable. You may debug your framework or analyze the internals of your database. You should not be afraid to do that. Studying how your toolset works before is even better.

  2. Passion. I firmly believe that you have to love computers and software to be successful as a software developer. Only if you are devoted to something, you can be successful at it. Some people say that they have never worked a day in their life because they love their job so much. In my opinion, passion helps help you wade through the hassle, which happens from time to time.

  3. Curiosity. Steve Jobs once said: “Stay hungry, stay foolish.” I continuously try to do better. Rewrite some method, refactor some piece of code, try to redesign architecture to be more resilient. Never assume that you are an expert in something - there is always a bigger fish. Experiment, discover new things and have fun.

Summary

Although we have smarter IDE’s, better tools to design software, more productive programming languages, core principals of software engineering didn’t change much. Our computers use Von Neumann architecture, introduced in 1945. For over 50 years, our programs are still often imperative focused on achieving specific goals.
Stability is a good thing because core concepts are the same; you only need to change tools. We place great value on configuring and designing our programs to be resilient to an emergency, network failures. Smaller or bigger innovation is a natural part of our process. The only constant in life is a change. Sooner or later, you are going to unlearn what you know and learn new things. What is going to be? I can not predict the future.

Photo credit

Two best books in 2019

According to The National Library of Poland research, only 10% of Poles read at least 7 books per year. I know that we are not reading a lot, and it was no surprise for me. Conversely, this fact motivated me to investigate to do some health-check about my reading.

The challenge

At the beginning of 2019, I thought that I read approximately about 10 books a year, but I wasn’t sure. My regular peace is one book a month, but I didn’t measure it in any way. I’ve decided to change it.

Goodreads reading challenge was a game-changer for me. This technique meets the SMART goal requirement. It was specific, measurable and since I know that I read about 10 books a year, I could set a target that was also achievable. I’ve decided to commit to reading 15 books in the year 2019. In fact, I’ve managed to read 18 books.

From all of those 18 read books, I’ve selected two books that have the biggest impact on my career as a Technical Leader.

Best technical book

Designing Data‑Intensive Applications by Martin Kleppmann is a must-read for every Software Engineer.

If you think that you are not building (or will never build) a distributed system, look at Single-Threaded CPU Performance, for the last ten years. The CPU performance is not increasing so rapidly that it used to. That’s why we should learn how to build complex and distributed systems, using only commodity class machines.

Before I start discussing this book, we need to distinguish between two terms: competence and knowledge.
Competence is about ability, a certain area of skills that allows you to perform well. Knowledge is just only knowing things. In most cases, you can Google for facts you are looking for. Take a look at the following answer at Quora.

Why should I hire a software engineer if I can just copy and paste code from Stack Overflow?

  • Access to StackOverflow is Knowledge.
  • Knowing which code to copy from StackOverflow is Competence.

This book helps you gain Competence.

Let’s discuss the book itself. To align the level of the reader’s understanding of a more advanced topic, this book introduces fundamental databases or data related algorithms. You may be surprised how many Software Engineers do not know these things.

The most precious thing that you get from this book is a big picture view of how many databases and data processing frameworks work under the hood. Also, you will learn by example how to create distributed systems.

What is more, the author is not focusing only on a single technology or database. He constantly contrasting and comparing different solutions, both used today and that we were using some time ago. If you would want to only read databases documentation to gain this knowledge, you would have to spend far more time reading and analyzing it, that reading that book.

Best leadership book

What Got You Here Won’t Get You There by Marshall Goldsmith is a good lecture if you work a lot with other people, and obligatory lecture if you want to be a leader or manager.

What you need to know about the author is that he was a coach for many CEOs in multiple companies. That is an impressive achievement and I thought that this book may be valuable. Furthermore, my friend recommended it to me, so it was a must-read for me. What is unusual about this book is that the author focuses on one and only one aspect of leadership: Changing Your behavior.

Dr. Marshall, in the first part of his book, explained 20 common behavioral mistakes, that not only many people do, but most importantly many managers or directors commit. I constantly asked myself: “What I would do If I were in that situation”. Realizing that I’m not perfect, was an important discovery for me, which forced me to change.

The second part of this book, helps you discover your weaknesses and support you to change your behavior. It presents many techniques, most of which may be implemented right away.

On the other hand, this book is sometimes boring. It spends too much talking about the same issue or repeating itself. It could be more condensed.

Feedback

Currently, I preparing my reading list for 2020. It will be only 12 books, but I want to select the best books that I can read. That is why I would welcome any suggestions from you.

If you are a Software Engineer, I would like to know your reading habits. Do you read more or less? What do you read? Can you recommend something? Leave a comment :)

Photo credit

Agile IT Organization Design

General impressions

Agile promises to deliver solutions through collaborative effort, cross functional team design, modern programming methods and probably many more things. Because of that, it is hard to distinguish between part and parcel of Agile, and optional techniques that were developed over time. Agile IT Organization Design by Sriram Narayan is a bird’s eye view on Agile topics and tries to organize them.

Agile IT Organization Design

As the author mentions multiple things that are somehow related to Agile, starting with project estimations, through project finance and software development practices, ending with team room layout, the reader should be ready to jump over the wide variety of topics. Putting so many subjects in one book causes that specific aspects are discussed from time to time without any case-studies or detailed guidance how to kick start those ideas. It is valuable if you want to get a general idea, but mediocre if you want to get in-depth knowledge. There is nothing to worry about, because there are a lot of references to other publications or books, which may help you to explore more.

The good thing is that at the end of each chapter there is a summary, which after short skimming, helped me to create my own book reading order.

Author very often, explain reasons for doing things in a certain fashion. This approach helped me to reflect what and how I did in the past, and hopefully, it will help me make better decisions in the future.

Recommendations

This book is written for people from all walks of life. You can read it if you are a leader, a product owner, a software engineer or a stakeholder. It would be favourable if every person read that book, but it is not possible. If you work in a modern organization and you have the general understanding of bolts and nuts of Agile processes, you will probably not lose much by skipping that book. It is a good book for someone who would like to take a break from technical books and read something without code listings inside. Also, I would recommend this book to reach out for a particular chapter which are in the area of your interest.

Key takeaways

I have highlighted over 50 sentences in this book. Some of them to remember, other to think about or to discuss them with colleagues. I have selected 9 that stuck in my mind:

  1. End-to-end cycle time is more important than team development velocity.
  2. Doing sprints doesn’t make it iterative development.
  3. An engineering team without a clear leader may have trouble settling down on a solution design.
  4. The product is never mature; it is always maturing until it is finally obsolete.
  5. Move from a project centric model of execution to a product-centric model.
  6. Labeling people with job titles like an engineer, a manager, a QA, may have side effects.
  7. Spoken language influences the way we think.
  8. People say tl;dr to anything longer than a tweet.