Software Engineer Checklist
When I started my career as a software engineer, I didn’t have any map or list of skills that are necessary to succeed. I’ve spent too much time on unimportant stuff, a far too little on crucial things. If you know at least something about software development - you can create web service in Django, or build microservice in Spring, crate some machine learning model, or write Spark Job - and you want to advance your career to the next level, this blog post might be helpful for you.
Before we start, I have good and bad news.
The good news is that at some point you will have to unlearn what you have learned. If you learn something, you are becoming blind to other things that you can learn. Remember, you are a creative person; your mind should be free and open for new waters. Do not be afraid to forget stuff.
The bad news is that you will probably never stop learning.
Over time, some things became similar, and we use similar design patterns in different contexts. Even that something is similar, it is not an excuse to stop learning. You need thousands of hours of practice and writing software to be good at it. Besides spending many years writing software, I know that I have deficiencies in some areas that I want to improve.
Disclaimer
I’ve tried to build this competence area map to be as universal as possible. Nevertheless, I realize that backend software engineers found it applicable, whether other specializations may found it less useful.
If any of those concepts are new for you, do not try to learn everything in one week. Learning about software engineering is like going down the rabbit hole. You think that you get it, but then you realize that there is another level of abstraction or other stuff to discover. If you are a person who likes following the rabbit holes, this guide may be for you.
Original idea
It all started with writing down the desired competency list for myself a few months ago. In the same way, many of us do a medical examination, and I wanted to do my professional skillset examination. I did it in Google Sheets. The total number of rows that I’ve written was 50 - each describing skill, knowledge area, or competence that I want to build or improve in myself.
Hard technical skills
Hard technical skills are a necessity when you are a software developer. While I was writing down technicals skills, I divided them into three groups:
- General. Those are universal skills that benefit you for many years. Understanding them is an important thing.
- Support. Although those skills are not necessary to do your job rightly, by having those skills, you gain a deeper understanding and be able to propose more suited solutions.
- Trendy. The IT profession is susceptible to trends. Today we use that technology, but tomorrow we may use different.
General technical
General technical skills are skills that are not going to be outdated promptly. On the other hand, practicing those skills consumes much time, and in my opinion, you need years to practice them, rather than months.
Algorithms. I often get alarmed whenever I hear that someone is stating that “Learning algorithms is a waste of time.” Studying algorithms is the essential thing that you can do when you learn computer science. Algorithms are everywhere. Algorithms are often quite different from one another. By studying more and more algorithms, you may develop different solutions for the problems which arise during a regular job. I have to admit that I learn mostly from my experience. Studying algorithms is an eye-opener for new approaches on how to solve things. If I hadn’t studied algorithms, I would propose worse code, simply because I wasn’t familiar with some techniques.
Data structures. Each data structure is designed to arrange data to suit a specific purpose. Sometimes we want to find data quickly or store quickly. Knowing fundamental data structures, like Hash Maps or Trees, is a necessity. Also, it is vital to be familiar with more advanced data structures like probabilistic data structures, immutable data structures, or when we talk about BigData - distributed data structures.
Programming. Can you write code in your primary programming language without an IDE (Integrated Development Environment)? Can you do that using only a text editor and compiler? Without knowing language syntax, builtin libraries, data structures, fundamental data types, it is impossible to be productive. Although I use IDE on my daily basis, it provides me little to none help when it comes to solving simple and primitive code mistakes.
- Framework. Can you write code without using StackOverflow? My primary programming framework is Spring Framework. Things like exposing an API, fetching data from a database, calling other services using HTTP Client, writing unit tests, are natural for me because I am familiar with this framework. If you need to Google everything you need to do and copy-paste the solutions - especially simple tasks - you are probably not familiar with your framework.
Architecture. It is easy to understand, maintain, and develop a system with clean and transparent architecture. There is no silver bullet here. You have to know at least a few architectural styles to communicate your ideas, and to understand others. You have to understand the basic concepts. When someone is saying to you, “Event sourcing is ideal for this problem.” you need to be on the same page to discuss details of that architectural style.
Database. In the initial steps of designing a new system, a database might be a nonrelevant detail to your application architecture. Sooner or later, the database is an element that cannot be ignored. Embrace the polyglot database style. Analyzing database queries, managing indexes, configuring partitioning, and replication is a piece of must-have knowledge. There are far too many different databases on the market to know each well, but you have to know one database at least well and know when to use which database.
Design patterns. If your primary language is object-oriented, you have to know basic design patters (reading a book about design patterns may help). Besides that, there are also more rules, like SOLID, KISS, DRY. There is also DDD and CQRS, which are more like architecture styles. Use those patterns, but do not overuse them.
Coding styles. There are different approaches on how to write code. There is TDD (Test Driven Development), Test First Approach, Pair programming, writing PoC (proof of concept), or writing scripts for one-time use. Each of these styles aims to achieve different things. Try different things and know the advantages and disadvantages of different approaches.
Concurrency, no wonder, is the scariest topic for many developers. It’s hard to test, debug, and reason. Not only you have to understand the behavior of concurrent programming in your language, but you also need to understand your platform well. For example, take JVM. At first glance, you download it, and it works. For me, this is a typical example of a rabbit hole. You have to investigate the Java Memory Model, Garbage Collection, Just-in-time compilers, bytecode. You can not study it in just one day. It may take you weeks to investigate all this stuff.
- Tests. There are many different tests that you can write for your code: Unit tests, Integration tests, End to End tests, Chaos testing, mutation testing. It is easy to write test code, but the tricky part is maintaining the test codebase in vital conditions. Test code should be treated with the same importance as your production code. Remember that if you have robust and reliable tests, you can rewrite your production code without any bother.
CI and CD. I don’t pay much attention, whether it is Jenkins, Bamboo, or GitLab CI. Pick one, and know it well. Understanding why we do Continues Integration or Continues Development is an important aspect. The tool does not matter. The right configuration to achieve the right goals is crucial.
Security. This a vast topic. Essential is asymmetric cryptography (public/private key), authentication methods, access delegation methods - OAuth. On the other hand, you have to know different vulnerabilities (like SQL Injection or Cross-Site Scripting). Then it is essential to realize what types of malware there are and what are the root causes of exploits. And last, you have to know your technology stack, to design and build safe applications.
Network communication. The first thing to know is “8 fallacies of Distributed Systems.” It helps you to understand the behavior of a network application and characteristic of network communication. Then, digging deep into the network stack helps you understand network communication via modern protocols and debug problems, which sooner or later happen.
Rapidity. Practice, practice, practice until you achieving fluency in general programming. After a few years in software development, you should be able to swiftly develop a new application, fix a bug, or re-engineer a complete system. Anyone can probably write any system, given no time limit.
Supporting
Supporting skills are skills that are not your core competence. Instead, knowing those things helps you see the complete spectrum of the Software Engineer Toolbox. Although you do not have to be an expert in each aspect, you have to be familiar with each aspect, and be able to develop something in other areas.
Distributed systems. Truth be told, I don’t think that I will ever write a system that runs on a single computer. Because we are not able to build faster CPUs and single machines in general, it is reasonable to host your services on commodity class machines or in a cloud in a shared environment. We all need to understand how to design, develop, and maintain a distributed system.
Statistics and math. I noticed that I often use mathematical statistics concepts, which I learn in School. It is Percentiles, standard deviation, quartiles, mean, distribution. I use it all the time, for example, in analyzing service response times or working with almost any kind of data.
BigData. It is good to have some experience with Hadoop, Spark, Job Scheduling mechanism (like Airflow), Stream processing. In the world where we treat data as a bar of gold, you have to know how to work it. A few years ago, it was hard to find someone doing big data, but now I think it is a necessity.
Web . A full-stack developer is a most wanted developer. In my opinion, people claiming to be full-stack developers, most of the time, prefer frontend or backend. It is good to understand both worlds. If you prefer backend, you shouldn’t be afraid to change the frontend and vice versa, but hardly I see experts in both words.
Machine Learning. If you are a Software Engineer, machine learning is an entirely different career path. Machine learning models are becoming more and accessible, so you will only benefit from knowing how machine learning models are built or how to train and work with an artificial neural network.
Unix. Why do I recommend Linux or Mac for a Software Engineer? Because of Unix Philosophy. For example, text Processing Toolset. You may state that a Software Engineer should write software and should not work with raw files. Far too often, I see a use-case for tools like sed, awk, or even simple grep does the job. That and many other tasks are simple in Unix.
DevOps toolbox. In your company, your infrastructure for services is probably ready and configured, and there is a dedicated person to run and manage that thing. Unfortunately, problems are unavoidable. Knowledge of how and where your services are running may be crucial when there is some failure.
Site Reliability Engineer mindset. By this, I understand how to release, monitor, manage an emergency, and all things that you can read in a free by Google Engineers Every software developer who has an application on production should embrace those concepts.
Computer architecture. Do you know a computer or server architecture? Do you know about CPU caches, RAM, or network bandwidth? Do you know the limitations of hard disks? It is essential to understand bare metal machines and their physical limitations. It is also good to know about computer construction. Nowadays, we tend to go to the shop and buy a notebook. It might be an unusual experience to build a PC from scratch by yourself.
Trendy
In this section, you have to answer what is trendy right now. Take a look and write your answers. Do not be sentimental about past technologies. It is not a confession.
- Core programming language: ……..
- Main framework: ……..
- Monitoring: ……..
- CI/CD: ……..
- Build tool: ……..
- Asynchronous and non blocking communication: ……..
- Version Control: ……..
Some choices are clear to me and rather stable - Git as a version control system. Your primary programming language probably influences other choices. That is fine, as soon as you are aware that you are not working in obsolete and deprecated technologies.
My programming language of choice is Java (but I have a remarkably pleasant experience with Kotlin), with Gradle as a built tool and git as version control, with project reactor. I know that the project reactor is ugly. Keep in mind that the topic of asynchronous and non-blocking communication is continuously changing. At the begging it was CompletableFuture, then it was RxJava. Now it is a choice between Project Reactor and Kotlin Coroutines.
People
Technical skills are not everything. Software Engineers work in teams, so communication skills are as relevant as the ability to code.
Communication. For me, excellent communication is not about speaking or giving a great presentation. Speaking may be a monologue. Dialogue is when you listen actively to others. Never assume that you know other person intentions or goals. Paraphrase and ask for clarification.
Be authentic. Do not pretend someone you are not.
Language: Business and technical. Communication is tricky. Business people tend to see profits and results, unlike technical people, who tend to see problems and potential bugs. You have to learn how to communicate your ideas to different groups of people.Respect. It is a magic glue that holds teams together. We are all equal, so we should treat everyone in the same way. Our workplaces should allow expressing ourselves freely. You have to do it professionally and cut down any insult or aggression.
Willing to help. Giving useful information or advice to your teammate is a great way to build a relationship. Soon you realize others became willing to help you in the moments when you need it.
Give and receive feedback. Providing high-quality, fact-based, structural feedback is hard. What is harder? Receiving feedback. No one likes to be criticized, and far too often we make it personal. When we make it that way, we became defensive. Thus we do not accept the feedback and remove any chance to improve ourselves.
Product
We - software developers - do not make software just for fun. We do not use design patterns because someone at the conference said so. Our product should be the most useful for our clients, with the right features; hence we built maintainable software that is easy to extend.
Focus on product, not tasks . It is easy to forget about the product when working with some issue tracker. Task after task, swiping from TODO column to DONE. Stop for the moment and look for business value gain in those tasks. If there is little to none product improvement in your tasks, there is something wrong. On the other hand, if there are only business tasks and no time for maintenance (or technical debt), there is also something not right.
Propose new features. Although stakeholders or business people are good at business, they are not experts in technology. I’ve noticed that the best ideas and solutions came from technical people - developers, designers, product owners. Having a good idea is one thing, but having the courage and charisma to propose your ideas is another thing. If you have good ideas, you also have to communicate them effectively.
Public relations. There is going to be a time when your system will not be working correctly or will be down. It may be due to deployment of the new version of the system, hardware or network failure, wrong configuration. No matter what was the cause, you have to clean up the mess, and you have to it professionally. This process usually has three steps:
a) Fix the problem. Make the system healthy again.
b) Repair potential damage.
c) Introduce changes to avoid similar emergencies.
What is essential during this time is communication. You have to communicate what happened, what was the impact when the solution is going to be deployed. In the end, writing a blameless postmortem is a thing that helps the whole organization to learn and improve.
Predict (next) requirements. You have written and deployed your first service from scratch. Soon after that, there are new requirements to develop in that service. There are features often omitted (sometimes on purpose) in the first version of the service and always required in the next iterations. I find that the most popular cases are:
a) Reporting and analytical module.
b) Authorization mechanism.
c) System readiness for A/B testing.
d) Data search features.
You have to predict or at least talk about those common features at the beginning of the project. Discussing those may help you better plan and organize the design of your system.
Learning
During my time at university (5 years at Warsaw University of Technology), the most important thing which I learned was learning how to learn. It is vital to know your possibilities in self-learning. Most of the things that I know are from self-study. It is apparent that I had and still have mentors that point me in a direction, but I’ve to learn by myself. Below are 6 things that I use to learn new things.
Books. My average peace of reading is about 15 books a year. It is not much. I’m trying to read books that suit me well. I use Goodreads to find new inspiration. You can see books which I read here
Blogs. There a lot of useful blogs to read online. I’ve discovered many exciting blogs from my friends, who shared with me interesting articles. I’ve also added those blogs to my RSS Reader: Feedly so that I can stay up to date.
Podcasts and vlogs are modern forms of blogs. I can listen to them on the bus on the headphones. It is the most convenient form for me.Conferences (offline). I like traveling to conferences. I’ve to admit that my attitude toward the conference changed. At the beginning of my career, I was eager to listen to every conference talk. Now, most value for me is meeting with people, discussing novelties and trends.
Conferences (online). Nowadays, conference talks are available online - on YouTube. Because you can speed up or pause the video, I prefer this form of watching conference talks.
Workshops. I love attending workshops. During the workshop, you have a unique ability to focus on a problem, experience new tools on hand. You can discuss the unknowns with the mentor.
Knowledge sharing. And by this, I do not mean to be a rock star speaker or world-famous blogger. It is about helping your colleagues every day, about small things. Whenever you are giving a piece of small advice, you often make sure that this is the right and correct, you often do some research and discover new things. Maybe someone else has a different point of view, and you can learn something new. The code review process is a great place to start. Famous polish speaker Jacek Walkiewicz said: “Kto przewiezie innego człowieka swoją łodzią na drugi brzeg, sam też tam dopływa.” My translation is: “Who will carry another man with his boat to the other side, he also arrives there.” You have to notice if you support someone else, you also support yourself.
Motivation
What motivates you? Are you desirous to wake up early and code, or maybe each day in the office is a nightmare for you? For me, if I weren’t a geek, I couldn’t be a software engineer.
Look what is under the hood. No matter what project you do, sooner or later, something goes wrong, and going deep is inevitable. You may debug your framework or analyze the internals of your database. You should not be afraid to do that. Studying how your toolset works before is even better.
Passion. I firmly believe that you have to love computers and software to be successful as a software developer. Only if you are devoted to something, you can be successful at it. Some people say that they have never worked a day in their life because they love their job so much. In my opinion, passion helps help you wade through the hassle, which happens from time to time.
Curiosity. Steve Jobs once said: “Stay hungry, stay foolish.” I continuously try to do better. Rewrite some method, refactor some piece of code, try to redesign architecture to be more resilient. Never assume that you are an expert in something - there is always a bigger fish. Experiment, discover new things and have fun.
Summary
Although we have smarter IDE’s, better tools to design software, more productive programming languages, core principals of software engineering didn’t change much. Our computers use Von Neumann architecture, introduced in 1945. For over 50 years, our programs are still often imperative focused on achieving specific goals.
Stability is a good thing because core concepts are the same; you only need to change tools. We place great value on configuring and designing our programs to be resilient to an emergency, network failures. Smaller or bigger innovation is a natural part of our process. The only constant in life is a change. Sooner or later, you are going to unlearn what you know and learn new things. What is going to be? I can not predict the future.
