Is LinkedIn Premium Worth It?

The Short Answer: No.

The Long Answer: As a job-seeking software engineer, I don't think LinkedIn Premium is worth the cost ($39.99/month). I reached that conclusion after using the free one-month trial and spending time with its exclusive features.

Opinions on the internet are mixed about whether LinkedIn Premium offers a meaningful advantage in the job market¹²³⁴⁵. Supporters point to features like seeing which recruiters viewed your profile, messaging those recruiters directly, and receiving application insights that estimate how well your resume matches a position. In my experience, however, none of these features had a meaningful impact on my job search.

First, the ability to see which recruiters viewed my profile didn't provide much of an advantage. Just because a recruiter viewed my profile doesn't mean they'll respond to a message. In fact, I've never received a response after reaching out to a recruiter first. If a recruiter is genuinely interested, they can message me—and as recruiters, they almost certainly already have LinkedIn Premium through their employer.

The feature I used most was "Jobs where you'd be a top applicant." It highlights recently posted jobs that LinkedIn believes closely match your experience. It's a useful feature, although not a perfect one. It frequently recommended full-stack and frontend positions, even though I'm exclusively a backend engineer.

I eventually realized that these recommendations rely heavily on the Skills section of your profile—a section I had largely neglected. Improving that section only slightly improved the recommendations. Even then, LinkedIn's search and filtering tools remain fairly limited. There's no equivalent of a negative regex or exclusion filter, so you'll still spend time filtering out irrelevant positions.

Some info censored for privacy reasons

Another selling point of LinkedIn Premium is access to LinkedIn Learning. Personally, I found most of the courses to be fairly introductory. For the topics I was interested in, classes I've found on LinkedIn were rudimentary and lower in quality than what you can find on YouTube for free.

Finally, paying LinkedIn for the privilege of fully using its own data feels a little backwards. It reminds me of paying a headhunter to help you find a job. Traditionally, job-search platforms have been funded primarily by employers and recruiters rather than by job seekers.

That said, the strongest reason to use LinkedIn isn't Premium—it's the network itself. Nearly every professional software engineer I've worked with has a LinkedIn profile. Over time, your coworkers become your professional network, giving you a large pool of current and former colleagues (along with the occasional recruiter) that you can reach out to during your next job search. None of that requires a Premium subscription.

LinkedIn's job board is also reasonably useful, although I still think it's underdeveloped due to its meager filtering options. Its biggest advantage is the integration with your network. Once you discover that a company you're interested in employs someone you know, you can ask them for that sweet, sweet job referral. Again, that's entirely possible with the free tier.

To be fair, there are situations where Premium may be worth the cost. If you're pursuing a particularly niche role or have been searching unsuccessfully for an extended period, spending $40 for a month or two to gain even a slight advantage may be a reasonable investment. Job searching can be exhausting, and even small improvements to the process can make it feel more manageable—even if they don't ultimately land you the job.

Footnotes

1. https://www.linkedin.com/posts/courtney-paraski-33544672_my-honest-review-was-linkedin-premium-worth-share-7381786673992429568-jUcL/ (not worth it)

2. https://www.trykondo.com/blog/linkedin-premium-worth-it (only if active seeker in "high demand" industry)

3. https://www.quora.com/Can-you-explain-what-LinkedIn-Premium-is-and-whether-it-is-worth-the-cost (mixed)

4. https://www.reddit.com/r/careerguidance/comments/104vai3/linkedin_premium_is_it_worth_it/ (mixed)

5. https://www.linkedin.com/posts/jarrodbestmitchell_is-linkedin-premium-worth-it-if-youre-job-ugcPost-7386736395865858048-oD_O/  (yes)

Evaluating AI Models for Production

The following article isn’t sponsored by any organization and is solely the opinions and observations of the author.

I recently attended a software seminar on generative AI (GenAI) model evaluation. The event was hosted by TrackIt and AWS. The session promised to explore “how modern AI teams are benchmarking, testing, and optimizing large language models for production environments,” but the most valuable takeaway was a discussion how best to evaluate LLMs (large language models; generative AI models like ChatGPT, Claude Code, etc.) for real-world business use.

I found this topic particularly interesting because there is little guidance available on how organizations should choose an LLM for a specific project or business need. While model capabilities are discussed extensively, the process of evaluating and selecting a model is often overlooked.

This slide was crucial:

Sorry that I didn't get a better shot

It summarizes the three primary approaches to LLM evaluation:

1. Algorithmic (deterministic)
2. LLM-as-Judge
3. Operational (cost, speed, etc.).

The first and third are relatively straightforward. Operational considerations such as cost, latency, and vendor support are often the first factors engineering teams evaluate when selecting software. Similarly, there are dozens of deterministic LLM-evaluating benchmarks, such as Humanity’s Last Exam, GPQA Diamond, and others. There are numerous websites dedicated to comparing LLMs using these benchmarks.

LLM Metrics from Vellum

What was novel to me was the LLM-as-judge approach: using one LLM to evaluate the output of another. In this approach, you define the evaluation criteria and have a separate LLM score the responses generated by the model under test. Although this requires some upfront effort to design effective evaluation criteria, those criteria can be reused across all of your future evaluations. 

This approach is especially valuable when your intended usage has no objective ground truth, or when quality depends on subjective factors. While traditional benchmarks can provide insight into general model performance, they may not accurately predict how well a model performs on a specialized business task. For example, a benchmark score will not tell you which model is best at transforming a chef’s rough notes into polished, consumer-ready recipes for a new cooking website.

Let’s imagine you are building a cooking platform and have a chef who sketches recipe ideas in shorthand. You want an LLM to convert those rough notes into clear, complete, and professionally written recipes. To evaluate candidate models, you would create a collection of representative inputs (the chef's notes and the model instructions) and ideal outputs (finished recipes). Each model under evaluation would generate responses for the same inputs, and the judge model would compare those responses against the expected outputs using criteria you define. The resulting scores would provide a task-specific evaluation tailored to your business needs.

Some tips:

• Pick one or two judge models. One might think that the comprehensive way would be to use many different models. This adds significant complexity while providing limited additional value. In most cases, using your current production model, or a capable, low-cost model is sufficient.
• Use a small continuous scoring range, such as 1–5. Quality assessments are inherently subjective and thus should be rated with a range rather than a binary pass/fail. Conversely, a huge range (e.g. 1-100) implies a false level of precision.
• Don’t rely solely on LLM-as-judge evaluations. Combine subjective evaluations with the algorithmic and operational metrics. Those quick, easy, and deterministic methods add important color to the subjective method discussed here.

The key lesson I took away from the seminar is that selecting an LLM should not be based solely on benchmark rankings. The best model for a business is often the one that performs best on the organization's specific tasks while balancing algorithmic performance, subjective quality, and operational requirements.

Code: Kotlin First Impressions

Introduction
Kotlin is one of the newest and most popular JVM languages and it might be one of my new favorite languages. Disclaimer: I haven't written much production-ready code in it yet.

As a JVM language like Scala, Groovy, and Clojure; Kotlin tries to take the portability of the Java Virtual Machine and familiarity of the Java language while improving upon its archaic design with the conciseness and power of modern programming languages. Simply put: it's Java but better—for real this time!

Scala was my favorite attempt at improving Java, but I was pretty unhappy with the build system. Since Kotlin is written by JetBrains, an IDE company, compiler and IDE support is first-class. You can even use JetBrains' IntelliJ to output Kotlin code to Java, and vice versa. For this reason alone, it is easy to pick up for Java developers.

Improvements on Java 8
So what does Kotlin improve upon? Firstly, Java's verbosity. Compare a hello world program in Java:

public class HelloWorld {
  public static void main(String[] args) {
    System.out.println("Hello world!");
  }
}

and in Kotlin:

fun main() {
    println("Hello world!")
}

The first thing to note is that no class definition is required. You can simply add top-level functions. Kotlin is a functional language, so a function definition is required, but the public access modifier is implied. Top-level functions are always static. And type inference also applies to functions, so no void is required here. Just fun to mark that it's a function. In the actual function itself, println is a top-level function, so no need to qualify the function with a package and class. Lastly, no semicolons.

This is not some neat trick about hello world programs, the rest of the language is written to be concise. Record classes (data classes), raw strings, type inference, default function parameter values, string interpolation, collection builders, array slices, range operators, and null safety operators all help make your code very concise and efficient. If you don't know what those mean take the following Kotlin code:

fun doSomething(param: String = "default") {
  val stringTemplate = "$param ${1 + 2} String"

  val nums = (0..20).toList() // A list of integers from 0 to 20
  println(nums.slice(1..5 step 2)) // Prints "[1, 3, 5]"

  val newStr: String? = getStringOrNull()
  println(newStr?.length) // Prints length or "null" if string is null

  val neverNull = getStringOrNull2() ?: "" // Converts nulls to ""

  val regex = """\w+@gmail\.com"""
}

Doing this in Java 8 without special libraries would be like:

public static void doSomething() {
  doSomething(null);
}

public static void doSomething(String param) {
  param = (param == null) ? "default" : param;
  String stringTemplate = String.format("%s %d String", param, (1 + 2));

  List<Integer> nums = IntStream.rangeClosed(0, 20).boxed()
      .collect(Collectors.toList()); // It's either this or a for loop
  // A for loop is simpler than the stream version
  List<Integer> slicedList = new ArrayList<>();
  for (int i = 1; i <= 5; i += 2) {
    slicedList.add(nums.get(i));
  }
  System.out.println(slicedList); // Prints "[1, 3, 5]"

  String newStr = getStringOrNull();
  System.out.println((newStr == null) ? null : newStr.length());

  // We need two lines for this, unless we call the function twice
  String intermediateVar = getStringOrNull2();
  String neverNull = (intermediateVar == null) ? "" : intermediateVar;

  String regex = "\\w+@gmail\\.com"; // We must escape special characters
}

Or at least that's my interpretation. With Java 8 streams, there are a few functions to create and manipulate collections. You can also short circuit nullable objects with Optional.ofNullable([nullable]).orElse([default]), but I prefer the ternary operator in Java.

Another great feature is destructuring declarations. In Python, you can return multiple values. In Kotlin, we get half of that feature by being able to "destructure" a returned value into multiple variables:

  val (name, age) = person
  for ((key, value) in map) {
    // stuff
  }

Lastly, it is interoperable with Java. You can call Java code from Kotlin and you can even call Kotlin code from Java. One of Java's biggest strengths is the massive set of libraries you can use. These libraries are available, too, in Kotlin.

Drawbacks
Sounds too good to exist? Well, Kotlin has some drawbacks. I probably will come up with more after using the language for a while, but for right now, the drawbacks are minimal.

For one, variable type declarations follow the variable name as in var a:Int. For many of us, we started programming in C, C++, or Java where variable types come before the variable name. I disliked this seemingly arbitrary change until it was explained to me.

When declaring a variable, the variable's name is the most important aspect. When your language has type inference, the type isn't always written. This means variables declarations are generally aligned well, regardless of type declaration or lack thereof. This alignment is maintained for variables (val or var) and functions (fun).

val a = 1   // Infers type from value
val b: Int  // No value to infer type
var c: String = "String" // Optionally specify type
// Infer return type of single-expression function
fun sum(arg1: Int, arg2: Int) = arg1 + arg2

Notice that the val/var/fun keywords are all aligned vertically, as are the names. So while this name-type order will take some getting used to, it's not a bad thing. Pretty much all modern languages (Go, Rust, Swift, Scala, Nim, and Python) use it.

One issue that I dislike is the amount of scope functions. let, run, with, apply, and also are all ways to call a block of code with a temporary scope and each is used slightly differently. This seems like overkill and I don't imagine I'll ever memorize which is which. You don't want to have too many ways to do the same thing in a language, as it makes reading and reviewing code difficult.

Overall, it seems like a very fun, concise language and an upgrade to Java.

Kotlin vs Newer Versions of Java

JetBrains 2021 Developer Study

Having said that, Java has progressed quite past version 8. Java 8 is still the most popular version, according to a JetBrains survey, being used by 72% of Java programmers (users were allowed to select multiple versions). SNYK's survey results says Java 11 slightly outweighs Java 8 with both being around 61%. In either case, the higher versions of Java have little use with both companies saying Java 15 use is around 13%.

SNYK 2021 Developer Study

Java 19 will drop in September, but for a company looking for stability should probably stick with Java 17, as it is will still be the most recent version with Long-Term Support from Oracle. So what does Java 17 add to the language that Java 8 and Java 11 users may be unfamiliar with? I will describe some of the notable language updates, ignoring preview features and compiler and JVM enhancements.

Java 9 to 11 updates:

• 9: Project Jigsaw: Modular system
  Introduces a module system to the language to define exports and dependencies.
• 9: Private methods in interfaces
• 9: Actual immutable collections
  Allows things like Set.of(item1, item2).
• 10: Type inference
  Introduces var keyword.
• 10: Root certificates.
  Allows TLS out of the box.
• 11: HttpClient
  A replacement for HttpUrlConnection.
• 11: More String and Files methods
  New methods like String::lines and Files::readString.

Java 12 to 17 updates:

• 12: New methods in String, Files, and Collectors
• 12: A number formatter
• 14: Switch expressions
  Makes switch expressions much less verbose.
  
• 15: Text blocks
  Allows setting Strings to nicely-formatted multiline blocks of text without inserting pesky "\n".
• 16: Records
  Allows the creation of data classes with default getters and setters in essentially one line.
• 16: Pattern matching for instanceof
  Decreases verbosity by allowing the declaration of a variable in an instanceof expression.
• 17: Sealed classes
  Increases control over class inheritance.

Most of this article was written with Java 8 in mind. It's fair to say that Java has come a long way since then and Java now has more syntactic sugar. Specifically, type inference, text blocks (multiline strings), and records (data classes). However, even these new features are weaker than their Kotlin equivalents:

• Type inference only applies to local variables, not top-level variables or lambda expressions
• Text blocks aren't raw strings and will still process escape sequences (i.e. you still have to escape all of your backslashes). Writing regex in Java still sucks.
• A record cannot contain any private instance variables. Admittedly, this is a tiny disadvantage.

Even the work that has been done to modernize Java doesn't bring it up to par with Kotlin. That's why I'm excited to start using Kotlin.

Software Engineering: RAED for Permissions

 In any given important software being used by many people, access control is an important part of security. For example, I have the proper permissions to edit this blog and you do not.

But there's many ways to implement access control. For example, Unix-like file systems generally have read-write-execute permissions. A user, group, or "other" can each have a combination of permissions to read, write, or execute a file. An admin might use the chmod command to edit a user's permissions.

Windows implements file access control differently. There are permissions such as "Full control", "Modify", "Read & execute", and just "Read" that can be allowed or denied to a user or group.

Windows File Permissions

CRUD is another way permissions might be implemented. A user can be allowed to create, read, update, or delete files or other types of data in any combination.

At my last job I was tasked with coming up with an authorization system for our microservices. Rather than use some framework for authorization, I decided to build our own. I was replacing WordPress permissions, which were similar to CRUD. For any given type of page, a user might be allowed any combination of the creation, reading, updating, or deleting permissions, depending on what an admin had checked. For example, our online magazine had CRUD permissions. All of our customers could read the magazine. Specific content creators could create new articles and upload them. Editors could update anyone's articles to fix grammar, spelling, or links. Finally, admins could delete articles.

Designing permissions was important. We didn't want to give a user the wrong set of permissions, otherwise they might be allowed to delete important data. Or, with insufficient permissions, they might not be able to do their job.

I decided to use something I had previously created: RAED, a superior access control design for general-use permissions.

RAED stands for read-add-edit-delete and is a set of eclipsing permissions to be used with file systems, RESTful APIs, or anything else that requires access control. It's basically CRUD, but much better.

Other permissions become confusing when certain combinations are used. For example, what does it mean when you can write something, but you cannot read something? When would you be allowed to delete data without being able to update it?

With RAED, there is no confusion because if you have one level of permission, you have all of the permissions below (or to the left of) it. If you can add, then you can read. If you can edit, then you can read and add. And finally, if you have the delete permission, you also have the read, add, and edit permissions for that particular thing. For any given permission level, just makes sense to have the lower permission level. When will a user need to delete data, but should be forbidden from reading or editing that same data?

RAED is simpler to display. Instead of four sets of radio buttons (Allow/Deny) in CRUD, you have five radio buttons (the first one being no permissions). Example:

Database Permission:
None Read Add Edit Delete

RAED also can be stored with fewer bits, since there are only 5 possible values as opposed to the 16 possible values with CRUD and Allow/Deny. This is a very small benefit, since we generally store data in single bytes or larger, but it's definitely more compressible.

Can't remember the order of permissions? It's always RAED, like the Arabic name for boys.

RAED doesn't work for all access control systems and permissions, for example, when you only need a simple binary Yes/No for Can-Upload-Photo-Permission or similar. However, if this meets your needs, I highly encourage you use RAED.

Job Seeking: Crafting the Perfect Resume

In this economy, job seekers are at an advantage. Businesses are desperately seeking employees, not just in the service industry, but also in the tech industry. But that doesn't mean it's not important to put your best foot forward and craft the perfect resume that highlights your talents.

I have interviewed scores of software engineers and managers across three companies over a 13-year career. As a director I've made the final call on hiring several QA analysts, QA engineers, and QA managers. I've represented my company at several career fairs. So I know what I want to see in a resume.

This is one area I wanted to make sure I had a lot of experience in before I spoke publicly about it. I feel I have finally achieved that level of experience. Here is my advice.

Basics

A resume should list your skills, your technologies you're familiar with, your work history, and your education. There are a lot of good samples on the Internet. I have posted one below.

Work History

I've been told to make sure I highlight concrete, measurable accomplishments rather than job duties. That is, don't say "Responsible for redesigning backends", say "Streamlined backends, reducing wait time by 50%" if you sped up a backend request from 0.8 seconds to 0.4 seconds. But when I'm reviewing resumes, I am less focused on those achievements since some are easier to accomplish, depending on the company's existing technology. Additionally, many of those tasks were given by bosses and measured afterwards. That measurable accomplishment doesn't tell me much more than a description of the task you were given; it just tells me that you were successful at it. When I read work history, what I'm generally looking for are the technologies worked with, tasks given, and roadblocks overcome.

Similarly, use action verbs. Don't just say "did", "made", and "told". Use "performed", "implemented", and "documented". Once again, I see through that kind of stuff, but it's a good sign when you can use strong action verbs truthfully to describe what you accomplished at previous companies. For example, "documented" implies a permanence that "told" does not. Are they still using your research? Probably so if it was "documented." Probably not if you only "told" someone about it.

Definitely don't sell yourself short. Highlight your top accomplishments and phrase your responsibilities and leadership so that readers don't think you just played a small part in any projects that you led or had a massive part in.

Words Words Words

There is an exception to my apathy to big words: fancy words and flowery grammar show me that you have an excellent command of the English language. If I have any bias, it's a bias towards excellent English communicators. As a software engineer, I have worked with many people who didn't know how to communicate properly and it has occasionally become a stumbling block. It is sublime working aside engineers who can disseminate information widely and individually in a manner that is concise, precise, and unambiguous. Yes, I did use some SAT words there to make a point.

Don't Lie

Oh my god, don't lie. At Google, I once did a "coaching call" (mock interview) with an applicant who claimed he had been the leader of a local student organization. Incidentally, not only had I participated in that same organization, I actually knew the then-leader of the organization. While on the phone with this applicant, I realized something was suspicious and asked for clarification. He backtracked from his lie.

I would never hire someone who would lie about something small like an extracurricular activity. Saying you know Java when you only wrote a small Java app is a small stretch of the truth. Saying you held a leadership position you didn't actually hold is such a meaningless lie that it shows you can't be trusted at all.

Cover Letters: Unnecessary

A cover letter has never, not once, affected whether I wanted to hire someone or not. I understand that there is a level of effort in creating one, but that effort could go towards improving your resume or leveling up your interviewing skills. Anything important to know should be in the resume. Most times, I don't even read cover letters.

Resume Length: 1-2 Pages

It's hard to fit your whole life onto one page, especially if you've been at multiple companies. Now that physical paper is a thing of the past, a two-page resume is completely fine or even expected. Three is a bit long and four is ridiculous, unless you've had a very long, illustrious career. Stick to 1-2 pages. My current resume is 1.5 pages.

Fancy Design

Should you use column headers, multiple colors, a sidebar? I shy away from that. My resume is super plain; maybe even too plain. I do enjoy reading a nice looking resume, but don't go too fancy and make things difficult to read. During my current job hunt, I've filled out online job applications that scan my resume and fill out the application with the parsed results. I imagine these apps struggle on overly-fancy resumes.

An exception is when the role requires good design skills. I am impressed when a frontend developer has a nice resume. This may be foolish, but I suspect the skills in resume design may carry over into web design. But it's not required.

Customize to the Job

If you're applying to different roles, customize your resume. For example, I've applied to QA manager roles and engineering manager roles recently. Each time, I tailor my resume to highlight what the hiring manager wants to see and remove experiences and skills that are irrelevant.

Example

Here's a sample resume from Monster:

[image removed]

I find that most sample tech resumes on the Internet are pretty good, so follow those for the basics.

Software Engineering: Good Reads

I'm not an avid book reader, so I feel that my book suggestions should be taken seriously, since there are so few books I can really recommend.  I think there are a few software-engineering-related books that every good programmer should have on their shelf.  I will list these books, roughly in order of importance.  I've also listed each book's latest edition (that I know of).

The C Programming Language, 2nd Edition by Brian W. Kernighan and Dennis M. Ritchie

The C bible. C is still an important language for many programmers to learn, and this is the book to use to learn it.  Never before or since has a programming language had a textbook so thorough.  I only wish that Java or C++ had a similar textbook that I could recommend as highly.  Maybe it's because C is a small language, but this book works well whether you're learning C for the first time, or you just need a reference.  The only downside is that the latest version of this book covers ANSI C, instead of either the newer revisions, C99 and C11.  For that reason, if you're using a C99 or C11 compiler, this book is somewhat deprecated. C99 made a lot of important changes to the language.  However, the book is still a good starting point if you don't know any C (or, again, if you're using an older compiler).

Introduction to Algorithms, 3rd Edition by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein

This book contains all the information you need to know about algorithms and data structures, including their pseudocode implementations and computational complexity.  I suggest it to everyone who is applying to Google as a software engineer.  The only issue is that it is too complete, and contains a lot of extraneous information.  It's not a book you read cover to cover.  Also, I'm not a big fan of their "pseudocode."  A more Java-like or Python-like language would be easier to read.  As a software engineer for any company that relies on speed for computing large amounts of data, you will need to know the most efficient way to solve your problem.  This book will help you get there.

Effective Java by Joshua Block

That's great that you can write a one-line method to sort an array.  Now, graduate from a programmer to a software engineer by learning why that is awful.  Effective Java tells readers about readability and other important issues that come up with writing code in a work environment.  Basically, it's a collection of tips for writing readable, maintainable Java code.  Do you swallow InterruptedExceptions, that is, catch them and then simply throw them away?  If so, Joshua Block will tell you why you're breaking his heart and the heart of everyone around you.  (For those who just want the answer, read this. Or the simple answer is that you may prevent a program from being killed when it needs to die.)

Software Project Survival Guide by Steve McConnell

This book is written more for managers and team leads, but is useful to all engineers.  It describes in detail the basic ideas and reasoning behind a waterfall software engineering process.  A lot of you may be using agile or scrum, but you can still incorporate most of the ideas in this book in your work process.  Testing, maintainability, requirements, tracking... this book has all you need to turn your cowboy coders into seasoned engineers.  Process, process, process.

Team Geek by Brian W. Fitzpatrick and Ben Collins-Sussman

This book written by two Google engineers seems to be written more for team leads, but, again, is useful to any software engineer or manager of software engineers.  It's all about the non-technical side of software engineering; the human element.  For example, they cover dealing with "poisonous" coworkers, building a strong team, how to have efficient meetings, and how to communicate and collaborate well.  This is the only book I didn't have to read for school or work that I'm recommending.  I think it's important because even a group of smart, trained software engineers can have interpersonal issues or leadership issues, and this book tries to train people to alleviate those issues.

Software Engineering: Automated Software Testing 101

My official title at Google is Software Engineer in Test Tools and Infrastructure, which means I spend most of my time writing automated test infrastructure and consulting with teams to help them test their products.  For the last couple years that I've been working in this role I've learned a lot about testing.  I'd like to disseminate some of this knowledge.

There are many ways to test your software--way too many to cover in one short blog post--, but some of them are more important and widely applicable than others.  In this article I'll talk about the major types of automated software testing and why they are important.

Why and When to Test
Firstly, why do we need to test software?  Software testing can sometimes seem less important than other tasks.  For example, is there any reason to spend expensive engineer-hours writing tests, when you already have design reviews, code reviews, and manual code inspection to assure product quality?  Additionally, the fact that bugs can still exist in well-tested code is disheartening.

Despite the drawbacks, testing should be done early and often.  Testing your software increases product quality (and in doing so, lowers maintenance costs), aids development speed (testing takes time, but bugs are easier to fix when found earlier), proves requirements have been met, and can reveal design flaws.  It's usually worth the time to write at least a few small tests.

"The problem with quick and dirty, as some people have said, is that dirty remains long after quick has been forgotten."
— Steve McConnell, Software Project Survival Guide

(By the way, the Software Project Survival Guide is a pretty good book for those in the software field. I recommend it for engineers and especially for project managers).

This article focuses on automated tests because it's my area of expertise, manual tests are pretty easy to figure out, and because automated tests have several advantages over manual tests: automated tests are generally faster to run, can be run more frequently when part of a continuous build, are easier to use for unit and integration testing, and free up engineers to do other things.  If you're writing a small program for yourself, relying solely on manual system tests is fine.  For major projects, you'll need automated tests.

What to Test
So by now you've been persuaded by my convincing words that testing should be done early and often. But what are we testing?  If we test everything, at what point do we write tests for tests?

Tests should be created for most things, especially complex and critical functionality.  If you've created a function that does nothing but add a digit to the end of a string, it's not critical to write a test for that function. Most other stuff should be tested.

While tests themselves can break, tests don't usually need their own tests because of their simplicity and because a false failure will be visible to your team when the test runs.  A falsely-passing test can be a disaster, but is an infrequent occurrence for well-written tests that were working to begin with.  The exception is to write tests for test infrastructure.  Sufficiently complex tests are usually built on some sort of testing infrastructure (JUnit, WebDriver, a fake database, etc.) that can easily contain bugs, and accordingly, they should be tested.

Who Does the Testing?
Who writes tests: software engineers or a QA/test team?  The answer is that it depends.  For small teams creating small projects, it is sufficient to have engineers test their own code.  After all, it can be difficult to obtain a test guy or gal, and there isn't much code to test, so writing tests is quick and easy.  But even for large teams creating large projects, it can be beneficial to have the programmers write tests.  They can gain insight into their own code, have a better idea of what can break, and are more familiar what the issue is when a test breaks.  As someone who writes tests for other teams' projects, I've often encountered teams that have no idea why a test is broken or how to fix it, even though they know the project code better than I do.  That issue diminishes when the team has a hand in writing the tests.

Alternatively, a separate test team gains a different perspective of the code.  Unlike the programmers that wrote the code, a test person's ego is not affected when a test finds a bug.  In fact, finding bugs is ego-boosting for the test team.  The drawback is that a test team may not write tests that sufficiently cover the weak points of the dev team's code.

So the answer is: either way is pretty good.  Get a test team when writing tests becomes burdensome for the project programmers.  This can happen when the software is extremely hard to test for whatever reason, or when it's complex enough to require special test infrastructure.

Test-Driven Development
So tests should be written early.  But how early?  Some programmers believe in test-driven development, where you write the test before writing the code.  I think that's a bit drastic, but not always a bad idea.  APIs and features may change slightly when actually writing code, so your tests will likely have to change anyway.  You don't necessarily need to write a test before the code is written, but you should have a test plan.  Writing tests should be done simultaneously when writing code; that's early enough to catch initial bugs and late enough to prevent overhauling tests.

Now, on to types of tests...

Unit Testing
First and foremost, if you're going to have any automated tests (i.e. you're not writing a small program for yourself), you should have unit tests.  The reason is that unit tests are the easy and quick to write and run, unlike other tests.  Unit tests are small tests, usually white-box style, that test a class, a few classes, or another small portion of code.  Using mocks or stubs is perfectly fine for unit tests.

Because unit tests are so lightweight and quick to run, they should be run often, preferably as part of a continuous integration process.  This allows the unit tests to catch newly introduced bugs quickly.  Several solutions for continuous integration exist, like Jenkins. (I have not used Jenkins myself).

Integration Testing
The downside of unit testing is that important parts like databases or dependencies on other binaries* are mocked out or nonexistent.  You don't get a good picture of how the software performs under real conditions.  In order to verify the different subsystems of a program, you should write integration tests.  Integration tests are large tests that test multiple binaries, or one binary that uses external resources.  Integration tests are helpful in catching integration errors (often API or design bugs).  Even if two pieces of a project are "bug-free," they may not integrate well, resulting in miscommunication that can only be caught by integration tests.  Integration tests can also find speed or memory issues that can't be found by simple unit tests.

Since integration tests are large and slow, it can be difficult to find a test framework that will bring up all the resources (a.k.a the environment) needed for the test and then tear them down after the test is done.  One workaround is to leave up servers or other external resources indefinitely and have the tests clean up any permanent effects in the environment after the tests finish.  This is a dangerous game, as the long-running environment can get into a bad state, giving inaccurate test results.  Sometimes the only way to run integration tests is to manually bring up and down the environment.  In this case, it might be worth it to manually run the integration tests as well.

System Testing
To get a complete picture of how the software will actually perform, system tests are required.  System tests are large tests, black-box style, that bringing up an actual environment and simulating an end-user using your product.  Not only do they help reveal memory and speed issues that may only occur in a real-world environment, they also may reveal UI and usability issues.

System tests are often run manually since it's hard to find good frameworks that can both setup a large environment and give human-like input (often by manipulating a GUI).  However, many GUI-manipulating tools exist.  I've used AutoIt to manipulate Windows GUIs with excellent results and I've also used WebDriver to manipulate webpages with very good results.  Combined with integration-test-style frameworks, you can potentially automate your system tests.

In summation:

• Write tests early and often
• Run unit, integration and system tests, automating them if you can
• Run automated tests as part of a continuous integration process if you can

Following these suggestions will help you get excellent code coverage and excellent feature coverage, which will result in stable, easy-to-maintain, and on-schedule software.

Update: Added What and Who testing sections.

*Binaries are individually-compiled programs. They may not do much alone, but work in conjunction with other programs to create useful output.  If they do useful work by themselves, they are standalone binaries, or executables.

6+ Ways to Make Your Company as Great to Work for as Google

This post is for the CEOs and people who have the CEO's ear.

For the last four years, Forbes has named Google the #1 best place to work.  I'm told that Google gets huge numbers eager applicants every year.  Most of the software engineers I know want to work at Google.  As a Google employee, I can confirm that Google is indeed a great place to work.

But Google doesn't have some magic bullet.  Sure, spending money on employee perks are an obvious way to increase employee happiness, but it's not the only way.  And not every company can do what Google does, but most companies that employ a set of skilled professionals--especially software companies--can implement some or all of the features that make Google great.  And by making your company an awesome place to work, you will perfect the art of employee recruitment and retention.

I will list six ways to make your workplace better.  And then I will list some more.  It was getting to be a rather long list, so I tried to boil it down to six essentials.  And then failed.

1. Hire the smartest people
Seems like a no-brainer, but I don't think many companies follow this rule like Google does.  Whereas some companies will hire mediocre engineers because they simply need warm bodies to code, Google (sadly) discards a lot of good candidates because we are so serious about hiring brilliant people.  And while passing on good candidates is definitely a bad thing, Google is full of brilliant people as a result of our rigorous hiring process.  The obvious benefit is that smart people work more efficiently and can tackle difficult problems, but as an added bonus, having lots of smart people at your company helps recruit other smart people.  Ray Kurzweil works at my company!  Maybe I'll get to meet him someday!

Hiring the smartest people is hard to do if your company isn't one of the best places to work for, which is what brings us to #2...

2. Treat your employees well (with pay and perks)
If you ask most people what their employer could do to make them happier, the first thing they'll say is to increase their salary.  Obviously, you should pay your employees the going rate for their respective jobs, if you can afford it.  But money isn't everything.  Giving employees perks is another creative way to spend that extra cash you have on employee happiness.

Google employees get tons of perks, from game rooms, work parties, free meals, free valet, matching 401k contributions, gyms, free meals, beautiful working environments, free meals, and, lastly, free meals.  I'm very big on the free meals.  On-site perks, like the meals, will keep employees at work longer and prevent them from taking 2 hour off-site lunches.  All of which can boost productivity.  The only problems with perks are that they cost money and some people may take advantage of the employer's generosity.  Which brings us to #3...

3. Hire honest, caring people
Honest, caring people care about the company they work for, especially when that company cares for them. They try not to cheat the company by overusing perks. It also can help team dynamics.  But how do you recruit good, honest people?

4. Don't be evil

When people talk about how evil a company Google is, I think it's amazingly hilarious and thoroughly frustrating.  Every single decision by Google that has ever been considered possibly evil by the public has been internally protested and ridiculed repeatedly by Google employees.  That's because the company is full of employees who care about the Google's economic, social, political, and psychological impact on the world and we hate it when a head executive makes a bad decision.  We honestly believe that, overall, we're doing good in the world.

Being good is hard and is often expensive, but it helps recruit and retain honest, caring people.  And it has a side benefit of helping the public image of your company.

5. Use the right tools

Google employees have tons of internal software and hardware tools that help us do our jobs.  Some companies are reluctant to upgrade software engineers' computers until they are too slow to run.  Some companies don't want to shell out money for software that can improve productivity.  Google generally keeps their engineers' hardware up to date, and buys or builds software tools that make our lives easier.  From our version control system, to our cloud code repository software, to our testing infrastructure, to our laptops; Google invests a lot of time, energy, and money into making sure we have the right tools for the job.  I'm not saying all our tools are perfect, but it's usually a question of which tool is best and not whether a manager will shell out money for it.

Good tools make employees productive and happy.

6. Share internal information

Sharing information means two things: managerial transparency and knowledge transfer with efficient, persistent interoffice communication.

Transparency is hard.  I once worked at a defense company where it was legally impossible because of security issues.  But at Google, we have a weekly all-hands meeting where we talk about what's going on in the company to a surprising detail, and low-level employees can ask CEOs and division managers questions.  I think it brings us together as a company and makes employees feel that they have a voice.  The downside is that some employees leak information about upcoming products and services.  These leakers are fired as soon as they are identified, of course, but the leaks are a known issue with our transparency.

Knowledge transfer is the most important part of sharing internal information.  Googlers teach other googlers through technical talks, postmortem talks after failures, coding tutorials for our different internal projects and lots of informal presentations.  I recently learned how to use a new integration test framework at Google.  As soon as I finally got it all working, I was told: "Good, now you can teach us all how to use it."  I strongly believe that the amount of engineer-to-engineer teaching we have at Google is what makes us an intellectual powerhouse.

And even when the teaching is not formal, we have a lot of interoffice communication.  We have online forums (implemented with Google Groups) where we can post any question we have about any tool or project and have it answered quickly by other people who are using the tool.  For internal tools, the person who answers your question will probably be the engineer who wrote the tool.  The great thing about forums is that the information and answers are online forever.

In addition to forums, we use email, chat, and video chat to ask experts how to get things working.  I'm working on a project now with a guy two states over from me.  I've sent and received code reviews from people all over the world.  This interoffice teaching and collaboration is incredibly powerful.

Knowledge transfer is more of a tool to make employees efficient, but I also think it makes Google a great place to work.  It makes our jobs easier, and we can build relationships with people we haven't met face-to-face.

Others
Now that I've detailed 6 excellent ways to make your company a great place to work, I'm going to list a few more that are almost as important:

• 20% Projects. Googlers will sometimes spend 20% of their time on a side project and 80% on their main task.  This is a win-win for companies because employees are happy to work on something that they think is cool, and the finished project is usually something that improves Google.
• Reuse Code. A lot of google code is stored in one repository that can be seen on internal websites.  This helps us not reinvent the wheel if we're writing something that's already been done before; and it helps to see different examples of how to implement something, when we're doing something slightly new.
• Make telecommuting work. Because of my laptop, our excellent videoconferencing software and all our internal websites, I can stay home all day, for days and still be 95% as effective as I am at work.  This also helps with knowledge transfer; being able to attend meetings from home or on the road.
• Dogfood.  If you have consumer products, use them.  It allows you to find bugs and poorly implemented features before the customer gets them.  It also saves money.  And if your products are cool, your employees will enjoy this.
• Use free and open source software (FOSS) when possible.  Free means you save money, open source means you can hack it if you don't like it.  This makes your company a better place to work because the money can go elsewhere, like to your employees! The one caveat: only do this when it's right.  Don't use crappy tools just because they are free and open source.

I hope that more companies follow Google's methods.  Who knows, making our workplaces great might improve the human condition!

This article was not requested, sponsored, or endorsed by Google, Inc.