The End of White Collar Jobs

There have been a great number of articles of late talking about how manual factory labor will be going away in the relentless march of robots, drones and autonomous vehicles. While these are all technologies that will have a huge impact upon manufacturing, it is tempting to think if you consider yourself a knowledge work or professional that the job that you do is safe. The problem is that this is simply not true.

If I was to counsel a young person today looking at where they want to explore as a career, it would actually be to avoid computer programming or data science, as well as marketing or management. I write this as a programmer with thirty five years of experience. Programmers, especially, are doomed.

Most professions involve both specialization and assuming responsibility. To be a doctor, for instance, you learn the basics of anatomy and physiology, biochemistry, pharmecology and other disciplines. You go back periodically and learn about new advances in the field, but the core knowledge that you learned in your twenties and early thirties generally doesn't change radically. The same can be said for being a lawyer, being a dentist, being an engineer.

There is, however, one profession where this isn't true: programming. Yes, there's a core area that does remain the same - algorithms may advance some, but there are few truly ground-breaking algorithms any more, knowledge of data types and storage and functional and declarative programming principals are broadly similar regardless of what era you're talking about.

What's different, however, is that programmers automate their knowledge routinely. That's what a program is. It's someone figuring out how to solve a certain problem, converting that into code, then incorporating that code into libraries that other programmers can use. That is almost the textbook definition of how a compiler works.

It used to be, not all that long ago, that old school programmers would regularly write machine language code to optimize performance for given platforms, but then other programmers took that knowledge and, well, automated it. Now, most compilers can reduce high level code to byte level code far faster and more efficiently than a human being can.

Not that long ago, people would create web pages by hand. Now, most web applications are built around platforms, usually utilizing templates that were written by someone else. The next stage has been happening for a while - you specify the kind of web site or web application you want, walk through a WYIWYG screen to drag and drop containers and fields, then press BUILD. Complex sites can be produced this way, and a person doing nothing but building websites is likely to go broke unless they happen to be exceptional designers. The next stage, where AI systems are able to reconfigure their systems to build websites dynamically in real time, is already here as well, and people looking to get a solid website up quickly at minimal cost are already killing the market for web developers by utilizing these.

The same thing applies for mobile applications. It applies to data analytics software. For a while there was a lot of chest beating by data analysts about how they were on top of the software stack, yet analytics software is moving beyond the ability to drag and drop pipelines for processing, it is now analyzing and identifying tools for both achieving a certain analytic result and for visualization of that data. These are AIs at work in the wild, automating knowledge and even automating the automation of that knowledge.

You see this increasingly as smaller and smaller teams of people are able to handle larger and larger software projects. Twenty years ago, you might need a team of a hundred developers to create a project like the Affordable Care Act exchange. Yet today, it could probably be done by five to ten people. Tomorrow, you can set up an exchange or similar type of data hub in a matter of hours with one person.

The list of software languages in use today are daunting - a few thousand at last count. However, 95% of programming is done in one of maybe five imperative languages - Java, Javascript, C#, C++ and PHP. Data messaging formats are down to maybe five as well - XML, JSON, Text (such as CSV), RDF and arguably ZIP. Images, Video, Audio, each of these are down to a handful of common permutations. Each time you get standardization, you reduce the need for programmers and reduce the complexity of that code.

This has had the subtle effect of hollowing out the middle of the developer arena. You either need commodity integration people to connect systems together (and that's fading) or you need very high end software developers and machine learning specialists, of which there are precious few. This cutting edge exists because we haven't standardized the knowledge yet that they embody, but that day is fast approaching. As it does, even those AI specialists will find themselves looking for jobs, because they have effectively written the software that replaces them.

"But ... but ... there are so many problem areas that need to be solved, yet. Right?"

I hear this refrain frequently, but the evidence doesn't bear that out. Most software being written today handles one of the hardest, most human activities out there - the process of dynamic categorization. That's what most AIs are. They are categorization machines that partition a particular set of data into meaningful sets. It turns out that this classification problem accounts for a huge percentage of "human" activity, from artificial vision and speech interpreters to the analysis of stocks to the building of models for business or government activities. This includes the process of managing corporations, of designing ad campaigns, of determining (and setting) social media trends. It affects the ability to recognize and diagnose diseases and conditions in the body, to determine gene sequences that control specific actions, that make autonomous cars and trucks work.

It also involves determining those patterns for making better software. The real problem with most software today is simply that there are still too many standards, which means potential permutations of applications based upon data format differences, but this isn't going to last all that much longer - we're very nearly to the point where the ability to recognize model and match data structures can be handled cleanly by an AI. Do this, and you eliminate the need for integration specialists, which, by some estimates, make up over half of all developers currently working today. This will happen in the next five years.

"Creative jobs will remain though, right?"

Take a look at the end-credits of any game, any movie, most TV shows, and you will see page after page with modelers, skinners, lighting specialists, visual effects specialists, matte artists and so forth. These are creative technical specialists, mostly devoted to activities like making water look natural, making hair move smoothly, making every potential nuance happen.

Most of them are heavily reliant upon AIs to do the bulk of the heavy lifting, and every movie, every game is moving to the day where the AI is doing nearly everything with the exception of setting the input parameters. That's still maybe twenty years out, enough for one more cycle of jobs or so before those jobs become redundant. That's because each of those activities are going into specialized databases, each of the complex rendering algorithms are stored and parameterized, and current hardware functions are increasingly becoming tomorrow's software and vice versa.

Writing is another area where we would like to think ourselves superior, but that barrier's falling fast. Chatbots and AI journalists are already being installed in media organizations handling the generation of natural sounding articles that can be gleaned from various data (), while Google has been working on an AI that reads romance novels and from them writes it's own. Indeed, systems that analyze writing styles of other writers have been around for a few years now, and increasingly AIs are able to apply that knowledge to determine in a slush pile what constitutes good writing. And some are even winning literary prizes.

Again, it is worth noting that these AIs are not weird aliens - they are simply the reflection of our increasing accurate analysis about what constitutes creative and innovative work. These systems are not "aware" in any meaningful sense, but rather represent the encoding of knowledge about knowledge in a form that a computer can then use to perform specific tasks.

"So, what about managing, marketing, recruiting or sales? Surely these are safe."

Management has already been decimated by older generation AIs, and is about to undergo yet another round. Marketing is now mostly managing social media. The Ghost of Sales Present is Salesforce, and the Ghost of Sales Future is Amazon. The only reason that most of these particular skills haven't been fully automated is because these positions are typically controlled by the ones making the buying decisions in companies, not because there is any innate superiority that managers have with regard to processing or making decisions within a company.

Yet the proliferation of startups, where you have a few kids in a garret building the next major company shows that over times, these microcompanies with minimal staffs are the future of business. This will become more pronounced as 3D printing becomes increasingly mainstream. The barrier to entry for most companies drops dramatically in that scenario, as you need a much smaller capital investment to provide a good or service.

The same thing applies to those in the public sphere. Again - a government is a machine, one that reprograms itself constantly by the use of laws, amendments and executive orders. Right now Federal and State governments employ armies of analysts, yet already the "data science" revolution is making many of them redundant as it is doing elsewhere. This will be a slower process, but it is taking place nonetheless. It's unlikely that elected officials will be rendered redundant any time (short of a complete collapse of government) but this situation has less to do with the ability of AIs to effectively run much of the government as it does with the understandable reluctance to let them do so, because of the loss of political power in the process.

It should be pointed out that this is the emerging reality of the Fourth Industrial Age. Not all jobs are going away, but certainly 30% or more will in the next twenty years, perhaps as much as 60% by 2050 (only thirty three years away, for what it's worth - thirty three years ago was 1984). While robots will account for some of those losses, AIs will likely account for the bulk of those losses, and will hit the white-collar professions - the professional, technical and management classes - the hardest.

Acknowledging this, and preparing for it, is essential. There will be many things that people will be able to do once AIs reach this level of complexity, but this will be a highly disruptive period as we try to develop a new form of economics to provide an alternative to a labor-based compensation system. There are others out there (I hope to cover a few of them in a subsequent article) but the reality is that without preparation, the professional middle class could face a period of extreme turmoil, and possibly extinction, in the decades ahead.