The Unwritten Rules of IT — Starting With What Nobody Tells You
BLUF: The Unwritten Rules of IT
You don't need to know everything. You need to keep learning, stay humble, and understand that technology rarely fails on its own — humans fail it, consistently, creatively, and at the worst possible time.
The rules break down into three unavoidable truths:
On entering the field: Start before you're ready. Experience beats preparation every time, "I don't know" is a legitimate answer, and the only skill with guaranteed shelf life is the ability to learn new ones.
On the humans: Almost every outage has a human at the root of it — one who changed something, skipped the test, refused to document, or scheduled the Friday deployment with misplaced confidence. The systems are largely fine. The people are the variable.
On AI: It's a fast, tireless, extremely confident tool that doesn't know what it doesn't know, will fabricate solutions that look exactly like real ones, and is already being handed credentials it shouldn't have by someone in your organization who was just trying to be helpful. Supervise it accordingly.
The through-line: IT is not a technology problem with occasional human interference. It is a human problem that happens to run on technology. Every rule in this document — from "never deploy on a Friday" to "it's always DNS" — is ultimately a rule about people: how they communicate, cut corners, assume, avoid, and occasionally, heroically, hold everything together with a 2am fix that nobody will ever fully understand.
0a. You don't need to know everything to get started. The single biggest reason talented people never enter IT is that they're waiting until they feel ready. That feeling never comes. Not on day one, not after your A+, not after your CCNA, not after twenty years. The people who look like they know everything have simply had more opportunities to figure things out under pressure — which is just experience wearing a confident expression.
0b. Experience is the certification that actually matters. You can memorize every answer on a practice exam and still freeze when a real problem lands in front of you. You can also fumble through your first year, break things, fix things, and emerge knowing more than any exam ever tested. The field rewards people who kept showing up. It is far less impressed by people who studied perfectly and then waited.
0c. "I don't know — let me find out" is a complete and professional answer. Users will assume you know everything. Managers will assume you know everything. The new hire shadowing you will assume you know everything. None of them know everything either. The most dangerous IT professional isn't the one who admits ignorance — it's the one who can't. Confidently incorrect is how outages happen.
0d. An inch thick and a mile long is the job. IT is not about mastering one thing completely. It's about knowing enough about everything to ask the right questions, recognize the right patterns, and know which expert to call. Your value isn't depth in isolation — it's the ability to connect dots across an impossibly wide landscape. Nobody hired you to know everything. They hired you to figure things out.
0e. The ability to learn is the only skill that doesn't expire. Every specific technology you know today will eventually be legacy, deprecated, or replaced by something a 24-year-old built last Tuesday. The sysadmin who thrives for thirty years isn't the one who mastered Windows NT — it's the one who was curious enough to keep learning when everything changed. Comfort with not-yet-knowing is the most durable skill in the field.
And Then You Meet the Humans
1. Never deploy on a Friday. Technically this is about timing, but really it's about humans. Specifically, the human who schedules a Friday deployment because "it'll only take five minutes" — the same human who has never, in recorded history, completed anything in five minutes. The deployment takes four hours. You know this. They know this. Nobody says it out loud.
2. "I didn't change anything." They changed something. The full archaeology of this statement is remarkable. Stage one: "I didn't change anything." Stage two: "Well, I updated one thing, but that shouldn't matter." Stage three: "Okay, I may have also restarted the service." Stage four: "...and installed some software." Stage five: quiet, prolonged eye contact as the full picture emerges. You will reach stage five every single time. Budget for it.
3. The 2am fix makes zero sense on Monday morning. The human who wrote the 2am fix was you, technically, but they were operating under conditions — panic, caffeine, a user screaming on Slack — that no longer exist. The real problem is the other human: the one who created the crisis at 2am because they "just needed to make one quick change before a big presentation" and didn't mention this to anyone until something exploded. That human went to bed at 9pm. They slept great.
4. Never say the Q word ("It's quiet"). The Q word is never said in isolation. It's always said in response to a human. Usually a manager wandering through asking how things are going, clearly hoping for reassurance. You, desperate to provide it, say "pretty quiet actually." The manager nods, satisfied, and leaves. Three things break simultaneously. The manager will not return to witness this. They are at lunch.
5. If it's working and nobody knows why — don't touch it. The dangerous human here is the enthusiastic new hire who wants to "clean things up" and "apply best practices." They are not wrong that it's messy. They are catastrophically wrong that this matters. The mess is the solution. The mess accumulated organically around an original problem like scar tissue. Disturbing it doesn't clean things up — it just moves the chaos somewhere less predictable.
6. The backup you never tested is the one you'll need. There are two humans in this story. The first said "we should test the backups regularly" and was told "we don't have time for that right now." The second is the executive calling at midnight asking why the data is gone. These two humans have never been in the same meeting about backup testing. They will, however, be in the same meeting about the incident report. It will be a long meeting.
7. Document it now or hate yourself in six months. Documentation fails because of humans at every stage. The human who built it didn't document because they were too busy. The human who inherited it didn't document because they were trying to understand it. The human who needs it now is opening a ticket marked URGENT while simultaneously asking a question that is answered nowhere. Somewhere, the original human has left the company and is living their best life, completely unreachable, on a beach.
8. Restarting fixes 80% of problems. Admitting it fixes 0% of credibility. The real issue is the human on the other end of the ticket who will, if told "we restarted it," immediately ask "why did it need restarting?" — a question with a real answer that will take forty-five minutes to explain and will not satisfy them. So instead you say "we identified and resolved an instability in the service layer," which is both technically accurate and completely unanswerable as a follow-up. Humans respond well to words that sound like effort. This is not cynicism. This is professional communication.
9. DNS. It's always DNS. The human problem with DNS is that explaining why it's DNS takes longer than just fixing it. So you fix it. The human asks what was wrong. You say "DNS issue." They nod as though this means something to them. It does not mean something to them. Next month they will make a change that breaks DNS again and tell you, with complete sincerity, that they didn't change anything.
10. The person who never raises a ticket has the biggest problem. This human has a workaround. The workaround involves copy-pasting something into Notepad, waiting 30 seconds, and then copying it back. They do this seventeen times a day. They have never mentioned it to anyone because "it's fine, I've got a system." The system took them four minutes to develop eighteen months ago and has since consumed approximately 140 hours of their life. They are fiercely protective of it. When you fix the underlying problem, they will be briefly annoyed that their system no longer works.
And Then AI Showed Up
11. AI is confidently wrong the way only a very fast, very agreeable intern can be. The danger isn't that AI doesn't know something — it's that it doesn't know that it doesn't know something, and will explain its wrongness to you in beautifully structured paragraphs with appropriate technical terminology. A human who doesn't know something will usually hesitate. AI will not hesitate. It will cite the hesitation in APA format if you ask nicely.
12. "The AI said so" is not a root cause. You will get tickets caused by someone copy-pasting AI-generated code directly into production without reading it. The code will be almost correct. "Almost" is doing a lot of work in that sentence. When you ask why they didn't test it first, they will explain that the AI seemed very sure. The AI is always very sure. That is not the same thing as being right.
13. Prompt garbage in, prompt garbage out. AI does not rescue bad requirements — it accelerates them. If you ask a vague question, you will receive a confident, thorough, beautifully formatted answer to a slightly different question than the one you meant to ask. This is not the AI's fault. This is Rule 2 in a new costume: the user didn't change anything. They just asked the wrong thing.
14. AI will hallucinate a library, a function, and three Stack Overflow posts that don't exist. It will do this calmly, with working-looking syntax and plausible version numbers. You will spend forty minutes trying to install a package that has never existed before you realize what happened. The AI, when informed of this, will apologize sincerely and suggest a different package that also doesn't exist. This is not malice. This is a very sophisticated form of making things up, which somehow makes it worse.
15. Someone has already given the AI admin credentials. You just don't know who yet. There is a human in your organization who, trying to be helpful, fed a system prompt containing environment variables, API keys, or database connection strings into a third-party AI tool. They did this because the tool asked for "context" and they wanted to be thorough. This happened. It may be happening right now. Check your logs. Check your logs again. Consider crying.
16. AI doesn't replace the need to understand the thing — it replaces the excuse not to. For twenty years the defense was "this is too complicated to document properly." AI can now draft your runbooks, summarize your architecture, and explain your legacy codebase in plain English in about four minutes. The humans who refused to document will now have to explain why they also refuse to have AI document it. This conversation will be awkward. Enjoy it.
17. The AI audit trail is "we asked it and it said yes." Traditional systems fail with logs, error codes, stack traces, and timestamps. AI fails with vibes. Something went wrong and the AI was involved, but reconstructing exactly what was asked, what was returned, and what the human did with that information is an exercise in archaeology with no artifacts. If you're deploying AI in any serious workflow, logging the inputs and outputs isn't optional. It's the only thing standing between you and an incident report that says "the AI told us to."
18. AI is the new junior developer who works at superhuman speed and has read every Stack Overflow post ever written but has never actually run anything in production. This is not an insult — junior developers are valuable, and so is AI. But you wouldn't give a junior developer unsupervised access to the production database on their first day, hand them the deployment keys, and go to lunch. The same logic applies here, just with higher throughput and more confident typing.
19. "We'll just use AI for that" is the new "we'll fix it in post." It sounds like a solution. It has the shape of a solution. It is a placeholder wearing a solution's clothes. Slotting AI into a broken workflow doesn't fix the workflow — it automates the broken parts at scale, adds a layer of opacity to the failures, and ensures that when something goes wrong, nobody is entirely sure which part of the system decided to do the thing that caused the problem.
20. AI changes every six months. The humans using it do not. New model, new capabilities, new limitations, new hallucination patterns, new things it's surprisingly good at, new things it confidently ruins. You will just have finished training your team on how to use the current version responsibly when a new one arrives that behaves differently in ways nobody has fully mapped yet. The humans will continue to use it exactly as they used the previous version, applying lessons that no longer apply, missing capabilities that now exist, and filing tickets about behavior that changed four months ago.
The uncomfortable truth no one will print:
The infrastructure almost never fails on its own. Servers don't get bored and decide to misbehave. Networks don't harbor resentment. Code doesn't act out of spite — though it occasionally feels that way at 2am.
Almost every outage, every mystery, every "how did this even happen" post-mortem traces back to a human decision. A skipped test. An undocumented change. A timeline someone invented and then treated as a physical law. A meeting where the words "do we have time to do this properly?" were asked and answered incorrectly.
No comments:
Post a Comment