Podcast: Tales from the NOCside
Whether you’ve worked in ITOps or in the NOC, you probably have some interesting stories about your time there. From crashes to errors and everything else in between during graveyard shifts, there’s most likely an abundance of spooky tales from you and your colleagues. In the latest episode of our That’s great IT technology podcast, “Tales from the NOC side,” host Craig Ferrera talks horror stories with several BigPanda employees. In parts one and two, they explore their time working in the NOC or in ITOps and share their most spooky tales from the NOCside.
Here’s a sample of a few scary stories from the NOCside to give you a taste of the horror the podcast will cover:
When trying to fix something else…
“I disabled the network connector on a remote machine one time when I was trying to restart a service. And I’ve never found myself panicking so fast than when the remote terminal I was in just went blank all of a sudden. It was good fun. Yeah, that was, again, in the middle of the night, and you didn’t really want to wake somebody up to tell them you nuked a network adapter on a machine halfway across the country. So we had to go do a little digging. And thankfully, there was a Dell box and we were able to jack in and restart the network adapter from there, which required a not insignificant amount of googling to figure out how to make that happen. So pro tip: don’t disable the network adapter on a machine that you remoted in the middle of the night. Fixing that took longer than it should have been to accomplish the task.”
On people you work with…
“There’s people going through these horrible things with you, right? They might not be the same company, you may never meet them, but you’re up at 3am having customers just send a [lot] at you and you’re trying to just mediate between some person in R&D who’s never talked to a customer in their life. Or maybe someone just cut through a cable and you have a data center that’s falling over. People are going through this stuff with you. You’re not alone. This is an industry that is like the backbone of just about everything in the world. And the people that are working with it are really, really regular people. They’re cool people that are just trying to either start a career, or they’re doing a solid for their team and working some weird hours on these shifts and stuff.”
On making food during an overnight shift…
“I think one of the more inventive things I saw was, we figured if you could crank the toaster on high and hold down the lever, button, whatever we want to call it, you could force it to stay hot. And we would actually crisp up corn dogs on top of the toaster slot because we had no kitchen. That and a whole lot of black coffee were really what fueled the team. So we didn’t eat super well, but we didn’t starve.”
Best advice to new IT professionals?
“I would encourage the people that are out there doing coverage, working on it today, to learn from those outages. It can [be] fuel to further your career and provide better advice. But share that advice with your peers. You have to talk to each other, and make sure you’re leveling up one another.”
We hope these stories didn’t send too many shivers down your spine. If you enjoyed them, there’s more where that came from! Check out our latest episode today for more tales on the NOCside, and keep listening for our other informative That’s great IT podcast episodes to stay up to date on all things AIOps and BigPanda.
