When the "Obvious" Slaps You in the Face!

Ryan King, CIO, Power Solutions International, Ryan is a passionate and progressive Information Technology executive with 20 years of experience in diverse, global  IT environments. Award winn...  More >>

More posts by this contributor:

• Reclaiming Control of ERP Implementations

Share your experiences through CIOReview

It is like a horror movie we have all seen where for some strange reason they are dumb and they go to the basement or they check that closet. Or someone gets robbed at their home and the police are reviewing the crime scene and the homeowner had an alarm system and they never activated it because they say “I just thought it would never happen to me!” Or here is the classic, “you lock the front door and the back door is wide open and you wonder to yourself how could that happen?”

It is like that in companies and securing your environment. We overlook the obvious! Or we do the dumbest things in our quest to secure our companies valuable intellectual property. From my role as a CIO of a manufacturing company in Chicago and my 20+ years of experience in the IT field, I have seen it all and still nothing surprises me even to this day. I have to always stand back from the controlled chaos of every day work life and review the obvious and make sure I am not overlooking anything that can bite me later. It is not about spending the most money, it is about good old fashion review and double check your work. I call it looking for the “Free Stuff”

A little good old fashion hands on review never hurt anyone - Here are some basic items to review for that “obvious” “aha moment” or the “Free Stuff” that I feel if always reviewed is your first line of defense.

In post incident review, it is often discovered that critical systems were left unpatched either since installation or over an extended period of time.

• Are my servers/workstations patched for all critical patches?

In a conversation I had with “Special Agent Aaron Van Hoff” on system patching who works “cyber matters” in the FBI Chicago field office and has 12 years with the FBI on investigating incidents where a company or persons are compromised by a technology hack, with his permission and I quote:  

“Keeping systems up-to-date with the latest patches and hotfixes is a baseline principle of IT security, but unfortunately at times, it is a glaring oversight that we see in some victim companies that have been hacked. In post incident review, it is often discovered that critical systems were left unpatched either since installation or over an extended period of time. With the window increasingly shrinking between when vulnerabilities are discovered and when malicious actors develop exploits for those known vulnerabilities, it is crucial for IT security professionals to work quickly within that window in order to safeguard their systems from known threats.”

“Although staying current with system patches and updates is only one of many facets to a healthy defense in-depth posture for system security, the practice of ‘set it and forget it’ is more than likely exposing your company or organization to risk that is unnecessary. An idea to keep in mind is that the longer you go without patching your systems, the more vulnerability you have on your networks. This is a needless risk that can be mitigated by a well planned and executed patch management process.”

• Is my anti-virus up to date and working? Have I reviewed the logs? Am I getting any automated emails that most vendors give for free? Take advantage of the “FREE STUFF”

• Am I reviewing my server logs looking for log entries that pose an alarm and further investigate them thoroughly? Sometimes the automated programs like Solar Winds miss items like a cookie crumble or morsel, which will lead to a larger event. They say you are hacked six to nine months prior to the “Big Event” that takes place.

• Do I have a good backup of my critical data – in the case you are attacked are your backups working? Review the logs daily or at least weekly?

• The software is saying I have a good backup, “but” have you tested it? Test your backups with a restore at least monthly or quarterly. Like I tell my staff all the time - the one thing that will get you fired is not having backups and making sure they work! OBVIOUS!!!

• Document critical software and hardware and understand it thoroughly and in the case of a disaster your downtime will be minimal. Nothing I hate more in IT environment is “tribal knowledge” of one person who seems to have the answers and when they go on vacation or quit “you are stuck” and have to try under stress to get a hold of them and they magically get it fixed in five minutes! Document and then document some more. Then after you have documented – test that documentation on someone who has never seen or tried the software or process and have them truly test how good the documentation is. This is an ongoing process, in the end it will pay for itself in multiples of 100s.

► For example, I just had a network engineer leave for an opportunity closer to his home and because we documented all of his responsibilities and tested them, his departure had a very minimal impact on our department because we knew and understood what he did.

• Here is a BIG ONE – PAY ATTENTION FOR THIS ONE: “Educate your employees of their role in security of the companies data infrastructure”

► I had an experience not too long ago where a user came to the IT department and said he went to a supplier website from Japan and when he clicked on a link from the website, his whole machine started to do strange things. It was changing the items on his desktop to different extensions. What he had contracted was ransomware and it was encrypting his entire machine and would have spread to the entire network, however; his quick thinking said “remove it from the network” he unplugged his network cable. He was somewhat computer savvy and knew that he needed to do something and fast. We were able to redo his laptop and get him back running rather quickly. Damage very minimal.

I love this quote and it applies to my discussion in my article: “By small and simple things are great things brought to pass” – Alma  

Paying attention to the little things makes all the difference!

Do not let “Obvious” slap you in the face, because it just may cost you your job!