Guide

UNIX/Linux Privileged Access Best Practices

PIM, PAM and PUM have different meanings, and interpretations, to different people. For the most part the concepts around these three far-ranging topics intersect, and for the most part we are talking about the same thing. PIM, privileged identity management; PUM, privileged user management; and PAM, privileged account management OR privileged access management.

All three of these acronyms revolve around a few simple concepts: who can get to a server, virtual machine or system image, how they can get into the operating system and what they can do when they get there.


MANAGE ACCOUNTS

If we want to talk about access control and privilege manage- ment, it has to start with the accounts that are set up to allow use of a system. Creating accounts is, in general, a pretty easy task. Disabling or removing accounts that are no longer re- quired has been a challenge for enterprises for years, and will continue to be a challenge. The risk introduced by leaving accounts active that should not be has played out numerous times in various ways, and a great historic example is a logic bomb set to do enormous damage by an employee who was notified that he was terminated, but allowed to continue working that day, with his full administrative access he had needed to do his job as an administrator. That is an example of only a ‘people’ account.

Linux and legacy UNIX systems require an account, somewhere, for people to use to access the system. Typically, Linux/UNIX systems are only accessed directly by people when they are performing administrative activity on the OS, and critical applications or data. ‘People’ are only one class of account on these system that are required.

Although managing ‘people’ accounts is what most people think of when they think of account management, leaving system and functional/application accounts out of the planning and process for account management leaves a gap, and introduces risk to the enterprise.

 

IMPLEMENT LEAST PRIVILEGE 

What an account is allowed to do once it has logged in is arguably more important that controlling who logs in. Imag- ine that everyone in the enterprise could log in to a system, but they could only do what is appropriate for their job; many people would not be allowed to do anything at all. The risk of allowing access is far less important than controlling what people can do.

Consider the following:

  • Does every system administrator need to be logged in as ‘root’?
  • Do they need to be able to run every command that ‘root’ has privilege to run?
  • Database administrators - Do they need root access, or should they be prevented from running system commands that are not a part of their job?
  • What about accounts used for app-to-app communications?
  • Does one computer talking to another require access to every command that could be run - or is there a small set of commands that these automated processes require?



MANAGING PASSWORDS

Although strong authentication is recommended for the ‘people’ to log in with, there are times and places where passwords still are, or must be, used. Where passwords exist they need to be managed, and most people have some experience with this, whether they are technical or not. Managing passwords includes all of the basics: require the password to be changed periodically, do not allow the reuse of old passwords, require that they be of some level of complexity. These are all the basics of password management.

Managing passwords needs to mean ALL the passwords: users, application/functional/service accounts, and the all- powerful root.


ELIMINATE USE OF SHARED ACCOUNTS

This is especially important in respect to the ‘people’ that access a system. Two administrators of a system should not share an account; this provides traceability and accountability for actions performed. And, not just administrators of the OS, but anyone who accesses a system to work on an application or data. For some organizations that could become a lot of accounts to manage across a lot of servers, but the goal of not having any two people share an account for access to a system should drive the planning and process of the preceding requirements for implementing PIM/ PAM/PUM.

There are some accounts that just do not go away. Every Linux/UNIX system has a root account, and many applications have an account set up to associate its process with and limit the application’s privilege on the system; and sometimes an OS application or data administrator will need to perform some activities with the privilege of the OS, application, or data service.

Implement a system where a person can execute commands as another account, without having to login as, or become, that identity. The traditional Linux/UNIX ‘sudo’ command does this, allowing a user to elevate privilege, or change identity, and run a command with the privilege or the identity they have called on. This type of model eliminates the need for a shared account, and it provides the required traceability and accountability for activities performed on the system.

However, people are not the only account, and there is at least one account that systems administrators may need to use -- root -- and there may be others, depending on what the system is doing and how it is used. When nothing else will substitute for logging in as the ‘root’ user, a model that provides the ability to attribute the use of the account to a person, providing accountability for its use, is required.


CONTROL ACCESS

Not every account is equal; not every account available to a system needs to be able to access it over the network. If you have properly implemented privilege control, then nobody needs to log in as root when they SSH to the system to perform maintenance. But, root will still need to be able to log in at the console, for that rare occasion that something has gone terribly wrong and an administrator needs to truck in to the datacenter and stand at the keyboard.

Not every account needs to access every server from every computer on the network. Those automated authentications, from app-to-app or system-to-system, don’t need to be available from everywhere. In fact, almost by definition, they should only be allowed from the system that is performing the task, so the target should only allow that account, connecting that way, and doing that stuff, to do it from a defined location.

Our administrators should have some constraints as well. An administrator in a secured location, on a secured network and a secured system, may be allowed to do much more than if they log in from their laptop over the VPN from a hotel room.

Access control should be granular, and the context of the access should feed in to the authorization decision. Where, when and how an account accesses a system should inform the authorization decision about what that account can do when it gets there.
 

LOG, LOG, LOG, LOG

Log changes. Log access. Log privilege. Log failures. Logging is just a good idea, and most security staff, compliance staff, and auditors get that, but it also take time and resources to store and evaluate all of those messages, which can be 100s of 1000s each day.

You need to be able to report on who has access to what and when. This is required by many forms of regulation, from the payment card industry’s PCI standard, to business rules like SOX, to new laws dealing with personal information and privacy, like GDPR. Reporting on the rules in place, however, is not an audit or log trail.

Recording/logging any changes to accounts, changes to entitlement, additions/deletions or other activity that pertains to the rules affecting who can access what from where and when, is a logging trail. All changes to accounts, access levels and other rules should be recorded so that the administrator that changed the rules can be identified.

Logging access is almost a no-brainer. Log who logs in. Know which account is being used, how it authenticates, where it comes from and how it connects. This is pretty much typical syslog (but, to note, this is considered the VERBOSE level of logging in an out-of- the-box OpenSSH installation).

Whenever an account needs to execute commands with a different level of privilege, that should be recorded. This goes back to the who does what, and when. This is beyond important, for a number of reasons. Privileged operations have the capacity to potentially access any data available to the system and have the capacity to destroy or damage the ability of the system to perform its functions. (i.e. this can take your enterprise down)! There are two things here that should be considered. First, you need to know who did a bad thing, if a bad thing happens. Second, people who are being watched (and know they are being watched) tend to behave better.

Lastly, and arguable most importantly, log failures! If an account performs automatic-authentication that uses an SSH-key for authentication, and suddenly that account is trying to access with the key from a host that it shouldn’t, that is a problem you want to know about. Of course, it would be best to block the access, and then send a notification. This type, and any type, of failure due to rules- in-place will tell you one of two things: your rules are wrong, or something is happening that shouldn’t.
 

MANAGE AND RECORD PRIVILEGE ACTIVITY

Knowing who access what host, when, from where, and how, are all good and fine. Knowing that that account then elevated the granted privilege to execute additional commands is good to know, too. But sometimes more is better.

If a bad thing happens, it is great to be able to analyse the data and figure out who did what. However, logging of activity at the key- stroke level allows you to see exactly what they did. When bad things happen -- the details matter.

Managing who has the right to what, where they can do it, and when, is a powerful tool. Administrators often need to run commands with a privilege other than those assigned to their own account. This could be application or database administrators that must perform maintenance or configuration operations, or it could be a system administrator. Generally, in the past, these administrators all wanted to be granted total control; root access. Generally speaking, however, this just is not true. Root access is a shortcut, and an easy way out. It is guaranteed to prevent privilege management from becoming an impediment to productivity. It is also a recipe for disaster.

There are two ways that excessive privilege can cause tremendous problems for an organization. First, the abuse of the account by the trusted party. We can think of episodes in recent history like the Snowden case. An administrator gone rogue who abuses their privilege and access to the detriment of the enterprise. Even today, after examples like this, it is hard for many organizations to focus skepticism on what often must be the most trusted people running technology. If you cannot trust the people that you have hired, it is just depressing, and tightly managing them may seem to be a blow to morale; nobody wants to feel that they are untrusted or watched.

However, even with the most trustworthy staff in the world, the account itself, the representation of the person on an operating system, simply cannot and should not be trusted in this day and age.

Administrator credentials, not data, are the crown jewels that are sought by attackers. Once administrative privilege on systems is usurped all else falls. Attackers do not target the data, not first at least; privileged accounts are the gateway to everything: email, data, databases, files, system configurations, applications. Everything becomes exposed once the privileged accounts are breached.

To this end, the insider ‘accounts’ must be treated with suspicion. Not because of the people who they are assigned to, but because they are the most valuable asset to those who would attack us.


ALERT AND NOTIFY

This is just the natural adjunct to a well implemented logging and audit system.

Not every minor violation needs to wake up the security administrator at 2am, but there certainly are those that should.

  • Is access from an account attempted from a host where it should not originate from?
  • Is an SSH key being used from a place where it should not?
  • Is an administrator’s account attempting to run privileged commands that are not granted to it?

First, a well implemented system with well-defined rules should stop the bad behavior. After all, if we can look at logs and know that it should not be happening, that also means that we could build rules to prevent it from happening. The difference is do we detect and alert proactively or reactively. This is the concept of detection through enforcement. The same rules that could be written to detect bad behavior should be used to prevent bad behavior. Defence and detection are baked in to a single effort.


CENTRALIE, UNIFY, AND BE EFFICIENT

Almost all of these valuable features are available natively in Linux and Unix.

  • SSH allows for the local configuration to control who can access, from where, how and what can be done once connected.
  • Accounts can be managed at the local system with ‘adduser’ and ‘userdel’, and there are various techniques for leaving an account in place but blocking access.
  • Privilege can be managed using ‘sudo’ and local configuration files.
  • Everything that can be done, can be done in the setup and configuration of the local system.

Tightly coupling access control and privilege management has distinct advantages over trying to manage them separately, and in the Linux/UNIX world, access control should be tightly coupled with account control. For that, you need a tool that allows you to centrally manage all three key aspects of access and privilege: account management, access control and privilege management.

This operationally, however, presents an enormous challenge.

Small Infrastructures: For one system and only a few users, you will be managing those few accounts, the SSH configuration and access rules, the sudo permissions, and more. Each additional server amplifies the effort to maintain the proper configuration exponentially. For one or a few servers, it might be a tedious but achievable challenge. The 451 Group has running metrics updated annually that tell management when automation tools become useful and very cost effective. As of 2018 a system administrator should be able to administer with scripts and manual processes no more than 500 concurrently running operating systems.

Mixed Infrastructures: Linux and the various UNIX like operating systems like IBM AIX, Oracle Solaris, and HPE’s HP-UX all manipulate users, groups, permissions, passwords, SUDO and SSH keys. Unfortunately, due to historical reasons going back decades each platform vendor made different choices in how operating system security is configured and maintained. For example

  • Each platform has a different way to define password length and complexity rules, using unique admin tools
  • Solaris stores SSH keys in a completely different technical format from all other platforms.

Attempting to maintain these differences with scripts or configuration management can be problematic, and platforms continue to have logarithmic or exponential deployment issues (e.g. distribution of SUDO config files, or user SSH keys). Scripting and change management tools as a result spend a lot of time doing basic file comparisons on each and every node to check if anything has changed, wasting LOTS of CPU and network traffic cycles trying to keep everything in sync.

These operational problems do not go away if you currently have a “clean” single platform monoculture of systems, in a mergers and acquisitions business culture worldwide, you just do not know what operating systems you need to control, and which technical platforms you will inherit from that next acquisition.

BoKS ServerControl is designed to operate cross-platform, maintaining security in the way each operating system expects, and administered from a central control point. In addition, the security administrator does not need to care if some of the systems are AIX, Linux, PowerLinux, or Solaris, our infrastructure takes care of the details.

Large Infrastructures: For hundreds or thousands of servers, and tens or hundreds of users, keeping servers properly configured becomes an enormous task, arguably impossible to achieve without an army of administrators. The default answer is to deploy configuration management software like Puppet, Ansible, or Chef. However, the 451 Group’s latest metrics say using a configuration management tool only doubles the productivity of a single administrator to no more than 1000 concurrent operating systems.

Web-Scale Infrastructures: If you have worldwide or net facing infrastructures in the tens of thousands of nodes another administrative tooling approach is required, and that is where BoKS ServerControl saves significant slices of time and energy away from administration staff.

REQUEST A DEMO
See BoKS ServerControl in action. Request a demo now.

Related Products

Related Solutions