cybersecurity on Digi Hunch

Firewall Deployment Patterns

Sat, 16 Nov 2024 16:24:15 -0400

The Hub-and-Spoke topology is the most common topic in the discussion for building cloud infrastructure design. This topology appeared in both AWS and Azure design papers and had been around as a very important option in physical networking design.

The AWS whitepaper Building a Scalable and Secure Multi-VPC AWS Network Infrastructure has thorough discussion on the topology. This topology often feature a Transit Gateway as the hub. In addition to workload VPCs, the network topology often includes some special-purpose VPCs, such as interface endpoints VPC, or shared tooling VPCs. One of the special-purpose VPC is the inspection VPC. It is a key design area to suit the need of inspection and traffic management for the business and the design may vary a lot depending on the available inspection tools such as a Firewall appliance.

Inspection Requirements

The most common situation with an enterprise is connecting with on-prem networking. Options including Direct Connect, site-to-site IPsec or SD-WAN overlay. The business decides whether and at what level they would like to inspect the traffic between on-prem and their VPCs. Here is an example.

Connectivity	Inspection Requirement
Between Workload VPCs (East-West)	No inspection
Between a workload VPC and a special-purpose VPC	Normal Inspection
Ingress Traffic from Internet to Workload VPC	Deep Packet Inspection
Egress Traffic from Workload VPC to Internet	Deep Packet Inspection
Between Workload VPC and on-prem networking over Direct Connect	Normal Inspection
……

With a normal inspection, the firewall appliance only checks the information in the packet’s header, such as the source and destination IP addresses, port number, etc. With deep packet inspection, the appliance examins a larger range of metadata as well as the data in each packet. DPI provides a more effective mechanism to perform network packet filtering and find otherwise hidden threats. It is however an expensive operations from a performance standpoint. Ultimately the business makes the call but it is important to identify ALL connectivity scenarios in this phase and explicitly document the decision and rationales. They can choose from an NGFW product or the Network Firewall service from AWS, depending on capability required.

Inspection Architecture

At minimum, inspection is required for ingress and egress traffic to and from workload VPC. The design must account for both routing and inspection. Many would use the same VPC for ingress/egress traffic and for inspection.

It is also possible to separate these two purposes into two different dedicated VPCs: an inspection VPC that hosts firewall services or appliances, and an ingress/egress VPC that directs traffic from and to the Internet but we must route the traffic to the inspection appliance. If all traffic to be inspected has to be routed through the Transit Gateway both ways, the cost would be high. In 2020 AWS introduced Gateway Load Balancer (GWLB) to address this use case.

The recommended pattern using GWLB allows you to place firewall appliance and a GWLB in one VPC, and place the GWLB endpoint (GWLBE) in a different VPC. The connectivity between GWLBE and GWLB is backed by HyperPlane, a technology that also enables other endpoint service such as PrivateLink. The connectivity between GWLB and the appliance take place with Geneve encapsulation. This pattern places any appliance behind an endpoint, so long as the appliance supports Geneve protocol.

The GWLB technology enables a number of inspection patterns based on distributed ingress paths, as summarized in this document. Distributed ingress/egress means each workload VPC can have their own Internet Gateway and NAT gateways. They must configure their route table so as to send the traffic via GWLBEs to inspection appliances. In general, I recommend this pattern over the centralized ingress/egress patterns where only the inspection VPC can take ingress traffic from Internet Gateway. The Network architectures for ingress traffic inspection presentation from 2021 ReInvent covered this topic as well, especially about the scaling benefit of distributed ingress.

Firewall deployment patterns

The firewall deployment pattern available differ between vendors and the requirements. Since the GWLB pattern places appliances behind the GWLB, the appliances rely on Geneve traffic that GWLB forward over. Some vendors may argue that this pattern keeps the NGFW product from performing other tasks that do not support Geneve traffic. One example is Network Address Translation. The native NAT gateway services is very expensive (consider fck-nat as an alternative for NAT). Many clients want to use the NAT feature of the NGFW product. The architecture therefore has to be adjusted in favour of centralized egress. Review this post about one-arm mode and two-arm mode.

If we have to go with central ingress/egress anyways, there are still numerous options. Take FortiGate for example, while the GWLB pattern of deployment is supported, other available options include:

Traditional pattern with multiple interfaces across different subnets in the inspection VPC (L3 mode)
Integration with Transit Gateway using Transit Gateway Connect Attachment
Integration with Transit Gateway using Transit Gateway VPN Attachment

I regard the first option as traditional because it does not directly integrate with Transit Gateway and it is very similar to how we deploy them in a physical networking environment. Fortigate refers to it as L3 (NAT/route) mode. In this mode the Firewall appliance can also influence network routing.

The second and the third options are similar except for different types of Transit Gateway attachments are used. The reason to directly integrate with Transit Gateway is so that the Transit Gateway can route the traffic for inspection therefore no need for a Gateway Load Balancer, and thus no dependency on the firewall features supporting Geneve.

The second option builds a GRE (Generic Routing Encapsulation) tunnel over a Transit Gateway Connect attachment as the transport tunnel, and uses BGP to exchange routes between the Transit Gateway and the appliance. It treats the firewall instances as SD-WAN appliance and has performance benefit. The third option uses VPN attachment with the main benefit of encryption if it is part of compliance requirement.

Rules

The rule configuration for Firewall is critical to the operation of the entire multi-VPC network configuration. Unfortunately, there is no standard with the rule syntax across majore NGFW vendors, leading to challenges for customers to swap vendors. Most flavours of rules have common elements such as

Action (Pass, drop, or alert)
Source and Destination
Protocol and Port

A very common open-source firewall rule syntax is the Suricata-compatible format. One important adopter is the AWS Network Firewall, which supports both stateful and stateless rule groups. With stateful rule group, there are two options for how the Suricata engine evaluates rules.

With “Action Order” option, Suricata engine evaluates the rules in the order of: pass, drop, reject and alert. You can use the priority attribute to influence evaluation;
With strict order, the rules are evaluated in the order of the rule definition;

It is important to be aware of the rule evaluation order since it impacts the firewall behaviour deeply.

Summary

In the networking infrastructure design, ingress and egress routing are the most critical one-way door decision. This decision must account for both routing and inspection. While there are many options, we usually start with capturing the key requirements. In this post we reviewed how to approach the requirement, a key technology Gateway Load Balancer and some firewall deployment patterns with FortiGate as an example. The approach is similar for other NGFW vendors, such as Palo Alto, Check Point or Cisco Secure Firewall.

NTLM and Kerberos protocols

Wed, 25 Mar 2020 20:33:00 -0400

This article explains how two most common authentication mechanisms (NTLM and Kerberos) work. Both authentication protocols are based on symmetric key cryptography. The protocols themselves are platform independent.

NTLM Authentication

NTLM is the default authentication protocol prior to Windows 2000 and still prevalent today as backup to Kerberos. It is based on challenge/response mechanism.

Authentication starts with user trying to log on from a client computer providing the username and password. The following steps will occur:

The client application hashes user’s password (with HMAC-MD5) and then discard the password. Then the client only sends the username to the server;
The server generates a challenge (16-byte random number) and sends it to the client;
The client encrypts this challenge with the hash (of user’s password from step 1), and send the result back to the server as response (encrypted challenge);
The server sends the following three items to the domain controller:
- username (in plain text)
- the challenge it had sent to the client
- the response from the client (encrypted challenge)
The domain controller has the hash of user’s password stored, and retrieved (by username). It uses the hash to encrypt the challenge by itself and get its own version of encrypted challenge. It then compares it with the one passed on from the server. If identical, the user is authenticated and the domain controller notifies the user;
The server sends the appropriate response back to the client.

Essentially, NTLM mechanism is comparing the result encrypted by the client the result encrypted by the credential stored in itself. The results are expected to be identical because the encryption key – hash of user’s password – should be identical. In this whole process, there is no password transmission on the wire. However, the disadvantage with this challenge-response based mechanism is that it does not let client authenticates the server, and is therefore considered less secure. This is what Kerberos is built to address.

Kerberos Authentication

Kerberos is a ticket-based authentication mechansim. In Kerberos, a Key Distribution Centre (KDC) consists of AS (Authentication Service) and TGT (Ticket Granting Service). Authentication takes place in the following steps:

The authentication starts with user trying to log on from a client computer. Note that the “client” here refers to Kerberos client relative to Kerberos server (KDC). The client machine can serve as application server that runs kerberos library. The user provides username and password.

The client sends KRB_AS_REQ as plain text to AS including:
- username
- timestamp
AS verifies timestamp, and validates username exists. If timestamp is too far (i.e. over 5 min) from current time, or user is not a legal principal, KRB_AS_REQ will be rejected. Otherwise, AS generates a random TGS session key and uses it to build a TGT. In KRB_AS_REP AS sends two messages to the client:
- Message 1 is the TGT, which is encrypted with TGS secret key (so the client cannot decrypt TGT). It includes:
  - username
  - TGS name
  - timestamp
  - client network address
  - lifetime of TGT
  - TGS session key
- Message 2 is encrypted with client secret key (stored in AS)
  - TGS name
  - timestamp
  - lifetime
  - TGS session key (same as message 1)
The client receives both message 1 (TGT) and message 2. It decrypts message 2 with its own secret key and obtains TGS session key. For message 1 (TGT), the client cannot decrypt it. The client simply stores it in the credential cache. Then the client prepares two messages to send to the KDC:
- Message 3 contains:
  - TGS name
  - lifetime
  - TGT (message1, encrypted with TGS secret key)
- Message 4 (aka Authenticator) is encrypted with TGS session key and contains:
  - username
  - timestamp
KDC processes message 3, determines TGS name is valid and forward TGT to TGS. TGS decrypts TGT using its own secret key and obtains TGS session key, along with username and timestamp. Then TGS decrypts message 4 using the newly obtained TGS session key, in order to get username and timestamp from message 4 as well. At this point, TGS has two sources of username and timestamp: one from TGT; the other from message 4. The TGS make sure they are identical, check if TGT is expired, and confirm that authenticator is not in the cache (to prevent replay). If all checks pass, the TGS then generates a random service session key. It will send two messages back to the client:
- Message 5 (the service ticket) is encrypted with service secret key (stored in TGS) and contains:
  - username
  - service name
  - timestamp
  - client network address
  - lifetime
  - service session key
- Message 6 is encrypted with the TGS session key containing:
  - service name
  - timestamp
  - lifetime
  - service session key
The client receives message 5 and message 6 but it cannot decrypt message 5. The client cached TGS session key from previous step so it can decrypt message 6 and obtain service session key. Now the client contact the server by sending the following two messages:
- Message 7: a new authenticator message encrypted with service session key that contains:
  - username
  - timestamp
- Message 8: the same as message 5, encrypted with service secret key
The server now receives message 7 and message 8. It decryptes message 8 to get service session key, along with username, service name, timestamp, etc. Using the service session key it descrypts message 7 to get a second source of username and timestamp. Similar to what happened in TGS, now the server compares username from the authenticator and from the ticket, checks if ticket is expired, and confirms that authenticator is not already in cache (to prevent replay attack). If all checks turn out okay. The service confirms its identity to the client with:
- Message 9: an authenticator message encrypted with service session key that contains:
  - service name
  - timestamp
Lastly, the client receives message 9 and decrypts it with the service session key in cache. The client then confirms the service name and timestamp are valid/expected. If they are good, the authentication is completed and the client starts to communicate with the server.

The steps above borrowed some information from this page. The Keberos authentication process involves many steps and several keys:

session (shared) keys:
- TGS session key
- Service session key
secret keys:
- client secret key
- TGS secret key
- Service secret key

By using these keys, no password is ever transmitted across the wire in the clear. The client and the server authenticate each other (mutual authentication). With a trusted third party, Kerberos ensures that the service ticket is only used by the intended client, and that only the intended server can validate the requested service ticket. Although, this sounds similar in two way authentication in TLS handshake, Kerberos does not encrypt the application traffic, neither is it intended to.

Kerberos Implementations

The most popular implementation is MIT Kerberos 5. The other well-known implementation is Heimdal. In addition, it is supported as API in GSS-API. In Windows applications, SSPI (Security Support Provider Interface) provides similar functionality to the GSS-API. SSPI can be viewed as Microsoft’s implementation of GSS-API, which can be virtually regarded as Kerberos API.

Kerberos is supported by many application protocols through GSS-API. If you build a Linux application, the server where the application is hosted acts as Kerberos client (i.e. requiring krb-libs package and /etc/krb5.conf configured correctly) to interact with customer’s Active Directory environment. If you need to configure Kerberos servers, you will need other packages such as krb5-admin-server, krb5-kdc, and krb5-user.

This page lists some pros and cons of Kerberos. Some important takeaways are:

Kerberos is legacy protocol, complex to set-up and maintain. It requires user accounts, user clients and the services on the server to all have a trusted relationship to the Kerberos token server. All must be in the same Kerberos realm or in domains that have a trust relationship between each other.

Kerberos cannot be used in a scenario where users want to connect to services from unknown/untrusted clients as in a typical Internet or cloud computing environment, where authentication provider typically does not have knowledge about the users client system. This implies Kerberos does not work will with modern REST applications and Authentication Methods such as SAML, and OAuth 2.0

Kerberos was created to accomplish authorization back in the days when no-one used a secure network connections.

Introduction to Authentication Frameworks (PAM and SSPI)

Tue, 24 Mar 2020 20:19:00 -0400

This article gives a very brief high-level introduction to PAM (Pluggable Authentication Module) and SSPI (Security Support Provider Interface) as authentication frameworks in Linux and Windows respectively.

PAM

The Pluggable Authentication Module (PAM) architecture provides a powerful abstraction for user IAM using pluggable authentication model Unix platforms. It defines a generic API for authentication and hides the underlying mechanisms. Thanks to PAM, administrators can plug different authentication modules and protocols into Linux. This makes different authentication methods and protocols available to applications running on Linux. Here is some of authentication methods and protocols that PAM supports:

Unix file-based authentication (using /etc/passwd or /etc/shadow)
LDAP-based authentication
Kerbero-based authentication
NTLM-based authentication

PAM obviates the need for a separate authentication schemes. It exports methods of the various libraries under its auspices to calling applications. Here is a diagram for PAM on Redhat.

PAM can also enbable single-sign-on(SSO) on the UNIX platform. If the password used for different services are identical, PAM can be used to share the password transparently between the application’s possibly different authentication mechanisms. PAM is configured in /etc/pam.d/ directory.

The Windows equivalent of PAM is the Security Support Provider Interface (SSPI) and its Security Support Provider (SSP) Modules. For example /etc/pam./login manages login module, /tec/pam.d/imap manages imap module.

SSPI

Windows Authentication Architecture involves Local Security Authority (LSA, to authenticate users to local computer only and is managed in local security policy) and Security Support Provider Interface (SSPI).

SSPI is the API that obtains integrated security service for authentication, message integrity, message privacy, and security quality-of-service for any distributed application protocol. SSPI is the implementation of the Generic Security Service API (GSS-API) in Windows operating system. Applications and infrastructure services authenticate users by using the SSPI to abstract calls for authentication. This way, developers do not need to understand the complexities of specific authentication protocols or build authentication protocols into their applications. Here is the architecture diagram:

The SSPI in Windows provides a mechansim that carries authentication token over the existing communication channel between the client computer and the server. When two computers or devices need to be authenticated so that they can communicate securely, the requests for authentication are routed to the SSPI, which completes the authentication process, regardless of the network protocol currently in use.

Here is some exampls of SSPs that are supported by SSPI:

Kerberos SSP (default for Active Directory)
NTLM SSP
Digest SSP
Negotiate SSP (based on SPNEGO, RFC4178)
Credential SSP
Negotiate Extension SSP

Some SSPs such as Kerberos SSP and NTLM SSP use a single protocol. Some (e.g. Negotiate SSP and Credential SSP) combine several protocols to allow application to select what security mechanism they wish to use and negotiate with authentication services.

SASL Authentication Mechanisms

Thu, 19 Mar 2020 22:53:00 -0400

Introduction

Authentication is used in many protocols (such as LDAP binding) and it usually involves sending password. Given the nature of authentication protocol, its traffic encryption is usually mandatory. Simple Authentication and Security Layer (SASL) is introduced to ensure the security during authentication. It is not a single protocol, but rather a framework for authentication and data security involving many protocols. The intent is to decouple authentication mechanisms from application protocols, thus allowing any authentication mechanism (under SASL) to be used in any application protocol (that supports SASL). Application protocols that support SASL typically can also be built on Transport Layer Security (TLS), whose latest versions (1.2 and 1.3) are considered more secure.

None (ANONYMOUS)

The server basically does not authenticate the client. The client connects to the server anonymously. Under SASL framework, this may also be referred to as ANONYMOUS mechanism.

Simple (PLAIN)

In simple authentication method the password is sent to server in the clear. This is subject to eavesdropping and is not secure. It is still surprisingly widespread in legacy configurations probably due to the simplicity of configuration. This option should not be available in cloud environment. Under SASL framework, this may also be referred to as PLAIN mechanism.

CRAM-MD5 and DIGEST-MD5

CRAM-MD5: described in RFC 2195, using HMAC-MD5 algorithm. In this challenge-response scheme based mechanism, the client’s password is protected during authentication, but the application session (e.g. LDAP) traffic is not encrypted. It includes random data from the server and is slightly better than Simple authentication. However, this authentication method is not recommended either.

DIGEST-MD5: described in RFC 2831. This is very similar to CRAM-MD5 but is is somewhat stronger because it includes random data from both the client and server. In addition, it also provides a provision to ensure connection integrity and confidentiality (a data security layer).

GSSAPI

Generic Security Service Application Program Interface (GSS-API) is an API specification for programs to access security services. GSS-API by itself does not provide any security. Instead, security-service vendors provide GSSAPI implementations – usually in the form of libraries installed with their security software. These libraries present a GSSAPI-compatible interface to application developers who can write their application to use only the vendor-independent GSSAPI.

Under SASL, the dominant GSSAPI mechanism implementation in use is Kerberos version 5. GSSAPI allows Kerberos implementations to be API compatible. In many contexts, GSSAPI simply implies Kerberos.

NTLM

NT LAN Manager (NTLM) is a challenge-response based Microsoft security protocols. It is implemented in a Security Support Provider (SSP), which combines the older LAN Manager authentication protocol, NTLMv1, NTLMv2 and NTLM2 Session protocols in a single package. Group policy manages whether these protocols are used or can be used. NTLM passwords are considered weak because they can be brute-forced very easily with modern hardware. It might still be enabled in server configuration as a backup mechanism to Kerberos.

TLS (EXTERNAL)

CRAM-MD5, DIGEST-MD5, GSSAPI, and NTLM are more commonly referred to as SASL mechanisms (in loose terms). These mechanisms allow for a secure password exchange without requiring TLS by trying to address the authentication traffic encryption problem at application layer. Using TLS this can also be address at transport layer. TLS can be used in combination with any of the mechanisms above but usually TLS/Simple mechanism is sufficient. In many occasions the mechanisms under SASL can be replaced by simple authentication encrypted with TLS. Under the SASL framework, this may also be referred to as EXTERNAL mechanism so TLS (in strict terms) is also considered a SASL mechanism.

Summary

This article outlined several authentication protocols under SASL. Since SASL is the framework that intends to govern all authentication protocols, the use case of these protocol can be widespread. For example, in LDAP you can find all of them. Here is a comparison across them:

	Password on wire	Session
Simple	Clear	No Encryption
SASL/CRAM-MD5	Encrypted	No Encryption
SASL/DIGEST-MD5	Encrypted	No Encryption
SASL/GSSAPI	Kerberos	Encryption
TLS:SIMPLE	Encrypted	Encrypted

Common SASL implementation includes Cyrus SASL and GNU SASL. There are also some API implementations that supports some of SASL mechanisms, such as SSPI.

OAuth 2.0 and OIDC 1 of 2

Sat, 14 Mar 2020 21:10:00 -0400

OAuth 2.0 and OpenID Connect (OIDC 1.0) are different but highly related protocols and they are often confused. When we talk about IAM (identity and access management), we should first distinguish between Authentication (AuthN) and Authorization (AuthZ):

AutheNtication (AuthN, aka Identity Management) is about validating user’s identity by verifying that the user trying to connect is actually who it claims itself to be;
AuthoriZation (AuthZ, aka Access Management) refers to granting or denying access to specific resources based on the requesting user’s identity. It is usually performed after a user is identified through authentication. The most common approach is Role-Based Access Control (RBAC).

In a nutshell, OAuth 2.0 deals with authorization. OIDC is a layer later developed on top of OAuth 2.0, to deal with authentication. This post is greatly influenced by a presentation (1 hour) delivered by Nate Barbettini from Okta, with the slides available here. There is also an abridged and illustrated video (16 min) by Okta available here.

OAuth 2.0

OAuth was originally developed by Twitter and Google in 2006 as an open standard for API authorization. OAuth 2.0 is published in 2012. It allows user to delegate authorization. The original scenario is a user signed up to a new application and allows it to automatically import her Gmail contact. The technical problem to solve is: how can a user (Resource Owner) let an app (Client) to access his contact list stored in Google server (Resource Server)? The proposal is that it redirects user to Google Account page (Authorization Server) for user to log in. Then Google Account issues token to the application (Client) with user’s approval. Note that the user did NOT log in to the application itself with her Google account. From the application’s standpoint, the user had been authenticated already, and was simply importing contact after logging in. The roles involved in OAuth 2.0 are:

Resource: the contact list of the user
Resource owner: the user
Client: the application
Resource Server: contact.google.com
Authorization Server: accounts.google.com

The diagram below illustrates the interactions:

OAuth 2.0 Authorization Code Flow

This page has further details for each step. Note that at step 3 to 5 may seem unnecessary because Auth Server could have send Access Token Grant to Application via User-Agent at step 3, which could have eliminate the need for step 4 and 5. In fact this design is to avoid sending critical information (Access Token Grant) to User-Agent (browser) which is considered in secure. In other words, it avoids front channel (User-agent to auth server) and prefers back channel (Client to Auth server) for security. This is the difference between Authorization code flow and the implicit flow.

With OAuth 2.0, there are a number of flows:

Authorization code (front channel and back channel)
Implicit (front channel only, token returned to user agent directly)
Resource owner password credentials (back channel only)
Client credentials (back channel only)

In Authorization Code Flow, the Application (client) needs a one-time registration with the Auth Server and is given a client ID and client secret, which are sent to Auth Server at step 4 along with Access Token Request, to prove the identity of the client application.

Note that OAuth 2.0 is an inherently insecure protocol since it does not support signature, encryption, channel binding or client verification. The protocol relies entirely on the underlying transport layer security (TLS) to provide confidentiality and integrity.

Also note that throughout the process (Authorization Code Flow as an example), the Client application eventually is granted access to user’s data. However, it does not know anything about the user itself. Neither the authorization code grant, nor the access token grant is obligated to present information about the user itself. Therefore, OAuth 2.0 is designed strictly for permission purpose without the intent to address identity issue. In the flow, the auth server does the authentication (for the purpose of granting access to resources, but none of the authentication. A user logs in to client application as Bob, when he requests to imports contact, he is redirected to account.google.com and there he could put in the credential of Alice and therefore load Alice’s Google contacts into Bob’s App account!

Pseudo-authentication with OAuth 2.0

In many real life OAuth 2.0 implementations, at step 3, the Auth server chooses to include a field about the user’s identity. This makes user’s identity visible to the client, and the client is therefore able to confirm user’s identity in its own code. This also allows client application to use OAuth 2.0 as an authentication method, which is referred to as pseudo-authentication. The access token acts as a kind of “valet key” that the application can include with its request to the auth server, as a proof that it has user’s permission to access the resources (or APIs).

Because the identity provider (auth server) typically (but not always) authenticates the user as part of the process of granting an OAuth access token, it’s tempting to view a successful OAuth access token request as an authentication method itself. However, because OAuth was not designed with this use case in mind, making this assumption can lead to major security flaws.

Nate’s presentation outlined some scenarios where OAuth 2.0 is applied up to 2012, and which ones are misuses:

Simple login – pseudo authentication with OAuth 2.0
Single sign-on across sites – pseudo authentication with OAuth 2.0
Mobile app login – pseudo authentication with OAuth 2.0
Delegated authorization – the only intended use case for OAuth 2.0

To address the authentication issue properly, and in a standard approach, we need OpenID Connect.

OpenID Connect (OIDC)

OpenID Connect is an open standard for authentication, promoted by the non-profit OpenID Foundation. It allows user to be authenticated using a third-party service called identity providers. User may choose to use their preferred OpenID Connect providers to log in to websites that accept the OpenID Connect authentication scheme. For example, a user uses her Facebook to login to an online application.

OpenID Connect is an extension to OAuth 2.0 with a just few additions:

In addition to access token, an ID token is returned by the authorization server;
Userinfo end point is provided in case Id token is not sufficient and more user information is needed;
“openid” is passed as a parameter in the Scope during the initial call to the authorization server;

Therefore OpenID Connect is considered an identity layer on top of OAuth 2.0. Many application supports OpenID Connect such as Apache Nifi. OIDC is comparable with SAML in the sense that both provide SSO feature (federated identity). Here is a comparison table:

	OpenID Connect	SAML
Main Purpose	SSO for consumer/mobile applications	SSO for enterprise applications
Load	Relatively light weight	Heavy weight due to the size of XML messages
Use case	Satisfies both authentication and authorization use cases, often combined with OAuth 2.0	Generally not used for API security
Transport	HTTP GET and HTTP POST	HTTP Redirect (GET) binding, SAML SOAP binding, HTTP POST binding, et

Here are more details about their differences. In general SAML is more common in the enterprise world for SSO and it has been around for a while. When developing new applications for enterprise it is advised to consider OIDC first.

OIDC also has authorization code flow, with the additional fields on top of its counterpart in OAuth 2.0. The authorization server returns both access and ID tokens, wrapped in a data structure named JWT (JSON Web Token). The JWT includes a signature field, allowing the client application to verify it with authorization server’s public key. Nate’s presentation proposes the following flows for each application type:

Web application with server backend: authorization code flow
Native mobile app: authorization code flow with PKCE
Java Script app (SPA) with API backend: implicit flow
Microservices and APIs: client credential flow

In addition, this page from Okta developer has a good summary of how to select flow type (grant) based on each use case.

Summary

OpenID Connect is an authentication protocol for the purpose of validating user’s identity. OAuth 2.0 is an authorization protocol. You should use OAuth 2.0 for granting access to your API, or access to user data in other systems. If you need to log user in, or make your accounts available in other systems, you need OIDC.

The next post about OAuth and OIDC was posted in 2023.

Lightweight Directory Access Protocol (LDAP)

Mon, 02 Mar 2020 21:11:00 -0400

Introduction

Originally LDAP only refers to the connectivity protocol to the directory server. This term is being used loosely today and it also refers to the actual directory service that supports and complies with LDAP. LDAP v3 is the current version developed in RFC 2251.

A directory is information about some set of entities such as people, organization, or stones. An example of directory would be /etc/passwd file in Linux. A directory server is simply an application with the main purpose of maintaining directories. Typically, the read traffic is high whereas write traffic is low. LDAP is a general-purpose directory server. It can store information about people, or cars, or rocks. You just need to define what a person’s entry looks like as well as what a rock’s entry looks like. The general architecture of LDAP provides the capability nedded for managing large amount of diverse directory entries.

An LDAP entry consists of DN (distinguished name) and attributes. An attribute may have one or more attribute names and they are defined in attribute definitions. Attribute names are not case-sensitive. An attribute may have one or more values if multiple values are allowed for that attribute. Attribute values may be case-sensitive depending on the definition.

A special attributed named objectclass attribute provides information about what type of record it is, and what attributes canbe given to the record. For example, the organization name (o) is required for any entry with an organization object class. While a record may have multiple object classes, one of these object classes must be the structural object class for the record. A structural object class determines what type of object the record is.

In addition to regular attributes, the directory server may also attach special operational attributes to an entry. Operational attributes are used by the directory server itself to store information about entries. Such attributes are not designed for use by end users, and are usually not returned during LDAP searches.

An LDAP schema defines types of records in a directory and how those records might relate to each other.

Information in an LDAP directory is organized into one or more hierarchies where, at the top of the hierarchy, there is a base entry, and other entries are organized in tree-like structures beneath the base entry. Each node on the hierarchy is an entry, with a DN and more than one attributes. This hierarchically organized collection of entries is called a directory information tree (DIT). In DIT, LDAP directories stores data in hierarchical relationships. The root entry sits at the top and subordinate entry is beneath that, which in turn may have its own subordinate entries. Each of these records has its own DN, and its own attributes. The DN of each entry is composed of two parts: the relative DN (RDN) and the full DN of the superior entry.

LDAP is nothing other than a special sort of database that organizes data into tree structures, like a file system hierarchy. This view is more easily seen by comparing an LDAP directory to a relational database system (RDB), where SQL is the protocol and RDBMS is the service. LDAP refers to both the protocol and the service.

OpenLDAP

A common LDAP implementation is openldap. OpenLDAP suite can be broken up into four components:

Servers: slapd (stand-alone LDAP Daemon) provides LDAP services.
Clients: ldapsearch is used to manipulate LDAP data
Utilities: support LDAP servers
Libraries: provide programming interfaces to LDAP

Installing OpenLDAP requires libldap-2.3-0, slapd, ldap-utiles packages. It is configured in /etc/ldap/. An HDB (hierarchical database) needs to be specified in the configuration.

To test as a client, the first thing that must happen is the client must authenticate to the server (via simple bind or SASL Bind). LDAP server verifies the identity, permission as well as password provided by the client.

LDAPTLS_REQCERT=never ldapsearch -x -o ldif-wrap=256 -H ldaps://ldap.digihunch:636/ -b "OU=Admin,OU=Service Department,DC=digihunch,DC=com" -D "gh\ldap-bind-user" -w 'S@f35+P@55w0rd' "(objectclass=user)" -s sub -d 9

The command above first sets client environment variable LDAPTLS_REQCERT to never, in case the client is being asked to provide certificate. Then the ldapsearch command performs the bind.

To search the directory, the client needs to provide the followings:

Base DN: where in the directory to start from
Scope: how deep in the tree to look
Attributes: what information to be retrieved per result
Filter: what to look for

Below is an example of ldapsearch (-b for Base DN, -s for Scope, -S for attributes, stdin for filter):

ldapsearch -x -o ldif-wrap=256 -H ldaps://ldap.digihunch:636/ -b "OU=Admin,OU=Service Department,DC=digihunch,DC=com" -D "gh\ldap-bind-user" -w 'S@f35+P@55w0rd' "(memberof=CN=Security-Admin,OU=Admin,OU=Service Department,DC=digihunch,DC=com)" -s sub -S name

Users with appropriate permissions may also other directory operations using ldapadd, ldapmodify, ldapdelete, ldapcompare, ldapmodrdn, ldappasswd, ldapwhoami, etc

Apart from those in Openldap toolkits, there are many other tools such as Apache Directory Studio that allows you to perform similar functionality with a user interface.

LDAP security

Historically LDAP servers listens to port 389 through which traffic is sent in clear text. This is a bad security practice known as “insecure bind”. To secure LDAP traffic, two prevalent approaches are Secure LDAP and StartTLS.

Secure LDAP was the original attempt to secure LDAP traffic as an addition to LDAP v2. It is also known as LDAPS, LDAP over TLS/SSL or LDAP channel binding (“channel binding” just refers to the establishment of encrypted channel following TLS handshake. It provides a facility to tie an authentication exchange to security services provided at a lower layer. Defined in RFC 5056). Secure LDAP operates on port 636 on the server side and TLS handshake must be established for traffic encryption. Client application usually need to import the certificate of LDAP server. As part of TLS 1.2 protocol, the server may also request client certificate during ServerHello message. The presence of CertificateRequest means the server either demands client certificate, or tries to get client certificate (i.e. TLSVerifyClient is set to demand or try, which is only visible on the server). If client cert is only attempted, the LDAP client may choose to ignore it. If client cert is demanded, then a two-way TLS authentication is required and thus the client must proof its identity to the server. This Secure LDAP configuration requires the server to listen to both 389 and 636 ports on the same server to support both secure and legacy applications, which is unnecessary. Secure LDAP therefore is not the preferred approach.

The standardized way of implementing SSL/TLS in LDAP v.3 is to use the StartTLS method. This method should be implemented whenever possible. If an AD server supports StartTLS, the client can start with a STARTTLS command to the server so that the server begins the TLS encryption process. In the binding phase, TLS handshake follows a LDAP_START_TLS_OID command through port 389.

Here’s the summary of the three LDAP configuration mode:

	Legacy	SecureLDAP (aka LDAPS, LDAP over TLS/SSL)	StartTLS
Listening port	389	636	389
Traffic Encrypted	No	Yes	Yes
Standard	Yes but this should always be avoided since it is insecure	Introduced in the time of LDAP v2, but the option is deprecated (although still supported) by RedHat	Introduced in LDAP v3. This may be left as the only valid option.

Note that one of the recent changes that drives may customer away from the legacy mode is the requirement for LDAP channel binding on Windows servers, with a target date of March 2020. Our current strategy at CS is to direct customer towards Secure LDAP as we do not support StartTLS yet and we know we do support LDAPS. Although Secure LDAP itself is somewhat legacy this would not hold long. According to this Wikipedia page:

The use of LDAP over SSL was common in LDAP Version 2 (LDAPv2) but it was never standardized in any formal specification. This usage has been deprecated along with LDAPv2, which was officially retired in 2003.

The trade off between StartTLS and TLS/SSL exists not only in LDAP protocol, but also in many other protocols such as SMTP (port 2525, 25, 587). StartTLS is also called Opportunistic TLS. The standard is in the relevant RFC documents.

Cryptography basics 2 of 2

Sun, 08 Sep 2019 21:21:41 -0400

My previous post outlines several core concepts around cryptography, such as asymmetric key encryption, digital certificate, the encoding formats and relevant file extensions. In this article, we continue to explore cryptography use cases, where these concepts are connected and put into application.

The most important use case is TLS handshake. I cannot stress enough how paramount this scenario is. This use case and its variation can be found in almost every situation where connection needs to be secured.

TLS handshake: the process in which client and server establish secure connection. During the handshake, two parties agree on TLS version, decide on cipher suite, authenticate the identity of each other (although client identity authentication is less common), and generate session key for symmetric encryption after the handshake. Details steps are very important. Here is a fairly thorough reference, and here is a great diagram:

TLS handshake

Secure browser connection: The most widespread use of TLS handshake is to secure browser connection with HTTPS:

Browser initiates connection to the server;
Server sends browser its certificate (public key + digital signature signed by CA);
Browser has preloaded public key of CA and uses it to decrypt digital signature and get the digest of public key;
Browser calculate digest of received public key and compares it against the digest from the previous step;
If the result is the same, public key is trusted, a green lock is displayed; otherwise, a warning is displayed;
proceed to the rest of the steps in TLS handshake.

Secure connection in Java: Java applications manages keys and certificates through two classes: java.security.KeyStore and java.security.TrustStore. Suppose a Java application client initiates TLS connection to server. The server application will present its certificates from server’s Key Store. The client will use certificates stored in client’s Trust Store to verify the identity of the server. Once validated, the client then presents certificates stored in client’s Key Store back to the server for validation.

A KeyStore keeps keys and certificates for your own application. Typically, you store a KeyPair in a KeyStore file.
A TrustStore keeps the certificates of external systems that your application trusts.

Below is a great diagram:

Secure Java application in TLS

JKS file is Java’s version of PKCS#12 (private key + certificate, password protected). Entries in a JKS file must have an “alias” that is unique. The JKS file type can be used for both Key Store and Trust Store. When it’s used in Key Store, it contains a certificate and private key for the Java application. When it’s used in Trust Store, it only contains certificate from external trusted applications. Note that JKS is the default keystore format until Java 8. Since Java 9 the default keystore format is PKCS12.

Finally, we can introduce some tools:

Command line Tools

openssl is a versatile tool for cryptography and keytool is a similar tool for Java applications.

keytool is for store keys/certificates in Java Key Store or Trust Store.

Conclusion

I do not include any openssl or keytool command in this article in order to remain theoretical. However, once through the conceptual hurdles, one should become fairly comfortable picking up the tools and understand why each command is needed to achieve its purpose.

Cryptography Basics 1 of 2

Wed, 10 Jul 2019 20:07:18 -0400

I have been dabbling with OpenSSL commands to achieve what I needed during IT implementation, but I decided to spent some time to overcome the conceptual hurdles around cryptography. In this domain, following other people’s instructions through the project does not produce much learning value when too many concepts cloud around. Let’s take the bull by the horn.

This article is purely conceptual. There are already lots of step-by-step guideline about acquiring a website certificate. The intention is to elucidate the core concepts on IT cryptography, and then connect the dots to form the big picture in cryptography.

First, let’s distinguish three basic concepts: encoding, hashing and encryption. They are in essence all mathematical functions, but one does not need to understand the underlying algorithm in order to understand what they are.

Encoding – Transform data into a format so it is readable by external system. Encoding is about interoperability. It is not about security whatsoever. Example: ASCII, BASE64, UNICODE.

Hashing – Mathematic algorithms to generate digest of content. A digest is usually fixed-size, non-reversible and deterministic. You cannot restore content from its digest (non-reversible, one-way calculation). Two different contents results guarantees different digests (deterministic). Digest as a result of hashing is mostly about data integrity. For example, in file download you can calculate MD5 hash and compare the result against the digest given by the source. Another common use, is to hash all password in database. In that sense, hashing has to do with security. Popular algorithms are MD5 and SHA-256.

Encryption – Mathematic algorithms that only succeeds if correct parameter (key) is provided. The function is deterministic and reversible (two-way). The parameter (key) used for calculation (encryption) and reverse calculation (decryption) can either be the same or different.

Cryptography involves all three concepts, but it mainly addresses issues around encryption. We break down encryption into two categories, symmetric key encryption (aka private key encryption) and asymmetric key encryption (aka public key encryption).

Symmetric Key Encryption: using the same key (shared secret) for encryption and decryption. Both parties need to keep it secret. Popular algorithm is AES.

Asymmetric Key Encryption: involves a pair of public key and private key. Public key is given out to external systems. Private key is kept secret. Popular algorithm is RSA. You can run some experiments on this page, and I’ve made some additional notes (all based on RSA algorithm):

Public Key and Private Key are NOT interchangeable. The size of an RSA private key is usually much larger than its public key;
You can encrypt with either key, and decrypt with the other, so long as you specify the key type at the time of encryption or decryption;
You can generate public key from private key (ssh-keygen -y); but not the other way round;
Using a wrong key to encrypt or decrypt leads to failure, instead of wrong result.

Obviously symmetric key encryption is the original form of encryption and requires both parties to keep the key secret, permanently. This is not realistic in real life between organizations. This challenge leads to the adoption of asymmetric key where the public key can be published, for external party to encrypt outgoing messages, whereas the private key is kept secret within the owner, only to decrypt incoming messages.

Knowing this distinction, we can introduce two concepts that are built on top of asymmetric key encryption.

Digital Signature: if an entity signs a document digitally. The digital signature is the digest of the document (hash of the content) encrypted with the signer’s private key.

Digital Certificate: contains owner’s public key and digital signature of issuer (digest of owner’s public key encrypted by issuer’s private key). Client (e.g. browser with CA’s public key preloaded) should not trust the owner’s public key until it compares its digest against decryption result of digital signature (using CA’s public key).

The key difference here is that a digital certificate involves third party. Digital signature by itself cannot address impersonation, which is addressed by digital certificate issued by third party (certified authority). This requires digital certificate must follow some standard, and the most prevalent one is:

X.509: a standard format for digital certificates. It contains:

a public key
an identity (a hostname, or an organization, or an individual)
a signature (either signed by CA or self-signed)

Since the advent of certificate, there are tons of global organizations that need to manage keys and certificates. When we create a secure connection now, we only need to get a certificate from an intermediate CA, thanks to existing certificate chain. It involves lots of work for large organizations to maintain keys and certificates, which requires:

Public Key Infrastructure (PKI): the IT infrastructure to create, manage, distribute, use, store and revoke digital certificates and public keys.

One of the important technique to manage keys and certificates is the encoding. Keys and certificates are usually wrapped with different encoding formats based on what need to be done on them. Here is a summary of encoding formats

Format	Encoding	what is stored	Possible Suffix
PEM	DER file (binary content) encoded in Base64. Certificate files typically include clear text statement “BEGIN CERTIFICATE” and “END CERTIFICATE”	single certificate, certificate chains or private keys	.pem, .crt, .cer, .key
DER	Binary format in early days; it’s complex and don’t use unless with a specific purpose.	single certificate, certificate chains, or private keys	.der, .cer
PKCS#7	Base64 encoded ASCII file, typically include clear text statement “BEGIN PKCS7” and “END PKCS7”	only certificates or certificate chains; no private keys	.p7b .p7s
PKCS#8	Similar to PEM as base64 encoded format but for storing private key only, can be password protected	Private key	.key
PKCS#12	a binary format, heavily used by Microsoft products	certificate, certificate chains or private keys; public private key pair	.pfx .p12
OpenSSH	used by OpenSSH to store public keys (as specified in RFC4253)		.pub

Note: above are just encoding formats. Just by file extension you cannot tell whether the file is a key or a X.509 certificate. When you are configuring certificates, you may come across the following file extensions as well:

.key this extension can indicate any kind of key, but usually it is a private key (used along with .crt file)

.csr certificate signing request, including a public key and an identity required by CA. CA needs this file to issue a certificate. CSR could be encoded in Base-64 or DER

.cer or .crt a certificate, usually in X.509 v3 (public key + identity + signature), the encoding could be PEM or DER.

.jks -> java key store file type. It can be either a key store (private key along with certificate) or trust store (certificate) for Java application. Refer to the section for Java applications.

In the next article, we will examine some use case involving the concepts introduced in this post.