Introduction to Symmetric Encryption for the Working Developer

Encryption is all about establishing confidentiality. That is, making sure only authorised parties have access to specific data. It involves transforming data into a form that is inscrutable to unauthorised parties but with the ability for authorised parties to transform the data back into its legible form.

Think about two parties communicating without wanting other persons privy to the messages. This is achieved via the cryptographic primitive of Encryption.

In this post, I will provide a whirlwind tour of what encryption is for the working developer. The goal is to provide the basic information needed to be able to properly wield this cryptography tool without going into the inner workings of the algorithm. It is part of the Cryptography101 series of posts.

This post contains the following

• What Encryption is and its main components
• What Encryption is not
• Symmetric Encryption vs Asymmetric Encryption.
• Block Cipher vs Stream Cipher.
• Overview of encryption Standards
• Components of AES
  • Key
  • Padding
  • Mode

Encryption and its main components

Encryption is the part of cryptography that deals with ensuring confidentiality of data. This usually involves taking data and transforming it into what looks like gibberish to unauthorised parties.

Encryption can be used as part of secured communication, where two parties communicate without any eavesdropper being able to make sense of the communication. It is also used to encrypt data at rest, for example the data on a harddisk; making sure it is only accessible by the owner or any other authorised parties. This can also be seen as securing communication, where the other party is the future self that needs to access the content of the harddisk.

It should be noted that encryption must also include the reversible component; that is decryption. This is the process of taking the gibberish output of the encryption process and transforming it back to the original data. Without being able to decrypt, encryption becomes useless.

Both the encryption and decryption process makes use of a critical component called the secret key. The secret key together with the encryption/decryption algorithm is what is used to transform a message into gibberish and also to recover the original message. As long as an eavesdropper does not know the secret key, there is no way to access the message. Hence the secrecy of the key is at the heart of the security of encryption.

The encryption, decryption and secret key generation algorithm is oftentimes together referred to as Encryption Scheme.

The data to encrypt is usually called plaintext, the algorithm that encrypts the data is usually called cipher and it makes use of a secret component called key. The encrypted data is usually called ciphertext.

What Encryption is not

Just as it is important to say what encryption is, it is also important to make clear what encryption is not.

Cryptographic Hashing is not Encryption

Cryptographic Hashing is another cryptography primitive that is sometimes confused with encryption. A cryptographic hash algorithm takes data and produces a random bit string, often called digest, which serves as a form of fingerprints of the data.

Hashing is a one way operation, that is, with a secure hashing algorithm, it should be practically impossible to retrieve the original data from the digest. This is in contrast with what you want with encryption. With encryption you need to be able to retrieve the original message.

See Introduction to Cryptographic Hash Functions for the Working Developer for more about Cryptographic Hash Functions.

Encoding/Decoding is not Encryption

Encoding/Decoding schemes like base64 or even representing data in hexadecimal or binary string is not encryption even though they might also appear to be hiding data. They are not encryption because they do not make use of any secret key, hence they provide no confidentiality since anyone can perform the reverse decoding operation on the encoded data.

Steganography is not Encryption

In infosec there is sometimes the need to hide the fact that communication between two parties is taking place. This usually involves hiding a secret message in something that is not secret. This is done via Steganography. This is different from Encryption which does not seek to hide that communication is taking place but only to hide the message that is being communicated.

With Steganography an eavesdropper won't be aware that communication is taking place.

With encryption an eavesdropper can observe communication taking place but won't be able to see what is being communicated.

So having touched on a couple of things that are not encryption, let us go on to see some different aspects of encryption that are often used to put them into different categories or types.

Symmetric Encryption vs Asymmetric Encryption

As mentioned earlier, the process of encryption depends on the usage of a secret key. You then have two flavours of encryption depending on how this secret key is used.

An encryption scheme where the same key is used for both encryption and decryption is called symmetric Encryption. While an encryption scheme where the key used for encryption is different from the key used for decryption is called asymmetric encryption (also known as asymmetric cryptography or public key cryptography).

This post is about symmetric encryption

Block Cipher vs Stream Cipher

Encryption schemes can also be categorised based on how the plaintext is passed on to the encryption algorithm.

Encryption schemes that require the plaintext to be encrypted a fixed size at a time are called Block ciphers.

This involves having the plaintext broken down to a certain fixed bit size, and each bit size encrypted. This is a hard requirement, hence if the plaintext or the last piece of the chopped up plaintext is not up to the required fixed size, it would have to be padded till it meets the required size.

Stream ciphers, on the other hand, can encrypt plaintexts at varied bits at a time. Since they do not require a fixed block to be passed in for encryption, stream ciphers hence do not have a need for padding.

This post is about symmetric encryption that are block ciphers.

Overview of Encryption Standards

As stated in Introduction to Cryptographic Hash Functions for the Working Developer there are different agencies that publish cryptographic related standards. In this post, we only limit to standards published by National Institute of Standards and Technology (NIST).

When it comes to symmetric encryption, the go to encryption scheme as standardised by NIST is Advanced Encryption Standard, usually referred to as just AES.

As with most standards by NIST, what later came to be the AES was as a result of an open competition that started in 1997. The competition was an open call to all interested cryptographers to come up with an encryption scheme to replace the then standard scheme: Data Encryption Standard usually referred to as DES.

In 2001, the competition was concluded and out of 15 different submissions, the Rijndael encryption scheme, designed by Vincent Rijmen and Joan Daemen emerged the winner and was named the AES.

AES has since then been the main encryption scheme. Replacing DES, which for all intents and purposes is broken and should not be used. Hence the remaining part of this post would be basically an overview of AES.

Components of AES

In this section, I will go over some of the key components of AES. This are key size, padding schemes, and mode of operation.

Keys in AES

Like any symmetric encryption scheme, AES also requires a secret key. AES comes in 3 different versions depending on the size of the key.

We have:

• AES-128 takes a key of 128 bits (16 bytes)
• AES-192 takes a key of 192 bits (24 bytes)
• AES-256 takes a key of 256 bits (32 bytes)

The size of the key dictates the level of security. Larger keys provide higher security but for almost all use cases, AES-128 will suffice.

AES keys are random bits. That means any method that can generate true cryptographic random numbers of 128, 192, and 256 bits can be used to generate AES keys. As mentioned earlier on, the security of AES depends on the key, hence the methods used to generate the key needs to be truly random and unguessable.

For example, in JavaScript environment with Web API available, like in the browser or Deno, window.crypto.getRandomValues(new Uint8Array(16)) can be used to generate random 256 bits that can be used as a AES-256 key.

In Deno, this look like:

Deno 1.16.4
exit using ctrl+d or close()
> let counter = window.crypto.getRandomValues(new Uint8Array(16));
undefined
> console.log(counter)
Uint8Array(16) [
  189,  59,  14, 161,  84,
  117, 160, 119, 175,  96,
  130, 233, 121,  99, 176,
  240
]

Padding scheme in AES

AES is also a block cipher, hence it can only encrypt a fixed size at a time. AES block size is 128 bit hence it can only encrypt plaintext 128 bit at a time.

This means if your plaintext is larger than 128 bit, it would have to be broken up into blocks of 128 bit each. If the plaintext is less than 128 bit, or after breaking into 128 bits blocks, there remains a last piece that is less than 128 bit, then that piece needs to be padded somehow to become 128 bit.

Note that the block size of 128 bit is independent of the key size. Meaning that even if a key size of 192 bits or 256 bits is used, the encryption/decryption would still be done 128 bits at a time.

The padding scheme needs to be such that upon decryption the padding portion can be removed and discarded to reveal the original plaintext.

Hence AES needs a padding scheme. One of the popular standardized padding scheme, which is also used in AES implementation is PKCS#7. Simply put the PKCS#7 algorithm states the value of each padding byte should be the length of the padding required. So in the case where you have a plaintext of 12 bytes, this will require 4 bytes of padding to make it 16 bytes (that is to make it 128 bit that is required by AES). Hence the value of the 4 bytes should be 4.

To prevent ambiguity in the case where the plaintext is already a multiple of 128 bits, PKCS#7 stipulates that we add a full block of padding set to the value 16. This ensures that, to remove the padding, one can always look at the last byte, and the value found there, is the number of bytes to discard in order to get to the main data.

Most implementations of AES do not make the padding scheme configurable to the developer, hence you won't need to fiddle around with the padding scheme most of the time. Irrespective it is good to know, to have a better understanding of AES.

Mode of operation

The consequence of AES being a block cipher, that is being able to only encrypt 128 bits at a time leads to two things: We need to provide a padding mechanism, which was mentioned in the previous section.

We also need a way to put the pieces of encrypted 128 bits together to get the final ciphertext. The way by which the individual encrypted 128 bits are assembled together is called Mode of operation and there are various ways to go about this.

There are various modes that have been defined for block ciphers (i.e. not limited only to AES), but to have a better understanding of what they are actually supposed to help with, we look at two modes: ECB mode and CBC mode.

ECB - Electronic Code Block - Mode

A naive way to piece back the encrypted blocks, is to assemble them back exactly the way the plaintext was broken up. This mode of encryption is called Electronic Code Book (ECB).

The ECB mode has the drawback that it leaks information about the plaintext.

If the plaintexts are broken apart, encrypted and assembled back the same way, patterns about the data can still be carried over into the encrypted version, because similar bits in the plaintext would encrypt to similar cipher text. This allows some information to still be learnt about the plaintext, which defeats the purpose of encryption. A good encryption scheme should not reveal any useful information about the encrypted message, none at all.

A very good example of illustrating this weakness of the ECB mode is the ECB penguin. An image of a penguin is encrypted using the ECB mode, but the shape of the penguin can still be seen in the encrypted image.

Hence for all intents and purposes, ECB mode is usually not used.

CBC - Cipher Block Chaining - Mode

The Cipher Block Chaining mode is another mode of operation that can be used with AES that does not suffer from the weakness of ECB.

The problem with the ECB mode is that identical portions of the plaintext would encrypt to identical cipher text. This is what enables patterns to still be carried over to the ciphertext.

The CBC mode avoids this problem by including a procedure in its algorithm that first randomises the plaintext before encryption. This process of randomisation ensures that identical portions of the plaintext will never encrypt to the same ciphertext.

Simply put, CBC achieves this randomisation by taking a previous encrypted block and using this to randomise (by performing an XOR operation) the next block to be encrypted. This then provides the output that is then fed as the next block to the encryption. This chaining goes on until the plaintext is fully encrypted into the ciphertext.

You might ask, how will the first block be randomised before encryption since there is no previous cipher text that can be used for the XOR operation? The answer is that CBC used with AES requires an additional 16 bytes value called the Initialisation vector (IV) to kickstart the encryption process. This is the value that is used to randomise the first block to kickstart the encryption process. It should be noted that the IV also needs to be random and unpredictable in order to have a secured encryption scheme.

So far we have seen what encryption is, what it is not, the difference between symmetric encryption and asymmetric encryption and also the difference between block ciphers and stream ciphers. We also saw how block ciphers lead to needing padding and mode of operation. The weakness of ECB mode and how CBC avoids that.

Next we will see how encrypting data with CBC looks like in code. To do this, we would again be using Deno.

The following code shows how to encrypt and deprect a piece of text using AES with the CBC mode.

// We generate a key
// See https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/generateKey
let key = await crypto.subtle.generateKey({
   name: 'AES-CBC',
   length: 128
}, false, ['encrypt', 'decrypt']);

// We generate random 16 bytes to be used as initialization vector
let iv = new Uint8Array(16);
// and fill it with random numbers
// see https://developer.mozilla.org/en-US/docs/Web/API/Crypto/getRandomValues
await crypto.getRandomValues(iv);

// The web crypto requires data to be in ArrayBuffer
let plaintext = new TextEncoder().encode("hello world");

// The encryption and decryption requires specifying
// the algorithm and initialization vector
let param = {
   name: 'AES-CBC',
   iv: iv
};

// We encrypt the plaintext to get the ciphertext
let ciphertext = await crypto.subtle.encrypt(param, key, plaintext);

// We decode and console logged the ciphertext
console.log(new TextDecoder("utf-8").decode(ciphertext))

// We decrypt the ciphertext back to plaintext
let deCryptedPlaintext = await window.crypto.subtle.decrypt(param, key, ciphertext);

// We tranform the plaintext from bytes to string and then print it out
console.log(new TextDecoder("utf-8").decode(deCryptedPlaintext));

If you copy this code into a filename enc_dec_aec_cbc.ts and then execute deno run enc_dec_aec_cbc.ts, you should see two lines printed out. One looks like gibberish binary blob, while the second line should be "hello world"

As the comment in the code shows, we are generating a key, and IV, and then used to encrypt the text "hello world". Then print the cipher text and then decrypt it back to the plain text "hello world".