End-to-End Encryption: A Technical Perspective

This post delves into the technical intricacies of E2EE, highlighting its significance in safeguarding digital communications against unauthorized access.

Definition and Distinction

While cryptography is a traditional research area in computer science, and E2EE has become a practical reality for many users worldwide with its implementation in popular messenger apps, it's surprising to note that a universally accepted definition of E2EE does not exist. As of the writing of this article, the IETF's "Definition of End-to-end Encryption" is in progress, but currently only holds informal status (Knodel et al., 2023).

This can be attributed to the changing meaning of the term over time. In the late 1980s, Nelson and Heilmann saw E2EE as situated either on the transport or the network layer of the OSI model (ISO, 1994; Nelson & Heimann, 1989, p. 357). Current definitions, however, often place E2EE at the application layer, as per the BSI: "[End-to-end encryption means] encrypting transmitted data across all transmission stations. Only the communication partners as endpoints can decrypt the data." (BSI, 2021, p. 17). "With good end-to-end encryption, an end user only needs to trust the endpoints of the communication: themselves, the counterpart, and the clients. In between – on the internet and the servers – the communication is secure." (BSI, 2021, p. 7). NIST shares this view: "In end-to-end encryption, data is encrypted when being passed through a network, but routing information remains visible. End-to-end encryption is generally performed by the end-user organization." (Nieles et al., 2017, p. 67).

At this point, E2EE can be distinguished from other types of encryption, which are often not clearly differentiated in general language use.

Encryption During Transit

Transport encryption is considered a best practice today, but this was not always the case, especially in web browsers. It was only after Google began considering encryption as a ranking factor in its search engine in 2014 that website operators started to widely adopt HTTPS. When browsers began to notify users about the security status of their connections to websites, HTTPS ultimately supplanted HTTP (Gupta, 2017, p. 79). However, HTTPS only encrypts the data traffic between the client (browser, mobile app, IoT device, etc.) and the server. The data remains unencrypted at both the client and the server, and any systems behind them.

Encryption During Storage

Data can also be encrypted while stored. Many database systems offer the necessary concepts for this (Bouganim & Guo, 2009). This ensures that data is protected when attackers gain passive access, such as in the case of theft of a backup copy. However, in an attack scenario where there is also active access to the server or surrounding systems, this approach risks the encryption keys required for operation falling into the hands of the attackers (Shmueli et al., 2010, p. 31).

Encryption During Transit and Storage

The approaches of encrypting data during transit and storage can be combined, and this is commonly practiced today (AWS, 2022; Google, n.d.). However, even assuming the client is completely trustworthy, the server remains a potential harmful actor in this scenario. This is because all data is processed in plaintext there, either after being received from the client or read from the database.

End-to-End Encryption

This is where the concept of E2EE shows its strengths. Encryption and decryption occur exclusively "at the ends" on the client side (Hale & Komlo, 2022, pp. 6ff). The server is eliminated as a weak point. Theoretically, encryption during transit and storage could be dispensed with in this scenario, since the data is already encrypted at the sending client. However, in practice, transport encryption is not only considered best practice but is often also a requirement. For example, Apple expects developers of native apps for its platforms to use transport encryption for communication between these apps and their servers (Conger, 2016).

Properties

What constitutes security in terms of end-to-end encryption? A closer look at the specific properties offered by various protocols and implementations is helpful here.

According to Hale and Komlo, the central property of E2EE is confidentiality: "In terminology, end-to-end encryption applies to the guarantee of endness for confidentiality only." (Hale & Komlo, 2022, p. 12). Confidentiality here means that the plaintext of encrypted information is only available to those for whom the information is intended (BSI, n.d.). However, Hale and Komlo note that many modern messaging applications also incorporate the protection of authenticity, the second central property (Hale & Komlo, 2022, p. 12). This means that recipients of an encrypted message can be sure that the sender is indeed who they claim to be (BSI, n.d.). The third central property is the protection of integrity, which ensures that the message is not unnoticedly manipulated between the sender and recipient (BSI, n.d.). Additionally, there are other properties that are considered optional but desirable, see the table below.

Encryption Methods

Symmetric Encryption

In symmetric encryption, both parties, the sender (Alice) and the recipient (Bob), use the same key to encrypt and then decrypt an exchanged message (Eckert, 2018, p. 300; Spitz et al., 2011, p. 22). This method is highly relevant in practice because "the encryption and decryption algorithms used are generally based on very simple operations (including shifts, XOR) that can be efficiently implemented in hardware or software" (Eckert, 2018, p. 300).

However, symmetric encryption also leads to one of the fundamental problems of cryptography, the so-called key exchange problem:

"It makes no sense for Alice and Bob to agree on a common key over the same transmission channel they will later use to send encrypted messages. Because if [a potential attacker] Mallory listens to the transmission channel, then he will get the key"

(Schmeh, 2013, p. 175).

This problem could be solved if Alice and Bob used a different channel to exchange the key beforehand, called out-of-band key exchange (Naor et al., 2019). However, if Alice communicates not only with Bob but with many other people, this approach quickly becomes impractical (Schmeh, 2013, p. 176).

Asymmetric Encryption

A better solution for the key exchange problem is the second fundamental method, asymmetric encryption:

"The central idea of asymmetric cryptosystems is the use of a pair of keys for each communication partner involved. Each key pair consists of a secret and a public key, with the public key being stored in publicly accessible databases. A plaintext is encrypted by the sender with the recipient's public key, and the recipient decrypts the ciphertext with their secret key. Thus, no secret key needs to be exchanged between the communication partners."

(Eckert, 2018, p. 331)

In short: using an asymmetric algorithm E, Alice can encrypt plaintext m using Bob's public key, BobPub, to get the ciphertext c. Bob can then decrypt c using the algorithm D and his private key, BobPriv, to convert it back into the plaintext m:

c = E(m, BobPub) => m = D(c, BobPriv)

Hybrid Encryption

Yet another problem remains to be solved:

"Encryption and decryption with asymmetric methods are about 1,000 times slower than with symmetric methods. Asymmetric methods are therefore far too costly for large amounts of data. The solution is to encrypt only a symmetric key with an asymmetric method and then use the symmetric key to encrypt the large amount of data."

(Spitz et al., 2011, p. 28)

This hybrid method also solves the key exchange problem of symmetric encryption. To send Bob a message of any size encrypted symmetrically, Alice no longer needs to worry about how to securely get the key to Bob. She generates the key herself and encrypts it asymmetrically. Bob can now use his private key to obtain the symmetric key and decrypt Alice's message in a second step.

Algorithms

When implementing the encryption methods discussed, it's crucial to select the most suitable algorithms available at the time of implementation. The best choice of algorithm in any given case depends on various factors.

Over time, some formerly popular algorithms have been ruled out due to successful cryptanalysis. This means attack methods have been found that can break the respective encryption, making these algorithms no longer considered secure. This includes algorithms like DES, Triple-DES, and SKIPJACK (Barker, 2020, pp. 24f).

Another aspect to consider is the assumed future-proofing of an algorithm and its parameters, such as the lengths of the keys used:

"The development of the performance of classical computers can nowadays be relatively well estimated. Fundamental scientific advances (either in terms of attack algorithms or the development of a cryptographically relevant quantum computer), on the other hand, are not predictable. Therefore, any prediction beyond a period of six to seven years is difficult, especially for asymmetric methods, and even for this period of six to seven years, the forecasts can prove to be wrong due to unforeseeable developments."

(BSI, 2022, p. 20)

The frequency and thoroughness of an algorithm's examination compared to possible alternatives should also be factored into its evaluation. For example, the BSI notes that although no negative findings are known for the block ciphers Serpent and Twofish, the Rijndael algorithm is considered to have been much more intensively studied and is therefore preferred (BSI, 2022, p. 27).

Ultimately, an algorithm must also be practically usable. That means appropriate libraries should be available, and support should be provided by the respective platform.

For the implementation of symmetric encryption, the BSI currently recommends the Advanced Encryption Standard (AES) with key lengths of either 128, 192, or 256 bits (BSI, 2022, p. 27).

Asymmetric encryption is more complex in its implementation and requires several algorithms – for generating the key pairs and for encryption and decryption (BSI, 2022, p. 31). These, in turn, utilize "further subcomponents, such as hash functions, message authentication codes, random number generation, key derivation functions, and/or block ciphers" (BSI, 2022, p. 33). A selection must be made for each of these as well. As of the time of writing this post, the BSI advocates for the Elliptic Curve Integrated Encryption Scheme (ECIES), the Discrete Logarithm Integrated Encryption Scheme (DLIES), and the Rivest Shamir Adleman method (RSA) (BSI, 2022, p. 32).

References

• AWS. (2022): Encrypting Data-at-Rest and -in-Transit—Logical Separation on AWS.
• Barker, E. (2020): Guideline for Using Cryptographic Standards in the Federal Government: Cryptographic Mechanisms(NIST SP 800-175B; S. NIST SP 800-175B). National Institute of Standards and Technology.
• Bienstock, A., Fairoze, J., Garg, S., Mukherjee, P., & Raghuraman, S. (2021): What is the Exact Security of the Signal Protocol?
• Bouganim, L., & Guo, Y. (2009): Database encryption. Encyclopedia of Cryptography and Security, Springer, 1–9.
• BSI. (n.d.): Verschlüsselt kommunizieren im Internet.
• BSI. (2021): Moderne Messenger – heute verschlüsselt, morgen interoperabel?.
• BSI. (2022): Kryptographische Verfahren: Empfehlungen und Schlüssellängen.
• Conger, K. (2016): Apple will require HTTPS connections for iOS apps by the end of 2016. TechCrunch.
• Eckert, C. (2018): IT-Sicherheit, Konzepte – Verfahren – Protokolle (10. Aufl.). De Gruyter.
• Google. (n.d.): Encryption in transit. Google Cloud.
• Gupta, C. (2017): The Market’s Law of Privacy: Case Studies in Privacy and Security Adoption. IEEE Security & Privacy, 15(3), 78–83.
• Hale, B., & Komlo, C. (2022): On End-to-End Encryption.
• ISO. (1994, November): ISO/IEC 7498-1:1994 Information technology—Open Systems Interconnection—Basic Reference Model: The Basic Model. ISO.
• Knodel, M., Celi, S., Kolkman, O., & Grover, G. (2023): Definition of End-to-end Encryption (Internet Draft draft-knodel-e2ee-definition-08). Internet Engineering Task Force.
• MacDermott, Á., Heath, H., & Akinbi, A. (2022): Disappearing Messages: Privacy or Piracy?CONF-IRM 2022 Proceedings, 10.
• Marlinspike, M., & Perrin, T. (2017): The Sesame Algorithm: Session Management for Asynchronous Message Encryption.
• Nabeel, M. (2017): The Many Faces of End-to-End Encryption and Their Security Analysis. 2017 IEEE International Conference on Edge Computing (EDGE), 252–259.
• Naor, M., Rotem, L., & Segev, G. (2019): Out-of-Band Authenticated Group Key Exchange: From Strong Authentication to Immediate Key Delivery. Cryptology ePrint Archive.
• Nelson, R., & Heimann, J. (1989): SDNS Architecture and End-to-end Encryption. In G. Brassard (Hrsg.), Advances in Cryptology—CRYPTO’ 89 Proceedings (Bd. 435, S. 356–366). Springer New York.
• Nieles, M., Dempsey, K., & Pillitteri, V. Y. (2017): An introduction to information security (NIST SP 800-12r1; S. NIST SP 800-12r1). National Institute of Standards and Technology.
• Park, D., Boyd, C., & Moon, S.-J. (2000): Forward Secrecy and Its Application to Future Mobile Communications Security. In H. Imai & Y. Zheng (Hrsg.), Public Key Cryptography (Bd.1751, S. 433–445). Springer Berlin Heidelberg.
• Schmeh, K. (2013): Kryptografie: Verfahren, Protokolle, Infrastrukturen (5., aktualisierte Auflage). dpunkt.verlag.
• Shmueli, E., Vaisenberg, R., Elovici, Y., & Glezer, C. (2010): Database encryption: An overview of contemporary challenges and design considerations. ACM SIGMOD Record, 38(3), 29–34.
• Spitz, S., Pramateftakis, M., & Swoboda, J. (2011): Kryptographie und IT-Sicherheit: Grundlagen und Anwendungen (2., überarb. Aufl). Vieweg + Teubner.
• Tyagi, N., Grubbs, P., Len, J., Miers, I., & Ristenpart, T. (2019): Asymmetric Message Franking: Content Moderation for Metadata-Private End-to-End Encryption. In A. Boldyreva & D. Micciancio (Hrsg.), Advances in Cryptology – CRYPTO 2019 (Bd. 11694, S. 222–250). Springer International Publishing.