Non-commutative cryptography is the area of cryptology where the cryptographic primitives, methods and systems are based on algebraic structures like semigroups, groups and rings which are non-commutative. One of the earliest applications of a non-commutative algebraic structure for cryptographic purposes was the use of braid groups to develop cryptographic protocols. Later several other non-commutative structures like Thompson groups, polycyclic groups, Grigorchuk groups, and matrix groups have been identified as potential candidates for cryptographic applications. In contrast to non-commutative cryptography, the currently widely used public-key cryptosystems like RSA cryptosystem, Diffie–Hellman key exchange and elliptic curve cryptography are based on number theory and hence depend on commutative algebraic structures.

Non-commutative cryptographic protocols have been developed for solving various cryptographic problems like key exchange, encryption-decryption, and authentication. These protocols are very similar to the corresponding protocols in the commutative case.

In these protocols it would be assumed that G is a non-abelian group. If w and a are elements of G the notation w^a would indicate the element a⁻¹wa.

The following protocol due to Ko, Lee, et al., establishes a common secret key K for Alice and Bob.

1. An element w of G is published.
2. Two subgroups A and B of G such that ab = ba for all a in A and b in B are published.
3. Alice chooses an element a from A and sends w^a to Bob. Alice keeps a private.
4. Bob chooses an element b from B and sends w^b to Alice. Bob keeps b private.
5. Alice computes K = (w^b)^a = w^ba.
6. Bob computes K' = (w^a)^b=w^ab.
7. Since ab = ba, K = K'. Alice and Bob share the common secret key K.

This a key exchange protocol using a non-abelian group G. It is significant because it does not require two commuting subgroups A and B of G as in the case of the protocol due to Ko, Lee, et al.

1. Elements a₁, a₂, . . . , a_k, b₁, b₂, . . . , b_m from G are selected and published.
2. Alice picks a private x in G as a word in a₁, a₂, . . . , a_k; that is, x = x( a₁, a₂, . . . , a_k ).
3. Alice sends b₁^x, b₂^x, . . . , b_m^x to Bob.
4. Bob picks a private y in G as a word in b₁, b₂, . . . , b_m; that is y = y ( b₁, b₂, . . . , b_m ).
5. Bob sends a₁^y, a₂^y, . . . , a_k^y to Alice.
6. Alice and Bob share the common secret key K = x⁻¹y⁻¹xy.
7. Alice computes x ( a₁^y, a₂^y, . . . , a_k^y ) = y⁻¹ xy. Pre-multiplying it with x⁻¹, Alice gets K.
8. Bob computes y ( b₁^x, b₂^x, . . . , b_m^x) = x⁻¹yx. Pre-multiplying it with y⁻¹ and then taking the inverse, Bob gets K.

In the original formulation of this protocol the group used was the group of invertible matrices over a finite field.

1. Let G be a public non-abelian finite group.
2. Let a, b be public elements of G such that ab ≠ ba. Let the orders of a and b be N and M respectively.
3. Alice chooses two random numbers n < N and m < M and sends u = a^mbⁿ to Bob.
4. Bob picks two random numbers r < N and s < M and sends v = a^rb^s to Alice.
5. The common key shared by Alice and Bob is K = a^{m + r}b^{n + s}.
6. Alice computes the key by K = a^mvbⁿ.
7. Bob computes the key by K = a^rub^s.

This protocol describes how to encrypt a secret message and then decrypt using a non-commutative group. Let Alice want to send a secret message m to Bob.

1. Let G be a non-commutative group. Let A and B be public subgroups of G such that ab = ba for all a in A and b in B.
2. An element x from G is chosen and published.
3. Bob chooses a secret key b from A and publishes z = x^b as his public key.
4. Alice chooses a random r from B and computes t = z^r.
5. The encrypted message is C = (x^r, H(t) m), where H is some hash function and denotes the XOR operation. Alice sends C to Bob.
6. To decrypt C, Bob recovers t as follows: (x^r)^b = x^rb = x^br = (x^b)^r = z^r = t. The plain text message send by Alice is P = ( H(t) m ) H(t) = m.

Let Bob want to check whether the sender of a message is really Alice.

1. Let G be a non-commutative group and let A and B be subgroups of G such that ab = ba for all a in A and b in B.
2. An element w from G is selected and published.
3. Alice chooses a private s from A and publishes the pair ( w, t ) where t = w ^s.
4. Bob chooses an r from B and sends a challenge w ' = w^r to Alice.
5. Alice sends the response w ' ' = (w ')^s to Bob.
6. Bob checks if w ' ' = t^r. If this true, then the identity of Alice is established.

The basis for the security and strength of the various protocols presented above is the difficulty of the following two problems:

• The conjugacy decision problem (also called the conjugacy problem): Given two elements u and v in a group G determine whether there exists an element x in G such that v = u^x, that is, such that v = x⁻¹ ux.
• The conjugacy search problem: Given two elements u and v in a group G find an element x in G such that v = u^x, that is, such that v = x⁻¹ ux.

If no algorithm is known to solve the conjugacy search problem, then the function x → u^x can be considered as a one-way function.

A non-commutative group that is used in a particular cryptographic protocol is called the platform group of that protocol. Only groups having certain properties can be used as the platform groups for the implementation of non-commutative cryptographic protocols. Let G be a group suggested as a platform group for a certain non-commutative cryptographic system. The following is a list of the properties expected of G.

1. The group G must be well-known and well-studied.
2. The word problem in G should have a fast solution by a deterministic algorithm. There should be an efficiently computable "normal form" for elements of G.
3. It should be impossible to recover the factors x and y from the product xy in G.
4. The number of elements of length n in G should grow faster than any polynomial in n. (Here "length n" is the length of a word representing a group element.)

Let n be a positive integer. The braid group B_n is a group generated by x₁, x₂, . . . , x_n-1 having the following presentation:

Thompson's group is an infinite group F having the following infinite presentation:

Let T denote the infinite rooted binary tree. The set V of vertices is the set of all finite binary sequences. Let A(T) denote the set of all automorphisms of T. (An automorphism of T permutes vertices preserving connectedness.) The Grigorchuk's group Γ is the subgroup of A(T) generated by the automorphisms a, b, c, d defined as follows:

An Artin group A(Γ) is a group with the following presentation:

where ( factors) and .

Let F be a finite field. Groups of matrices over F have been used as the platform groups of certain non-commutative cryptographic protocols.

^[1]

• Group-based cryptography