Table I.—Normal frequency table. Frequency for ten thousand letters and for two hundred letters. This latter is put in graphic form and is necessarily an approximation. Taken from military orders and reports, English text. [6]

Vowels AEIOU = 38.37%; consonants LNRST = 31.86%; consonants JKQXZ = 1.77%.

The vowels may be safely taken as 40%, consonants LNRST as 30% and consonants JKQXZ as 2%.

Order of letters: E T O A N I R S H D L U C M P F Y W G B V K J X Z Q.

Table II.—Frequency table for telegraph messages, English text. This table varies slightly from the standard frequency table because the common word “the” is rarely used in telegrams and there is a tendency to use longer and less common words in preparing telegraph messages. [7]

In this table the vowels AEIOU = 40.08%, consonants LNRST = 30.77% and consonants JKQXZ = 2.25%.

Orders of letters: E O A N I R S T D L H U C M P Y F G W B V K X J Q Z.

Table III.—Table of frequency of digraphs, duals or pairs (English). This table was prepared from 20,000 letters, but the figures shown are on the basis of 2,000 letters. For this reason they are, to a certain extent, approximate; that is, merely because no figures are shown for certain combinations, we should not assume that such combinations never occur but rather that they are rare. The letters in the horizontal line at the top and bottom are the leading letters; those in the vertical columns at the sides are the following letters. Thus in two thousand letters we may expect to find AH once and HA twenty-six times. [8]

Table IV.—Order of frequency of common pairs to be expected in a count of 2,000 letters of military or semi-military English text. (Based on a count of 20,000 letters).

Table V.—Table of recurrence of groups of three letters to be expected in a count of 10,000 letters of English text.

[9]

Table VI.—Table of frequency of occurrence of letters as initials and finals of English words. Based on a count of 4,000 words; this table gives the figures for an average 100 words and is necessarily an approximation, like Table III. English words are derived from so many sources that it is not impossible for any letter to occur as an initial or final of a word, although Q, X and Z are rare as initials and B, I, J, Q, V, X and Z are rare as finals.

It is practically impossible to find five consecutive letters in an English text without a vowel and we may expect from one to three with two as the general average. In any twenty letters we may expect to find from 6 to 9 vowels with 8 as an average. Among themselves the relative frequency of occurrence of each of the vowels, (including Y when a vowel) is as follows:

The foregoing tables give all the essential facts about the mechanism of the English language from the standpoint of the solution of ciphers. The use to be made of these tables will be evident when the solution of different types of ciphers is taken up.

The Spanish language is written with the following alphabet:

while the exact sense often depends upon the use of accents over the vowels. However, in cipher work it is exceedingly inconvenient to use the permanent digraphs, CH, LL and RR and they do not appear as such in any specimens of Spanish or Mexican cipher [10]examined. Accented vowels and Ñ are also not found and we may, in general, say that a cipher whose text is Spanish will be prepared with the following alphabet:

and the receiver must supply the accents and the tilde over the N to conform to the general sense.

However, many Mexican cipher alphabets contain the letters K and W. This is particularly true of the ciphers in use by secret service agents who must be prepared to handle words like NEW YORK, WILSON and WASHINGTON. The letters K and W will, however, have a negligible frequency except in short messages where words like these occur more than once.

In this connection, if a cipher contains Mexican geographical names like CHIHUAHUA, MEXICO, MUZQUIZ, the letters H, X and Z will have a somewhat exaggerated frequency.

In Spanish, the letter Q is always followed by U and the U is always followed by one of the other vowels, A, E, I or O. As QUE or QUI occurs not infrequently in Spanish text, particularly in telegraphic correspondence, it is well worth noting that, if a Q occurs in a transposition cipher, we must connect it with U and another vowel. The clue to several transposition ciphers has been found from this simple relation.

Table VII.—Normal frequency table for military orders and reports, calculated on a basis of 10,000 letters of Spanish text. The graphic form is on a basis of 200 letters. [11]

In this table the vowels AEIOU = 45.65%; consonants LNRST = 30.33%; consonants JKQXZ = 1.76%.

Order of letters:

E A O R S N I D L C T U M P G Y (BH) F Q V Z J X.

Table VIII.—Table of frequency of digraphs, duals or pairs, Spanish text. Like Table III, this table is on the basis of 2,000 letters although prepared from a count of 20,000 letters. For this reason it is, to a certain extent an approximation; that is, merely because no figures are shown for certain combinations, we should not assume that such combinations never occur but rather that they are rare. The letters in the horizontal lines at the top and bottom are the leading letters; those in the vertical columns at the sides are the following letters. Thus, in two thousand letters, we may expect to find AI twice and IA twenty-three times. [12]

Table IX.—Order of frequency of common pairs to be expected in a count of 2,000 letters of Spanish military orders and reports. Based on Table VIII.

[13]

Frequency of occurrence in 1,000 letters of text:

French

German

Italian

Portuguese

[14]

Frequency of occurrence in 200 letters of text.

French

Italian

[15]

German

Portuguese

[16]

Case 1.—Geometrical ciphers. This case includes all ciphers in which a certain number of the characters are chosen so that they will form a square or rectangle of predetermined dimensions; and then these characters are arranged according to a geometrical design.

Taking the message:

A B C D E F G H I J K L M N O P Q R S T U V W X

of twenty-four letters and assuming a rectangle of six letters horizontally, and four letters vertically, we may have:

(a) Simple Horizontal:

(b) Simple Vertical:

[27]

(c) Alternate Horizontal:

(d) Alternate Vertical:

(e) Simple Diagonal:

(f) Alternate Diagonal:

(g) Spiral, clockwise:

(h) Spiral, counter clockwise:

It is simply a matter of inspection to read a message in a cipher of this type, once the dimensions of the rectangles have been determined. We place the whole or a portion of the message in such rectangles and read horizontally, vertically and diagonally forward and backward. Parts of words will at once be apparent and the whole message is soon deciphered. Two examples will show the process. [28]

Message

ILVGIOIAEITSRNMANHMNG

This message contains eight vowels or 38% out of twenty-one letters, and the letters LNRST occur 7 times or 33%, the letters XQJKZ not appearing. It is therefore a transposition cipher. Twenty-one letters immediately suggest seven columns of three letters each or three columns of seven letters each. Trying the former we have:

and reading down each column in succession (Case 1-b) reveals the message to be “I am leaving this morning.”

Message

There are 120 letters in this message with 57 vowels or 47% vowels, and the letters LNRST occur 31 times or 26% of the whole.

Non-occurrence of K and W and vowel proportion leads us to the assumption that it is a transposition cipher of a Spanish text. The factors of 120 are 5 × 3 × 2 × 2 × 2. We may then have one rectangle of 4 × 30 or one of 5 × 24 or two of 5 × 12, or three of 5 × 8, or four of 5 × 6, or five of 3 × 8, or ten of 3 × 4, or twenty of 3 × 2. The message being in a rectangle of 4 × 30, we can inspect it as it stands and this is clearly not the arrangement if it be a geometrical transposition cipher at all. It is best however to try the largest possible rectangles first so we will put it in the form 5 × 24, thus:

[29]

Here an inspection shows this to be Case 1-f, alternate diagonal, and the text to be “ME SITUO SOBRE PARRAL PORQUE ME PRESENCIA FUE REVELADA POR U”; here the sense breaks but note that U is the twelfth letter of the line and continue as if the rectangle were 5 × 12 and we have “NA PAREJA QU.” Now inspect the second rectangle of 5 × 12 in the same way and the sense continues “E SE ME ACERCO Y HUBO QUE RECHAZAR POR EL FUEGO ALLI ESRERO ORDENES FINISX”.

The practical way of examining a cipher of this type is to have several men prepare rectangles of different dimensions, using the letters of the cipher in the order received. The rectangles can be inspected very rapidly when once prepared. Note that the dimensions of any rectangle will rarely be such as to contain more than fifty letters, on account of the necessity of filling up a rectangle with nulls if the number of letters of the message is just a little greater than a multiple of the rectangle. Also large rectangles give, for all but the diagonal method, whole words in a line or column and these are easily noted.

The following ciphers come under Case 1:

Case 1-i.—The rail fence cipher, useful as an operators’ cipher but permits of no variation and is therefore read almost as easily as straight text when the method is known. The message:

HOSTILE CAVALRY HAS RETIRED

is written:

and is sent:

OTLCV LYART RDHSI EAARH SEIEX [30]

Case 1-j.

Message

To solve this cipher, read down the columns in this order 8, 1, 15, 2, 14, 3, 13, 4, 12, etc. A variation is to arrange the cipher so the columns are read upwards. Another is to arrange the ciphers so the columns are read alternately upward and downward. The factors of the number of letters in this case give the shape of the rectangle as usual.

It will be seen that there are a great number of possible transposition ciphers that come under Case 1 but practically all of them are useless from a military standpoint because they do not depend on a key which can be readily and frequently changed. However such ciphers constantly crop up in cipher examination, being used for special communication between parties who consider the regular military ciphers too complicated. Thus some of these expedients have been used.

Reversed Writing.—(Special case of Case 1-a).

LEAVING TONIGHT is enciphered THGINOT GNIVAEL or it may be reversed by words, thus GNIVAEL THGINOT or by groups of five letters, thus IVAEL NOTGN XTHGI.

Vertical Writing.—(Special case of Case 1-b). Same message is enciphered,

[31]

Case 2.—This case includes all transposition ciphers in which lines and columns of the text are rearranged according to some key word or key number. There are many varieties of this case but their solution usually is arrived at through the methods suggested for Case 1, that is, arrangement into appropriate rectangles and examination of lines and columns for words or syllables. Rearrangement of columns or lines follows until the solution is completed.

Case 2-a.

Message

There are 108 letters in this message and examination shows it to be a transposition cipher, English text. The number of letters, 108, immediately suggests a rectangle of 12 × 9 or 9 × 12 letters. Put into this form we have:

The vowel count of the lines shows the first arrangement to be the more likely. We will now number the columns and try pairing off certain ones which in no line would give impossible combinations of letters.

[32]

These combinations appear among others:

The word FIGHT stares at us from the first line; let us arrange the columns thus:

We have the words FIGHTI(NG), VICIN(ITY), RENEWE(D), ANDVIL(LA), RDINGT(O), RECE(IVED). With this to go on, we must choose column 11 as the next one and then in order, columns 8, 10, 7, 12, 9. But note that the order 11, 8, 10, 7, 12, 9, is the same as the order 5, 2, 4, 1, 6, 3. The message was written in twelve columns and the columns have been transposed in that order. We may, although it is entirely unnecessary, speculate on the key word used. It was probably

meaning that the 4th column of the plain text was transferred in enciphering so it became our 1st, the 2d column remained the 2d; the 6th column became our 3d, etc.

Actually, this cipher was solved because the word VILLA was suspected and all the necessary letters were found in line six of the arrangement in [33]twelve columns. The order 1, 6, 3, 11, 8 was tried and gave this result.

The remainder of the solution followed the lines already laid down and, naturally, offered no difficulties, in view of the large number of connected syllables available.

Case 2-b.

Message

This is a transposition cipher, English text, and contains 105 letters. The factors of 105 are 5 × 3 × 7 so that we must investigate the following rectangles; 5 × 21, 15 × 7, three of 5 × 7, five of 3 × 7 and seven of 5 × 3.

[34]

This is also excellent, so we will try three blocks 5 × 7 and see if rearrangement of horizontal lines will give results reading the columns vertically.

Among other combinations are:

The addition of line 6 above line 3 and line 4 below line 7 will complete this cipher. The successive columns should be read downward.

Case 2-c. In this case, both lines and columns are rearranged by means of a key word or key words. The method of solution is the same as Case 2-a and 2-b except that the lines must be rearranged after the columns have been correctly arranged, or in some cases, vice versa. This cipher is not infrequently met with because it seems to offer safety by use of two key words and by the great but only apparent complexity of the method.

Message

This is a transposition cipher, English text and [35]the number of letters, 70, leads us to try rectangles of 10 × 7 and 7 × 10.

The first form looks the more likely from the vowel count. We proceed to number the columns and lines and try rearrangement of columns so as to obtain possible letter combinations from every line.

Among other combinations we have these:

A very casual inspection of the lines shows that they should be rearranged in order 6, 1, 2, 7, 3, 5, 4, as follows:

Although of no particular importance, it may be stated that the column key in this case was GRAND [36]and the line key was CENTRAL, both used as in enciphering Case 2-a.

Case 3. Route ciphers. In this case, whole words of the message are transposed according to some of the methods of Case 1 or 2 or their equivalents. The route cipher is little used at present. Its development and use during the Civil War was caused by the inability of the telegraphers of that day to handle regular cipher matter correctly and rapidly. It was, even in those days, frankly only a delaying cipher and, to be of any value, had to be filled with meaningless words to conceal the message proper. An example from the Signal Book will suffice to show the general character of route ciphers. To one familiar with monoliteral transposition ciphers, even the best of route ciphers offers but little difficulty.

“To encipher the message ‘MOVE DAYLIGHT. ENEMY APPROACHING FROM NORTH. PRISONERS SAY STRENGTH ONE HUNDRED THOUSAND. MEET HIM AS PLANNED.’ arrange as follows:

Here the route is down the first column, up the fourth, down the second and up the third.”

This cipher was often complicated by the introduction of nulls for every fifth word. Thus the above message might be sent:

The words in italics are nulls and not a part of [37]the message and the receiver eliminates them before arranging his message in columns to get the sense of it.

As an additional complication, it was customary for each correspondent to have a dictionary or code in which the names of all prominent generals and places and many of the prominent verbs,—as to march, to sail, to encamp, to attack, to retreat,—were represented by other words.

A route cipher using the code words of the War Department code might have some advantages over the method of enciphering code messages as prescribed in that Code.

It is the consensus of opinion of experts that the transposition cipher is not the best one for military purposes. It does not fulfill the first, second, and third of Kerckhoffs’ requirements as to indecipherability, safety when apparatus and method fall into the hands of the enemy, and dependability on a readily changeable key word.

However, transposition ciphers are often encountered. They are favorites with those who find the substitution ciphers too difficult and too tedious to handle and who believe that their transposition methods are either absolutely indecipherable or sufficiently so for the purpose of concealing the text of a message for the time being. They seem to be particularly popular with secret agents and spies, presumably because special apparatus is rarely necessary in enciphering and deciphering.

Although the number of transposition methods is legion, they can practically all be considered under one of the three cases already discussed. It is surprising how often transposition ciphers prepared [38]by complicated rules, will, on analysis, be seen to be very simple.

To be successful in solving transposition ciphers, one should constantly practice reading backward and up and down columns, so that the common combinations of letters are as quickly identified when seen thus as when encountered in straight text. Combinations like EHT, LLIW, ROF, DNA, etc., should be appreciated immediately as common words written backward.

A study of the table of frequency of digraphs or pairs is also excellent practice and such a table should be at hand when a transposition cipher is under consideration. It assists greatly if Case 2 be encountered and is of considerable use in solving Case 1.

The solution of route ciphers is necessarily one of try and fit, with the knowledge that such ciphers usually are read up and down columns. It is not believed that route ciphers will often be met with at the present day. [39]

Case 4-a.

Message

OBQFO BPBRP QBAML OBHIF PILFQ FJBOX OFLNR BIXOZ EL

From the recurrence of B, F and O, we may conclude that a single substitution alphabet was used for this message. If so and if the alphabet runs in the same order and direction as the regular alphabet, the simplest way to discover the meaning of the message is to take the first two words and write alphabets under each letter as follows, until some line makes sense:

The word RETIRESE occurs in the fourth line, and, if the whole message be handled in this way we find the rest of the fourth line to read USTED POR EL MISMO ITINERARIO QUE MARCHO. The message was enciphered using an alphabet where A = X, B = Y, C = Z, D = A, etc. noting that as this message is in Spanish the letters K and W do not appear in the alphabet. [41]

Case 4-b.

Message

HUJZH UIUPN OZYTS VQXMI SMOMX MQHUD UMREI SESJU AG

This is a message in Spanish. We will handle it as in case 4-a, setting down the whole message.

Here each word of the message comes out on a different line, and noting in each case the letter corresponding to A, we have the word QUEMADOS which is the key. The cipher alphabet changed with each word of the message.

A variation of this case is where the cipher alphabet changes according to a key word but the change comes every five letters or every ten letters of the message instead of every word. The text of the message can be picked up in this case with a little study.

Note in using case 4 that if we are deciphering a Spanish message we use the alphabet without K or W as a rule, altho if the letters K or W appear in [42]the cipher it is evidence that the regular English alphabet is used.

Case 5-a.

Message

DNWLW MXYQJ ANRSA RLPTE CABCQ RLNEC LMIWL XZQTT QIWRY ZWNSM BKNWR YMAPL ASDAN

This message contains K and W and therefore we expect the English alphabet to be used. The frequency of occurrence of A, L, N, R and W has lead us to examine it under case 4 but without result. Let us set down the first two words and decipher them with a cipher disk set A to A and then proceed as in case 4.

The message is thus found to be enciphered with a cipher disk set A to E and the text is: BRITISH GOVERNMENT PLACED CONTRACTS WITH FOLLOWING FIRMS DURING SEPTEMBER.

Case 5-b.

Same as case 4-b except that the cipher message must be deciphered by means of a cipher disk set A to A before proceeding to make up the columns of alphabets. The words of the deciphered message will be found on separate lines, the lines being indicated as a rule by a key word which can be determined as in case 4-b.

The question of alphabetic frequency has already been discussed in considering the mechanism of language. It is a convenient thing to put the frequency tables in a graphic form and to use a similar graphic form in comparing unknown alphabets with the standard frequency tables. For instance the standard Spanish frequency table put in graphic [43]form is here presented in order to compare with it the frequency table for the message discussed in case 4-a.

Our first assumption might be that B = A and F = E but it is evident at once that in that case, S, T, U and V (equal to R, S, T and U) do not occur and a message even this short without R, S, T or U is practically impossible. By trying B = E we find that the two tables agree in a general way very well and this is all that can be expected with such a short message. The longer the message the nearer would its frequency table agree with the standard table. Note that if a cipher disk has been used, the alphabet runs the other way and we must count upward in working with a graphic table. Note also that if, in a fairly long message, it is impossible to coördinate the graphic table, reading either up or down, with the standard table and yet some letters occur much more frequently than others and some do not occur at all, we have a mixed alphabet to deal with. The example chosen for case 6-a is of this character. An examination of the frequency table given under that case shows that it bears no graphic resemblance to [44]the standard table. However, as will be seen in case 7-b, the preparation of graphic tables enables us to state definitely that the same order of letters is followed in each of a number of mixed alphabets.

Any substitution cipher, enciphered by a single alphabet composed of letters, figures or conventional signs, can be handled by the methods of case 6. For example, the messages under case 4-a and 5-a are easily solved by these methods. But note that the messages under case 4-b and 5-b cannot so be solved because several alphabets are used. We will see later that there are methods of segregating the different alphabets in some cases where several are used and then each of the alphabets is to be handled as below.

Case 6-a.

Message

This message was received from a source which makes us sure it is in Spanish. The occurrence of B, H, P and S has tempted us to try the first two words as in case 4 and 5 but without result. We now prepare a frequency table, noting at the same time the preceding and following letter. This latter proceeding takes little longer than the preparation of an ordinary frequency table and gives most valuable information. [45]

Frequency Table

It is clear from an examination of this table that we have to deal with a single alphabet but one in which the letters do not occur in their regular order.

We may assume that P and S are probably A and E, both on account of the frequency with which they occur and the variety of their prefixes and suffixes. If this is so, then B and H, are probably consonants and may represent R and N respectively. D and X are then vowels by the same method of analysis. Noting that HC occurs three times and taking H as N we conclude that C is probably T. Substitute these values in the last three words of [46]the message because the letters assumed occur rather frequently there.

Now Z is always prefixed by S and may be L. Taking X=I and D=O, (they are certainly vowels), V=G and Y=M, we have

Substituting these values in the rest of the message we have

We may now take Q=F, O=D, E=S, R=B, T=C, A=P and J=U and the message is complete. We are assisted in our last assumption by noting that S=E and E=S, etc., and we may on that basis reconstruct the entire alphabet. The letters in parenthesis do not occur in the message but may be safely assumed to be correct.

It is always well to attempt the reconstruction of the entire alphabet for use in case any more cipher messages written in it are received.—— [47]

Case 6-b.

Message

The address and signature indicate that this message is in English.

There are 250 letters in the cipher; the vowels AEIOU occur 109 times or 43.6%, the letters LNRST occur 62 times or 24.8%, and the letters KQVXZ occur 5 times or 2%. The proportion in the case of the vowels is somewhat too large and, in the case of the letters LRNST, it is too small. It is then questionable whether this is a transposition cipher altho, at first glance it might appear to be one.

On examination for parts of possible words we are at once struck by the occurrence at irregular intervals of recurring groups, viz:

This is a strong indication that the cipher is a substitution cipher, so, to make an examination a frequency table will be constructed.

[48]

Superficially, this looks like a normal frequency table, but O is the dominant letter, followed by H, E, A, T, I, N, S, in the order named. It is certainly Case 6 if it is a substitution cipher at all.

Let us see what can be done by assuming O=E; the triplet ENO, occurring six times might well be THE and E=T and N=H. A glance at the frequency table shows this to be reasonable. Now substitute these letters in some likely groups. FNOHOENO becomes _HE_ETHE; FTEN becomes _TH; ENOENHO becomes THETH_E; ENOHO becomes THE_E. A bit of study will show that F=W, T=I and H=R and the frequency table bears this out except that H(=R) seems to occur too frequently. The recurring groups containing DAC (see above) occur in such a way that we may be sure DAC is one word, FTRR is another and FTEN(=WITH) is a third. Now FTRR becomes WI__, which can only be completed by a double letter. LL fills the bill and we may say R=L. As DAC starts the message and is followed by FTRR (=WILL) it is reasonable to try DAC=YOU. Looking up DAC in the frequency table it is evident that we strain nothing by this assumption. We now have:

Now take the group ENOUTHOMEAH which occurs twice. This becomes THE_IRE_TOR and if we substitute U=D and M=C we have THE DIRECTOR. Next the group (FTRR)BHAMOOUEA becomes (WILL) _ROCEEDTO and the context gives word with missing letter as PROCEED, from which B=P. Next the group (ENO) IZTIETASMOSEOHIEYOCK(FNOHO) becomes (THE)__I_TIO_CE_TER_T_EU_(WHERE) and the group (FTEN)EFAPHOSMNIZTIEAHL becomes (WITH)TWO_RE_CH__I_TOR_. [49]The substitution of A for I, V for Z, N for S and F for P makes the latter group read (WITH TWO FRENCH AVIATORS and the former read (THE)AVIATION CENTER AT _EU_(WHERE).

Now the word YOCK = (_EU_) is the name of a place, evidently. We find another group containing Y, viz: ENOSTSMAYBISD which becomes THENINCO_PANY so that evidently we should substitute M for Y. The other occurrence of Y (=M) is in the group EAYOEQISU which becomes TOMET_AND. A reasonable knowledge of geography gives us the words MEUX and METZ so that X should be substituted for K and Z for Q.

We now have sufficient letters for a complete deciphering of the message.

The message deciphers:

The substitution of B for G, G for W and K for V completes the cipher. This cipher is difficult only because the cipher alphabet is made up, not haphazard, but scientifically with proper consideration for the natural frequency of occurrence of the letters. In cipher work it is dangerous to neglect proper analysis and jump at conclusions.

In the study of Mexican substitution ciphers, several alphabets have been found which are made up in a general way, like the one discussed in this case.

Case 6-c.—It is a convenience in dealing with ciphers made up of numbers or conventional signs to substitute arbitrary letters for the numbers and signs. Suppose we have the message:

By arbitrary substitution of letters this is made

This message is now in convenient shape to handle as Case 6-a and on solution is found to read:

In the same way the message

is found to be made up entirely of numbers between 11 and 36 with the numbers 23, 28 and 30 occurring most frequently. This immediately suggests an alphabet made up of the numbers from 11 to 36 inclusive and each cipher group of figures represents two letters. By arbitrary substitution of letters for groups of two numbers we obtain:

and this message is also in shape to handle as Case 6-a. It reads, on solution,

[51]