Moderator: mosher

Announcing: First-ever collection of declassified Chaocipher-related correspondences

mosher
Super member
mosher
Super member
Joined: May 26th, 2009, 10:24 am

October 1st, 2009, 7:45 pm #1

[Here's the major part of a posting I uploaded to the sci.crypt newsgroup a few minutes ago]

===== Start =========================================

Cryptologic books and texts that mention John F. Byrne's Chaocipher (e.g., David Kahn's "The Codebreakers", articles in Cryptologia) refer tantalizingly to historical letters between Byrne and William F. Friedman, the father of American cryptology. The only letters readers have had access to are snippets quoted by Byrne in his autobiography, "Silent Years". The full correspondences between Byrne and Friedman, unfortunately, have not been readily available to the cryptologic community.

As of today, this is no longer the case.

The Chaocipher Clearing House is proud to present a first-ever comprehensive collection of Chaocipher-related correspondences between John F. Byrne, William F. Friedman, and other persons.

Thanks to numerous Chaocipher researchers in the past, these letters have been assembled from numerous sources, including declassified Freedom Of Information Act (FOIA) requests to NSA, private collections across the United States, and John F. Byrne's own autobiographical "Silent Years".

Reading these letters will give you an unprecedented historical perspective on the intriguing cipher mystery called Chaocipher.

The collection can be found at:

http://www.mountainvistasoft.com/chaoci ... spondences

===== End ====================================

I believe all Crypto Forum readers interested in Chaocipher will find the collection fascinating.

Enjoy!

Moshe


Quote
Like
Share

mosher
Super member
mosher
Super member
Joined: May 26th, 2009, 10:24 am

October 1st, 2009, 7:52 pm #2

Here are some thoughts I have about the Byrne-Friedman collection of letters.

The correspondences, spanning the years 1922-1957, illuminate the interaction between John F. Byrne and William F. Friedman in his capacity as Head Cryptanalyst, Chief Signal Office. Byrne's "Silent Years" provides a biased view of their interactions, leaving one with a sense of great injustice being done to Byrne by Friedman. Reading the full set of correspondences over the years goes a long way to restoring the historical balance, casting Friedman in a professional and fair light.

The following chronology is divided into five (5) different periods:

* 1922: Byrne's initial contact with Friedman and the Army Signal Corps.
* 1937-1938: Byrne's attempt to interest the U.S. Navy in his improved model
* 1942: Byrne contacts Friedman during World War II, after a 20 year hiatus, with the aim of submitting his machine
* 1954: Friedman delivers a talk about Byrne's "Silent Years"
* 1957: Byrne contacts Friedman for the last time, demanding an explanation from Friedman why Chaocipher was not being seriously considered

In 1922, the correspondences show that Byrne's first model was not reliable enough for consideration. Friedman did indeed analyze the system, submitting an internal paper in which he considered the system breakable. As far as can be seen, Byrne did not take Friedman up on his request for fifty messages of twenty five words each. Had Byrne done that at the time, Friedman would have conducted a thorough analysis of the cipher, notifying Byrne of the ultimate outcome. Had Byrne submitted the messages he would have saved himself much headache in the future.

1937-1938 finds Byrne trying to interest the U.S. Navy in his improved system. It seems he was dismissed without the courtesy of subjecting his system to a thorough examination. Jeffrey Hill posits in his seminal paper "Chaocipher: Analysis and Models" (page 1):
wrote:The Navy, it seems, had already selected a cryptograph from among those demonstrated. The following day, May 4, 1938, a patent application for an electro-mechanical cipher machine was filed by inventors Bern Anderson and Donald Seiler listing as assignee the United States as represented by the Secretary of the Navy. This device was an electric typewriter with a set of rotors interposed between the keyboard and the printing mechanism so that messages could be automatically enciphered or deciphered while being typed. Byrne's device had to match or exceed the sophistication of this device in order to hold the Navy's attention for more than a few minutes, but it failed to do so.
In 1942, Byrne, partly motivated by patriotism and partly to further his invention, attempted to interest Friedman once again in his system. By this time, Byrne had no doubt improved the machine and corrected its functional problems encountered in 1922. Friedman sent Byrne the two standard enclosures that explain the reasons for thorough testing and the number/types or messages required to evaluate the system. Once again, Byrne refused to comply. This was self-defeating and simply prolonged Byrne's frustration and disappointment. Had Byrne complied, the technical issues would have been put to rest one way or the other.

In 1954 Friedman was asked to present a talk about Byrne's "Silent Years", presumably because he was mentioned as a protagonist in chapter 21 of the book. Although the talk does not reveal any technical details, it is interesting to know that Friedman was fully aware of the final chapter of the book.

The 1957 correspondences bring the 35 year saga to its close, as far as it concerned Byrne and Friedman. It is sad to see how Byrne, in his self-inposed frustration, is forced to demand a straight answer from Friedman regarding Chaocipher. At this point Friedman no longer had the will or interest to pursue it at all, showing far more interest in James Joyce than in Byrne's Chaocipher.

The tragedy of it all is that Byrne could have subjected his system to a systematic, possibly sobering evaluation, thereby saving himself many years of anguish, disappointment, and frustration. It ultimately boiled down to his non-adherence to Kerckhoff's dictum that the security of a system not reside in the secrecy of its inner principles. True, Byrne's Chaocipher has not yet been broken without knowledge of its working principles. But he did not live to see the day where his cryptographic principle, whatever it was, was recognized for its originality like Jefferson's Wheel Cipher or Wheatstone's Cryptograph.

I would like to believe that one day soon, Byrne will receive credit for his underlying cryptographic principle. When this happens, there will hopefully be a sense of closure to the end game of a legendary cipher mystery.
Quote
Like
Share

Gerry StPierre
Just registered
Gerry StPierre
Just registered
Joined: June 5th, 2009, 1:24 am

October 2nd, 2009, 2:27 am #3

Thanks to everyone who contributed to making materials available and mosher for putting it all together. Some interesting reading for the weekend.
Quote
Like
Share

cmdline
Just registered
cmdline
Just registered
Joined: July 17th, 2008, 4:42 am

October 3rd, 2009, 3:06 am #4

I'd like to add my thanks to moshe and everyone who contributed to this amazing collection of correspondence!
Quote
Like
Share

osric
Super member
osric
Super member
Joined: March 15th, 2009, 11:05 am

October 8th, 2009, 2:37 pm #5

In his memo to G M Campbell in March 1942, Byrne says:

“Transmission and reception of the matter of the document ‘Chaocipher -- the Ultimate Elusion’ as it stands would require 100% accuracy. But I want it understood that my system can produce indecipherable cipher which would not require 100% accuracy either in transmission or reception.

I will discuss the subject of transmission errors, which would be of relatively little importance, and also of coping with errors in reception, which is more vital.”

It sounds from ‘100% accuracy’ as if Byrne’s Exhibits have used a method of encryption where the key for the n’th encipherment depends on the setting of the machine from the previous encipherment --- or, perhaps, on the plain or cipher letter of the previous encipherment. Accordingly a mistaken letter of ciphertext would throw out decryption of everything that followed. An impossible situation that would lead to instant rejection of the system, for example by Friedman.

Byrne then tells us that there’s another way of using his machine which also produces indecipherable text but where transmission errors don’t matter. Perhaps in this second way the machine is set up for each encipherment from an initial position according to a new key. Now if there's a mistaken cipher letter the rest are unaffected, so all is OK.

I can readily envisage model machines which have both these properties, so no problem there.

What is inexplicable to me is the final paragraph where we are told that transmission errors don’t matter but reception errors do. Can anyone get a handle on that one?
Quote
Like
Share

mosher
Super member
mosher
Super member
Joined: May 26th, 2009, 10:24 am

October 9th, 2009, 1:20 pm #6

Hi osric,

Before I relate to your good questions, I just want to thank Gerry and cmdline for their kind words about the Byrne correspondences on TCCH. I know exactly what they (and others) experienced when reading the letters -- I had precisely the same feeling (something akin to a kid in a candy store), only it was a few weeks before them <g>.

Regarding your questions:

(1) I understand "100% accuracy required" to mean there is an autokeying feature being used. An error in the received ciphertext will cause error propagation from that point onwards (or from a point several characters ahead). If the autokeying feature is removed then 100% accuracy is not required, because an error does not propagate.

(2) I admit Byrne's statement that "... transmission errors, which would be of relatively little importance, and also of coping with errors in reception, which is more vital” is quite puzzling. In essence, a transmission error correctly received is the same as a correctly transmitted letter incorrectly received! With such symmetry, why _is_ there a difference between transmission and reception? An unlikely explanation is that a "transmission error" according to Byrne means an incorrect letter sent (e.g., "JTECS" is transmitted as "JTEXS"), while a "reception error" means a letter was dropped (e.g., "JTECS" is transmitted as "JTES"). In a Vigenere, for example, an incorrect letter will only corrupt that single letter, while a dropped letter will shift all subsequent ciphertext letters into the incorrect column, leading to corrupt plaintext from that point onwards. I just don't see why Byrne would have referred to these concepts with the terms he used.

Best regards,

Moshe
Last edited by mosher on October 10th, 2009, 5:12 pm, edited 1 time in total.
Quote
Like
Share

jhll
Just registered
jhll
Just registered
Joined: October 8th, 2009, 10:01 pm

October 9th, 2009, 2:59 pm #7

<< I will discuss the subject of transmission errors, which would be of little importance, and also of coping with errors in reception, which is more vital. -- John Byrne, March, 1942 >>

I take everything that Byrne said with a grain of salt, but the way I would interpret this is as follows:

(1) Byrne has a machine which is set to an initial state and from the first step to the last step, the machine supplies the key for the next step. The key is used to manually align a PT disk and a CT disk.

(2) At each step, the pt/ct encipherment is read from the alignment of the PT/CT disks.

(3) The pt/ct encipherment in some way influences the process that produces the next key.

If there is a transmission error, that would mean, in Byrne's view, that an incorrect ct letter was transmitted. It doesn't mean that the ct was read incorrectly from the originating machine, so the originating station will have the correct letter on file. The person receiving the transmission error can report that the message became garbled at that point and request retransmission of a particular five-letter block. Perhaps this seemed like a problem "of little importance" to Byrne in the sense that the transmitting station has the correct letter on file, so the cipher clerk can easily retransmit the requested block.

Suppose, however, a message is transmitted correctly and there are five reception errors. The message becomes garbled at five different places in the cipher, but the receiving code clerk can only discover them one at a time in sequence and can only request the retransmission of one block at a time as he decrypts the message. In other words, he has to put the message aside five times and request retransmission of a particular block before he can proceed. If a message requires "action this hour", it is obviously "vital" that reception errors be avoided to the extent possible. How Byrne would "cope" with a situation like this is unknown.

Of course, this leaves open the possibility that the clerk at the originating station misread the machine in the first place and does not have the correct letter on file. If that happens, then the only thing that the clerk could do, as far as I know, would be to repeat the encryption process up to the point where the message became garbled and then transmit the correct letter block. Whether Byrne would find this to be "of little importance" is unknown.

Quote
Like
Share

osric
Super member
osric
Super member
Joined: March 15th, 2009, 11:05 am

October 10th, 2009, 3:25 pm #8

Hi Mosher and jhll,

Many thanks for your responses on the curious words of Byrne that, as far as his Chaocipher is concerned, transmission errors don’t matter but reception errors do.

From your responses I feel that there is no rational explanation of this contention! Mosher puts up a possible thought to explain what was in Byrne’s mind but the link is tenuous to say the least. And I believe jhll’s view from the receiving cipher-clerk’s chair just shows how ridiculous Byrne’s contention is. For this clerk it doesn’t matter a fig whether an error came from transmission or reception – the result is the same inability to decipher any more of the message. And the remedial step is the same: asking for a repeated transmission (anathema to a professional cryptographer).

My conclusion is that this statement of Byrne’s is just another in a long list of ‘near nonsense’ (to use Friedman’s words delivered in his 13 March 1954 talk on Chaocipher) we have received from him – and no doubt jhll’s recipe is correct of taking it with a pinch of salt.

There is just one aspect of Chaocipher that to me is interesting but I will embark on that in a separate and more positive thread.
Quote
Like
Share

kryptosfan
NSA worthy
kryptosfan
NSA worthy
Joined: September 4th, 2011, 6:09 am

September 17th, 2011, 12:39 am #9

Sounds like Byrne suffered from some of the same self-imposed limitations that Babbage did.
OBKR
UOXOGHULBSOLIFBBWFLRVQQPRNGKSSO
TWTQSJQSSEKZZWATJKLUDIAWINFBNYP
VTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
Quote
Like
Share

kryptosfan
NSA worthy
kryptosfan
NSA worthy
Joined: September 4th, 2011, 6:09 am

September 17th, 2011, 12:39 am #10

i.e. personality
OBKR
UOXOGHULBSOLIFBBWFLRVQQPRNGKSSO
TWTQSJQSSEKZZWATJKLUDIAWINFBNYP
VTTMZFPKWGDKZXTJCDIGKUHUAUEKCAR
Quote
Like
Share