Cryptography ( Codebreaking ) - Deciphering messages encrypted with Lorenz SZ40 cipher machine

[SergeyKostrov]

Cryptography ( Codebreaking ) - Deciphering messages encrypted with Lorenz SZ40 cipher machine The Vernam cipher implemented by the Lorenz SZ40 machines utilizes the Boolean "exclusive or" ( XOR ) function. Note: Tests only

[SergeyKostrov]

 [ Summary - IVB workstation ]

  Number of OpenMP Threads :    4  Test Completed in 281660 ms
  Number of OpenMP Threads :    8  Test Completed in 195219 ms  Not Best time ( but it should be... )
  Number of OpenMP Threads :   16  Test Completed in 193597 ms  Oversubscription / Processing Saturation
  Number of OpenMP Threads :   32  Test Completed in 200695 ms  Oversubscription / Processing Saturation
  Number of OpenMP Threads :   64  Test Completed in 188730 ms  Oversubscription / Best time
  Number of OpenMP Threads :  128  Test Completed in 192771 ms  Oversubscription / Processing Saturation
  Number of OpenMP Threads :  256  Not tested
  Number of OpenMP Threads :  512  Not tested
  Number of OpenMP Threads : 1024  Not tested
  Number of OpenMP Threads : 2048  Not tested

[SergeyKostrov]

 Application - ScaLibTestApp - NOS64_MSC ( 64-bit ) - Release
 Tests: Start
 > Test1155 Start <
 Sub-Test 01 - Generates KeySet values
      Keys         : 16 13 03 10 10 02 12 11 10 06 01 04 03 04 04 14 06 04 14 16 12 10 09 06
 Sub-Test 02 - Displays KeySet values
      Min Key value: 01
      Max Key value: 16
      Keys         : 16 13 03 10 10 02 12 11 10 06 01 04 03 04 04 14 06 04 14 16 12 10 09 06
 Sub-Test 03.1 - Displays PlainSet and CipherSet messages
      Plain  Text: HELLO LORENZ SZXX CIPHER
      Cipher Text: XHOFE"@DXCO^#W^V^$MY\BLT
 Sub-Test 04.3 - Deciphers CipherSet message
 Number of OpenMP Threads : 8
 Number of Characters (N) : 27
 Length of Sub-String (L) : 4
 Number of Cases to Verify: 531441       where NofCV = N^L
 Memory Allocated         : 0.002 GB
 Generating DataSet
 Identification
      Cipher: 'XHOF' - Crib: 'HELL' - Keys: 16 13 03 10 - Offset: 0
      Cipher: 'HOFE' - Crib: 'FALL' - Keys: 14 14 10 09 - Offset: 1
      Cipher: '@DXC' - Crib: 'LORE' - Keys: 12 11 10 06 - Offset: 6
      Cipher: '@DXC' - Crib: 'CIPH' - Keys: 03 13 08 11 - Offset: 6
      Cipher: 'XCO^' - Crib: 'WINT' - Keys: 15 10 01 10 - Offset: 8
      Cipher: 'W^V^' - Crib: 'SZXX' - Keys: 04 04 14 06 - Offset: 13
      Cipher: 'MY\B' - Crib: 'CIPH' - Keys: 14 16 12 10 - Offset: 18
      Cipher: 'Y\BL' - Crib: 'SPRI' - Keys: 10 12 16 05 - Offset: 19
      Cipher: 'Y\BL' - Crib: 'SUMM' - Keys: 10 09 15 01 - Offset: 19
 Deallocates Memory
 Test Completed in 509296 ms
 > Test1155 End <
 Tests: Completed

 //

    Offsets:        000000000011111111112222
                    012345678901234567890123

    Plain  Text:    HELLO LORENZ SZXX CIPHER

    Cipher Text:    XHOFE"@DXCO^#W^V^$MY\BLT

    Deciphered Segments:

                    HELL??
                    ?FALL?
                          LORE???
                          CIPH???
                          ??WINT?
                                 SZXX?
                                      CIPH??
                                      ?SPRI?
                                      ?SUMM?

[SergeyKostrov]

Here is an example how processing duplicates of deciphered text segments look like...

[SergeyKostrov]

Here are a couple of examples on a Total Number of Keys Verified for a segment of German text SPRUCHNUMMER ( English translation: Message Number )...

[SergeyKostrov]

Intel C++ compiler Application - IccTestApp - NOS64_ICC ( 64-bit ) - Release Tests: Start > Test1155 Start < Number of Characters : 36 Length of Sub-String : 4 Number of Cases to Verify: 1679616 where NofCV = N^L Memory Allocated : 0.15 GB Number of OpenMP Threads : 8 Identification Started ThreadId 3 - Cipher: JSH5 - Plain: CHNU - Dictionary: CHNU - Keys: 009 027 006 096 - K-Value: 1611012873 ThreadId 3 - Cipher: JSH5 - Plain: SPRU - Dictionary: SPRU - Keys: 025 003 026 096 - K-Value: 1612317465 ThreadId 3 - Cipher: JSH5 - Plain: MMER - Dictionary: MMER - Keys: 007 030 013 103 - K-Value: 1728912903 ThreadId 0 - Cipher: NZYZ - Plain: MMER - Dictionary: MMER - Keys: 003 023 028 008 - K-Value: 136058627 ThreadId 0 - Cipher: NZYZ - Plain: SPRU - Dictionary: SPRU - Keys: 029 010 011 015 - K-Value: 252381725 ThreadId 0 - Cipher: NZYZ - Plain: CHNU - Dictionary: CHNU - Keys: 013 018 023 015 - K-Value: 253170189 ThreadId 0 - Cipher: Y5GL - Plain: CHNU - Dictionary: CHNU - Keys: 026 125 009 025 - K-Value: 420052250 ThreadId 0 - Cipher: Y5GL - Plain: SPRU - Dictionary: SPRU - Keys: 010 101 021 025 - K-Value: 420832522 ThreadId 0 - Cipher: Y5GL - Plain: MMER - Dictionary: MMER - Keys: 020 120 002 030 - K-Value: 503478292 Number of Keys Verified : 51539607552 Identification Completed Deallocates Memory Test Completed in 15725 ms > Test1155 End < Tests: Completed

[SergeyKostrov]

Microsoft C++ compiler Application - ScaLibTestApp - NOS64_MSC ( 64-bit ) - Release Tests: Start > Test1155 Start < Number of Characters : 36 Length of Sub-String : 4 Number of Cases to Verify: 1679616 where NofCV = N^L Memory Allocated : 0.15 GB Number of OpenMP Threads : 8 Identification Started ThreadId 3 - Cipher: JSH5 - Plain: CHNU - Dictionary: CHNU - Keys: 009 027 006 096 - K-Value: 1611012873 ThreadId 3 - Cipher: JSH5 - Plain: SPRU - Dictionary: SPRU - Keys: 025 003 026 096 - K-Value: 1612317465 ThreadId 3 - Cipher: JSH5 - Plain: MMER - Dictionary: MMER - Keys: 007 030 013 103 - K-Value: 1728912903 ThreadId 0 - Cipher: NZYZ - Plain: MMER - Dictionary: MMER - Keys: 003 023 028 008 - K-Value: 136058627 ThreadId 0 - Cipher: NZYZ - Plain: SPRU - Dictionary: SPRU - Keys: 029 010 011 015 - K-Value: 252381725 ThreadId 0 - Cipher: NZYZ - Plain: CHNU - Dictionary: CHNU - Keys: 013 018 023 015 - K-Value: 253170189 ThreadId 0 - Cipher: Y5GL - Plain: CHNU - Dictionary: CHNU - Keys: 026 125 009 025 - K-Value: 420052250 ThreadId 0 - Cipher: Y5GL - Plain: SPRU - Dictionary: SPRU - Keys: 010 101 021 025 - K-Value: 420832522 ThreadId 0 - Cipher: Y5GL - Plain: MMER - Dictionary: MMER - Keys: 020 120 002 030 - K-Value: 503478292 Number of Keys Verified : 51539607552 Identification Completed Deallocates Memory Test Completed in 24415 ms > Test1155 End < Tests: Completed

[SergeyKostrov]

MinGW C++ compiler Application - MgwTestApp - NOS64_MGW ( 64-bit ) - Release Tests: Start > Test1155 Start < Number of Characters : 36 Length of Sub-String : 4 Number of Cases to Verify: 1679616 where NofCV = N^L Memory Allocated : 0.15 GB Number of OpenMP Threads : 8 Identification Started ThreadId 3 - Cipher: JSH5 - Plain: CHNU - Dictionary: CHNU - Keys: 009 027 006 096 - K-Value: 1611012873 ThreadId 3 - Cipher: JSH5 - Plain: SPRU - Dictionary: SPRU - Keys: 025 003 026 096 - K-Value: 1612317465 ThreadId 3 - Cipher: JSH5 - Plain: MMER - Dictionary: MMER - Keys: 007 030 013 103 - K-Value: 1728912903 ThreadId 0 - Cipher: NZYZ - Plain: MMER - Dictionary: MMER - Keys: 003 023 028 008 - K-Value: 136058627 ThreadId 0 - Cipher: NZYZ - Plain: SPRU - Dictionary: SPRU - Keys: 029 010 011 015 - K-Value: 252381725 ThreadId 0 - Cipher: NZYZ - Plain: CHNU - Dictionary: CHNU - Keys: 013 018 023 015 - K-Value: 253170189 ThreadId 0 - Cipher: Y5GL - Plain: CHNU - Dictionary: CHNU - Keys: 026 125 009 025 - K-Value: 420052250 ThreadId 0 - Cipher: Y5GL - Plain: SPRU - Dictionary: SPRU - Keys: 010 101 021 025 - K-Value: 420832522 ThreadId 0 - Cipher: Y5GL - Plain: MMER - Dictionary: MMER - Keys: 020 120 002 030 - K-Value: 503478292 Number of Keys Verified : 51539607552 Identification Completed Deallocates Memory Test Completed in 33417 ms > Test1155 End < Tests: Completed

[SergeyKostrov]

Intel C++ compiler Application - IccTestApp - NOS64_ICC ( 64-bit ) - Release Tests: Start > Test1155 Start < Number of Characters : 36 Length of Sub-String : 5 Number of Cases to Verify: 60466176 where NofCV = N^L Memory Allocated : 6.31 GB Number of OpenMP Threads : 8 Identification Started ThreadId 0 - Cipher: JSH5N - Plain: HNUMM - Dictionary: HNUMM - Keys: 002 029 029 120 003 - K-Value: 2015173890 ThreadId 0 - Cipher: GLFRG - Plain: SPRUC - Dictionary: SPRUC - Keys: 020 028 020 007 004 - K-Value: 118758420 ThreadId 0 - Cipher: GLFRG - Plain: HNUMM - Dictionary: HNUMM - Keys: 015 002 019 031 010 - K-Value: 521339407 ThreadId 2 - Cipher: ZYZY5 - Plain: ERxxx - Dictionary: ERxxx - Keys: 031 011 034 033 077 - K-Value: 555879199 ThreadId 0 - Cipher: JSH5N - Plain: SPRUC - Dictionary: SPRUC - Keys: 025 003 026 096 013 - K-Value: 1612317465 ThreadId 3 - Cipher: ZYZY5 - Plain: SPRUC - Dictionary: SPRUC - Keys: 009 009 008 012 118 - K-Value: 201853193 ThreadId 1 - Cipher: JSH5N - Plain: ERxxx - Dictionary: ERxxx - Keys: 015 001 048 077 054 - K-Value: 1294991631 ThreadId 3 - Cipher: ZYZY5 - Plain: HNUMM - Dictionary: HNUMM - Keys: 018 023 015 020 120 - K-Value: 336533266 ThreadId 1 - Cipher: GLFRG - Plain: ERxxx - Dictionary: ERxxx - Keys: 002 030 062 042 063 - K-Value: 708713986 Number of Keys Verified : 13194139533312 Identification Completed Deallocates Memory Test Completed in 4499708 ms > Test1155 End < Tests: Completed

[SergeyKostrov]

Microsoft C++ compiler Application - ScaLibTestApp - NOS64_MSC ( 64-bit ) - Release Tests: Start > Test1155 Start < Number of Characters : 36 Length of Sub-String : 5 Number of Cases to Verify: 60466176 where NofCV = N^L Memory Allocated : 6.31 GB Number of OpenMP Threads : 8 Identification Started ThreadId 0 - Cipher: JSH5N - Plain: HNUMM - Dictionary: HNUMM - Keys: 002 029 029 120 003 - K-Value: 2015173890 ThreadId 0 - Cipher: GLFRG - Plain: SPRUC - Dictionary: SPRUC - Keys: 020 028 020 007 004 - K-Value: 118758420 ThreadId 0 - Cipher: GLFRG - Plain: HNUMM - Dictionary: HNUMM - Keys: 015 002 019 031 010 - K-Value: 521339407 ThreadId 2 - Cipher: ZYZY5 - Plain: ERxxx - Dictionary: ERxxx - Keys: 031 011 034 033 077 - K-Value: 555879199 ThreadId 0 - Cipher: JSH5N - Plain: SPRUC - Dictionary: SPRUC - Keys: 025 003 026 096 013 - K-Value: 1612317465 ThreadId 3 - Cipher: ZYZY5 - Plain: SPRUC - Dictionary: SPRUC - Keys: 009 009 008 012 118 - K-Value: 201853193 ThreadId 1 - Cipher: JSH5N - Plain: ERxxx - Dictionary: ERxxx - Keys: 015 001 048 077 054 - K-Value: 1294991631 ThreadId 3 - Cipher: ZYZY5 - Plain: HNUMM - Dictionary: HNUMM - Keys: 018 023 015 020 120 - K-Value: 336533266 ThreadId 1 - Cipher: GLFRG - Plain: ERxxx - Dictionary: ERxxx - Keys: 002 030 062 042 063 - K-Value: 708713986 Number of Keys Verified : 13194139533312 Identification Completed Deallocates Memory Test Completed in 6599825 ms > Test1155 End < Tests: Completed

[SergeyKostrov]

MinGW C++ compiler Application - MgwTestApp - NOS64_MGW ( 64-bit ) - Release Tests: Start > Test1155 Start < Number of Characters : 36 Length of Sub-String : 5 Number of Cases to Verify: 60466176 where NofCV = N^L Memory Allocated : 6.31 GB Number of OpenMP Threads : 8 Identification Started ThreadId 0 - Cipher: JSH5N - Plain: HNUMM - Dictionary: HNUMM - Keys: 002 029 029 120 003 - K-Value: 2015173890 ThreadId 0 - Cipher: GLFRG - Plain: SPRUC - Dictionary: SPRUC - Keys: 020 028 020 007 004 - K-Value: 118758420 ThreadId 0 - Cipher: GLFRG - Plain: HNUMM - Dictionary: HNUMM - Keys: 015 002 019 031 010 - K-Value: 521339407 ThreadId 2 - Cipher: ZYZY5 - Plain: ERxxx - Dictionary: ERxxx - Keys: 031 011 034 033 077 - K-Value: 555879199 ThreadId 0 - Cipher: JSH5N - Plain: SPRUC - Dictionary: SPRUC - Keys: 025 003 026 096 013 - K-Value: 1612317465 ThreadId 3 - Cipher: ZYZY5 - Plain: SPRUC - Dictionary: SPRUC - Keys: 009 009 008 012 118 - K-Value: 201853193 ThreadId 1 - Cipher: JSH5N - Plain: ERxxx - Dictionary: ERxxx - Keys: 015 001 048 077 054 - K-Value: 1294991631 ThreadId 3 - Cipher: ZYZY5 - Plain: HNUMM - Dictionary: HNUMM - Keys: 018 023 015 020 120 - K-Value: 336533266 ThreadId 1 - Cipher: GLFRG - Plain: ERxxx - Dictionary: ERxxx - Keys: 002 030 062 042 063 - K-Value: 708713986 Number of Keys Verified : 13194139533312 Identification Completed Deallocates Memory Test Completed in 8879156 ms > Test1155 End < Tests: Completed

[SergeyKostrov]

A note for Eric M: Eric, Please take a look at Posts 21 and 22 first, and if you have time, take a look at all the rest performance results. Even if the Vernam cipher strength realized in Lorenz SZ4x cipher machines is only 5-bit ( looks like simple! / characters and key-numbers are from 0 to 32 ) deciphering still could be a challenging process even with modern hardware! Once again, it is very-very simple in essence, just XOR, but number of key verifications significant. Another Undocumented property of the Vernam cipher are Processing Duplicates ( deciphered duplicates of the same Crib ) and that is why codebreakers at the Bletchley Park in UK in 1940th had to recover initial rotor settings of the Lorenz SZ4x cipher machines. They had hundreds of messages every day and manual processing was Not possible in a timely manner and that is why the whole process, Key recovering and Deciphering, was almost fully automated only with Colossus computing machine.

[SergeyKostrov]

An example of Processing Duplicates is as follows ( from Post 4 ): ... Cipher: '@UDD' - Crib: 'HELL' - Keys: 08 16 08 08 - Offset: 0 Cipher: 'K,BG' - Crib: 'O LO' - Keys: 04 12 14 08 - Offset: 4 Cipher: 'VULR' - Crib: 'RENZ' - Keys: 04 16 02 08 - Offset: 8 Cipher: '%CRT' - Crib: ' SZX' - Keys: 05 16 08 12 - Offset: 12 Cipher: '\(SY' - Crib: 'X CI' - Keys: 04 08 16 16 - Offset: 16 Cipher: 'X@KZ' - Crib: 'PHER' - Keys: 08 08 14 08 - Offset: 20 _____________________________Correct Identification of a segment of Plain text Cipher: 'X@KZ' - Crib: 'WINT' - Keys: 15 09 05 14 - Offset: 20 _____________________________Also Correct Identification of a segment of Plain text but this is a Duplicate and result needs to be Removed ... This is because: Crib: 'PHER' XOR ( Keys: 08 08 14 08 ) = 'X@KZ' = Crib: 'WINT' XOR ( Keys: 15 09 05 14 ), where 'X@KZ' is a Cipher segment of a text.