I was playing with a few SMTP authentication mechanisms while setting up my SMTP server. I found CRAM-MD5 authentication mechanism particularly interesting as it is a challenge-response authentication mechanism and involves HMAC-MD5. Let me first show a complete session with an SMTP server before coming to CRAM-MD5. Note that I am not using TLS or SSL in the sessions displayed below so that the SMTP traffic can be seen as plaintext. In practice, any email program should be configured to use TLS or SSL while having a session with an SMTP server. In the following session, I have selected the PLAIN authentication mechanism.

susam@nifty:~$ telnet susam.in 25
Trying 106.187.41.241...
Connected to susam.in.
Escape character is '^]'.
220 tesseract.susam.in ESMTP Exim 4.72 Mon, 07 Nov 2011 20:27:56 +0530
EHLO nifty.localdomain
250-tesseract.susam.in Hello nifty.localdomain [122.167.80.194]
250-SIZE 52428800
250-PIPELINING
250-AUTH PLAIN LOGIN CRAM-MD5
250-STARTTLS
250 HELP
AUTH PLAIN AGFsaWNlAHdvbmRlcmxhbmQ=
235 Authentication succeeded
MAIL FROM:<alice@susam.in>
250 OK
RCPT TO:<example.recipient@gmail.com>
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
Date: Mon, 07 Nov 2011 20:28:00 +0530
From: Alice <alice@susam.in>
To: Example Recipient <example.recipient@gmail.com>
Subject: Test email 

This is a test email.
.
250 OK id=1RNQef-0004e7-7s
QUIT
221 tesseract.susam.in closing connection
Connection closed by foreign host.

In the AUTH PLAIN line I have sent the base64 encoding of the string \0alice\0wonderland. \0 indicates a null character, alice is the sender's user name and wonderland is the sender's password. If an eavesdropper intercepts this network traffic, he can easily find the user's password by simply decoding the base64 response sent by the client. This is also susceptible to replay attacks as the eavesdropper can use the AUTH PLAIN line containing the base64 encoded credentials to log into the server in future. This is why it is very important to secure the connection with TLS or SSL while having a session with the SMTP server.

Now, let me quickly show only the relevant lines for an authentication using the LOGIN mechanism.

AUTH LOGIN
334 VXNlcm5hbWU6
YWxpY2U=
334 UGFzc3dvcmQ6
d29uZGVybGFuZA==
235 Authentication succeeded

What is happening here becomes clear by decoding the base64 encoded messages. The following statements in Python 2.7.2 decode these messages.

>>> 'VXNlcm5hbWU6'.decode('base64')
'Username:'
>>> 'YWxpY2U='.decode('base64')
'alice'
>>> 'UGFzc3dvcmQ6'.decode('base64')
'Password:'
>>> 'd29uZGVybGFuZA=='.decode('base64')
'wonderland'

If the session isn't encrypted, LOGIN authentication mechanism is susceptible to the same problems that PLAIN authentication mechanism is susceptible to. Now, let me describe the CRAM-MD5 authentication mechanism. When the client selects the CRAM-MD5 authentication mechanism, the server sends a base64 encoded challenge as shown below.

AUTH CRAM-MD5
334 PDE3ODkzLjEzMjA2NzkxMjNAdGVzc2VyYWN0LnN1c2FtLmluPg==

An HMAC is calculated for this challenge with the password as the key and MD5 as the hash function. A string is formed by concatenating the user name, a space and the hexadecimal representation of the HMAC. The base64 encoding of this string is sent as the response by the client. The following statements I tried in Python 2.7.2 show how a response can be formed for the above challenge.

>>> 'PDE3ODkzLjEzMjA2NzkxMjNAdGVzc2VyYWN0LnN1c2FtLmluPg=='.decode('base64')
'<17893.1320679123@tesseract.susam.in>'
>>> import hmac, hashlib
>>> hmac.new('wonderland', '<17893.1320679123@tesseract.susam.in>', hashlib.md5).hexdigest()
'64b2a43c1f6ed6806a980914e23e75f0'
>>> 'alice 64b2a43c1f6ed6806a980914e23e75f0'.encode('base64')
'YWxpY2UgNjRiMmE0M2MxZjZlZDY4MDZhOTgwOTE0ZTIzZTc1ZjA=\n'

Of course, this can be written as a small function:

import hmac, hashlib
def cram_md5_response(username, password, base64challenge):
    return (username + ' ' +
            hmac.new(password,
                     base64challenge.decode('base64'),
                     hashlib.md5).hexdigest()).encode('base64')

The following snippet shows the SMTP server accepting the client-response.

AUTH CRAM-MD5
334 PDE3ODkzLjEzMjA2NzkxMjNAdGVzc2VyYWN0LnN1c2FtLmluPg==
YWxpY2UgNjRiMmE0M2MxZjZlZDY4MDZhOTgwOTE0ZTIzZTc1ZjA=
235 Authentication succeeded

CRAM-MD5 authentication mechanism is relatively more secure than the other two mechanisms even if the connection is not secured because the password cannot be retrieved by decoding the base64 encoded client-response. The password is used as the key to calculate the HMAC but the password is not present anywhere in the response. It prevents replay attacks too because the server sends an unpredictable challenge for every authentication. The client-response computed for a certain challenge is invalid for further authentications which will involve different unpredictable challenges.

Here is a list of hyperlinks for further reading:

  1. RFC 4954: SMTP Service Extension for Authentication
  2. RFC 4616: The PLAIN Simple Authentication and Security Layer (SASL) Mechanism
  3. RFC 2195: IMAP/POP AUTHorize Extension for Simple Challenge/Response
  4. RFC 2104: HMAC: Keyed-Hashing for Message Authentication

I have two blogs: blog.susam.in (this one) and notes.susam.in. I use the latter to take notes without caring much about comprehensibility. I wanted a nifty TODO list somewhere in that blog to keep track of what I would like to learn or write about in future. However, I didn't want the TODO list to appear by default. I wanted it to be hidden like an easter egg. In this post, I'll explain how I solved this problem.

First, let me show you how the TODO list behaves. If you visit http://notes.susam.in/, you wouldn't find any TODO list anywhere in the web page. However, if you append ?todo to the URL or in other words visit http://notes.susam.in/?todo, you'll find a TODO list in the top right corner.

I created the TODO list as an HTML/JavaScript gadget. This could be done by visiting the Blogger dashboard, clicking the 'Design' link under the required blog, clicking 'Add a Gadget' under the 'Page Elements' tab and selecting the 'HTML/JavaScript' gadget.

Next, I moved to 'Edit HTML' tab and found that the id attribute of the new widget had a value of HTML7 in the blog template.
So, I edited the blog template to add this CSS.
#HTML7 {
    display: none;
}

#layout #HTML7 { display: block; }

The CSS for the #HTML7 selector ensures that the new gadget is not displayed when the page loads. However, this causes one problem. This makes the gadget invisible in the 'Page Elements' tab as well and hence makes it difficult for us to edit this gadget. So, I needed some way to make the widget visible in the 'Page Elements' tab only.

The panel in the 'Page Elements' tab where we can drag and rearrange gadgets is an HTML iframe element.

On viewing the source of this frame, I found that the body element had an id attribute with a value of #layout.

Thus, I came up with the CSS for #layout #HTML7 selector to make the gadget visibile in this frame.

So, with the gadget visible in the 'Page Elements' tab and invisible in the blog, I needed a quick way to make the gadget visible in the blog whenever I wanted. I wrote and appended the following JavaScript code to the HTML/JavaScript gadget I used to create the TODO list.

var todoGadget = document.getElementById('HTML7')
if (document.URL.search(/[?&]todo\b/i) != -1) {
    todoGadget.style.display = 'block'
} else {
    todoGadget.style.display = 'none'
}

SEX

The DEC engineers had a SEX instruction in PDP-11. SEX here stands for 'Sign Extend'. However, before the assembler went out to the market, the marketing folks forced them to change the mnemonic.

In fact, Intel 8086 engineers also used a SEX instruction but again, SEX lost, marketing won. It was changed to two separate instructions CBW (convert byte to word, signed) and CWD (convert word to double-word, signed).

However, there was a case in which SEX won. It was Motorola 6809 which had 59 instructions including a SEX instruction. The RCA 1802 microprocessors also had a SEX instruction which stood for 'SEt X register'. This was used to select one of 16-bit registers as the index (X) register.

For more interesting facts about this, read http://catb.org/jargon/html/S/SEX.html

Vim users can use the :Sex command to split the current window into two and explore the directory of the current file. This is very useful while editing a file if we want to browse the directory that contains it.

Often I need to search all hidden files and directories within a particular directory and its sub-directories recursively, and remove them. On GNU/Linux, the operating system I use, files and directories beginning with a dot are hidden. They do not appear in the directory listings by default.

For a long time, I used the find command but I wasn't happy with the way I used it. Let me demonstrate what I used to do by showing a sample directory with some hidden directories and the command I used to run.

susam@nifty:~/lab$ tree
-aF
.
|-- .bar/
|   `-- sample/
`-- foo/
    `-- .bar/
        `-- .bar/

5 directories, 0 files susam@nifty:~/lab$ find -name ".*" -exec rm -rf {} \; rm: cannot remove directory `.' find: `./foo/.bar': No such file or directory find: `./.bar': No such file or directory susam@nifty:~/lab$ tree -aF . `-- foo/

1 directory, 0 files

I was not happy with the warnings. There are three of them. The first one complains that rm can not remove the current directory. That's fair. rm indeed can not remove the current directory. Try executing rm -rf . and you'll find the same warning. But why does it complain about the other two directories? The directories however are removed as can be seen in the output of the tree command I executed next.

If this problem appeals to you as a puzzle, you might want to stop here for a while, think over it and then proceed because you are going to find spoilers below.

Let us see what is going on and how to fix this command. With the same setup, this time I won't ask find to actually remove the directories but instead print the rm command with the directory names as the argument. This would tell us a little about the order in which find is trying to remove the directories.

susam@nifty:~/lab$ tree
-aF
.
|-- .bar/
|   `-- sample/
`-- foo/
    `-- .bar/
        `-- .bar/

5 directories, 0 files susam@nifty:~/lab$ find -name ".*" -exec echo rm -rf {} \; rm -rf . rm -rf ./foo/.bar rm -rf ./foo/.bar/.bar rm -rf ./.bar

So, it tries to remove a parent directory first and then its sub-directories. This explains the other two warnings. It removes foo/.bar first and then tries to search hidden files within foo/.bar but this directory no longer exists and hence the second warning: find: `./foo/.bar': No such file or directory. Similarly, it removes .bar first and then tries to search within it and so we see the third warning.

find command provides a -depth option to fix this. Let us try it.

susam@nifty:~/lab$ find -depth
-name ".*" -exec echo rm -rf {} \;
rm -rf ./foo/.bar/.bar
rm -rf ./foo/.bar
rm -rf ./.bar
rm -rf .
But, rm would still complain that it can not delete the current directory. We can fix that by refining the pattern.
susam@nifty:~/lab$ find -depth
-name ".?*" -exec echo rm -rf {} \;
rm -rf ./foo/.bar/.bar
rm -rf ./foo/.bar
rm -rf ./.bar
That's perfect. So, here is the proper command in execution.
susam@nifty:~/lab$ find -depth
-name ".?*" -exec rm -rf {} \;
susam@nifty:~/lab$ tree -aF
.
`-- foo/

1 directory, 0 files

No warnings this time.

I just finished solving this puzzle: http://dow.ngra.de/2010/02/18/a-job-offer-and-a-kinda-cool-puzzle/. I have included my solution below. All input I have entered appears in italics. The first input I have entered is the puzzle. The last output is the solution.

susam@swift:~/puzzle$ cat >
puzzle.txt

UEsDBBQAAAAIAIVyUjx/yWgsbQIAAP0CAAAHABwAcHJvYmxlbVVUCQADeTB9S3ow
fUt1eAsAAQToAwAABOgDAAAdUsmumlAA3fcrGtNFX1zABblgXm2jOACXSRBEY9Jc
5knmSb6+vi7OkJyzO+fXnx8T6W4WrB3LbiuOBxXMfNRNW8Wn9NipMa4QbVv8MQVL
05STYRfqADEysKpWFm/3mViqu5EtXlzGPVl6zauMly7vUNt3S9ji1LESwdn63jCN
DZkNHrzCc78lb8/mucZtFFM45ckrs1XnmtOYo34WENfnXkzC9vQ6s3sJEpCifalp
BtvFDjI50bJunpHhegTuhAKuKsh49uGhN9tXPJpkNxknpCaHjgoxivp4MNg4YQSG
V+K8IXs3KvSzK2IcaJOxvWny89ztEn66vIpYcNpw1jnkr7CplWUCGGNneZ0Y3mdK
du/0Ss2S8Zqve2ZQWWQcIsDMh/spoPTCEujcunKTWHZ9JZryK6YlAqi7Z+4noaHM
+SC2KyYws6Vk3gqt4FuqESqyfQ6jSjfkIO0yXJgXr6zUI0L6+CKOcmjZvkFyrmxd
92QoKQ0WHcmRqgiKJ7+I6J0xK8d70ALbNBQh8LMlPLlBefMRVfOVkupsao181s+X
alqrQprbp3QH0CuTxEminYhUtGLm8htNsEVWaVvbzxrTTGTuAqlSEbsXQiMdjSCh
SxWaNYe6hknmS6ND+6g01+aoFnZ/qaURCYTRQBEf9vEp9UmrIxTb7bnzMOFzaFE1
cagV4M+WbexrQrDLPf/eqjROlTqeNYFVbwpYLvV+HTgdbcxavlan9ZhtFp/vb3qb
hYsfE0sHcPXXf0yQeYAV/dbQfxvmfyfYLKIHYKkHYMA7Cd6Ab3i4Cxaf34MB5z+/
Wl/kfZH78fHx+ef3t39QSwECHgMUAAAACACFclI8f8loLG0CAAD9AgAABwAYAAAA
AAABAAAApIEAAAAAcHJvYmxlbVVUBQADeTB9S3V4CwABBOgDAAAE6AMAAFBLBQYA
AAAAAQABAE0AAACuAgAAAAA=

susam@swift:~/puzzle$ cat puzzle.txt | tr -d "
" |
base64 -d > x

susam@swift:~/puzzle$ file x

x: Zip archive data, at least v2.0 to extract

susam@swift:~/puzzle$ unzip x

Archive:  x

  inflating: problem

susam@swift:~/puzzle$ file problem

problem: ASCII text, with very long lines

susam@swift:~/puzzle$ fold -w 75 problem

<?$x0b="7VhLbsIwEN1zCgtxAMd2PhXqaapK3VUCFj1+SSLivBfP1K5L1UpsLIYZz/+NBw7ny8k

8m739CN5cj+Z6ODt+6sajXUiHXAdcvxwr0kvc6W6QuA0Ymrm9asgh2ajC0W5ANzq8O5FPQHK8ul

ch06sGyQ7DJ6/623dJrrvVbaXKS8IUUYcRkaqw1bxKe8pn0hzd6EuSsyhwS0txRGKNiEt2faKgu

hvR7hi5H5CMhlr0ubgnPQbIaaeOxRAYOLmQtBiCxTynhHXsfzP8Kd4aSOooi15RBUctIfZz2LbZ

34NkiwWl9u5vN7KREg15zEZGe2PnUH3lUW8xIotupISLyh3J/1NBmldifRMzlvIs45eSk+JSYnO

nCs2rHp0smvwN3r0vJBkanSTcopNFKKPwy/FLfUVdR08bLUWD0fJMraIXJXJpg6IGdng3BRzMFZ

es1VSRMHedk+6GbeoYdK2qCpMjP7jUwCkuzTpx9NHjlVGjB1KykJIxJ3Xg0MOnz8lY3/7nkpOAV

dkrSSiL8T62oMItyKKw3gw1i3oN6Sq8Ktr5izTrP6VFMrWrFNnVuTqJwKH/Rr6IaEDhGjd0Ut/M

Vbu8QvxaQfU2q/EqM1dzUVRDq/HVoDCSsyoRGpNwQOH7NYM1++Pu9eXt3RzOl9Nx9wk=";$x0c=

"base64_decode";$x0e="gzinflate";
eval($x0e($

x0c($x0b)));?>

susam@swift:~/puzzle$ php5 problem > y

susam@swift:~/puzzle$ file y

y: ASCII text, with very long lines, with no line terminators

susam@swift:~/puzzle$ fold -w 75 y | head -5

0x43 0x41 0x20 0x46 0x45 0x20 0x42 0x41 0x20 0x42 0x45 0x20 0x30 0x30
0x20

0x30 0x33 0x20 0x30 0x30 0x20 0x32 0x44 0x20 0x30 0x30 0x20 0x31 0x41
0x20

0x30 0x37 0x20 0x30 0x30 0x20 0x30 0x32 0x20 0x30 0x31 0x20 0x30 0x30
0x20

0x30 0x34 0x20 0x34 0x44 0x20 0x36 0x31 0x20 0x36 0x39 0x20 0x36 0x45
0x20

0x30 0x37 0x20 0x30 0x30 0x20 0x30 0x34 0x20 0x30 0x31 0x20 0x30 0x30
0x20

susam@swift:~/puzzle$ echo -ne `cat y | tr -d "
"
| tr -d " " | sed 's/0x/\\x/g'` > z

susam@swift:~/puzzle$ file z

z: ASCII text, with very long lines, with no line terminators

susam@swift:~/puzzle$ fold -w 75 z | head -5

CA FE BA BE 00 03 00 2D 00 1A 07 00 02 01 00 04 4D 61 69 6E 07 00 04 01
00

10 6A 61 76 61 2F 6C 61 6E 67 2F 4F 62 6A 65 63 74 01 00 06 3C 69 6E 69
74

3E 01 00 03 28 29 56 01 00 04 43 6F 64 65 0A 00 03 00 09 0C 00 05 00 06
01

00 04 6D 61 69 6E 01 00 16 28 5B 4C 6A 61 76 61 2F 6C 61 6E 67 2F 53 74
72

69 6E 67 3B 29 56 09 00 0D 00 0F 07 00 0E 01 00 10 6A 61 76 61 2F 6C 61
6E

susam@swift:~/puzzle$ echo -ne `cat z | tr -d " " |
sed 's/../\\x&/g'` > Foo.class

susam@swift:~/puzzle$ xxd -g1 Foo.class

0000000: ca fe ba be 00 03 00 2d 00 1a 07 00 02 01 00 04 
.......-........

0000010: 4d 61 69 6e 07 00 04 01 00 10 6a 61 76 61 2f 6c 
Main......java/l

0000020: 61 6e 67 2f 4f 62 6a 65 63 74 01 00 06 3c 69 6e 
ang/Object...<in

0000030: 69 74 3e 01 00 03 28 29 56 01 00 04 43 6f 64 65 
it>...()V...Code

0000040: 0a 00 03 00 09 0c 00 05 00 06 01 00 04 6d 61 69 
.............mai

0000050: 6e 01 00 16 28 5b 4c 6a 61 76 61 2f 6c 61 6e 67 
n...([Ljava/lang

0000060: 2f 53 74 72 69 6e 67 3b 29 56 09 00 0d 00 0f 07 
/String;)V......

0000070: 00 0e 01 00 10 6a 61 76 61 2f 6c 61 6e 67 2f 53 
.....java/lang/S

0000080: 79 73 74 65 6d 0c 00 10 00 11 01 00 03 6f 75 74 
ystem........out

0000090: 01 00 15 4c 6a 61 76 61 2f 69 6f 2f 50 72 69 6e 
...Ljava/io/Prin

00000a0: 74 53 74 72 65 61 6d 3b 08 00 13 01 00 11 4d 79 
tStream;......My

00000b0: 20 6c 69 66 65 20 66 6f 72 20 41 69 75 72 21 0a   life for
Aiur!.

00000c0: 00 15 00 17 07 00 16 01 00 13 6a 61 76 61 2f 69 
..........java/i

00000d0: 6f 2f 50 72 69 6e 74 53 74 72 65 61 6d 0c 00 18 
o/PrintStream...

00000e0: 00 19 01 00 07 70 72 69 6e 74 6c 6e 01 00 15 28 
.....println...(

00000f0: 4c 6a 61 76 61 2f 6c 61 6e 67 2f 53 74 72 69 6e 
Ljava/lang/Strin

0000100: 67 3b 29 56 00 21 00 01 00 03 00 00 00 00 00 02 
g;)V.!..........

0000110: 00 01 00 05 00 06 00 01 00 07 00 00 00 11 00 01 
................

0000120: 00 01 00 00 00 05 2a b7 00 08 b1 00 00 00 00 00 
......*.........

0000130: 09 00 0a 00 0b 00 01 00 07 00 00 00 15 00 02 00 
................

0000140: 01 00 00 00 09 b2 00 0c 12 12 b6 00 14 b1 00 00 
................

0000150: 00 00 00 00                                      ....

susam@swift:~/puzzle$ mv Foo.class
Main.class

susam@swift:~/puzzle$ java Main

My life for Aiur!

Newer | Older
RSS