Tuesday, November 15, 2011

The Complete Idiots Guide to Correctly Validating Your Customer's Email Addresses

Email address validation shouldn't be this hard.  Yet, obviously it's difficult enough for companies, webmasters and IT departments the world over to get it wrong almost without exception.

I'll try to spell it out very clearly.

First things first.

Beginner programming 101

Raise your right hand and repeat after me:

Me: "I will NEVER..."
You: "I will NEVER..."

Me: "...trust user input."
You: "...trust user input."

It may seem difficult to believe that someone could either not know or not remember their correct email address.  It happens often.  VERY often.

Validating email addresses during new account registration

Step 1

Collect the user's email address from a web form of some kind.  This web form will:

  • Be secured with https
  • Be comprised of two fields, forcing the user to type the email address twice.
  • Be able to perform at least a cursory validation, though the validation will not reject VALID email addresses, such as those with symbols like . or +.

Step 2

Since you don't trust the user to type their email address correctly, you will make the assumption that the email validation message will be sent to a user other than the intended recipient.

Because of this, your web form will additionally prompt the user for information only they know.  This can be:

  • A "security" question.
  • A validation code number (or any string) that you displayed to the user on their browser page after the user submitted their email address on the web form.
  • The user's chosen password, providing that you make certain that you never email the password to the user for any reason.  (Since you're NOT storing the password in plain text this is not possible anyway, right?)
  • Any piece of identifying information that is kept private.

This secret identifying information will never, under any circumstances, be emailed to the user, either before or after successful validation.

The email can contain a clickable link to get them back to finishing the validation process.  But first, again, raise your right hand and repeat after me:

Me: "I will NEVER..."
You: "I will NEVER..."

Me: "...send the user a simple clickable link in an email and assume that the clicking of the link establishes validity."
You: "...send the user a simple clickable link in an email and assume that the clicking of the link establishes validity."

Why does this not establish validity, you ask?

Because if you SEND THE DAMN LINK TO THE WRONG PERSON they can still click it!

Step 3

Send the address validation email message to the user.

Step 4

To finish the validation process, the user comes back to your site via the clickable link you provided in the email to the user in the previous step.  Now, you will prompt the user with a web form asking for the piece of secret identifying data from Step 2 that was never and could never be transferred via email.

If the values match, the user can now be considered validated.

Else, goto Step 5.

Step 5

If the email address validation fails because the user typed the piece of identifying data incorrectly, or for any other reason, indicate this to the user with an error message.

Step 6

If the user fails to provide the correct identifying information after N-number of attempts, the validation process will be aborted.  You will then delete the invalidated profile completely and point the user to the beginning of the account creation process again.  FULLY DELETE THE PROFILE INFORMATION.

You will NOT volunteer to email the user their secret identifying data.  Instead, you will FULLY DELETE THE PROFILE INFORMATION and force the user to start from scratch.  Suggest to the user that this time, they write down this identifying piece of information on a piece of paper.  This is somewhat acceptable (providing that the piece of identifying information is not the password) since it is only used to validate the email address this one time.

Failure to fully delete the profile information properly runs a high risk of data collision and personal information leakage.  If one John Smith attempts to sign up an account with jsmith@exampel.com, fails validation and re-registers with a corrected jsmith@example.com, then another John Smith attempts to register with jsmith@exampel.com, this new John Smith should NOT be able to see or otherwise collide with any information that may have been entered and associated with the first John Smith's account. 

Forgotten password procedures

Again, repeat after me:

Me: "I will NEVER..."
You: "I will NEVER..."

Me: "...store users' passwords in plain text for conveniently mailing the password to them."
You: "...store users' passwords in plain text for conveniently mailing the password to them."

Step 1

After the user has clicked the link to initiate the forgotten password procedure, you will prompt the user for their username, or, ideally, their email address.

However, if you are collecting email addresses from your forgotten password form, you WILL NOT display an error message indicating whether or not the email address was found in your records.  Simply indicate that, if found, the forgotten password procedure for the indicated email address will be initiated.  Optionally, indicate the email address from which the user can expect to receive the forgotten password procedure email so they can check their anti-spam measures as needed.

Step 2

Send the forgotten password initiation email message to the provided email address.

The amount of personal information contained in this email will be absolutely NONE.

Optionally, you can now include the helpful clickable link to take the user directly to your site.

Step 3

At this point, you can go back to Step 4 of the Validation process above.

Monday, August 22, 2011

TCSH isn't dead...

My tcsh configs have been growing for years.  There's probably still a few lines in there from my first .tcshrc config 15+ years ago.  It's an amalgamation of years of collecting little snippets from here there and everywhere, as well as plenty of my own additions.

I cleaned it up a bit and made it somewhat presentable sharing with others.  It's a modular configuration that allows dropping in chunks of config very easily as well as a crude distribution mechanism to copy it to lots of destinations.

Despite how modular it is and the fact that it's sourcing lots of files, it's actually quite fast so there's no delay opening a shell using this config.


tcshrc.d on github

Friday, April 15, 2011

Whack whacking and IPv6...

I saw this in a forum and had to look it up. As hard as it is to believe, the following is true.

Quote: IPv6 Address Nomenclature Used for a UNC Path

Follow these steps when specifying a literal IPv6 address in a UNC path:

Replace any colon ":" characters with a dash "-" character.

Append the text ".ipv6-literal.net" to the IP address.

For example, the nomenclature for a URI that points to a file share on a computer with the IPv6 address 2001:DB8:2a:1005:230:48ff:fe73:989d would be:

\\2001-DB8-2a-1005-230-48ff-fe73-989d.ipv6-literal.net\<sharename> Where <sharename> is the name of the file share on the target computer.

Thursday, March 17, 2011

"IPv6 no cause for alarm" - Melvyn Wray, 2011

My response to this article by idiot Melvyn Wray.



I hope you aren't paid to write for this website.  If so, you need to be fired immediately.

2000 wasn't a meltdown because corporations the world over spent BILLIONS on their software and infrastructure.  The company for which I was working at the time started hiring contract programmers in 1994 and still crossed their fingers hoping it would be enough time to audit and rewrite millions of lines of code.

My current company has allocated around $10 million dollars in infrastructure over the next couple of years.  As an ISP, you'd think we'd be able to find SOMEONE with a turnkey solution to give us a smooth ipv6 migration path.  Instead, despite the imminence of ipv6, we're starting to feel like pioneers in the industry, a position that we do not desire in the least.  For example, there are currently NO vendors offering a full-featured large scale nat solution to buy us some time to get fully underway with ipv6.  When you ask any of them about PCP, you're met with blank stares or "why would anyone want to do that?"  Granted, the initial draft of the IETF document was written only a month or so ago, ( https://datatracker.ietf.org/doc/draft-ietf-pcp-base/ ) but it should not be the burden of us as the customer to explain why this is critical to a vendor who is boasting that they have the best LSN solution.

The burden is on us to come up with a solution that allows our customers to access both ipv6 addresses as well as ipv4 addresses for at least the next decade.  I'd ask you to think of the ramifications of that task, but you're not capable.  Suffice it to say, for the forseeable future, access to both ipv4 and ipv6 is mission critical.

Every single piece of hardware serving customers is affected.  Every cable modem earlier than docsis 3.0 needs to eventually be replaced. Every cable tv set top box will eventually need to be replaced.  Not only does the cable modem need to be replaced, we're discussing the very real possibility of having to put a nat/fw at each customer premises.  The reason for this?  If, today, you walk into your local Staples, Best Buy, Dixon's, or Curry's, *almost none* of their devices are fully ipv6 capable.  Are you the one who is willing to tell the customer that, sure, your Roku, Tivo, Slingbox, Kindle, iPod, wifi Galaxy Tab, Xbox, Wii, Airport Extreme, and Magic Jack will all definitely work with ipv6?  You'd bet your paycheck on it?

Every one of our servers needs some attention because even the most recent 5.x version, Red Hat Enterprise and their glorious patch hostage business model is *STILL* based on linux kernel 2.6.18, which has some ipv6 related weaknesses.  Every piece of software that serves our customers needs at least some rewriting, especially with regards to anti-spam.  There is very little in the way of good, solid support for ipv6 blacklists.  The author of rbldnsd, for example, just committed some early support for ipv6 in his code a little over a week ago.  It's just enough to get us some working blacklist functionality.

Many of the most expensive pieces of our core networking infrastructure needs to be replaced.  While Cisco may have stamped "ipv6 ready" on the side of much of their gear, the act of turning on ipv6 support nearly brings the box to its knees.  The reason is because instead of having each line card being capable of hardware flows that require very little interaction with the supervisor engine, ipv6 is implemented in software-only, so suddenly every single packet needs to be sent to the supervisor card for handling.  This alone may cost us millions.  We're by no means a Cisco-only shop, so we're hoping that our testing doesn't reveal too many more surprises like this.

And you've obviously been reading nothing but puff pieces regarding ipv6 and security.  While it has a small number of nice features, it offers not one single bit of increased security that should cause ANYONE to breathe easier.  If anything, it should stress people out even more.  We're going to be finding fundamental flaws in the handling of ipv6 for years to come.  As recently as a couple of years ago, I discovered a critical flaw with the version of glibc that openSUSE was using at the time and it's handling of AF_INET, AF_INET6, and AF_UNSPEC.  It was easily missed because this flaw only surfaced when interacting with oddly behaving DNS servers and so only affected, at best, 1% of their userbase.  To my knowledge, no one ever figured out the entire cause in any sort of forensic detail, and was eventually fixed. 

This also glosses over a very serious shortcoming in ipv6 to which none of the original designers are willing to admit.  The problem is that ipv6 was designed in the early to mid 90's, when nat didn't even exist.  While the designers lauded themselves for guaranteeing that everything could have a public, internet facing ip address, they forgot to take into consideration whether or not everything *should* have a public, internet facing ip address.  Do you think the hacker's at the recent Pwn2Own conference were balked in any way by ipv6?  Do you think the recent, critical icon vulnerability in Windows is in any way stopped by ipv6?  Does the display of a favicon.ico file that exploits this vulnerability somehow become subverted because it was loaded from an ipv6 website?  Using nat provides a fundamental level of security that truly makes the internet a safer place.  Insisting, even despite proof to the contrary, that every device should be accessible via the internet is just plain wrong, no matter how many phd's can't get their own heads out of their asses.

All you're aware of is the little bit of a golf ball sized chunk of ice that you've been able to perceive.  The reality of the iceberg underneath is absolutely cause for alarm.  Keep your finger pointing and chicken little accusations to yourself and let the big boys who do the real work take care of business.

Here's to your writing success in a field other than technology.


Monday, March 14, 2011

Anyone used mysql with 128-bit binary strings? Riddle me this...

I was doing a little experimentation wielding ipv6 with perl and mysql:

So, given this table schema:



mysql> desc binary_i;


| Field | Type             | Null | Key | Default | Extra          |


| id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |

| ip    | binary(255)      | NO   |     | NULL    |                |

| mask  | binary(255)      | NO   |     | NULL    |                |


3 rows in set (0.00 sec)





Can anyone explain this??


Show a row:


mysql> select * from binary_i limit 1\G

*************************** 1. row ***************************

  id: 1

  ip: 00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101

mask: 11111111111111111111111111111111111111111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000

1 row in set (0.00 sec)





Copy the ip value from that into a select statement:



mysql> select * from binary_i where ip = '00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101';

Empty set (0.01 sec)





Try again with double quotes:


mysql> select * from binary_i where ip = "00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101";

Empty set (0.01 sec)



Still Empty??

Try it as a like statement with preceding and succeeding % symbols:


mysql> select * from binary_i where ip like '100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101%' limit 1;


| id | ip


|  1 | 00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101


1 row in set (0.00 sec)




Success.  Using the same string as I did in the previous WHERE clauses. I'd be a lot less baffled if I weren't COPY and PASTING the binary string directly from an existing entry in the db.