SmashTech

Here’s a quick refresher on why it’s a good idea to stay away from enable mode on routers before you’ve had a couple cups of coffee:

drvr-wifi#show flash
System flash directory:
File  Length   Name
  1   4707392  cxxxx-bnt-3mz.122-5f.bin
[4707456 bytes used, 3681152 available, 8388608 total]
8192K bytes of processor board System flash (Read/Write)
drvr-wifi#delete cxxxx-bnt-3mz.122-5f.bin
Delete filename [cxxxx-bnt-3mz.122-5f.bin]?
Delete flash:cxxxx-bnt-3mz.122-5f.bin? [confirm]
drvr-wifi#copy tftp://10.254.1.5/cxxxx-bnt-3mz.122-5f.bin flash:cxxxx-bnt-3mz.122-5f.bin
Destination filename [cxxxx-bnt-3mz.122-5f.bin]?
Loading cxxxx-bnt-3mz.122-5f.bin from 10.254.1.5 (via FastEthernet0): !
%Error copying tftp://10.254.1.5/cxxxx-bnt-3mz.122-5f.bin (Not enough space on device)
drvr-wifi#reload
Proceed with reload? [confirm]
System Bootstrap, Version 12.0(3)T, RELEASE SOFTWARE (fc1)
Copyright (c) 1999 by cisco Systems, Inc.
C1700 platform with 32768 Kbytes of main memory
boot: cannot determine first file name on device "flash:"

For those of you not familiar with Cisco routers, what I inadvertently did was delete the firmware on the router, try to copy a new firmware to it, and missed the ‘Not enough space’ error that popped up. Upon reboot, the router couldn’t boot anymore and I was dropped to a rommon prompt.

For anyone else who might run into this, there is a handy guide on reflashing IOS from rommon and TFTP available at the Cisco Blog.

Yesterday, I learned a valuable lesson on why you should never assume that a situtation will ‘never happen’ when it comes to a server or network.  In this case, it revolves around a POP3 mailserver and email attachments.

One remote site accesses all of its email via POP3, unlike all the other sites which are on Exchange.  This site is connected to the core office by a T1 link.  Normally, problems are few, as most traffic on the link is telnet.  Then, one user sent an email message.  A very large message.  Around 23MB large.  To all 89 employees at that site.

In going over the logs for that timeframe, the source message took about 10 minutes to send.  That didn’t cause any problems.  Its what happened once it hit the server that brought everything to a crawl.  All the users are setup on a Linux VM.  When sendmail received that 23MB attachment, for all 89 users, it made 89 distinct copies and gave one to each user (this here is why I love Single Instance Storage in Exchange).  The copies immediately chewed up a little over 2GB of space. 

Within minutes, the T1 link was suddenly brought to a standstill by the other 89 users’ Outlook doing a send/receive operation automatically.   The phones began ringing, and the problem was quickly tracked down.  However, the network link for the server did have to be disconnected for a few minutes to prevent users from getting a lock on their mailboxes, so we could clean them up.

Had the message been allowed to sit in place, it would have taken slightly over 3 hours and 5 minutes, at full saturation of the T1, for everyone to get their mail (and do nothing else during that time).  The results of this little fiasco?  Attachments now have a file size limit to match our Exchange limits, and POP3 traffic is rate-limited on the link to 768kbps.

I will be so glad once this location has been switched to Exchange.

Well, Im finally getting a widespread infection of SDBot under control here at work. Spreading via various DCOM and RPC exploits, SDBot caught us completely off guard, as we were used to email viruses instead (we still get ILOVEYOU.vbs opened every now and then).

Partly, it was our fault for not keeping our Win2K/XP machines up to date. We figured they are all behind the firewall, so nothing on port 445, 5000, or whatever it uses can get in. This overlooked the fact that someone might (against company policy) bring in a home PC and plug it into the network (which is probably how this got in).

To clean each system takes about 2 hours, which consists of installing SP2 (SP4 + an RPC patch on Win2K), resetting all Internet Explorer settings, and running SpyBot and AdAware on the system (because this variant drops about 6 different spyware programs). It is a royal pain in the ass. The person who wrote such a beast must be subjected to horrendous pain and torture by my hand.

Nintendo has announced its next portable system, the Nintendo DS, or Dual Screen. It features dual processors, dual 3″ LCD screens, and possibly wireless integration. In a press release, the company states that this is not designed to replace the GameCube or Gameboy Advance. No word if the screens will be backlit or not.

This is already being called the next Virtual Boy by many pundits, and Im tempted to agree with them. Why buy a new system that cant play your old games? We’re now spoiled by the backwards compatability that the PS2 and GBA gave us. This will flop. Big time.