United States
Anatomy of an ARP Poisoning Attack
WatchGuard Technologies, Inc.
WatchGuard Technologies, Inc.
Products  

Security Articles

Video Tutorials

WatchGuard Feeds

White Papers

Case Studies

Network Security Glossary

Dustin and the Tunnel of Curses (Part 1)

by the LiveSecurity Content Team

[Editor's note: The educational (but fictitious) adventures of network administrator Dustin Barnes remain LiveSecurity's most popular recurring feature. Dustin's next adventure in network security begins here and finishes this Friday. And if you can solve Dustin's problem, you could win a kinda nice prize (details at the end). In our last contest, readers accused us of making the problem too easy. We think this brain-buster will balance the scales. Good luck! --Scott Pinzon]

Afterward, Dustin Barnes realized the email message was rude. But he first figured the author was merely in a rush, and hadn't realized that short messages look curt. Short? Heck, the contact information in the signature file was longer than the text body:

Why haven't you got the VPN tunnel up yet?

Sam Powers, MSCE, CNA, OMFC, WWBSA, NPC, DP

Executive Director

World Wide Technologies Integration Group

Consolidated Consortium of Global Mining and Manufacturing Companies

http://www.ConGlomCo.com/

The endless contact information included an address in Singapore and a mobile phone number, an office number, a fax number, and a pager number.

Dustin could deal with rudeness, anyway. He was getting used to it. The atmosphere at Kunstler & Sons Musical Instruments had grown tense in the months since Dustin and his assistant Nandi prevented a serious data leak [recounted in "Dustin and the Secret Plans"]. CEO Karl Kunstler III was dragging his family's century-old brass instrument business kicking and screaming into the modern era. He had lost several respected (but computer-phobic) old-timers when he hired Dustin to install and oversee a state-of-the-art Just In Time (JIT) system -- but now the parts inventory stayed lean. Next he had managed to gain an exclusive alliance with a Global 100 company, which was allowing Kunstler's company to test and pioneer uses of a new alloy that could make marching band instruments lighter, shinier, and better-toned. Kunstler had had to sign the devil's own non-disclosure agreement to close the deal. The twin strains of keeping a sizable manufacturing overhaul secret, while playing up to the level of one of the world's biggest companies, had changed the environment at ol' K & S. Morale-wise, the new alloy deal had morphed from "We landed a key contract!" to "We must prove we're worthy of working with a Global 100 company!"

Kunstler had given Dustin the name Sam Powers as the contact at Consolidated Consortium of Global Mining etc. etc. -- known worldwide as ConGlomCo. Dustin and Sam were supposed to coordinate setting up the tight new extranet relationship between the two companies.

But Dustin and Sam had never spoken. Instead, Dustin had spent hours yesterday on phone calls with a subordinate of Sam's, named Lee Boon Sen, a ConGlomCo technician in Singapore. Their relationship had started off roughly. Dustin had no clue about which language people spoke in Singapore (Singaporian, he assumed). Early in the call from Lee, Dustin had made the mistake of blurting, "Wow, you speak English as good as me!" To which Lee replied, "You mean 'as well as I,' and I happen to speak English better than you." But they'd managed to set up an IPSec VPN tunnel to handle the data between the two companies in confidentiality. Dustin had cheated, using WatchGuard's Online Help, especially the chapter in the VPN Guide labeled "Configuring BOVPN with Manual IPSec." Lee remained haughty throughout the process, but Dustin overlooked that. Lee knew his stuff, and that's what mattered.

But the tunnel was already up, so why the blunt email from Sam Powers, HIJK, LMNOP? Better check it out.

Dustin used his KVM switch to change his display from his PC to his Management Station. He double-clicked the WatchGuard Control Center icon, which he kept on the computer's desktop for easy access. A window popped up, labeled with the IP address and model number of his Firebox, listing the status of its three interfaces:

He clicked the plus sign next to Branch Office VPN Tunnels, and stopped in mild surprise. There should have been details under his ConGlomCo tunnel describing how much traffic had passed. There were no details. The VPN tunnel he had diligently crafted with Lee -- gone.

Puzzling, but not alarming. In this early phase, live production data was not coursing through the tunnel yet. The connection might've simply timed out due to lack of traffic.

He sighed. Troubleshooting IPSec was neither his strength, nor his idea of fun. But he'd see how far he could get.

From the Control Center, Dustin clicked the Policy Manager button to open the Service Arena. A colorful array of icons greeted him. Ignoring them, he clicked on the Network menu, selected Branch Office VPN, then chose Manual IPSec...

The IPSec Configuration window displayed. He studied his IPSec routing policies, searching for any obvious errors, making sure the local and remote network information was correct. It looked OK to him. Not that that meant a lot, given his confidence level about his IPSec knowledge.

He clicked the Gateway button to see what IP had been entered for ConGlomCo's box on the other end. Lee Boon Sen claimed that ConGlomCo security policy forbade him from revealing what specific device, or even what brand of VPN appliance, Dustin's Firebox was trying to terminate at. In Dustin's experience, tech guys usually bent the rules to help each other. But Lee had stuck firmly to policy. So Dustin had no idea what kind of box the tunnel was terminating at, so he couldn't use WatchGuard's VPN Interoperability page. But he remembered what IP address he had used when the tunnel worked yesterday. It was still entered accurately.

He closed the Gateway window and clicked the Tunnel button. Yep, the policy was still set up.

He couldn't see anything wrong.

He sighed, propped his left elbow on his desk and plopped his chin heavily into his left palm. The fingers of his right hand drummed a soft but frantic pattern on the desk. What to do? He ignored his ringing phone in the interest of completing one entire thought for the day. A chiming reminder popped up from Microsoft Office. He ignored that, too.

When his telephone stopped ringing, he lifted the receiver and hit the speed dial button for Lee. Dustin checked his watch and did a little mental math. Midnight in Singapore. He'd have to leave a voice mail message.

"Consolidated Consortium of Global Mining and Manufacturing Companies Data Operations Center. This is Lee Boon Sen speaking. How may I help you?"

Wow. Did he say all that every time he answered the phone? "This is Dustin Barnes at Kunstler and Sons Musical Instruments. You're sure working late!"

"Oh. Dustin." In the tone of a grump explaining something for the tenth time to a slow child, Lee said, "We specified project deadlines in our contract, and we intend to meet them."

Apparently the long hours weren't helping Lee's disposition. Trying to establish some sympathetic bond, Dustin said, "Guess that means less hours of free time, huh?"

"You mean 'fewer hours,'" Lee corrected. "Did you call merely to determine the length of my shift?"

Geez. Better cut to the chase. "The VPN tunnel we set up yesterday has gone down. I'm trying to find out why. Does everything look OK on your side?"

"Please hold the phone."

Suddenly Dustin was hearing 101 Strings play Culture Club's "Do You Really Want to Hurt Me?" He indulged in further finger-drumming, trying not to feel Kunstler & Sons profits trickling away into an endless international telephone call. He read and deleted email until a little voice in his ear said, "I confirm that the tunnel is down. But our end is configured perfectly."

Dustin sighed. He was sighing a lot lately.

Lee suggested, "Perhaps you should reboot your Firebox."

"OK," Dustin said. In the Policy Manager, he clicked File, then Reboot. The Control Center window, poking up behind the Policy Manager, changed to say "Connection Lost." He waited a minute in silence, figuring that no matter what he said, Lee would object.  Then he clicked the little green Play triangle. Suddenly an IP address appeared where "Connection Lost" had been. "OK, my Firebox is back up. Can you send me some traffic to see if the tunnel is back?"

Lee exhaled heavily, as if Dustin had made this request a thousand times. Dustin could practically hear the guy's eyes roll. "I'll ping your server."

Dustin did the little Alt + Tab trick to bring Control Center to the front. After a second, the Branch Office VPN Tunnel entry for ConGlomCo sprouted a plus sign next to it, plus some traffic details:

"It's up!" Dustin realized. "Cool!"

"Glad to be of service," Lee said, in a tone that meant the opposite.

"So how's the weather there?" Dustin asked, cringing as he said it. Poor impulse control. His psyche persisted in trying for a buddy-buddy relationship.

"Tropical, as one would expect, " Lee said, as if amazed that anyone would ask something so inane. Pause. "Is there anything else?"

Amazing how the guy could make the politest question imply, "you dunce." But another thought had struck Dustin. "Is Sam there?"

"No. Sam is en route to you." Lee said "en route" in a flawless French accent.

"Oh. Well, if you talk to Sam, let him know we have the tunnel working, OK?"

A silence. Then, with an evident smile in Lee's voice: "Why yes, I will let Sam know."

Was Lee finally warming up to him? Whatever. Dustin suddenly remembered that this call was getting expensive. "I'll also send Sam an email. Thanks for all your help. Have a good day!"

"In Singapore, it is night," Lee pointed out. "Goodbye."

Dustin gazed at the now-silent phone receiver, thinking, Ooo-kay. Attention to detail is a good trait in a technician, right? Could've been much worse. Icy tone of voice aside, Lee had helped him. He still didn't know why the tunnel had gone down, but getting it back had been easy. He hung up.

He KVM-switched back to his PC running Outlook, and typed a quick email to Sam informing him that the tunnel was indeed up. Click Send. There. Another crisis handled.

Only 999 more to go, and then he could go home.

Dustin arrived in a good mood the next morning. But by the time he'd walked from the lobby of Kunstler & Sons to his office, he'd handled a "can't print" complaint, rebuffed an aggressive co-worker selling fund-raising candy bars, explained to a trio of factory workers why he would not allow them to use IRC to communicate across the shop floor, and learned that Larry Trank in Accounting had broken headquarter's only Jaz drive by somehow rolling his office chair over it. The good mood was ancient history.

When Dustin opened Outlook, the usual couple hundred emails had piled up overnight. He had set up an Outlook rule that listed in red any email from CEO Karl Kunstler III. So his eye went right to an unread email from Kunstler, with Dustin's boss cc'd. The subject line read, "Please get on this":

Cannot risk blowing this deal -- let's keep Sam happy.

Everything OK there?

KKIII

-----Original Message-----

From: Sam Powers

Sent: Friday, June 20, 2003 3:36 AM

To: Karl Kunstler III

Subject: Must insist on VPN

The VPN tunnel between our networks is still not up. I must insist that your people get this working ASAP or we jeopardize the timeline of ConGlomCo deliverables.

VPN has not been a problem for our previous partners. Will check to see if your team needs help when I arrive.

Sam Powers

The email finished with Sam's long cavalcade of contact info.

Shizznit. Suddenly Dustin felt very small-time. Then indignation trickled adrenaline into his gut. Why had Sam ignored his friendly email reply and gone over his head to complain? And why did Sam think the tunnel was still down?

Because he left his Management Station running all the time, Dustin didn't have to wait for it to start up. He KVM'd to it, logged into the Control Center, and -- what?

No tunnel.

He practiced a little deep breathing.

Then he ran through all the same steps he had yesterday: check the routing policy, check the gateway, check the tunnel policy. Everything still looked right.

This time, instead of calling Lee, Dustin rebooted the Firebox. When it came back up, so did the tunnel -- apparently. He needed to initiate a connection to something behind ConGlomCo's Mystery Firewall in order to test the tunnel. So, working from information he had gathered two days ago, he pinged the IP address of Lee's PC, then pulled up Control Center's Traffic Monitor to watch the results.

After a moment, his ping timed out. According to Traffic Monitor, he had a Phase 2 error (shown right here).

"Great," Dustin grumbled. He adopted his chin-in-palm-propped-on-elbow posture and thought hard. What kind of tunnel existed some of the time, but went away sometimes, apparently at its own whim?

Whatever it was, he'd better figure it out fast. Otherwise, it seemed Sam Powers, UVW, XYZ, Executive Director in a Global 100 company, was flying all the way from Singapore just to get him fired.


Take me directly to Part 2!