rebelpeon.com

Category: work

  • More ISA Site-to-Site IPSec VPN Configuration

    The last two months, traffic on this site has increased by almost 50%.  Honestly, I don’t know specifically what it’s related to, but my number one search item is “ISA” followed by #4 which is “2006”.  Therefore, I thought I’d post a little bit more, since there are still some issues that I had run into.

    First thing is the issue I discussed last time, which was about IPsec not creating the filters it needed to.  I think I may have found the solution to that, but I haven’t verified it (which I may do this week while I’m on the bench, in between various training).  Now because of the fact that ISA relies on Windows 2003 IPsec, there are some pretty awful problems.  The first being “adjacent ranges” in your IPsec rules.  Windows IPsec does not allow you to have two adjacent IPSec policies.  Instead it believes it should be one continuous policy.  When you attempt to create an adjacent policy in ISA 2006 you will recieve the error message below.

    As an example, let’s say that you need access to the individual hosts 10.10.10.150 and 10.10.10.151 at the remote site (or two ranges like 10.10.10.0/24 and 10.10.11.0/24).  Now, if the remote site happened to have a Cisco Concentrator, they would be able to publish each of those hosts (or subnets) as separate IPsec policies.  However, with ISA, they have to be in the same remote networks range.

    Many times, there isn’t a problem, especially with a small number of hosts or ranges like this.  However, the problem comes into play with subnetting.  Typically hosts are designated with a 32 bit mask (255.255.255.255).  However, since we’ve now created a range, we may see a different mask (255.255.255.254).  It’s when the different, unexpected mask comes into play, that we have issues.  If the mask is wrong, Phase II negotiations fail, and you’ll not be able to create a Phase II tunnel.  However, if you don’t put the the hosts into the range and ignore the warning that ISA gives, the IPsec policies won’t be created, and you’ll have to manually create them whenever the IPsec service restarts (specifically if/when the machine restarts).

    Finally, there’s yet another IPsec issue with Windows 2003, that again manifests itself with ISA.  There are multiple ways you may see this.  One way is that no matter what you set as your Phase II timeout policy from within ISA, you’re seeing Phase II rekeying happen about every 300 seconds.  Another way is that you IPsec Site-to-Site VPN connections drop a lot and in the logs you see the error “0xc0040014 FWX_E_FWE_SPOOFING_PACKET_DROPPED”.

    The first thing I tried was to disable IP Spoof Detection.  However, that didn’t seem to fix it, plus, since this is an external firewall, I wanted to keep the spoof features on to do application filtering.  The part that was really frustrating was that rekeying was happening every 300 seconds, instead of the 3600 I had specified in ISA.

    Well, it turns out it’s a bug with ISA and/or Windows 2003 IPsec.  This was actually a bug with ISA 2004, but apparently it wasn’t deemed big enough to fix with 2006, since there’s a workaround that works well.  Microsoft KB article 917025 goes over exactly what to do, but the gist of it is, is that you need to edit the SAIdleTime registry key and change it to 3600 (default is 300).  The downside is that 3600 is the max (trust me, I’ve tried to set it higher, it doesn’t work at all), so plan your IPsec Site-to-Site VPNs accordingly (let your peer know that the max will be 3600 seconds).

    Hopefully those two nuggets will help anyone having other issues.  I’m sure I’ll post more things too, as they come up.

  • Email

    I just love coming back from a week long vacation, containing 5 business days, to over 1000 emails.  I feel so, so, overwhelmed.

  • ISA Site-to-Site IPSec VPN

    I wasn’t necessarily going to post this, but since ISA seems to be the most linked to thing on this site because of only 2 articles, I figure it can’t hurt to talk about it. Especially since it was a very strange problem I had with it and I’m sure I won’t be the only one with it.

    Anyways, at work I am utilizing ISA 2006 Std edition in a front and back wall scenario. Site-to-Site VPN terminate on the external firewall, and all of our local VLANs (55 of them) are routed off of the internal firewall. So far, nothing that complex. It’s just a simple DMZ between the external and internal network setup.

    Anyways, I had a site-to-site VPN (IP pre-shared key) between a customer and us. Basically, we just need to hit a single machine, so the remote network contained two IP addresses, one for the client’s gateway (this is added by default in ISA 2006, DO NOT delete it, also be sure that the remote site has added your gateway in as a remote network too!) and another for the machine we needed to hit on their local network. Anyways, it was working fine. Well, actually, nobody was using it quite yet, but testing had been completed, and I was able to access everything that the developers would need. Anyways, the customer decides that they need to add another IP address that we’ll need to access. Again, no big deal. I’ll just add the IP to the network list for this client. Just to make sure everything’s working, I test it. Nothing works to the new IP. However, the old IP still works fine. What the hell?!

    For those of you unfamiliar with ISA, it’s not like I created a new VPN for this new IP addition, or anything like that. I simply added the new IP to the existing network. All the routing and firewall rules remained the same. Adding the new IP to the list of remote networks should have allowed it to work.

    Working with the IT person at the customer, I learn that when I try to hit the new IP address, the Quick Mode authentication was failing because the ISA server was sending the wrong local network that the request was coming from. The local network that was defined in the rule (by putting a subnet destination in the network rule) was 10.254.95.192/27. However, on the client’s side, he was seeing the request coming from 10.254.64.0/19. In order to create the IPsec tunnel, both the local and remote networks on each end of the tunnel must be identical, but switched (i.e. my local is his remote, and his local is my remote). Needless to say, this 10.254.64.0/27 was screwing everything up. However, when I connected to the original IP that worked, it was sending the correct network of 10.254.95.192/27.

    Of course, no where in ISA 2006’s logging can you see it making the IKE requests. All I could see is that requests were being routed correctly from the internal ISA to the external ISA, and then from the external out to the correct network for the customer VPN. In essence, traffic was going in to a black hole. I could also see that the VPN connection (Main Mode) was up and running. I was completely reliant on the customer to let me know what was coming down the pipe to him. That right there is not really something I’m comfortable with, but he seemed to be OK with it. I’m sure it’s because he knew it wasn’t on his end, but on mine.

    After deleting the VPN multiple times and recreating it to no avail, restarting the machine, etc, I knew that I would have to get some help from someplace else. Thankfully we have an awesome community of people at work that I could bounce ideas off of. Unfortunately, I never received a response. Also, ISAServer.org is a great place to get information. They have forums there that people keep an eye on. Unfortunately, ISA 2006 is still quite new and not as many people deal with it. I also did not receive a response from there. Needless to say, I was on my own for this one. Not a place I really wanted to be, since I thought I was at the end of my ability.

    Actually, the IP isn’t really done in ISA server at all. Much like everything else that ISA server does, it’s just an application that sits on top of the OS and utilizes things that are already built into the OS (in my case Windows 2003 R2). This means that all IP policies, rules, etc are done by Windows and this can be monitored using the IP Security Monitor MMC Snap-in.

    Since the VPN tunnel was being created successfully, I knew that Main Mode IKE Policies were correct, it was the Quick Mode policies that were causing me grief. Since we have multiple VPN connections terminating on this firewall, there are a lot of Quick Mode IP policies in place. Especially since all of them use pre-shared keys, which require that two IP policies are created, one for inbound and outbound (otherwise you can have one policy that does both inbound and outbound).

    Scanning through the policies I was able to find the inbound and outbound policies for the original customer IP address to the 10.254.95.192/27 network, but I wasn’t able to find it for the new customer IP address. Alas, the problem! The next best policy for the new IP address was for the 10.254.64.0/19, since this policy encompasses the 10.254.95.192/27 subnet. Finally, I felt like I was making progress. Unfortunately, ISA should have been creating these policies when I edit the customer VPN networks. Actually, I still have no idea why ISA isn’t creating these policies. This is why I think there’s a bug which I’m going to submit to Microsoft (via this post actually).

    Now that I knew the source of the problem, I had to fix it. Some days diagnosing the problems take longer than fixing them, and some days it’s the other way around. Since it had already taken me about a day to find the problem, I hoped that it wouldn’t take that long to actually fix it.

    Needless to say, you can’t add IP policies from the IP Security Monitor MMC Snap-in, because, well, it’s a Monitor not an editor. The IP Policy Manager MMC Snap-in was no use either, as it defines computer level policies. Doh. Well, I can finally say that one of my certifications actually came in handy. That “+ Security” portion of my MCSE gave me the knowledge that there is a way to edit IP policies from the command line. Going on this, a quick Google search gave me exactly what I was searching for. Now which command to actually use?

    At first I tried to just create a filter. However, I didn’t know of any filterlist, and none of the current filters were a member of a filterlist. Thankfully you can just make up a name and it creates on. Unfortunately this didn’t solve anything. Nothing showed up in the Quick Mode filters. Lets try again, yeah?

    Turns out it’s not a static setting, but a dynamic setting, which makes more sense. Anyways, you can add Quick Mode rules pretty much the same. In that I mean, the command is just as long and gross. Just be aware, that since I wanted to add a Quick Mode rule and not a Main Mode rule, I had to put in the Quick Mode Policy variable.

    Another thing that made this so confusing was that in IP Monitor, they are called Quick Mode Filters and at the command line they’re called Rules. Ugh. At least it’s taken care of. And now I think I know more than I ever wanted to about ISA and IP.

  • NetApp Certified

    Well, I have another acronym to stick at the end of my name, NACA.

  • Busy

    You can definitely tell when I’m busy and when I’m not based on how often I update this site.  As you can image, from how infrequently I’ve been doing updates, that I’ve become incredibly busy.  I received a promotion, and now I’ve taken on about 10x more responsibility than I had before.  It’s definitely different than what I was used to, but it’s a good different (at least it still is right now).  Instead of just being a worker bee assigned to a project, I now, in so few words, manage all infrastructure aspects of projects that come into the department. 

    This has resulted in a lot more work on my plate.  And not just more work, but more work that I’ve never done before (at least not at this scale).  Things like quotes and estimations.  Plus, I’ve been traveling some and trying to get some training done through NetApp.  Oh, did I mention all this responsibility was done by a person that was twice my level before my promotion?  Well, I can’t say that exactly because everything that he did didn’t just jump onto my plate, there’s a project manager that’s actually doing some of the work.  However, regardless, everything needs to cross my plate because of my technical abilities and my knowledge of what my team is doing.

    We’re also getting a NetApp FAS3050C cabinet in on Tuesday.  This is why I’ve been doing some training on it, since I’ll be point-man on getting that all setup.  We’ll primarily be using that as an iSCSI target for virtual machine hosts and SQL machines.  I’m personally really stoked about that.  ~30 TB of raw data to play with.  I’m glad that I’ll be doing that though, as it helps to balance out the ever increasing management responsibilities.  Toys are good.

    In non-work related news, I’ve been taking another photography class: Portrait & Lighting I.  It was actually one that I wasn’t really going to take, but I’ve definitely learned a lot.  I was skeptical of taking it at first because I had heard that the teacher didn’t go into much of the technical aspects.  However, that’s what I really like.  I enjoy learning all the technical stuff and then be able to play with that knowledge for my own images.  Well, the normal teacher is actually taking a break, so the same guy who taught our Photo II class is teaching this one.  This is probably one of the reasons I decided to take the class, because I like the way he teaches.  Unfortunately, there’s not a lot of “good shots” from this class as we’re basically just learning how to correctly use lights and many of the pictures are of different types of lighting or of a dummy head.  It’s possible that something from this next shoot will show up on aaron spruit (.com) though.  We’ll just have to see.

    In other photography news, I’m looking at getting a new 105mm macro lens and flash.  I’ve also been contemplating getting a light meter, and I’ve actually bid on a few on eBay, but I just don’t think I’d use it all that much.

    I think that’s about it, and be sure to check out aaron spruit (.com), since it gets updated every weekday.  Oh, and there should be some pictures taken in the last month showing up now, as I finally got out last weekend to take some pics.  Out of 100ish pictures, I’m probably going to throw up 29 or so.  It was nice last weekend; I just put some newish music on my iPod, threw on the headphones and took the camera out with me.  It was actually quite relaxing, but still amazing how many strange looks people gave me.

  • Six Flags

    Headed up to Six Flags yesterday for a corporate event.  The company I work for’s parent rented out the whole park for the day.  Normally I really have no desire to go to Six Flags.  It’s fun and all, but I’ve never been a fine of waiting in lines.  Especially for rollercoasters, which I’m not really the biggest fan of to begin with.  Plus you have to pay to park, pay for the tickets ($55), food, etc.  It really becomes expensive for what you get out of it.

    However, I’ll definitely go on a corporate event.  There were no lines, it only cost $25 per ticket, and parking was free.  Such a deal!  I rode every ‘coaster there at least once, and there were points where you didn’t even have to exit the ride, you could just ride it two times in a row.  There also weren’t that many people walking around.  It didn’t feel swamped and the weather was nice (despite a few minute showers).  There’s nothing like riding the American Eagle getting pelted with rain.

    Next time I go, though, I need to wear contacts.  There were a few times I felt a lil’ sick, but I think that had a lot to do with not being able to see.  Superman definitely made me feel that way, but as JoeJohn pointed out, it was probably because of the orientation of your body which made you feel as though you were hovering over the toilet.

    Best complete rides were definitely American Eagle which I rode 3 times or so and Raging Bull which I rode twice.  The best single drop though was definitely the first drop of Deja Vu.  It was the only ride with a long line, and it was closed for most of the day, but it was worth it.  Well, that is until we got stuck on it and it wouldn’t let us off.

    Rides we rode: Raging Bull, Superman, Batman (ugh, that one made me feel bad too), Deja Vu, Virtical Velocity (not really all the fun, but good on technical merit), Viper (snakes on a mf’in train!), American Eagle, Iron Wolf (painful for the ears), Demon, the bumpercars, and the Whizzer. 

  • Vacation Post Vacation

    Today is the perfect reason why I hate taking vacations for more than a 3-4 day weekend.  I took all of last week off, and now that I come back, everything is broken, and it’s like I’ve taken 3 steps back.  I had a plan for the next two weeks when I left, but now that I’m actually back, I won’t be able to get to any of that. 

    Two hours after getting to work today, I wished that I was on vacation again.  How fun is that?

    Oh, and did I mention that I’m starting to get the itch again?

  • Searching for Colocation

    I found this awesome service while searching for a new colocation for the project I’m currently on.  Since I’m sure everyone has seen the LendingTree commercials, I’ll compare it with that.  Basically, it’s the same thing, only with colocation instead of loans.  You go to the website, fill in the requirements, and wait for the offers to roll in. 

    A colotraq representative contacted me since I didn’t fill out all the requirements correctly, but that went smooth.  In under 24 hours, I’ve had 5 different companies giving me quotes.  This is crazy easier than individually calling places.  Heck, just even finding places was a major PITA.  This way, I really don’t have to do anything.  Awesome.

  • LiveMeeting 2005

    LiveMeeting 2005: when it works, it’s great, when it doesn’t, talk about horrible.  I’m trying to listen to some TechEd webcasts, but I get no audio.  The error message comes up and says it can’t download the codec that is used and I should click on “Web Help”.  Where is this mysterious “Web Help” button?  Also, why does it just not tell me what codec the audio stream is using so, well, I don’t know, go look for it?

    Talk about annoying.

  • Drive Compression

    Reason #1 why you DO NOT enable windows drive compression: uncompressing.

    Just to leave this entry not so cryptic…

    Windows compression is definitely not the way to go in any scenario.  Between the speed hit, the little space you actually get, and how cheap hard drive space is now, there is no reason to use it.

    Oh, did I mention how long it takes to uncompress a drive?