As an example, the following command will find all computers in Active Directory that have not been logged into during the past 8 weeks:

dsquery computer -inactive 8 -limit 0

After reviewing this list to make sure these computers no longer exist on your network you can use the following command to find and delete them:

dsquery computer -inactive 8 -limit 0 | dsrm

The DSQUERY utility comes with the Windows Server 2003 Support Tools package (Adminpak.msi) which can be installed directly from your Windows Server 2003 installation media or downloaded from the Microsoft website.

A lot of people use the Cisco IOS rate-limit feature. They have a real need. What they don't have is real documentation. Cisco itself hasn't produced a decent explanation of how rate-limit works. Yeah, there are some books that pretend to explain what the 'normal burst' and 'extended burst' settings do. But they STILL don't tell you how to figure them out in real life.

I'm going to tell you. And after I tell you, you're going to feel better. The network will be faster.

Why the burst settings matter?

A normal router interface is limited by the inherent speed of the wire. When incoming packets arrive faster than they can head out the interface, they are placed in the outgoing packet queue. Each interface has one, typically 40 packets in size. As packet flow reaches and surpasses the wire speed, this queue begins filling up. Those packets belong to someone -- a particular TCP connection. (We are ignoring other packet types, but that is okay.) When those packets end up in the queue, they are being delayed. This would be similar to ping times starting to rise. The TCP protocol notices this delay (because the receiving computer will take slightly longer to send back an ACKnowledgement packet), and the flow slows slightly. If too many packets fill the queue, they start dropping. When TCP sees dropped packets, it slows more significantly. This is how TCP works, and it works well.

What happens when you rate-limit instead of letting an interface reach line-speed?

When you rate-limit, the outgoing packet queue never comes in to play. When the rate-limit sees a flow exceeding the limit, it simply drops a packet. There is no outgoing queue to slow the packet momentarily. This is why there are two settings: the normal burst size, and the extended burst size. The size of the burst is essentially telling the router how tightly you want to apply the rate-limit. Too loose, and flows may momentarily exceed the limit by an undesirable amount. (What constitutes undesirable is completely up to you; some folks are using rate-limit to sell bandwidth, and others are using it to keep a line from ever reaching saturation and hence keep latency down.) When the extended burst is exceeded, ALL packets are dropped until the flow comes back under the limit. That will happen from time to time, but it is bad for consistent performance. The normal burst drop can occur many times as the flow oscillates around the rate-limit. The smaller the normal burst is, the more drops it can accomplish, which is good in moderation, bad when overdone. A good rate-limit can actually improve performance under heavy load versus no rate-limit. But a bad rate-limit screws up everything.

Too tight

Usually, the burst settings end up too tight. When they are set too tight, three things may happen: 1) The combined traffic flows will have trouble reaching the limit. 2) TCP flows will be jerky, resulting in un-smooth web page loading or wildly fluctuating download speeds. 3) A single TCP flow (ie a speed test) will NEVER get close to the limit. (Insufficient TCP Window size on server or client can also cause this)

Burst settings end up set too low because people are concerned that setting them higher is going to cause the rate-limit to be too loose. This isn't true; for every byte sent above the rate-limit, a byte must not be sent (or worse, dropped!) below the rate-limit. So don't be too tight with burst!

Just right

Cisco IOS reference manuals (certain editions, at least) say that normal burst should be (RATE-SPEED/8 * 1.5), and that extended burst should be double the normal burst. These suggested settings are great for smooth TCP, but they are sometimes too loose.

I'm a tightwad; what's the lowest burst I should use?

There is one more variable you should know in order to figure this out: The typical or highest-typical ping time. If you are in the US, accessing primarily US sites, a ping-time of 150ms is probably a safe tight-wad number.

So here is your formula for normal burst: (RATE-SPEED/8 / 20)

The formula for extended burst: NormalBurst + (RATE-SPEED/8 * 0.150)

Using an example of 20Mbps, your rate limit would be:

rate-limit out 20000000 125000 500000 conform-action transmit exceed-action drop

Cisco says extended burst should be double the normal burst, but that assumes you are also using their formula for normal burst. I don't assume that you are, which is why I specify extended burst as an amount that should be added to normal burst. In this way, you can use very small normal burst values, and still know what your extended burst should be in order to ensure smooth TCP performance.

Why is that the lowest?

150ms is your round-trip time (RTT). That is how long it takes a packet to be sent, and the acknowledgement to be received. Once a flow reaches this speed, and exceeds for a burst of the size/duration for one twentieth of a second, a packet will be dropped. The extended burst is the normal burst, plus the total bytes that can be transferred in the RTT. That is how long it takes TCP to notice that a packet was dropped and was never acknowledged. So, when you set the parameters thus, you are giving TCP enough time to "take the hint" of a single dropped packet before it gets the major punishment of all packets being dropped.

If you don't like these burst settings, and they are causing the flow to exceed the rate-limit too much or too often, I would suggest that it is NOT the burst sizes that you want to reduce, but the rate-limit speed itself. Knock it down a percent or two and see if that helps you out.

Oh, so you want to push the envelope, and want to have even lower settings? Well, in that case, you could use 24000 for the normal, and 399000 for the extended. The important thing is that you've still got one RTT between the first packet drop and the complete packet drop. You may not get full speed at this level, but it will be smooth. The potential harm is that with only a 24000 burst, the router is looking too closely at the flow. In our 20Mbps example, 24000 bytes is only one-hundredth of the per second speed of the flow. When you measure the flow this tightly, the rate limit will end up being applied too often, when the overall speed is actually not in excess of the rate-limit. For instance, a web browser that suddenly opens a bunch of connections as a page is loaded could (although statistically unlikely) cause a 24,000 byte burst as that handful of connections are opened and their first 2 or 3 packets are sent without waiting for acknowledgement. Use your judgment. If your judgment sucks, use bigger numbers for the burst, and smaller numbers for the speed.

An exception: These rules don't apply on high speed links with hundreds of TCP (or other) connections. If the router starts to drop all packets, the randomness works in your favor. It is likely that every packet dropped is part of a different TCP connection than every other packet dropped. No single download through this link should expect to use more than 2% of the link capacity in order for this to apply. In this case, the extended burst value can approach the normal burst value without impeding TCP performance.

 

Good explanation - Notes

Using Network trace to troubleshoot intermittent connectivity issues

Snehadeep

18 Jun 2010 8:59 AM

• Comments 0

Intermittent connectivity issues from a client machine when connecting to a remote database server can be often attributed to the network layer. By collecting the network traces and reviewing them we can tell why the connections failed to succeed or if the existing connections were getting closed. Recently I worked on a case where the customer was seeing intermittent GNE from an application that was trying to connect to the SQL Server remotely. Here is the error message they were getting.

OleDB Error Microsoft OLE DB Provider for SQL Server [0x80004005][11] : [DBNETLIB][ConnectionWrite (send()).]General network error. Check your network documentation.

After doing some initial troubleshooting like checking if the TCP chimney was disabled, if the application server was running out of max client port number etc. we collected simultaneous Network traces from both the application server and the SQL server. When you collect network traces always make sure to collect the following two as well.

1. ipconfig /all from all the servers involved.

2. Exact error message that shows the timestamp. If an error message with the timestamp is not available then request to record the exact time of the issue and send to you.

Below I will show step by step how we reviewed the collected network traces. You can use the same techniques to analyze the traces collected for the intermittent connectivity issue to the SQL server.

You can use Netmon or wireshark to capture and review the traces. Personally I prefer to use Netmon. However, in this case we used wireshark since traces sent by the customer could not be opened using the Netmon tool (due to a different file extension). You can download the Netmon tool or wireshark from the web.

IP addresses and exact time stamp of the issue:

Let’s check the IPconfig’s first to find out the IP addresses.

SQL server:

IP Address. . . . . . . . . . . . : 10.10.100.131

App server:

IP Address. . . . . . . . . . . . : 10.10.100.59

Now find out when exactly the issue happened from the error message.

Error message:

02/24/2010 09:28:08 DataBase Warning OleDB Error Microsoft OLE DB Provider for SQL Server [0x80004005][11] : [DBNETLIB][ConnectionWrite (send()).]General network error. Check your network documentation.

So the issue happened at 09:28:08 on 02/24/2010

Often time you will get significant number of trace files from each server when collecting network traces for intermittent issues because you may have to capture the trace for a long period of time and this can generate quite a number of trace files when configured to save the trace files as chain (for this particular case the customer sent us more than 60 trace files each of size 25MB for the app sever and 6 such files for the SQL Server J). The exact time will help you to find out which trace file you need to start with. So getting the exact time of the issue is very important when you have too much data to review.

Analyzing Application Server Traces:

Let’s start with the trace file from the application server. From the time stamp we narrowed down what trace file to review first. For intermittent connectivity issues the first thing we are interested to check in a network trace is if there are any RESETs. From SQL server error log we found out that the SQL server is listening to port 1433. So we started with the following filter.

“tcp.port eq 1433 && tcp.flags.reset==1”

This filter should list all the RESETs to the SQL server (we are assuming that the application server is connecting only to the problem SQL server and no other SQL server that is also listening to port 1433). With this filter we found about 20 RESETs.

Now let’s check the full conversation for any of the RESETs . To get the conversation of a RESET frame do the following.

Select the frame with RESET--> right click --> Conversation Filter --> TCP

It will show you all the frames in that conversation. We found only two frames.

Now if you have collected traces in chain and have more trace files collected before and after this conversation then you should check those trace files to see if there are other frames from this conversation. We need to find out what was happening before the RESET was sent and that may tell us what caused the RESET. To do this, copy the filter for this conversation from the current trace file.

(ip.addr eq 10.10.100.59 and ip.addr eq 10.10.100.131) and (tcp.port eq 1194 and tcp.port eq 1433)

We opened the trace file collected in the application server just before the current trace file and used the above filter to check the frames that are part of the same conversation.

If you see no issues and normal traffic (like keep-alive packets) then open the trace file previous to that and use the same filter. Keep doing this until you see some sort of issue or reach at the beginning of the conversation (three way handshake to establish connection).

In our case we saw a lot of keep-alive traffic. SQL Server (10.10.100.131) and application server (10.10.100.59) were sending [TCP Keep-Alive] and [TCP Keep-Alive ACK] packets back and forth. But at the end we saw application server (10.10.100.59) sent five [TCP Keep-Alive] packets to the SQL server but did not get any response from the SQL server as shown below.

We reviewed the trace file before the current trace and it showed SQL Server (10.10.100.131) and application server (10.10.100.59) were sending [TCP Keep-Alive] and [TCP Keep-Alive ACK] packets back and forth. Then we checked the trace file after the first trace file we reviewed (where we found the RESET) there were no frames for this conversation. So this conversation ended after this RESET.

Analyzing SQL Server Traces:

Then we checked the trace file that we collected from the SQL server. Once again using the error time stamp we selected the trace file that we need to start with.

We started with the same filter to look for frames from the same conversation we are investigating in the application server trace.

(ip.addr eq 10.10.100.59 and ip.addr eq 10.10.100.131) and (tcp.port eq 1194 and tcp.port eq 1433)

It showed a lot of keep-alive traffic and the same pattern as we saw in the application server trace. SQL Server (10.10.100.131) and application server (10.10.100.59) were sending [TCP Keep-Alive] and [TCP Keep-Alive ACK] packets back and forth.

Then we checked the next file using the same filter for the same conversation. Initially it showed the same pattern as in the previous file; SQL Server (10.10.100.131) and application server (10.10.100.59) were sending [TCP Keep-Alive] and [TCP Keep-Alive ACK] packets back and forth. But at the end we saw application server (10.10.100.59) sent a RESET to the SQL server (10.10.100.131).

The time stamp for the RESET frame is different but it is the same frame as we found in the application server trace, because we are tracking the same conversation. So the keep-alive packets sent from the application server were getting dropped and did not reach to the SQL server. The application server sent 5 keep-alive packages (which is the default setting) and then sent RESET as it did not receive an acknowledgement from the SQL server.

If an open connection remains idle for some time TCP attempts to verify if the connection is still intact by sending a keep alive packet to its peer. If the remote server is still reachable and functioning, an acknowledge packet is sent back. Otherwise, the local TCP will keep sending “Keep Alive” packet at an interval of KeepAliveInterval for TcpMaxDataRetransmissions times. If no acknowledge packet is received during this period, the local TCP will reset the connection. For every SQL Server TCP connections, Keep Alive is 30,000 millisecond by default and KeepAliveInterval is hard-coded 1,000 millisecond. TcpMaxDataRetransmissions is 5 by default. It is configurable for entire machine through HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters. As an example, under default configuration,

The following blog post has more information on this.

http://blogs.msdn.com/b/sql_protocols/archive/2006/03/09/546852.aspx

So in our case application server is RESETing the connection as it did not get a response for 5 keep-alive packets it sent to the SQL server. So the next step would be to find out where exactly in the network these [TCP Keep-Alive] packets are getting dropped. To do this you need to collect precise network architecture details, including any intermediate hubs, firewalls, routers, switches between client and SQL Server. You need to have a clear picture of all the networking hardware between the client and the SQL Server, for example:
Web Server: Network Card(s)->Local Hub->Cisco Firewall->Network Card(s)->SQL Server

Then we need to capture simultaneous network traces from all intermediate server(s)/hardware(s) along with the client and SQL server. By reviewing these traces we can further narrow down where the packets are getting dropped.

For this particular case both the SQL Server and Application server were hosted on the same physical machine using 3^rd party visualization software. So ideally there shouldn’t be anything in between the SQL server and the application server. So our conclusion was, somehow the virtualization software was causing the issue and we suggested that the customer either to reproduce the issue outside virtual environment or involve virtualization software provider to find why the packets are getting dropped.

Customer decided to involve the virtualization software provider. However, as the customer was waiting to get a resolution from the virtualization software provider we were able to implement a work around by increasing the “TcpMaxDataRetransmissions” setting to a higher value.

Notes

Bandwidth Throttling / Policing on Cisco ASA

August 17th, 2009 David Leave a comment Go to comments

If If you are looking to control the amount of bandwidth for a particular host using a Cisco ASA Security Appliance, you’ve come to the right place.  When I was first asked to look into this capability on the ASA I knew that I could perform some sort of Quality of Service (QOS).  In fact, all of the documentation that I came across either on Cisco’s website or from third party integrators have detailed information on controlling quality for VoIP, traffic shaping, and how to do those things across a VPN tunnel.  While the information on these great features of the ASA is helpful, finding articles on limiting bandwidth to a particular IP address was more difficult to track down.  In fact, it took a TAC case and several hours of reading papers on the above services until I was able to figure out how to police bandwidth using my ASA.  In the example below I am throttling bandwidth to 1Mb for the host 1.1.1.1:

For the sake of simplicity, I will show you how to limit inbound and outbound bandwidth for one host.  In order to do this for multiple hosts you simply replicate the steps making a few changes to access-list names, class-maps, and policy-maps.

The first step is to create the access list that define “interesting traffic” or what IP you want to control.

access-list throttle_me extended permit ip host 1.1.1.1 any
access-list throttle_me extended permit ip any host 1.1.1.1

The second step is to define the class-map.

class-map throttle-me
match access-list throttle_me

Now you need to define your policy-map and call the class-map.

policy-map throttle-policy
class throttle-me
police output 1000000 2000
police input 1000000 2000

The final step is to apply the new service-policy to the PHYSICAL interface where the traffic will flow.  You CANNOT apply this to a sub-interface.

service-policy throttle-policy interface outside

In summary, this configuration was applied to the outside interface of my ASA.  This is the “choke point” for traffic and can be considered the edge of my network.  As stated above, you must apply the policy to a physical interface on your ASA.  The IP address 1.1.1.1 represents a public address that is statically mapped to a private address behind a sub-interface on my ASA.  The method above combines a little bit of each QOS function from the ASA to get what I want it to do.

Using this on a WSUS server, works great.

Note: The NETSTAT command will show you whatever ports are open or in use, but it is NOT a port scanning tool! If you want to have your computer scanned for open ports see this page instead (link will follow shortly).

Open Command Prompt and type:

C:'WINDOWS>netstat -an |find /i "listening"
TCP   0.0.0.0:135   0.0.0.0:0   LISTENING
TCP   0.0.0.0:445   0.0.0.0:0   LISTENING
TCP   0.0.0.0:1025   0.0.0.0:0   LISTENING
TCP   0.0.0.0:1084   0.0.0.0:0   LISTENING
TCP   0.0.0.0:2094   0.0.0.0:0   LISTENING
TCP   0.0.0.0:3389   0.0.0.0:0   LISTENING
TCP   0.0.0.0:5000   0.0.0.0:0   LISTENING

You can redirect it to a text file by adding >c:'openports.txt to the command, if you want to:

netstat -an |find /i "listening" > c:'openports.txt

netstat -an |find /i "listening" > c:'openports.txt

You can also change "listening" to "established" to see what ports your computer actually communicates with:

C:'WINDOWS>netstat -an |find /i "established"
    TCP   192.168.0.100:1084   192.168.0.200:1026   ESTABLISHED
    TCP   192.168.0.100:2094   192.168.0.200:1166   ESTABLISHED
    TCP   192.168.0.100:2305   209.211.250.3:80   ESTABLISHED
    TCP   192.168.0.100:2316   212.179.112.230:80   ESTABLISHED
    TCP   192.168.0.100:2340   209.211.250.3:110   ESTABLISHED

Note: In Windows XP and Windows Server 2003, you can type NETSTAT -O to get a list of all the owning process ID associated with each connection:

C:'WINDOWS>netstat -ao |find /i "listening"
TCP   pro1:epmap   pro1.dpetri.net:0   LISTENING   860
TCP   pro1:microsoft-ds   pro1.dpetri.net:0   LISTENING   4
TCP   pro1:1025   pro1.dpetri.net:0   LISTENING   908
TCP   pro1:1084   pro1.dpetri.net:0   LISTENING   596
TCP   pro1:2094   pro1.dpetri.net:0   LISTENING   596
TCP   pro1:3389   pro1.dpetri.net:0   LISTENING   908
TCP   pro1:5000   pro1.dpetri.net:0   LISTENING   1068

You can use PULIST from the W2K Resource Kit (Download Free Windows 2000 Resource Kit Tools) to find the PID and see what process uses it and who started it. For example, you found out that your computer had an open connection to a remote IP address on TCP port 80, and you don't have any Internet Explorer or other browser windows open. You want to find out what process is using that session.

C:'WINDOWS>netstat -no

Active Connections

Proto Local Address Foreign Address State PID
TCP   192.168.0.100:2496   212.179.4.7:80   ESTABLISHED   1536

You can then use PULIST with the FIND command:

C:'WINDOWS>pulist |find /i "1536"

Process   PID   User
LUCOMS~1.EXE   1536   DPETRI'danielp

In this case, LUCOMS~1.EXE is run by DANIELP (myself) and as it happens, it's the Symantec Live Update process.

You can also look in Task Manager for the respective PID.

1. To set up Task Manager to show the PID column open Task Manager by using CTRL+SHIFT+ESC.
2. Go to the Processes tab, click View and then Select Columns.

1. In the Select Columns windows click to select PID and then click Ok.

1. You can sort the PID column to display the PIDs in descending or ascending order.

Notes

Document Title
Getting Support for Form Types in a Remote Output Queue (RMTOUTQ)

Document Description
When printing operating system spooled files to a LAN attached printer using a Remote Output Queue (RMTOUTQ), you will not normally receive message CPA3394 - Load form type '&4' device &5 writer &1 when the form type changes.

Getting Support for Form Types on One or More Remote Output Queues

Remote writers can support form types; however, you must start the writer with the message option set to *INQMSG (so you get an inquiry message whenever the form type changes) or change the writer after it has started. To start the writer with form type support, use the following operating system command:

STRRMTWTR OUTQ(remote-outq-name) FORMTYPE(*ALL *INQMSG)

To change an existing writer to have form type support, use the following operating system command:

CHGWTR WTR(remote-outq-name) FORMTYPE(*ALL *INQMSG)

After you have done either of these, you will receive message CPA3394 - Load form type '&4' device &5 writer &1 in the writer's message queue whenever the form type changes. However, if the writer ends and restarts automatically or manually without specifying FORMTYPE(*ALL *INQMSG), you will no longer receive message CPA3394.

Getting Support for Form Types on All Remote Output Queues

To get form type support on all of your remote output queues each time they are started, you can change the Start Remote Writer (STRRMTWTR) command to specify FORMTYPE(*ALL *INQMSG), rather than FORMTYPE(*ALL *NOMSG). That way, you will get form type support unless the STRRMTWTR command is specifically overridden to not support form types. This can be done by copying the command to another library that is higher in the library list than QSYS and then bby using the following operating system command:

CHGCMDDFT CMD(cmd-library/STRRMTWTR) NEWDFT('FORMTYPE(*ALL *INQMSG)')

Note: You can use the Change Command Default (CHGCMDDFT) command on the STRRMTWTR command that is in library QSYS, but there are two potential dangers with this, namely:

1.	If you install a newer version of the operating system or reinstall the current version of the operating system, your changes to the STRRMTWTR command will be replaced by the settings as shipped by IBM.
2.	If you make an error when changing the command defaults for a command, you can cause that command to fail every time it is run. Therefore, in this case, you might not be able to get remote writers started again without changing the command defaults back or restoring the STRRMTWTR command from your installation media.

Note: Another option is to use the LPR Print Driver (TSPLPRD) utility in a *LAN 3812 device description. This is an unsupported utility that provides the function of sending spooled files from an output queue to an ASCII printer that is attached to an LPD server. This is a similar function to what is provided using Remote Output Queues (RMTOUTQs), but it differs because this driver exit program is specified on a printer device description. Thus, it can make use of information specified about the printer in the device description that is not available on a remote output queue. Also, because it uses LPR within a printer writer, it supports page ranges, multiple copies and form types, yet it can be used with any printer that currently uses a RMTOUTQ including dot matrix and line printers.

For additional information on using the LPR Print Driver (TSPLPRD) utility, refer to the following Rochester Support Center knowledgebase document:

18762910, Configuring a *LAN 3812 Device Description that Uses the LPR Print Driver (TSPLPRD) Exit Program:

For additional information on the capabilities and limitations of RMTOUTQs and *LAN 3812 printer device descriptions, refer to the following Rochester Support Center knowledgebase documents:

14143423, Capabilities and Limitations of Remote Output Queues (RMTOUTQs):
412453666, Capabilities and Limitations of *LAN 3812 Printer Device Descriptions:

Notes

forward a port on the ASA 5505 running version 8.3 from the CLI.  Some of you prefer to use the ASDM to do you changes, so I guess I'll show you how to do it from there.  The ASDM is a bit of a learning curve for someone that's used to the CLI, and most CLI guys hate a GUI with a great passion.  I can go either way.  I use the ASDM to make some changes simply because I want to learn it and there's some guys coming into the field today that were taught on the GUI rather than a command line.

In this lesson I'm using ASDM version 6.3(1) and ASA version 8.3(1).  Since we added a  web server in the last post, let's make this one an FTP server.  The FTP server's IP is the same as the web server, 10.9.8.7/24 and we're running over the standard FTP port, 21.

First off, we want to start up the ASDM and connect to the ASA.  Once there, click on the button at the top of the screen, then the button near the bottom left, and finally select near the top left.  You'll now be at a screen that looks something like this:

Click for larger version

Now we need to create a new object, so click on "Add" under Addresses, then "Network Object".
Now we need to fill out our new window.  Once you fill out the name, IP address and description, you need to drop down the NAT box and fill it out.  Click the "Add Automatic Address Translation Rules" box, leave the type as "static" and set the translated address as the outside interface.
We now need to go to the Advanced menu from the Add Network Object window and setup the port forwarding.  The source will be inside, destination is outside.  Protocol in this instance is TCP and our port is 21, both real and mapped.
Click "OK" twice and your object will be created as well as the port forward.  Now we just need to add the access rule.  On the left side of the screen, just above the NAT Rules is your Access Rules. From there we want to click "Add" and "Access Rule".
We need to create the rule on the outside interface, coming from any IP to the FTPServer using FTP as the service.
Once you click OK, your rule is added.  You don't have to add a description like I did in the image above this one, I just did that for the hell of it.  When you click "Apply" at the bottom of the screen, the ASDM will issue the commands to the ASA.  I have preview turned on, so I can always see what commands are being sent to the device before they are actually sent.  If you followed all the steps above and you have preview turned on, you'll see the following:
And you'll notice that those are the exact 4 commands that I gave in the last post about doing it from the CLI!  Now you can forward any port you want from either the CLI or the ASDM!

On a side note, I know a lot of guys hate the ASDM.  When I was writing this post and going through all of this I was kinda upset when I saw that I had 10 pictures for 4 lines of code.  The good thing about the ASDM is that you have everything right there at your disposal and you really don't need to know the vernacular of IOS.  The drawback is that it will take you longer to get things done at first, but once you get used to it, it can be just as fast.

Notes

Many people (read 'geeks') have heterogeneous environments with a Mac and/or Windows and/or Linux box running in their homes.

I have a similar one and I wanted to backup my mac's /Users directory (I used to do it on my iPod, but my music got bigger). So I thought, well, I have about 100gigs empty on the windows box, why not use it!

So I tried the simplest thing. I shared a folder on windows (gave full permissions to it for the account I use, no one else.) Then mounted the folder in OS X, and started copying the /Users folder on it. It worked happily for a while, but then failed while it was trying to copy a file with a name like "abc|def.txt" -- problems with file name characters. I had to look for other options...

What I needed was to format a portion of that NTFS formatted hard drive as HFS and that would do it. So I reached for "Disk Utility" and followed these steps:
- Mount the windows shared folder
Tip: If possible try to connect over a wired Lan. I tried to create a big image (about 20gig) over the 802.11g network and it timeout twice. I connected the two via ethernet cable and it worked fine and fast. (obviously)
- Start "Disk Utility"
- Hit "New Image"
- Save As:
(name of the image you are creating i.e. it will be .dmg)
- Size: GB
(Tip: I read somewhere, that if you want to pre-allocate disk space for backup of files, a rule of thumb is allocate 3 times the space you currently use. It sounded reasonable and I did so.)
-Encryption: none
-Format: read/write disk image
-Hit "Create"

This will effectively:
-Create a volume called /Volumes/ that is visible as "Mac OS Extended" of the size that you created.
-You will see a file .dmg in the windows share.

Now I had the bytes ready, in the format I wanted, I just needed a tool transfer stuff. I tried a few but am currently using iBackup:
- iBackup - pretty good: you can select specific files/folder to backup
- rsync: I have previously used rysnc in shell. Works great, very powerful with the switches e.g.: rsync -av --delete --exclude "temp" --exclude ".Trash"
- RsyncX - basic UI for rsync; but can't exclude sub folders
- Other software I've heard raves about are SuperDuper, CarbonCopyCloner (CCC), etc.

The whole thing is pretty cool, so I thought I'd share it. Hope it helps ...

Works great - Notes

La version française est disponible ici: Migration de Xen version 3 vers la version 4 sous GNU/Debian Lenny

Introduction

Xen is an opensource hypervisor. Version 4 has been released on the 7th of april 2010.

One of the main new feature concerns high availability. With the intergation of project Remus code, Xen now supports fault tolerance for virtual machines, using live transactional synchronization of VM state between physical servers.

Unfortunatly, Debian is only shipped with Xen version 3.2 for current stable Lenny, and version 3.4 for next stable Squeeze. Let's see how to install Xen 4 on Lenny.

Current system inventory

The target Xen platform we want to upgrade is discribed here (sorry for my non french readers, theses 2 documents are not translated yet):

• Virtualisation avec Xen sous GNU/Debian Linux
• Cluster Xen sous Debian GNU/Linux avec DRBD, Corosync et Pacemaker

Despite the use of cluster stack in the second case, installation will follow the same steps for both architectures.

But we have to take care about some things in order to complete migration:

• We are about to upgrade Xen hypervisor. We will also have to upgrade Dom0 kernel (during my tests, Debian 2.6.26 kernel crashed while booting with Xen4 and I don't have time to investigate it for now).
• Because we update Dom0 kernel, we must build a new DRBD module accordingly
• Finally, because we'll build Xen from sources, we have to clean Debian install in order to avoid libraries conflicts.

All compilations will be done on a specific computer. For obvious security reasons, it's of course out of question to compile Xen on the Dom0. Remember ? "Never install dev tools on productions servers" ;-)

Environment setup

Inside our dedicated VM, tou must install needed packages for compilation:

Dev environment setup Needed packages for Xen compilation

aptitude install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo
             texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial
             build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev
             libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev gcc-multilib

Xen compilation

Xen compilation steps Getting Xen sources

mkdir ~/Xen
cd ~/Xen
wget http://bits.xensource.com/oss-xen/release/4.0.0/xen-4.0.0.tar.gz
tar -xzf xen-4.0.0.tar.gz
cd xen-4.0.0/

Compile Xen

make xen
make tools

As we'll use Debian provided Dom0 kernel, we don't need to build it. The make stubdom command is therefore not required.

Binaries have copied into dist directory. You'll find a script called install.sh which will take care of copying files in destination system tree. System files are located dist/install. In order to make deployment easier, we'll build a tar.gz archive from direcory dist.

But first, we have to modify some files to respect Debian's usages for configuration files stroage: we have to move etc/sysconfig to etc/default. Then, we have to adapt init scripts.

Adaptation de l'installation aux normes Debian

cd dist/install/etc
mv sysconfig default
cd init.d

Replace sysconfig with default (line 23) in xend.

Replace sysconfig with default (line 38) in xendomains.

Tar.gz archive creation for Xen 4

tar -czf xen4.tar.gz dist

Then, you can copy this archive on each dom0 you want to update. Let's deal now with DRBD module.

DRBD compilation

Our next dom0 kernel will be 2.6.32 from backports repos. We have to build it. As our dev VM does not use it, we must specify kernel version module-assistant will have to use to build against.

I only write a summary of needed commands, details are already here (still only in french, sorry): Installation de DRBD > 8.3.2 sur Debian GNU/Linux Lenny

DRBD packages build

apt-get install linux-headers-2.6.32-bpo.4-xen-amd64 module-assistant
dpkg -i drbd8-module-source_8.3.7-0_all.deb
m-a build -l 2.6.32-bpo.4-xen-amd64 drbd8-module

Dom0 configuration

Before beginning, ensure all your VMs have been migrated (or stopped, depending on you configuration).

First install new kernel from backports:

Kernel 2.3.32 install

echo "deb http://www.backports.org/debian lenny-backports main" >> /etc/apt/sources.list
aptitude update
aptitude install debian-backports-keyring
aptitude update
aptitude install linux-image-2.6.32-bpo.4-amd64 linux-image-2.6.32-bpo.4-xen-amd64

Now reboot the server. If everything is fine, you can uninstall all xen related packages as well as the old kernel. As usual, do not forget to backup your Xen configuration !

Once done, you can install DRBD module and Xen4 archive:

DRBD install

dpkg -i drbd8*.deb

Xen4 install Archive extraction

tar -xzf xen4.tar.gz
cd dist
./install.sh

Xen FS setup

echo "none /proc/xen xenfs defaults 0 0" >> /etc/fstab

Without the last command, xend process will be unable to communicate with hypervisor and therefore, nothing will work. You may have to add some others modules in /etc/modules:

Adding other Xen modules

blktab
xen-evtchn

Now it's time to restore Xen's config files. Finally, setup xen related services start xend and xendomains.

Xen4 services setup

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 20 21

When done and before rebooting on hypervisor, you should install some libraries. They are needed for Xen to work, but have been uninstalled by Debian (because you remove packages).

Xen4 needed librairies install

aptitude install libpci3 libsdl1.2debian libx11-6 libxext6 libgl1-mesa-glx

Now you can reboot and start Xen4 hypervisor. Let's check Xen status with command xm list.

Et voilà, now you have a completly new Xen4 dom0 running Debian Lenny. Have fun !

Sources and references

Xen

• http://wiki.xensource.com/xenwiki/Xen4.0

DRBD

• http://www.drbd.org/

GNU/Debian Linux

• http://www.debian.org/

Good doc - Notes

How To create a DID for PIKA FXO lines

It is easy to use DIDs with SIP and BRI lines as they send there own number in the call messaging. When using FXO lines though, this has to be done based on hardware indexes. This can be done in FreePBX with zaptel trunks quite easily but as the chan_pika trunks are only supported as custom trunks, this functionality is not available through the GUI.

This thread is to show you how to do it by editing the asterisk config files by hand. For this example I will assign a DID of 1234567 for FXO line 1 and a DID of 5551234 for FXO line 2.

To get our PIKA FXO trunks to behave like zaptel ones, add the following to the /etc/asterisk/extensions_custom.conf file. Do not add it to /etc/asterisk/extensions_additional.conf file because this file is overwritten every time changes are applied in the FreePBX GUI. The /etc/asterisk/extensions_custom.conf file is not over written.

NOTE: The "**" below should be a close curly brace. For some reason the forum replaces them.

Code:

[macro-from-pika-fxo-1]
include => macro-from-pika-fxo-1-custom
exten => s,1,Noop(Entering macro-from-pika-fxo-1 with DID = ${DID** and setting to: 1234567)
exten => s,n,Set(__FROM_DID=1234567)
exten => s,n,Goto(from-trunk,1234567,1)

; end of [macro-from-pika-fxo-1]

[macro-from-pika-fxo-2]
include => macro-from-pika-fxo-2-custom
exten => s,1,Noop(Entering macro-from-pika-fxo-2 with DID = ${DID** and setting to: 5551234)
exten => s,n,Set(__FROM_DID=5551234)
exten => s,n,Goto(from-trunk,5551234,1)

; end of [macro-from-pika-fxo-2]


[from-pika-fxo]
include => from-pika-fxo-custom
exten => _X.,1,Set(DID=${EXTEN**)
exten => _X.,n,Goto(s,1)
exten => s,1,Noop(Entering from-pika-fxo with DID == ${DID**)
exten => s,n,Ringing()
exten => s,n,Set(DID=${IF($["${DID**"= ""]?s:${DID**)**)
exten => s,n,Noop(DID is now ${DID**)
exten => s,n,Set(CHAN=${CHANNEL:4**)
exten => s,n,Set(CHAN=${CUT(CHAN,/,3)**)
exten => s,n,Macro(from-pika-fxo-${CHAN**,${DID**,1)
exten => s,n,Noop(Returned from Macro from-pika-fxo-${CHAN**)
exten => s,n,Goto(from-pstn,${DID**,1)
exten => fax,1,Goto(ext-fax,in_fax,1)

; end of [from-pika-fxo]

Now you need to go into the /etc/asterisk/pika.conf file and change the context to from-pika-fxo instead of from-pstn as highlighted below.

Code:

[fxo]
gp_group=0
conf_ref=TRUNK_23800186
context=from-pika-fxo  
extension=s

Now you need to create your incoming routes for the two DIDs you just created using the FreePBX GUI and apply configuration changes. You can now route based on the FXO line the call comes in.

If you want to create additional DID numbers for your other PIKA FXO lines, just add additional macro-from-pika-fxo-# as shown above where # is the line number. Any line that does not have a macro-from-pika-fxo-# associated with it will route through the from-pstn context the way it did before.

Hope this helps,
Skar

Works great - Notes