A basic walkthrough to using lftp

Author: Fryth (adrian at fryth dot com)

Revision: 5 (8/11/2005)

lftp is an open-source FTP client that is fully featured, supporting FTP, FXP, SFTP, SSL/TLS, FISH, passive/active modes, ipv6 and firewall/proxy support. It runs on Linux and Unix-based operating systems, including the BSD's, Mac OS X and Windows through cygwin. In other words, it runs on all major operating systems. It is fully persistent, meaning it will keep trying to connect to all your servers until all your commands are processed and all the transfers are complete. Although the command line interface can be somewhat difficult to learn at first, it is a robust, reliable, free alternative to other FTP software.

Installing lftp

lftp is included in the packaging systems of most Linux distributions, including the Cygwin setup application in Windows.

In Gentoo, type emerge lftp.

In Debian, type apt-get install lftp.

In Mac OS X, you can obtain lftp through fink. Type fink install lftp.

In Windows, run the cygwin setup.exe program and look for 'lftp' in the net section of the package list.

Configuring lftp

lftp has a default configuration that is adequate for most people. All the settings are accessible from the lftp command line, but they will be lost on exit. To keep the settings each time you start lftp, append your settings to the end of your /etc/lftp.conf file. Here are a couple of settings I use:

set net:reconnect-interval-base 60
set net:reconnect-interval-multiplier 1
set bmk:save-passwords yes
# Active (PORT) mode settings
set ftp:port-ipv4 66.192.67.111 # Replace with your IP address if you are behind a NAT.
set ftp:port-range 2040-2050 # I've forwarded these ports on my router, for use with active mode
# This allows you to see welcome banners, directory messages, etc.
debug 3

# Or you can use an even higher debug level to see all FTP messages. # debug 5

To get a listing of all settings, type set -a, or try the lftp man page.

Opening a connection to a site

lftp is entirely driven by the command line. The command to open a server connection is 'open' and it takes some parameters. TLS/SSL is automatically negotiated if supported by the server. You can type 'help command' at any time to see command help and syntax, so do that now with the open command. Type help open. (Simply typing 'help' will give you a list of all commands. Remember that you can use a lot of bash-style commands as well, such as piping ls output through grep, more/less, lp, etc.)

To connect to a server running on port 27 at localhost, with the username 'foo' and password 'bar', enter this:

open -u foo,bar -p 27 localhost

If the server is running on the default FTP port, 21, you can omit the -p setting entirely. The password can be omitted (the ,bar part of the above line) but you will be prompted for it immediately. Useful if someone is looking over your shoulder or the console is being logged, since screen output is disabled for the password prompt.

Once you press enter, lftp will look up the site address but will not connect to the site until you run a command. Type 'ls' to get a directory listing, which will trigger a connect to the site. If the site is not accepting connections, lftp will reconnect based on the net:reconnect-interval-base and net:reconnect-interval-multiplier settings, which I have set to 60 and 1 respectively in my lftp.conf. This will set the reconnect interval to a constant 60 seconds per attempt, a setting I feel is nice and polite to FTP sites (and their admins).

Bookmarking sites: With the bmk:save-passwords variable set to "yes" as above, bookmarking becomes easy. Once connected, type bookmark add sitename. Now you can connect to the site by typing open sitename or lftp sitename from the shell.

Downloading

You can get a directory listing by typing:

ls

To scroll the screen one at a time, use:

ls | less
OR
ls | more

To search in a directory listing, type

ls | grep searchstring

To search a site recursively, use:

find /path/of/base-directory | grep searchstring

To dump the directory contents into a file:

ls > file.txt

To get a file, type:

get filename

This will download the file to whatever directory you were in before you launched lftp at the shell.

To download or upload an entire directory of files, use the powerful mirror command.

mirror "directoryname"

This creates a directory called directoryname locally, and transfers all the files into it, preserving the directory structure and file permissions. Uploading is as simple as adding the -R switch to the mirror command:

mirror -R "directoryname"

(Tip: If you are in a unix-based system that uses file permissions, you would be wise to use the -p option in mirror when uploading to avoid mirroring the file permissions. FTP sites usually have their own permissions for incoming files.)

You can use the tab key to autocomplete file and directory names. lftp will automatically fetch a directory listing for you when you hit the tab key, if you don't have one in the current directory yet.

Queues

An important feature of any ftp client is the ability to queue multiple commands, and lftp allows you to do this with ease. Don't worry about queuing cd commands to change directories between transfers; lftp will cd to the right directory for you automatically, along the way. So, let's say I'm downloading a file, called file1, and then I want to get all files in the directory "Directory" and put them in a directory on my hard drive with the same name. After that, I want to upload a directory called "Directory2", which resides in the directory I was in when I launched lftp, to the server. I would issue these five commands in a row:

queue stop
queue get "file1"
queue mirror "Directory"
queue mirror -R "Directory2"
queue start

'queue stop' is needed as the first command, to create a stopped queue. If you do not enter this first, the queue will start processing when you add your first entry, which will occupy your control connection. If the control connection is occupied by a task, you won't be able to change directories, get listings, and do the things you generally need to do to queue up more stuff. 'queue start' will start the queue in the background. Nothing will seem to happen, but now you can type jobs -v to see what's going on. You'll see your current queue position, as well as the currect command, current file, and transfer speed. You can move queue items around, delete them, and insert to anywhere in the queue. Type help queue for a full explanation of the queue commands.

Multiple Connections

lftp handles multiple server connections, and you can switch between them by holding alt and pressing the number keys 0 to 9.

Alternatively, if you are in a GUI, you may find it easier to start additional copies of lftp in other shell windows. Either method works.

The Passive/Active FTP Nightmare

lftp uses passive mode by default, which works in 99% of cases. Some sites require an active internet connection (open ports on your end) to work properly (usually a misconfiguration by the admin). If this is true for you, just type:

set ftp:passive-mode on/off

The lftp default is on. Remember if you turn this off, the remote server will try to initiate connections to you, so if you are behind a firewall or NAT, lftp needs to know two extra pieces of information: a port range that's open to you, and your public IP address. Don't forget to forward ports in your firewall or NAT.

set ftp:port-ipv4 your.ip.address
set ftp:port-range minport-maxport (eg. 2000-2050)

Here's another problem you may encounter: A site administrator has enabled PASV mode in their server configuration, and the ports are available, but the IP has not been updated in the FTPd, which is sending out the wrong one in the PASV command.

set ftp:fix-pasv-address yes

This forces lftp to use the address specified when logging into the site with the open command (in other words, the verified working address of the server) and not the one the server sends with the PASV command.