sponsor Vim development Vim logo Vim Book Ad

basic Tip #145: Changing DOS style end of line to UNIX, or vise-versa

 tip karma   Rating 823/323, Viewed by 54410 

Read and edit this tip on the Vim tip wiki. The wiki may have a more recent version of this tip.

created:   October 22, 2001 12:06      complexity:   basic
author:   Douglas Potts      as of Vim:   5.7

Those of us doomed to work in both the Unix and Windows world have many times encountered
files that were create/editted on systems other that the one we are on at the time of our edits.  We
can easily correct the dreaded '^M' at the end of our Unix lines, or make files have more than one
line in DOS by:

To change from <CR><LF> (DOS) to just <LF> (Unix):
:set fileformat=unix
:w

Or to change back the other way:
:set fileformat=dos
:w

It also works for Apple land:
:set fileformat=mac
:w


And to tell the difference:
set statusline=%<%f%h%m%r%=%{&ff}\ %l,%c%V\ %P
                                              ^^^^^  This shows what the current file's format is.

Happy Vimming!

 rate this tip  Life Changing Helpful Unfulfilling 

<< recording keystrokes by "q" for repested jobs | opening multiple files from a single command-line >>

Additional Notes

Anonymous, October 22, 2001 13:27
I unfortunately do not find this to be true.  I still have ^M in my files after doing this.
pottsdl@yahoo.com, October 23, 2001 4:50
This may be the case if an extra set of ctrl-m (^M) were at the end of
line.

That being the case:
:%s/^M$//

will take care of that.

and that is ctrl-m not caret-M.
Or if there are more than one ^M at the end:
:%s/^M\+$//

HTH
cec@NgrOyphSon.gPsfAc.nMasa.gov, October 23, 2001 7:16
In order to process a bunch of files I usually
qb
:%s/^M$//
:set ff=unix
:w
:n
q
(then use the number of files)@b

To speed things up I also

:set lz
:set ul=-1

This converts dos->unix.  To go the other way, remove the substitute
and :set ff=dos .  I haven't ever used a mac, but :set ff=mac is
also supported in case you need to do that.
pottsdl@yahoo.com, October 25, 2001 5:20
Yet another way to do it for multiple files, if in Vim 6.0+ is to use the
:bufdo or :windo commands.

See
:h bufdo

and
:h windo

respectively
eferna1@platea.pntic.mec.es, October 25, 2001 10:19
Hmm, Vim automagically detects the format of a text file (DO$, UNIX, MAC) and it tells that at the command line. To change the format issue the command :set fileformat=<format> and save the file. ;-)). Anyway, update to the 6.0 version now!
--
It is illegal for anyone to give lighted cigars to dogs, cats, and other
domesticated animal kept as pets.
                [real standing law in Illinois, United States of America]
drini@br.crashed.net, November 14, 2001 19:39
yet another way to check  the file kind:
:set ff?
anonymous, September 26, 2002 14:59
None of this really works. I still get the ^M... I think I try lemmy which autodetects UNIX files and does the conversions...
area51@braintransplants.com.nospam, October 20, 2002 2:01
None of these tips worked for me either. I still have ^M all over the place in my file.
area51@braintransplants.com.nospam, October 20, 2002 2:02
None of these tips worked for me either. I still have ^M all over the place in my file.
mperry4@mail.lesley.edu, January 2, 2003 8:57
Umm, I try to use the replace option but how do you enter ctrl-M?  What to you type before it so it will write the character and not execute a ctrl-M ?
raxxy@nospam.shaw.ca, January 10, 2003 17:30
The proper substitution (worked for me) is:
:%s/[^M]$//

and the fileformat change;
:set fileformat=unix
Anonymous, March 17, 2003 15:09
I has the same problems with ^M at the end of a line in a Windows install. After some fiddling I found out that:

1. If you have a _vimrc file (even if it's empty) you won't be in vi compatible mode.
2. If you are not in vi compatible mode and you don't have the vimrc set up correctly vim always thinks you are in Unix mode even when open a DOS file on a Windows machine
3. An easy way to see this is to type :help fileformats to see the ^M chars at the ends of the line in help.
4. To fix this you need to add
        :set fileformats=dos,unix
    to your _vimrc. Your platform default file should be first (if you did this on a mac you would say :set fileformats=mac,dos,unix
5. Close vim and reopen it.

Problem should be solved. (Worked for me :-)

Of course now you need to add more commands to your _vimrc :-)
Anonymous, July 14, 2003 18:48
fantastic tip.  simple and useful.
a.lacey@man.ac.uk, September 22, 2003 8:28
One addition that might help some people.  There are two ways to get a ^M symbol.  One is doing

CTRL-V CTRL-M  - generate  ^M as might  be present in a dos file read on Unix

the other is

CTRL-V CTRL-RETURN - generate the ^M representation of a Unix carriage-return

So you might find that you want to replace CTRL-V CTRL-M with CTRL-V CTRL-RETURN

i.e.

:%s/CTRL-V CTRL-M/CTRL-V CTRL-RETURN/
emvy@richmondbc.info, September 24, 2004 9:39
OK, I've been vi-ing for many years, but I'm getting frustrated with this subject:

How do I insert a new line in Windows?
In UNIX I would do a :s/BeforeAfter/Before\^V^JAfter/
In set ff=DOS  mode, I tried :s/BeforeAfter/Before\^Q^M\^Q^JAfter/

...but it seems to xlate the ^J to a ^@ (WTF is a ^@?), so the line
just looks like Before^M^QAfter (which is obviously not what I want)

*sigh* back to Notepad :-(

PS while I'm on the subject, does *anyone* know how to insert a new line using sed commands?
Anonymous, September 24, 2004 9:51
(sorry, a typo - the result looks like: Before^M^@After)
(I'm losing it over this one)
Anonymous, October 4, 2004 9:32
In Windows, just %type :
:s/BeforeAfter/Before^Q^MAfter/

No more Notepad :)
NdrchipO@ScampbellPfamily.AbizM - NOSPAM, October 8, 2004 7:26
s/...\n.../...\r.../

Use \n instead of ^V^M etc on the left, and \r on the right with substitutes (that's backslash-n, backslash-r).
dtruter at gmail.com, December 1, 2004 5:30
The best tip so far, thanks!

(:/s/<ctrl +V><ctrl+M>/<ctrl +V><ctrl+enter>/g)
cheers!
me@nowhere.com, December 10, 2004 9:46
Vim is a bit different from the old Vi
For getting rid of ^M characters you can use
:0,$s/<ctrl v><ctrl m>//g
Vim hides them from you :o) so you need to do as this article says
You can always FTP files in ASCII mode
me@windows.be, February 19, 2005 22:48
Just found out that on windows CTRL-V is already used to paste text. Where CTRL-V is mentioned in one of the tips, a windows user has to press CTRL-Q instead.
programmer@us.site.com, May 20, 2005 13:22
The Vim search and replace is an implementation of REGEX so you can use normal regex special escape characters to represent the control characters. In the case of ^M at the end of lines, you can use the regex escape code \r to represent the raw linefeeds.

:%s/\r$//

That should take care of them.
hooligan0069@yahoo.co.uk, May 25, 2005 6:51
I used :%s/.$// and this got rid of all my ^M in my files.
Mike
scottjamesturn@aol.com, June 12, 2005 8:21
I'm finding that :$s/^M/ /g' removes instances of ^M from the file, but does not actually move the intervening text to a newline by itself.  I have tab-column exel files which spill over into one long line.  How do I remove the ^M AND have the spilling line break up into seperate lines for UNIX, so all the columns re-align?

honk, June 23, 2005 16:02
honk
g_without_the_underscores_barros@catho.com.br, August 2, 2005 11:48
I don't think this Tip is valid anymore.

Testing in gVim 6.3 under Windows  and vim 6.3.78 under debian

%s/.$// -> removes the last characther not the \n nor \r
/\r -> finds nothing, even if there's a \r char
/^M -> finds nothing
/[^M] -> finds everything


ctrl-v ctrl+m is a terminal flow control so it's valid as long as you're on a real terminal  with support to flow control :)

gabriel
Anonymous, August 2, 2005 11:53
just adding what _does_ still work

open dos file
:set ff=unix
:wq

it does the trick here with 6.3

Gabriel
Anonymous, August 17, 2005 8:49
it doesn't work for me using 6.3

oddly enough this worked:

%s/^M/^M/g

Anonymous, August 26, 2005 20:54
Weird, the above poster's suggestion of:

:%s/^M/^M/g

actually got rid of my ^M's. Something strange going on here, common sense says that shouldn't work! :P
RichardBronosky (first@last.com), March 17, 2006 10:31
All I care about is if the file format is NOT unix.  If it's not, I want a big red warning.  That way I'm not the jerk who checks in a file that causes every line to get modified by the diff patch.

So, I added this to my existing statusline:
  %9*%{&ff=='unix'?'':&ff.'\ format'}%*

Here's what is does:
  %9*
  \- Change highlighting to user setting #9 (see :he hl-User1..9)
  %{
  \- Begin evaluating as expression until } is encountered
  &ff=='unix'?'':&ff.'\ format'
  \- This is a ternary that returns either an empty string, or 'XX format'
  }
  \- This marks the end of the expression
  %*
  \- Restores normal highlight

So, how do you use it?  First I call:
  :set statusline?

Which returns:
       statusline=%<%f :: %{TagName()} %(%h%m%r %)%=%-15.15(%l,%c%V%)%P

But, remember that if you want to set a status line you must escape all white space.  So that line would have to be entered as:
       statusline=%<%f\ ::\ %{TagName()}\ %(%h%m%r\ %)%=%-15.15(%l,%c%V%)%P

So when I added my modification I have:
  :set statusline=%<%f\ :%9*%{&ff=='unix'?'':&ff.'\ format'}%*:\ %{TagName()}\ %(%h%m%r\ %)%=%-15.15(%l,%c%V%)%P

But then, to make the user highlighting #9 big and red I started by viewing all the existing highlighting configurations (I'm too lazy to write my own) by calling:
  :hi

The entry titled ErrorMsg looked good to me so I copied its settings which were:
  term=standout cterm=bold ctermfg=7 ctermbg=1

I then called:
  :hi User9 term=standout cterm=bold ctermfg=7 ctermbg=1

Now my status line is unchanged and uncluttered, unless I have opened a dos file.  That's pretty cool.

I wrote this is REAL basic terms, because I really wish someone had explained it to me like this.  I hope it's well received.
Nujoom, April 10, 2006 4:25
Can use dos2unix command also for changing the format of the file from dos to unix. It is needed to give the execute permission to the file if it needed to executed.
joe.schmoe@nunya.biz, May 9, 2006 10:44
I use gvim on windows and this tip worked for me:

Just found out that on windows CTRL-V is already used to paste text. Where CTRL-V is mentioned in one of the tips, a windows user has to press CTRL-Q instead.
kedar.mhaswade@sun.com, August 26, 2006 8:35
Hi me@windows.be,

Thank you very much for telling me that one must use CTRL+Q instead of CTRL+V to capture new lines on Windows. I was scratching my Head!

Go Google.
Go VIM.

Kedar
crephoto at gmail dot com, December 21, 2006 7:16
I've been converting end of line characters (dos/mac/unix) for years using a variety of approaches listed here already including the vi/vim <ctrl-v><ctrl-m> substitution (running vim 6.2 under MacOSX, 6.3.81 under GNU/Linux) , sed, troff, and others.  I recently ran into a very stubborn file (created by Adobe PhotoShop CS "Web Gallery" action) for which the vim substitution failed.

Since I abhor a mystery, I opened up the file in a raw hex editor. Lo and behold, the ornery file had Carriage Returns but no Line Feed characters. For the uninitiated out there, this is the Mac standard end-of-line character.

Having seen the suggetions of using :set fileformat=mac,dos, unix I tried it.  I even tried creating a .vimrc file and adding set fileformats=mac,dos,unix

None of that seemed to change how vim interpreted the Mac end-of-lines.  The only explanation I could find for any of this is that the "setformat" command can be fooled by Macintosh files since a <CR> (\r, ASCII 13, 0x0D) can appear anywhere in a valid unix file. So vim apparently sees the ^M as just another character. So much for the "set fileformats" option.

The only tip listed on this page that worked for me was the following:

:%s/^M/^M/g

I agree that's weird. Again, unwilling to accept "weirdness", I searched some more and discovered the following tip (Thanks Mike) on http://eandem.co.uk/mrw/

Open the vim editor with no active document. Issue the command ":edit ++ff=mac foobar.txt"

This instructs vim to forcibly interpret foobar.txt as a Mac file, converting the Carriage Returns into standard Unix Line Feeds.

For more useful (and useless) Mac OS X/Unix tips, visit the Wax Drum Wiki at waxdrum.net

Charles

If you have questions or remarks about this site, visit the vimonline development pages. Please use this site responsibly.
Questions about Vim should go to the maillist. Help Bram help Uganda.
   
SourceForge.net Logo