Find/Replace queries

[TimK]

I am reformatting some data a took from a data base, and I don't understand some things that are happening when I use search and replace.

1) I want to replace return marks with tab marks. I do this with Find and Replace. However, while it replaces most of the return marks, it doesn't replace all of them. These are definitely return marks. I see them when I turn on invisibles, and Nisus highlights them all when I run Find only. However, when I ask for replace, it does not replace all of them. Instead, it seems to put a space there (although the space dot is not there until I do a backspace and then type space).
Which return marks it misses depends on what font the edited copy ends up as. If the font and size changes, the selection of missed replacements changes.
If I try to do the same thing with Word: ie changing all ^p to ^t, it replaces all it with no problem at all.

This is a real pain. I bought Nisus to get away from Word, and now I find that I have to use Word again. I hope there is a solution.

2) This is probably something that is in the manual (at least I hope so) but I have spent so much time trying to sort out the above that I don't really have time to hunt, and I hope someone can point me in the right direction:
How can I control the font/size of the edited text (after find/replace). The original text has mixed fonts and sizes, and I want to keep to these. This is no problem in Word, but in Nisus Express, all the text seems to go the same character formatting as the first line in the block. I don't want that at all. How can I keep to the original formatting?

3) One more thing: in find/replace, is it possible to input codes like ^p and ^t in the search fields? I find having to hunt in the menu to be a pain.

Thank you

TK.

[TimK]

Sorry to bump this, but I 'd appreciate it if someone could try to replicate it, and tell me if it is a bug. I have spent a while trying to fix it, and I would fix it if I could.
I really don't want to have to return to word, but I will I have to.

[Steve]

Hi Tim,

<<This is all speculation but it might jar Nisus to investigate>>

I suspect when you import the text it is coming into Nisus with the "return" character actually being a carriage return followed by a linefeed. The Linefeed character shows up with the "paragraph" symbol when you show invisibles, but the carriage return remains invisible. I suspect internallyl, as far as the laying out of the text in the document, the carriage-return/linefeed combination is treated as a single character meaning "advance to the next line". The problem is in the find and replace, in that during that Nisus appears to be finding and replacing the linefeed character alone, leaving the carriage return character as is. After the find and replace, though, when Nisus sees an individual carriage return, it internally treats it in the same way, telling itself to "advance to the next line".

This doesn't fully explain your situation as I don't know why the results of "find and replace" would differ depending on font selection, since the choice of font wouldn't change the underlying codes that represent your actual text.

Anyway, as I say, it's just a thought. You can check this behavior by opening a document, opening the character palette from your input menu (the one that appears as a flag in your menubar after selecting several keyboard layouts to show in your Internation preference's Input Menu tab). Go to the character palette, select the Unicode Table tab, scroll to the very top and drag the character for linefeed (000A) followed by a carriage return (000D). Do a find and replace and see if it seems similar to what you're seeing.

I believe this behavior should be considered a bug.

as far as the third problem, which might solve the first one if what I suspect is true. to get rid of the combination Carriage-return/Linefeed go to the find and replace dialog (using Powerfind) and into the find field type the four characters \n\r

\n will find the carriage returns and \r will find the linefeeds.

The combination of \n\r should then get rid of any situations where you have CrLf.

To find the tab character, type in the two characters \t

As for the second question, I'm not sure as I haven't looked at it yet (I'm new to Nisus as well) but I'll see what I can find out.

Hope all this helps.

Steve.

[TimK]

After some thought, I think I understand what you are saying, but as you say, it doesn't really explain everything.

I tried the test you mention. When i drag the (invisible) 000A mark to the document, I see what seems to be the end of paragraph mark, and the cursor moves to the next line. Then when I drag the (invisible) 000D mark to the document, no character appears, but the cursor moves up to the beginning of the document (before the paragraph mark)

Then, when I Find 'Return' and Replace with 'Tab', (clicking Replace all), the paragraph character disappears, and the tab mark appears on the second line. I'm not sure if this is what you would expect to happen, but as I look at the document, I would expect the tab mark to simply replace the paragraph mark on the first line.

If I then do an Undo, first the tab mark disappears, and the cursor returns to the beginning of the second line. (not the first, where it was when the find/replace command was made). Then when I click Undo (actually now 'Undo Paste') the cursor just moves slightly upwards. I'm not sure what it is doing: it just seems to click into place on the second line). Then when I click 'Undo Paste' again, the paragraph mark disappears and i am left with an empty document.

Incidentally, all this is with 'Powerfind'. I hadn't actually noticed this in my original document, which is not here with me right now. However, if I try Normal find with the test you describe, the 'Return' in the Find field changes to /n and when I click replace all, all I get is a beep.

Well, there you go. Thanks for your help. I hope we have progressed.

[Steve]

After some thought, I think I understand what you are saying, but as you say, it doesn't really explain everything.

I tried the test you mention. When i drag the (invisible) 000A mark to the document, I see what seems to be the end of paragraph mark, and the cursor moves to the next line. Then when I drag the (invisible) 000D mark to the document, no character appears, but the cursor moves up to the beginning of the document (before the paragraph mark)

Then, when I Find 'Return' and Replace with 'Tab', (clicking Replace all), the paragraph character disappears, and the tab mark appears on the second line. I'm not sure if this is what you would expect to happen, but as I look at the document, I would expect the tab mark to simply replace the paragraph mark on the first line.

That's what I see happen, also. Think of it this way, there are (apparently) 3 ways to "advance to the next line": 1) a single linefeed (what is inserted when we hit the <return> key), 2) a carriage return, and 3) a carriage return - linefeed combination. When you are importing I think you are getting the carriage return/linefeed combination. When you do the find and replace, you are stripping only the linefeed character away, leaving the carriage return behind. It is that carriage return that is left behind that is causing your problems. Say you start out with "Hello<Cr><Lf>Steve". This text would have "Hello" on the first line, and "Steve" on the second line (because Nisus sees the <Cr><Lf> essentially as a single character). AFter the find and replace, where we replace the linefeed with tab, we would have "Hello"<Cr><Tab>"Steve". But since Nisus sees the <Cr> character as a "advance to the next line" indicator, the <Tab>"Steve" appears on the second line. As to why the caret jumps up to the first line when we insert the 000D character, I haven't a clue

If I then do an Undo, first the tab mark disappears, and the cursor returns to the beginning of the second line. (not the first, where it was when the find/replace command was made). Then when I click Undo (actually now 'Undo Paste') the cursor just moves slightly upwards. I'm not sure what it is doing: it just seems to click into place on the second line). Then when I click 'Undo Paste' again, the paragraph mark disappears and i am left with an empty document.


Incidentally, all this is with 'Powerfind'. I hadn't actually noticed this in my original document, which is not here with me right now. However, if I try Normal find with the test you describe, the 'Return' in the Find field changes to /n and when I click replace all, all I get is a beep.

Let's stick with this and I think we'll figure all this wierdism out. In the meantime, just to test our hypothesis, create a script file with the following text and save it in your Macros folder. Use one of your documents that you're having trouble with. Select all the text and then run the script. It should replace all <Cr><Lf> combinations with <Carriage Return><Cr><Linefeed><Lf> (where <Cr> and <Lf> are the actual carriage return and linefeed characters, respectively). The script follows:

#Nisus Macro Block
#source clipboard
#destination clipboard
#Before Execution
#Clipboard 1
#Copy
#After Execution
#Paste
#Clipboard 0
#End Nisus Macro Block
while (my $line = <>)
{
$line =~ s/\n/<Carriage Return>\n/ig;
$line =~ s/\r/<Linefeed>\r/ig;
print $line;
}


If what I think is happening, each one of the trouble spots will now be identified by:
<Carriage Return>
<Linefeed>

We're making progress. By the way, I meant to say drag the 000D character first (the carriage return) and then the 000A character (the linefeed). For some reason I did it backwards Ooops.

Later,
STeve.

[Steve]

OK. I think I've got things a little messed . When we type a <return> it puts in a linefeed character (or newline character, or 000A in unicode). This is the character that Nisus displays with the "paragraph" symbol. The macro should be changed:

The macro should be:

#Nisus Macro Block
#source clipboard
#destination clipboard
#Before Execution
#Clipboard 1
#Copy
#After Execution
#Paste
#Clipboard 0
#End Nisus Macro Block
#This macro template lets you replace text with other text
#You can use regular expressions to do text pattern matching
while (my $line = <>) { #Reads each line of the selection from the clipboard
$line =~ s/\n/<LineFeed>\n/ig;
$line =~ s/\r/<Carriage Return>\r/ig;
print $line; #writes out each line to the clipboard
}


Sorry about that. Gotta go get the little 'un from school. I'll get back to this tonight.

Steve.

[TimK]

OK, I'll do that in a few days when I have the document again.

Thanks for this work. It seems to be something that you enjoy doing. I, on the other hand, do not enjoy this sort of thing one little bit.

Are you saying that Nisus does appear to have a problem here, and this Macro will help me solve it?
I must say that I would rather Nisus solve the problem. As I look back over the forum I see that there have been lots of problems with Find/replace.

Anyway, I'll forge on, and report back soon.

[Steve]

Hi Tim,

I'm not sure if the macro itself will help you, but it might tell us what is up with your document. I'm hoping you see
<Carriage Return>
<Linefeed>
where you have your problems. If that's the case then all you need to do in your find and replace is to do a find and replace where you find \r\n and replace it with \n. Then you can do a find and replace as you would normally.

Bear in mind that this problem is likely to be seen only when you import text from other operating systems (Windows, Linux, etc.), or other applications that might convert a newline (what we have been calling linefeed) character into a <cr><lf> pair (perhaps like a database that aims for cross-platform compatibility).

Anyway, I hope we're getting close on this. Let me know how things go.

Steve.