Bash command history problems with long commands

Support and Help
,
1

When I use the up-arrow and down-arrow to scroll through the command history, I have a problem with commands longer than 41 characters (the actual character length seems to vary, but that’s irrelevant to the problem). With up-arrow, the typical behavior is to erase the currently displayed command and display the previous command from the history. However, there is a problem with long commands. In such a case, all but the first 16 characters are erased, and the previous command now starts at position 17 after the prompt. If I press to accept the command, it accepts the command starting at position 17, ignoring the preceding text. It is like the first 16 characters of the long command became part of my prompt. If I continue to press up-arrow, the previous commands are shown, starting at position 17, regardless of how long or short the commands are.

FYI, I’m using:
Ubuntu 24.04.2 LTS
Linux 6.8.0-55-generic x86_64
Xfce DE
Gnome Terminal Version 3.52.0

Here’s an example of entering two commands (my prompt is two lines):

[~/computer/test]
[home5-randy]:ls *.tmp
temp.tmp
[~/computer/test]
[home5-randy]:cat 1234567890.tmp
cat: 1234567890.tmp: No such file or directory
[~/computer/test]
[home5-randy]:

Now if I press up-arrow twice, I get:

(up-arrow)
[~/computer/test]
[home5-randy]:cat 1234567890.tmp
(up-arrow)
[~/computer/test]
[home5-randy]:ls *.tmp

everything is as it should be with the previous commands listed in order.

But now I enter the following two commands:

[~/computer/test]
[home5-randy]:ls *.txt
temp.txt test.txt
[~/computer/test]
[home5-randy]:echo “123456789012345678901234567890”
123456789012345678901234567890
[~/computer/test]
[home5-randy]:

and press up-arrow twice:

(up-arrow)
[~/computer/test]
[home5-randy]:echo “123456789012345678901234567890”
(up-arrow)
[~/computer/test]
[home5-randy]:echo "1234567890ls *.txt

You can see that after the first (long) command in the history is displayed, the first 16 characters of that command essentially become appended to my prompt, and the next (previous) command in the history starts at position 17 following my original prompt. In fact, if I press -A to take me to the beginning of the line, it takes me to position 17, the start of the currently displayed command from the history buffer. If I press enter, the command is executed as if the first 16 characters are in fact part of the prompt and not part of the command:

[~/computer/test]
[home5-randy]:echo "1234567890ls *.txt
temp.txt test.txt

This has been an ongoing problem for years, which I would think would be a simple fix that plagues everyone who works in a terminal, but it has not been addressed. I tried changing my prompt to something simple, and that did change the character position but didn’t solve the problem:

[randy][~/computer/test]–:ls *.txt
temp.txt test.txt
[randy][~/computer/test]–:echo “123456789012345678901234567890”
123456789012345678901234567890
(up-arrow)
[randy][~/computer/test]–:echo “123456789012345678901234567890”
(up-arrow)
[randy][~/computer/test]–:echols *.txt

Interestingly, the starting column is 32 for the actual command following a long command for both cases, so it seems as if the prompt is appended to make it exactly 31 characters long.

Okay, I changed my prompt again to make it exactly 31 characters long, and that “solves the problem” in that no additional characters are appended to my prompt.

[randy]1234567[~/computer/test]–:ls *.txt
temp.txt test.txt
[randy]1234567[~/computer/test]–:echo “123456789012345678901234567890”
123456789012345678901234567890
(up-arrow)
[randy]1234567[~/computer/test]–:echo “123456789012345678901234567890”
(up-arrow)
[randy]1234567[~/computer/test]–:ls *.txt

But this is dependent on the length of the prompt, which is variable with the pwd. If I go to the root directory and issue the same commands, I get:

[randy]1234567[~/computer/test]–:cd /
[randy]1234567[/]–:ls .txt
ls: cannot access '.txt’: No such file or directory
[randy]1234567[/]–:echo “123456789012345678901234567890”
123456789012345678901234567890
(up-arrow)
[randy]1234567[/]–:echo “123456789012345678901234567890”
(up-arrow)
[randy]1234567[/]–:echols *.txt

So I could make my prompt even longer, but really, that’s not what I want. I want my prompt to be what I want to set it to and not be appended. Could I reset my prompt after each command to return to my desired state? That would solve the problem, but seems like overkill for what should be an easier fix.

Also, I thought my commands were stored in ~/.bash_history, but that doesn’t seem to be the case. I have a very long history file, but wasn’t able to find my most recent commands. I cleared the file with

/dev/null > ~/.bash_history

and that cleared out the file, but I still have a history I can scroll through. I don’t know where they are stored. Go figure. But that’s a separate issue. Regardless the commands stored in .bash_history looked fine even with extended command lengths beyond 41 characters.

If anyone has a solution to the prompt/history issue, I would appreciate learning of it. Thank you.

2

The reply is only about your tangent query, and not the main question:

Also, I thought my commands were stored in ~/.bash_history, but that doesn’t seem to be the case.

By default ~/.bash_history is only updated with the command history of the current session when the session ends. Myself, I find that incredibly annoying, and so modified my ~/.bashrc file to append commands on the fly. The cost being interleaving of commands from multiple simultaneous sessions.

doug@s19:~$ diff .bashrc.24.04.original .bashrc
19,20c19,20
< HISTSIZE=1000
< HISTFILESIZE=2000
---
> HISTSIZE=10000
> HISTFILESIZE=20000
117a118,126
>
... deleted some private stuff ...
>
> PROMPT_COMMAND="history -a;$PROMPT_COMMAND"
1 Like
3

@dsmythies, after I posted, I thought it might be something like the current session being held only in cache. I agree that’s annoying. Thanks for the tip.

4

Sounds like you have some non-printing characters – probably escape-sequences for the terminal – in your prompt and haven’t properly marked them as non-printing for the shell. Because of this the shell calculates the length of the prompt to be longer than it actually is and doesn’t position the text right. Insert ‘[’ before the beginning of any such sequence and ‘]’ after the end (search the manual page for bash for ‘PROMPTING’ for details). I had similar problems in the past and it was always something in the prompt …

2 Likes
5

@hdd-gehrke, thank you, this explains a lot, and gets me closer to a solution. The gnu manual, section 6.9, is the relevant test that says to use \[ and \] to surround non-printing sequences. I notice that the interpreter for this text window removes the backslashes before the brackets, so I had to double-slash them.

https://www.gnu.org/software/bash/manual/bash.html#Controlling-the-Prompt

Here’s my original prompt declaration from .bashrc:

PS1=“${IYellow}${On_Blue}[\w]$Color_Off\n${IWhite}${On_Blue}[\h-\u]:$Color_Off”

I have the following relevant color definitions in a file called .dircolors

IYellow=‘\e[0;93m’ # Intense Yellow
IWhite=‘\e[0;97m’ # Intense White
On_Blue=‘\e[44m’ # Blue background

This file is referenced by the following line in .bashrc:

eval $(dircolors ~/.dircolors)

I changed my prompt to:

PS1=“\[${IYellow}\]\[${On_Blue}\][\w]\[$Color_Off\][\n\]\[${IWhite}\]\[${On_Blue}\][\h-\u]:\[$Color_Off\]”

This almost solves the problem. If I up-arrow through history and come across a long command, it will still extend one character past the prompt on all subsequent commands in the history.

I entered the following sequence of commands:

[~/computer/test]
[home5-randy]:PPS1=“[${IYellow}][${On_Blue}][\w][$Color_Off][\n][${IWhite}][${On_Blue}][\h-\u]:[$Color_Off]”
[~/computer/test]
[home5-randy]:echo “1123456789012345678901234567890123456789012345”
123456789012345678901234567890123456789012345
[~/computer/test]
[home5-randy]:ls *.txt
temp.txt test.txt

Then stepped back through the history:

<up-arrow>
[~/computer/test]
[home5-randy]:ls *.txt
<up-arrow>
[~/computer/test]
[home5-randy]:echo “123456789012345678901234567890123456789012345”
<up-arrow>
[~/computer/test]
[home5-randy]:ePS1="\[${IYellow}\]\[${On_Blue}\][\w]\[$Color_Off\]\[\n]\[${IWhite}\]\[${On_Blue}\][\h-\u]:\[$Color_Off\]
<down-arrow>
[~/computer/test]
[home5-randy]:echo "123456789012345678901234567890123456789012345o
<down-arrow>
[~/computer/test]
[home5-randy]:els *.txt

Again, I’m having to double-backslash to have it show up properly in this textbox.
You’ll notice the extra “o” and lack of ending quote of the echo command when I’m scrolling forward through the history. This is only the display and has no effect if I were to hit return to accept the command, which comes from the history buffer and not the display.

I thought maybe my prompt was too complicated, so I simplified it by only having one line:

PS1=“[${IYellow}][${On_Blue}][\u][\w]:[$Color_Off]”

This works perfectly. Hmmm… The “\n” linefeed in Linux is not only doing a linefeed but also a carriage return. Are two escape sequences embedded in one? I tried various combinations of linefeed (\n), carriage return (/r), open one line below (\o), and move down one line without carriage return (\j). All permutations made the problem worse, not better.

I can get rid of the line feed entirely, so I have a single line prompt. This solves the problem, but I’m just being an-- about it. I want my prompt to start in the same column always. If I get deep into a directory tree, my cursor is over to the right side of the screen, not what I want. And I like having the extra information for when I remote into other computers on my LAN.

So the problem seems to be with how BASH interprets the line feed when it’s used in a prompt. BASH sees it as a single character, when really it’s two. I don’t know how to solve that.

As a side not, I tried putting the \[ and \] around the escape sequences in the color definitions in .dircolors, but that didn’t work. They need to be in the prompt definition.

6

There is an interesting solution to this via adding the second half of your prompt to a function that uses printf:

https://askubuntu.com/a/1197614

7

I don’t know if this is relevant but I note you are using Xfce as your DE but the problem is in gnome-terminal.

Do you have the same problem if you use xfce4-terminal instead of gnome-terminal?

1 Like
8

@ajgreeny, yes, I get the same behavior from xfce4-terminal, XTerm, and UXTerm. They all perform exactly the same way with regard to the prompt.

9

Which isn’t surprising at all since it’s the shell and not the terminal emulator that’s the cause of the problem.

10

@ogra, I tried that, and it does put my cursor down on the net line, but it has the same problem with scrolling through long prompts in the history buffer, but now the issue is on the next line down.

I also looked at the other solution which is calling chroot for every time a new prompt is created, and that just strikes me as crazy without even trying it.

11

LOL, you mis-read it … this is actually the default prompt on debian systems, if you chroot (or go into a container) on debian it populates the $debian_chroot variable so that the chroot name is in front of your prompt, it does not call chroot :wink:

1 Like
12

I might have found a solution to this problem. To recap, my original prompt had special characters which did not print, such as changing colors and line feeds. @hdd-gehrke clued me in that I needed to surround these non-printing characters, and the variables I used to reference them, with “\[” and “\]”. These special characters told bash to not count the enclosed characters when computing the prompt length. I did so, and this worked well, except that there was still one extra non-printing character that wasn’t being accounted for. When I would scroll back through my command history and come across a longer command, the first character of that command would “stick” to the end of my prompt. This was still annoying, but better than it was before.

I guessed that the linefeed character “\n” was the problem, because in Linux this one character handles both linefeed and carriage return, so perhaps under the hood it was being translated into two special characters. I could find I could issue a carriage return with “\r”, but couldn’t find an escape character for a line feed alone without a carriage return.

Today, I had an idea to enclose a normal, printing character within the non-printing counting brackets, as a way to fool the underlying bash prompt engine as to what the real prompt length is. I did so with the final colon as the non-printing (but really printing) character. In my short, 15 minutes of testing, my prompt is now behaving exactly as I want it.

Here’s what my prompt string looks like now:

PS1=“\[${IYellow}\]\[${On_Blue}\][\w]\[$Color_Off\]\[\n\]\[${IWhite}\]\[${On_Blue}\][\h-\u]\[:\]\[$Color_Off\]”

I know it looks confusing as spaghetti code, but that’s what does the job. I tried combining bracketing between adjacent non-printing characters, such as "\[${IYellow}${On_Blue}\], but that didn’t work. I will give it another day or two of testing before accepting this solution, if in fact it continues to behave properly.

13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.